From 56bcbcd5689b449ee8f6202b9498056f9a0a8d28 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Fri, 11 Sep 2026 10:35:28 -0400 Subject: [PATCH 001/338] ci: fork automation (upstream sync + OCR AI review) Squashes the four CI commits this fork carried on top of upstream into one, and cuts the carry from 3776 lines to ~1000. Removed: - .github/{README,QUICKSTART}.md and .github/docs/* (2273 lines). Beyond being bulk we rebase hourly, they had drifted into being wrong: they described a daily (not hourly) sync, Claude 3.5 Sonnet via the direct Anthropic API (we run Opus on Bedrock), a static IAM user (we use OIDC), and a Windows dependency-builder workflow that the same commit which documented it had already deleted. 16 referenced paths did not exist. An AI reviewer reading this repo ingests that as truth. - sync-upstream-manual.yml (252 lines). sync-upstream.yml already has workflow_dispatch; the sole behavioral difference was a force_push toggle whose "false" setting cannot work after a rebase anyway. - .github/.gitignore, which only ignored scripts/ai-review/, a directory that does not exist here. Rewritten: - sync-upstream.yml -> fork-sync-upstream.yml, 260 lines to 47. git rebase already handles the fast-forward, no-op and diverged cases that the ahead/behind arithmetic hand-rolled, so all that goes away, as do the issue open/comment/close machinery (a scheduled-run failure is already emailed to the one consumer, and the run log carries what the issue body would) and the "dev setup|dev v[0-9]" commit allowlist, which matched zero commits on master and had since it was written. Sync drops to every three hours: hourly meant 24 full-history clones of an 840MB repo per day, each force-push also re-triggering pg-ci.yml on master, to collect a handful of upstream commits. Renamed: - ocr-review.yml, ocr-model-check.yml -> fork-*.yml. Upstream owns .github/ too (it has touched pg-ci.yml three times this year), so fork-owned workflows now live under a name prefix upstream will not collide with, making the rebase structurally conflict-free and letting the sync guard match fork-owned paths exactly rather than allowing all of .github/. Unchanged: the OCR review workflows' logic and .github/ocr/* (rule.json, context.md, litellm.yaml, pg-history.py) - the part that carries value. --- .github/ocr/context.md | 126 ++++++ .github/ocr/litellm.yaml | 41 ++ .github/ocr/pg-history.py | 225 +++++++++++ .github/ocr/rule.json | 65 ++++ .github/workflows/fork-ocr-model-check.yml | 89 +++++ .github/workflows/fork-ocr-review.yml | 427 +++++++++++++++++++++ .github/workflows/fork-sync-upstream.yml | 47 +++ 7 files changed, 1020 insertions(+) create mode 100644 .github/ocr/context.md create mode 100644 .github/ocr/litellm.yaml create mode 100644 .github/ocr/pg-history.py create mode 100644 .github/ocr/rule.json create mode 100644 .github/workflows/fork-ocr-model-check.yml create mode 100644 .github/workflows/fork-ocr-review.yml create mode 100644 .github/workflows/fork-sync-upstream.yml diff --git a/.github/ocr/context.md b/.github/ocr/context.md new file mode 100644 index 0000000000000..c4a83b85b124e --- /dev/null +++ b/.github/ocr/context.md @@ -0,0 +1,126 @@ +# OCR review context — PostgreSQL contribution standards + +You are reviewing a change to a **PostgreSQL** fork. Every PR here is destined to +become a patch posted to the **pgsql-hackers** mailing list and tracked in a +**commitfest**. Review with the combined rigor, taste, and attention to detail of +the PostgreSQL committers. This context applies to the *whole* change, on top of +the per-file rules. + +## Review discipline +- Be precise and blunt; lead with the most serious problem. No praise, no + validation of the author, no disclaimers — accuracy is the only metric. +- Verify every claim against the actual diff. Confirm names, signatures, line + numbers, and APIs before asserting. Never invent behavior or cite code not in + the change. If unsure, say so, and tag each finding **high / moderate / low** + confidence. +- Judge the change on its merits regardless of how the PR frames it. A draft PR + is WIP: weight design/approach feedback over style nits. + +## Patch hygiene (top rejection reasons on -hackers) +1. **Minimal diff.** The fastest way to get a patch rejected is unrelated + changes: reformatting untouched lines, rewording unrelated comments, touching + code not required by the change. Flag any hunk not needed for the stated + purpose. After the patch, the code should read as if it had always been + written that way. +2. **Atomic, bisectable commits.** Each commit must build and pass tests on its + own — a broken intermediate commit breaks `git bisect`, revert, and + cherry-pick. Flag a commit that only compiles once a later commit lands. + Prefer one focused patch, or a clearly-ordered series of + independently-committable pieces. +3. **Tests + docs are mandatory.** A user-visible change without regression/TAP + tests **and** documentation is WIP, not commit-ready. New behavior needs + tests that cover edge and error paths, not just the happy path. +4. **DRY / reuse.** Prefer existing infrastructure (`List` in `pg_list.h`, + `StringInfo`, `dynahash`/`simplehash`, `palloc`/`MemoryContext`, `foreach`) + over reinventing it. Flag copy-paste and speculative abstraction alike — the + community wants minimal, targeted changes that fit the subsystem's existing + patterns. +5. **Whitespace.** No trailing whitespace; tabs (width 4) for C indentation; + `git diff --check` must be clean. Whitespace-only churn on untouched lines is + a defect. + +## Committer-owned files — do NOT touch in a patch (flag if present) +These are the committer's job at push time; including them causes needless +merge conflicts and is a mistake: +- **`src/include/catalog/catversion.h`** — the `CATALOG_VERSION_NO` bump is done + by the **committer** when pushing. A catversion bump in the PR is **wrong** — + flag it. (This is the single most common author mistake in catalog patches.) +- **Release notes** (`doc/src/sgml/release-*.sgml`) and version strings + (`configure.ac` `AC_INIT` version, `meson.build` `version`, `PG_VERSION`). + +## Generated files — never hand-edit; edit the source +Flag direct edits to generated output; point the author at the source instead: +- Catalog headers `src/include/catalog/*_d.h`, `postgres.bki`, `schemapg.h`, + `system_constraints.sql` → edit the `pg_*.dat` files. +- `src/backend/nodes/{copy,equal,out,read}funcs.c` and other + `gen_node_support.pl` output → annotate the `Node` struct in its header. +- `fmgroids.h`, `fmgrprotos.h`, `fmgrtab.c` → edit `pg_proc.dat`. +- `utils/errcodes.h` → `errcodes.txt`; wait-event headers → + `wait_event_names.txt`; `lwlocknames.h` → `lwlocknames.txt`. +- `configure` → `configure.ac`; `*.po` translations are handled separately; + generated Unicode tables come from their source scripts. + +## Portability is a hard gate +PostgreSQL runs on Linux, Windows (MSVC), macOS, the BSDs and Solaris, across +**x86_64, ARM64, RISC-V, PPC64, s390x**, both endiannesses and 32/64-bit. Any +change must be portable across all of them: +- No unaligned memory access; no dependence on `char` signedness, integer/pointer + width, endianness, or struct padding for on-disk/wire formats. +- Use `int16/int32/int64`, `Size`, and `INT64_FORMAT`/`UINT64_FORMAT` (never + `%ld` for `int64`). +- Atomics/barriers only via `port/atomics` (`pg_atomic_*`, `pg_read/write_barrier`). +- **Windows/MSVC:** any `extern` variable used from another module or an + extension needs `PGDLLIMPORT` in its header; no VLAs or compiler-specific + extensions beyond the tree's C99 baseline. + +## Backward compatibility — the strongest constraint +Do not break SQL behavior, the libpq wire protocol, the logical-replication +protocol, dump/restore, `pg_upgrade`, or exported/`PGDLLIMPORT` APIs without +extraordinary justification. **ABI** matters for back-branches: changing the +size/layout of an exported struct or the signature of an exported function +breaks installed extensions. + +## Mailing-list context & etiquette +Because each PR becomes a pgsql-hackers email read by a busy, expert, opinionated +audience, also flag what reliably wastes reviewer time or draws rejection: +- A patch that **does more than one thing** or bundles unrelated cleanup — split it. +- **Footguns**: easy-to-misuse APIs, silent data-loss/corruption hazards, unsafe + defaults — name them explicitly. +- **Performance claims without a reproducible benchmark.** +- No reference to the **design discussion / prior -hackers thread** (Message-Id) + for a non-trivial change. +- **Do not bikeshed:** keep style nits proportionate and clearly separated from + substantive correctness findings. + +## Minimalism — the "ponytail" discipline +The best code is the code you never wrote (YAGNI). Before accepting new code, +apply the ladder: (1) Does this need to exist at all? (2) Can existing +code/infrastructure already do it? (3) Is this the simplest thing that works? +Flag: speculative scaffolding and config for a path that isn't wired yet; dead +code and unused "flexibility" (fields, params, abstractions, options with no +caller); premature abstraction (a helper used exactly once); knobs/GUCs/flags +nobody asked for. Minimal, targeted changes that fit the existing patterns beat +clever or general-purpose ones. + +## Comment & identity accuracy +- Comments must describe what the code does **now**. Flag aspirational/ + future-tense comments for behavior that already shipped ("will be", "for now", + "not yet", "future", and stale "TODO/FIXME/XXX/HACK"); comments that drifted + from the code they sit above; and incomplete/trailing comments. Comments + explain **why**, not what. No commented-out code. +- **ASCII only** in source and diffs — no smart quotes, em-dashes, or ellipsis + characters. + +## Commit & versioning discipline +- Conventional-commit style, imperative subject, one logical change per commit, + each commit building on its own. +- Do **not** bump version numbers or generated version stamps (including + `catversion.h`) — that is the maintainer's job at commit/release time. + +Understand common list shorthand so your comments are precise and not +miscommunicated: WIP (work in progress), GUC (config variable), WAL, LSN, OID, +TOAST, FSM, TAM (table access method), RLS, DSM, 2PC, PITR, CIC (concurrent index +creation), SAOP, ABI/API, backpatch (apply to supported back-branches), HEAD +(master tip), catversion (catalog version), pgindent, buildfarm, cfbot, +`s/x/y/` (suggested text substitution), footgun, bikeshedding, POLA (principle of +least astonishment). diff --git a/.github/ocr/litellm.yaml b/.github/ocr/litellm.yaml new file mode 100644 index 0000000000000..e23cc4eee6fe2 --- /dev/null +++ b/.github/ocr/litellm.yaml @@ -0,0 +1,41 @@ +# LiteLLM proxy config — bridges Open Code Review (OpenAI protocol) to AWS Bedrock. +# +# This proxy is NOT a hosted service. The ocr-review.yml workflow installs it +# (`pip install 'litellm[proxy]'`) and runs it as a background process bound to +# 127.0.0.1:4000 for the duration of a single GitHub Actions job, then it exits. +# +# Auth to Bedrock: LiteLLM uses boto3's default credential chain, which reads +# the temporary AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_SESSION_TOKEN +# minted by the workflow's OIDC "Configure AWS credentials" step; region from +# AWS_REGION. + +model_list: + - model_name: ocr-bedrock + litellm_params: + # Set the repo variable OCR_BEDROCK_MODEL to an Opus inference-profile id + # your account has access to, e.g.: + # bedrock/converse/us.anthropic.claude-opus-4-8 + # The 'converse/' prefix uses Bedrock's Converse API, which is the most + # reliable path for Claude tool-use (what OCR relies on). + model: os.environ/OCR_BEDROCK_MODEL + aws_region_name: os.environ/AWS_REGION + + # "High effort" review. Claude Opus 4.8 on Bedrock uses *adaptive* thinking + # controlled by output_config.effort. Set it DIRECTLY here — NOT via + # reasoning_effort, which LiteLLM still maps to the legacy + # thinking.type.enabled that Opus 4.8 rejects. LiteLLM forwards + # output_config into additionalModelRequestFields for Anthropic models; if + # the build doesn't recognize the effort param it is dropped with a warning + # (no error) and the model reviews at its default effort. + # Valid: low|medium|high|max|xhigh (auto-clamped to the model ceiling). + output_config: + effort: xhigh + max_tokens: 32000 + +litellm_settings: + drop_params: true # silently drop params a model doesn't support + modify_params: true # auto-fix minor request incompatibilities + request_timeout: 600 + +general_settings: + master_key: os.environ/LITELLM_MASTER_KEY diff --git a/.github/ocr/pg-history.py b/.github/ocr/pg-history.py new file mode 100644 index 0000000000000..5794f8a920bd7 --- /dev/null +++ b/.github/ocr/pg-history.py @@ -0,0 +1,225 @@ +#!/usr/bin/env python3 +""" +pg-history: tie a PR's changes to PostgreSQL git + pgsql-hackers email history. + +OCR (the code reviewer) cannot call MCP servers, so this is a separate agent: +it runs a Bedrock (Claude Opus) tool-use loop wired to the Agora MCP server at +https://pg.ddx.io/mcp, lets the model search the mailing-list archives / commit +history / commitfest data, and emits a Markdown summary linking the changes to +the relevant threads (https://pg.ddx.io/m/pgsql-hackers/). + +Env: + PG_HISTORY_MCP_URL MCP endpoint (default https://pg.ddx.io/mcp) + PG_HISTORY_MODEL Bedrock model id (e.g. us.anthropic.claude-opus-4-8) + AWS_REGION region (creds come from the OIDC step's env) + BASE_REF, HEAD_SHA PR base ref and head sha (for the git diff context) + GH_PR_TITLE PR title (optional, adds context) + PG_HISTORY_OUT output markdown path (default /tmp/pg-history.md) +Writes the markdown to PG_HISTORY_OUT; exits 0 even on soft failures (writes a note). +""" +import json, os, subprocess, sys, urllib.request + +MCP_URL = os.environ.get("PG_HISTORY_MCP_URL", "https://pg.ddx.io/mcp") +MODEL = os.environ.get("PG_HISTORY_MODEL", "us.anthropic.claude-opus-4-8").replace("bedrock/converse/", "").replace("bedrock/", "") +REGION = os.environ.get("AWS_REGION", "us-east-1") +BASE_REF = os.environ.get("BASE_REF", "") +HEAD_SHA = os.environ.get("HEAD_SHA", "") +PR_TITLE = os.environ.get("GH_PR_TITLE", "") +OUT = os.environ.get("PG_HISTORY_OUT", "/tmp/pg-history.md") +UA = "pg-history/0.1 (+github-actions)" + +# Curated subset of the 108 Agora tools — the ones useful for connecting a +# change to its discussion/commit history. Intersected with what the server +# actually exposes, so unknown names are harmless. +TOOL_WHITELIST = { + "find_related_discussions", "find_similar_messages", "get_thread", + "discussion_links", "get_author_messages", "browse_by_date", + "blame_symbol", "check_upstream_status", "find_related", + "find_entries_for_thread", "find_entries_for_author", "get_commit", + "search", "hybrid_search", "get_callers", "get_callees", "find_pattern", +} +MAX_ROUNDS = 14 +TOOL_RESULT_CAP = 8000 # chars per tool result fed back to the model + + +def _mcp_post(body, sid=None): + headers = {"Content-Type": "application/json", + "Accept": "application/json, text/event-stream", "User-Agent": UA} + if sid: + headers["Mcp-Session-Id"] = sid + req = urllib.request.Request(MCP_URL, data=json.dumps(body).encode(), headers=headers, method="POST") + resp = urllib.request.urlopen(req, timeout=60) + sid_out = resp.headers.get("Mcp-Session-Id") + result = None + for line in resp.read().decode().splitlines(): + line = line.strip() + if line.startswith("data:"): + line = line[5:].strip() + if not line or line.startswith("event:"): + continue + try: + obj = json.loads(line) + except Exception: + continue + if isinstance(obj, dict) and ("result" in obj or "error" in obj): + result = obj + return result, sid_out + + +class MCP: + def __init__(self): + init, self.sid = _mcp_post({"jsonrpc": "2.0", "id": 1, "method": "initialize", + "params": {"protocolVersion": "2025-06-18", "capabilities": {}, + "clientInfo": {"name": "pg-history", "version": "0.1"}}}) + if not init or "result" not in init: + raise RuntimeError(f"MCP initialize failed: {init}") + try: + _mcp_post({"jsonrpc": "2.0", "method": "notifications/initialized", "params": {}}, self.sid) + except Exception: + pass + self._id = 1 + + def list_tools(self): + self._id += 1 + res, _ = _mcp_post({"jsonrpc": "2.0", "id": self._id, "method": "tools/list", "params": {}}, self.sid) + return (res or {}).get("result", {}).get("tools", []) + + def call(self, name, args): + self._id += 1 + res, _ = _mcp_post({"jsonrpc": "2.0", "id": self._id, "method": "tools/call", + "params": {"name": name, "arguments": args or {}}}, self.sid) + if not res: + return "(no response)" + if "error" in res: + return f"ERROR: {json.dumps(res['error'])[:500]}" + parts = [] + for c in res.get("result", {}).get("content", []): + if c.get("type") == "text": + parts.append(c["text"]) + return ("\n".join(parts) or "(empty)")[:TOOL_RESULT_CAP] + + +def git(*args): + try: + return subprocess.check_output(["git", *args], text=True, stderr=subprocess.DEVNULL).strip() + except Exception: + return "" + + +def pr_context(): + base = f"origin/{BASE_REF}" if BASE_REF else "" + rng = f"{base}..{HEAD_SHA}" if base and HEAD_SHA else HEAD_SHA + commits = git("log", "--no-merges", "--format=%h %s", f"{rng}") if rng else "" + stat = git("diff", "--stat", rng) if rng else "" + files = git("diff", "--name-only", rng) if rng else "" + return commits[:4000], stat[:3000], files[:2000] + + +SYSTEM = """You are a PostgreSQL community research assistant. Given a pull request's +commits and changed files, use the available tools (backed by the Agora index of +pgsql-hackers mail, commit history, and commitfest data) to connect the change to +its history. Your goal: + +- Find the mailing-list thread(s) and prior discussion behind this change. +- Identify related/superseded prior commits and any commitfest entry. +- Note relevant prior art, rejected approaches, or design rationale. + +Rules (voice & rigor): +- Be precise and blunt. No praise, no filler, no hedging, no disclaimers. Accuracy is + the only success metric — not the author's approval. Lead with the most important finding. +- NEVER hallucinate. Verify every Message-ID, thread subject, commit hash, author name, + and date against an actual tool result before citing it. If a search returns nothing, + say so plainly — do not guess or fabricate a plausible-looking link. +- Assess the change on its merits, independent of how the PR frames it. +- Tag any inferred (not tool-confirmed) linkage with an explicit confidence level: + high / moderate / low. +- Be decisive and efficient: a handful of targeted tool calls, not exhaustive search. +- Cite every mailing-list message as a Markdown link: [subject](https://pg.ddx.io/m/pgsql-hackers/MESSAGE_ID). +- If you find nothing relevant, say so in one line — do not pad. + +When done, output ONLY Markdown (no preamble) with these sections, omitting any that are empty: +## 🧵 Related discussion +## 🔗 Related commits / prior art +## 📋 Commitfest +## 🧭 Context for reviewers +Keep it tight (use bullets; link generously).""" + + +def to_toolspec(t): + schema = t.get("inputSchema") or {"type": "object", "properties": {}} + return {"toolSpec": {"name": t["name"], + "description": (t.get("description") or "")[:600], + "inputSchema": {"json": schema}}} + + +def main(): + commits, stat, files = pr_context() + if not commits and not files: + open(OUT, "w").write("") # nothing to do + print("No PR diff context; skipping.") + return + user = (f"PR title: {PR_TITLE}\n\n" if PR_TITLE else "") + \ + f"Commits:\n{commits or '(none)'}\n\nChanged files:\n{files or '(none)'}\n\nDiffstat:\n{stat or '(none)'}\n" + + try: + mcp = MCP() + tools = [to_toolspec(t) for t in mcp.list_tools() if t.get("name") in TOOL_WHITELIST] + except Exception as e: + open(OUT, "w").write(f"_pg-history: could not reach the Agora MCP server ({MCP_URL}): {e}_\n") + print(f"MCP unavailable: {e}") + return + if not tools: + open(OUT, "w").write("_pg-history: no usable MCP tools available._\n") + return + + import boto3 + from botocore.config import Config + + # botocore's default read timeout (60s) is too short for a multi-round + # (MAX_ROUNDS) tool-use loop against a large PR diff on a reasoning model; + # each converse() call alone can take several minutes. Bump it well past + # what a single round needs; connect_timeout stays short since a stuck + # TCP handshake is a different (and much cheaper to detect) failure mode. + brt = boto3.client("bedrock-runtime", region_name=REGION, + config=Config(read_timeout=900, connect_timeout=10)) + messages = [{"role": "user", "content": [{"text": user}]}] + final_text = "" + try: + for _ in range(MAX_ROUNDS): + resp = brt.converse( + modelId=MODEL, + system=[{"text": SYSTEM}], + messages=messages, + toolConfig={"tools": tools}, + inferenceConfig={"maxTokens": 4096}, + ) + out = resp["output"]["message"] + messages.append(out) + if resp.get("stopReason") == "tool_use": + results = [] + for blk in out["content"]: + tu = blk.get("toolUse") + if not tu: + continue + res_text = mcp.call(tu["name"], tu.get("input") or {}) + results.append({"toolResult": {"toolUseId": tu["toolUseId"], + "content": [{"text": res_text}]}}) + messages.append({"role": "user", "content": results}) + continue + final_text = "".join(b.get("text", "") for b in out["content"]).strip() + break + except Exception as e: + open(OUT, "w").write(f"_pg-history: Bedrock call failed: {e}_\n") + print(f"Bedrock error: {e}") + return + + if not final_text: + final_text = "_pg-history: no related history found._" + body = "## 📜 Change history & discussion (Agora / pg.ddx.io)\n\n" + final_text + \ + "\n\nGenerated by pg-history via the Agora MCP server (pg.ddx.io).\n" + open(OUT, "w").write(body) + print(body) + + +if __name__ == "__main__": + main() diff --git a/.github/ocr/rule.json b/.github/ocr/rule.json new file mode 100644 index 0000000000000..60e13e73dcbe0 --- /dev/null +++ b/.github/ocr/rule.json @@ -0,0 +1,65 @@ +{ + "_comment": "OCR per-file review rules for PostgreSQL core + extensions. Cross-cutting contribution standards & mailing-list etiquette live in .github/ocr/context.md, passed via --background-file. OCR uses FIRST-MATCH-WINS in declaration order, so rules are ordered most-specific first. merge_system_rule:true keeps OCR's built-in fine-tuned checks (thread-safety, injection, NPE) alongside these PostgreSQL-specific rules.", + "rules": [ + { + "path": "src/test/**", + "merge_system_rule": true, + "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL tests. Coverage is mandatory for any behavioral change and must include edge cases (NULL, empty, boundary/overflow) and ERROR paths, not just the happy path. A test that still passes with the feature reverted is worthless — confirm it actually exercises and would catch regressions in the new code. Regression (.sql/expected): deterministic, portable output — ORDER BY where row order matters, no timing/plan-dependent output except intentional EXPLAIN, no absolute paths, locale-independent (C collation or explicit COLLATE), DROP objects the test creates; expected/ output must stay stable across platforms and under the parallel schedule. Concurrency/locking belongs in isolation tests (src/test/isolation, .spec + permutations). End-to-end/crash/replication/CLI behavior belongs in TAP tests (t/*.pl with PostgreSQL::Test::Cluster/Utils) — no hardcoded ports/paths, no sleep as synchronization (use poll_query_until/wait_for), skip cleanly when prerequisites are missing, and clean up nodes." + }, + { + "path": "**/*.{c,h}", + "merge_system_rule": true, + "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL backend/frontend C — review as pgsql-hackers committers do, in priority order.\n\n(1) CORRECTNESS (highest): Memory — every palloc lives in the right MemoryContext; error paths via ereport/elog(ERROR) must not leak memory/buffers/locks/fds (rely on MemoryContext/ResourceOwner reset or PG_TRY/PG_FINALLY); no use-after-free; delete temp contexts. Concurrency — consistent lock ordering (deadlock-free), correct lock levels, balanced LWLockAcquire/Release and START_/END_CRIT_SECTION, no TOCTOU, CHECK_FOR_INTERRUPTS in long loops, async-signal-safe signal handlers (volatile sig_atomic_t). WAL — any change to shared on-disk state must be WAL-logged AND correctly replayed (redo path), crash- and replica-consistent. NULL/edge/overflow handling.\n\n(2) BACKWARD COMPATIBILITY / ABI: don't break behavior, dump/restore, pg_upgrade, libpq wire protocol, logical-replication protocol, or exported/PGDLLIMPORT'd APIs (struct size/layout, function signatures) without extraordinary justification.\n\n(3) CATALOG / GENERATED: new/changed catalog data goes in pg_*.dat, NOT the generated *_d.h/.bki. New Node types: ANNOTATE the struct in its header so gen_node_support.pl regenerates copy/equal/out/read — do NOT hand-edit *funcs.c. New SQL-callable functions: add to pg_proc.dat with an OID from the 8000-9999 developer range (src/include/catalog/unused_oids; check duplicate_oids); committer renumbers at commit. DO NOT bump CATALOG_VERSION_NO in the patch — flag any catversion.h change as a mistake (committer's job).\n\n(4) PERFORMANCE: no regression on hot paths; avoid O(n^2) where better is feasible; minimize work under contended locks; avoid needless palloc churn and large struct copies in hot paths.\n\n(5) SECURITY: bounded string ops (snprintf/strlcpy/strlcat — never strcpy/strcat/sprintf); integer/size-overflow checks before allocation; never user input as a format string; privilege checks via pg_*_aclcheck; beware search_path and SECURITY DEFINER.\n\n(6) PORTABILITY (hard gate): no unaligned access; no dependence on char signedness, int/long/pointer width, endianness, or struct padding for on-disk/wire formats; use int16/int32/int64 + INT64_FORMAT/UINT64_FORMAT (never %ld for int64); align contended shared structs (pg_attribute_aligned/cache-line pad). Atomics/barriers only via port/atomics (pg_atomic_*, pg_read/write_barrier) — never raw intrinsics or volatile-as-barrier. WINDOWS/MSVC: extern vars used cross-module/extension need PGDLLIMPORT; no VLAs or features beyond the C99 baseline the tree targets; use pg_pread/pg_pwrite. Applies across x86_64/ARM64/RISC-V/PPC64/s390x, big/little endian, 32/64-bit.\n\n(7) CONVENTIONS: errmsg starts lowercase, no trailing period, no embedded newlines; errdetail/errhint are complete capitalized sentences; correct ERRCODE_*; wrap user-facing text in _(); errmsg_plural for counts. Assert() only for can't-happen invariants (never user-reachable). Naming: snake_case with subsystem prefix (heap_insert) or CamelCase for major subsystems (ExecInitNode); ALL_CAPS macros. Must pgindent cleanly (tabs, width 4). Comments explain WHY not WHAT; no #ifdef 0 blocks, no commented-out code, no #ifdef fencing your feature. Reuse existing helpers (DRY)." + }, + { + "path": "**/*.dat", + "merge_system_rule": true, + "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL catalog data (pg_proc.dat, pg_type.dat, etc.) — the SOURCE for generated headers. The generated *_d.h, postgres.bki, fmgroids.h, fmgrtab.c must NOT be hand-edited (they regenerate from these files). OIDs: use a value from the developer range 8000-9999 (src/include/catalog/unused_oids; verify with duplicate_oids); committer renumbers to a final contiguous block, so stay in-range and unique but don't over-optimize the exact number. Keep proc entries complete/consistent (prosrc, provolatile, proparallel, prorettype/proargtypes, matching description). DO NOT bump CATALOG_VERSION_NO / catversion.h — committer's job at push time; flag any such change. New catalog columns/views need documentation in doc/src/sgml/catalogs.sgml." + }, + { + "path": "**/*.{sql,pgsql}", + "merge_system_rule": true, + "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL SQL. Valid PostgreSQL dialect (not MySQL/Oracle); correct types (bigint vs int, text vs varchar); sound transaction/isolation and CTE-materialization assumptions. SECURITY: flag SQL injection in dynamic SQL (require quote_identifier/quote_literal or format() with %I/%L), SECURITY DEFINER without a locked-down search_path, inappropriate RLS bypass. Prefer set-based over row-at-a-time/N+1. BACKWARD COMPATIBILITY (a top rejection reason): changing existing SQL behavior, the output of existing functions, or default GUCs needs extraordinary justification. New SQL-callable objects belong in pg_*.dat with OIDs from the 8000-9999 range, not in generated files. Minimal diff; add regression tests + docs." + }, + { + "path": "**/*.{pl,pm}", + "merge_system_rule": true, + "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL Perl (TAP tests and build/catalog tooling). Require 'use strict; use warnings;'. Must be perltidy-clean with the tree's src/tools/pgindent/perltidyrc and pass src/tools/perlcheck/pgperlcritic. Use the framework: PostgreSQL::Test::Cluster, PostgreSQL::Test::Utils, Test::More; no hardcoded ports/paths/PIDs; use safe_psql/poll_query_until, not sleep; skippable without optional prerequisites; clean up nodes. PORTABILITY: run on Windows (no fork-only constructs, use File::Spec, avoid unavailable signals) and the minimum supported Perl. Robustness: avoid two-arg open and string system()/qx with interpolated data (use list forms). Generator scripts (gen_node_support.pl, catalog Perl) must be deterministic and stay in sync with inputs; do not commit their generated output." + }, + { + "path": "**/*.py", + "merge_system_rule": true, + "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL Python (build/test tooling, oauth/pytest tests, src/tools). Follow surrounding style; keep imports to the standard library unless the dependency is already required by the tree (no surprise third-party deps in build/test tooling). PORTABILITY: support the project's minimum Python 3 and run on Windows and the BSDs (use os.path/pathlib, avoid POSIX-only calls and shell=True with interpolated input). Deterministic, self-cleaning tests; no hardcoded ports/paths; skip cleanly without prerequisites. For the Perl->pytest porting effort, confirm behavior parity with the TAP test replaced (same assertions/coverage), not a superficial translation. Minimal diff; match the tree's ruff/black config if present." + }, + { + "path": "**/*.{rs,toml}", + "merge_system_rule": true, + "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. Rust PostgreSQL extension (pgrx) or Rust support crate. Not core C, but it runs inside/alongside the backend, so backend safety applies. SAFETY: in code reachable from an SQL call, a Rust panic aborts the Postgres process — forbid unwrap()/expect()/panic!/unreachable!/todo! and index-panics on reachable paths; use Result and pgrx error reporting (error!/ereport!). Every `unsafe` block needs a comment justifying its invariant; scrutinize raw pointers and FFI across the pg_sys boundary. pgrx: honor #[pg_guard] on extern C fns (correct panic/longjmp handling); never hold Rust references across SPI or anything that can longjmp (skips Rust destructors -> leaks); respect MemoryContext lifetimes for palloc'd data; datum<->Rust conversions must handle NULL. Concurrency uses Postgres shmem/LWLocks (pgrx shmem API), not std::sync alone. Lints: must pass `cargo clippy --all-targets --all-features -- -D warnings` and `cargo fmt --check`; deny unwrap_used/expect_used/panic in libraries; thiserror (libs) / anyhow (bins). Justify every new dependency. Tests: #[pg_test] for in-backend behavior, #[test] for pure logic; cover error and NULL paths. Minimal, idiomatic diff." + }, + { + "path": "**/{configure.ac,*.m4,aclocal.m4}", + "merge_system_rule": true, + "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL Autoconf. Edit configure.ac / the m4 macros — do NOT hand-edit generated 'configure' or pg_config.h.in in the same patch (regeneration is the committer's step; a patch that also rewrites generated configure output is suspect). Feature/header/function probes must be portable and not assume a specific OS/compiler. Every configure knob must be mirrored on the Meson side (meson_options.txt/meson.build) and documented. Minimal diff." + }, + { + "path": "**/{meson.build,meson_options.txt}", + "merge_system_rule": true, + "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL Meson build. Valid syntax; correct subdir()/dependency()/declare_dependency and install paths; new source files must be added here. CRITICAL: PostgreSQL maintains BOTH Meson and Autoconf/Make — any new file, option, or feature check must be mirrored on the configure.ac/Makefile side so the two never drift (a file built by only one system is a common defect). New options need matching docs and sensible defaults. Minimal diff." + }, + { + "path": "**/{Makefile,GNUmakefile,*.mk}", + "merge_system_rule": true, + "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL Makefile (GNU Make). $(VAR) refs; correct .PHONY; accurate dependencies (no parallel -j races); $(MAKE) for recursion; VPATH/out-of-tree build support; no hardcoded paths (use standard PostgreSQL makefile vars and $(top_builddir)); clean/distclean/maintainer-clean must remove new artifacts; extensions use PGXS. Must stay in sync with meson.build. Minimal diff." + }, + { + "path": "doc/**/*.sgml", + "merge_system_rule": true, + "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL documentation (DocBook SGML). Technically accurate/complete (parameters, limitations, version/compat notes); correct tag usage/nesting (, , , , , /); working cross-references; spell it 'PostgreSQL' in prose; SQL keywords uppercase in examples. Coverage: a new GUC -> config.sgml (and postgresql.conf.sample); new/changed catalogs or views -> catalogs.sgml; new SQL syntax -> the matching ref/*.sgml; new functions -> func.sgml. Do NOT edit release-notes (release-*.sgml) — written by the release team/committers; flag such edits. New user-facing behavior in this PR should ship with matching docs." + }, + { + "path": "**/*.md", + "merge_system_rule": true, + "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. Markdown docs. Clear heading hierarchy; fenced code blocks with language hints; accurate instructions/prerequisites; consistent PostgreSQL terminology; no broken relative links or stale claims. Minimal diff." + } + ] +} diff --git a/.github/workflows/fork-ocr-model-check.yml b/.github/workflows/fork-ocr-model-check.yml new file mode 100644 index 0000000000000..10d250528cf7c --- /dev/null +++ b/.github/workflows/fork-ocr-model-check.yml @@ -0,0 +1,89 @@ +# Checks AWS Bedrock weekly for a newer Claude Opus inference profile than the +# one OCR currently uses (vars.OCR_BEDROCK_MODEL) and, if found, opens/updates a +# single GitHub issue telling the maintainer to bump the variable. It does NOT +# change the model automatically: GITHUB_TOKEN cannot write Actions *variables* +# (that needs a PAT with admin), so this is a notify-only mechanism by design. +name: OCR model self-check + +on: + schedule: + - cron: '0 12 * * 1' # Mondays 12:00 UTC + workflow_dispatch: + +permissions: + id-token: write + contents: read + issues: write + +jobs: + check-model: + runs-on: ubuntu-latest + steps: + - name: Configure AWS credentials (OIDC) + uses: aws-actions/configure-aws-credentials@v6 + with: + role-to-assume: ${{ vars.AWS_ROLE_ARN }} + aws-region: ${{ vars.AWS_REGION }} + role-session-name: ocr-model-check-${{ github.run_id }} + + - name: Find newest Opus vs configured + id: check + env: + CURRENT: ${{ vars.OCR_BEDROCK_MODEL }} + AWS_REGION: ${{ vars.AWS_REGION }} + run: | + python3 - <<'PY' >> "$GITHUB_OUTPUT" + import os, re, subprocess, json + region = os.environ.get("AWS_REGION", "us-east-1") + current = os.environ.get("CURRENT", "") + out = subprocess.run( + ["aws", "bedrock", "list-inference-profiles", "--region", region, + "--query", "inferenceProfileSummaries[].inferenceProfileId", "--output", "json"], + capture_output=True, text=True) + ids = json.loads(out.stdout or "[]") + # Parse claude-opus-- from any profile id (prefix us./global. ok). + def ver(s): + m = re.search(r"claude-opus-(\d+)-(\d+)", s) + return (int(m.group(1)), int(m.group(2))) if m else None + opus = [(ver(i), i) for i in ids if ver(i) and i.startswith(("us.", "global."))] + if not opus: + print("newer=false"); raise SystemExit(0) + best_ver, best_id = max(opus, key=lambda x: x[0]) + cur = ver(current) + newer = (cur is None) or (best_ver > cur) + print(f"newer={'true' if newer else 'false'}") + print(f"best_id={best_id}") + print(f"best_ver={best_ver[0]}.{best_ver[1]}") + print(f"cur_ver={'unknown' if cur is None else f'{cur[0]}.{cur[1]}'}") + PY + + - name: Open/update issue if a newer model exists + if: steps.check.outputs.newer == 'true' + uses: actions/github-script@v9 + with: + script: | + const best = '${{ steps.check.outputs.best_id }}'; + const bestVer = '${{ steps.check.outputs.best_ver }}'; + const curVer = '${{ steps.check.outputs.cur_ver }}'; + const marker = ''; + const title = `OCR: newer Claude Opus available (${bestVer} > ${curVer})`; + const body = `${marker}\n` + + `A newer Claude Opus inference profile is available on Bedrock.\n\n` + + `- **Configured** (\`vars.OCR_BEDROCK_MODEL\`): Opus ${curVer}\n` + + `- **Newest on Bedrock**: \`${best}\` (Opus ${bestVer})\n\n` + + `To upgrade, set the repo variable:\n\n` + + '```\n' + + `gh variable set OCR_BEDROCK_MODEL -R ${context.repo.owner}/${context.repo.repo} \\\n` + + ` -b "bedrock/converse/${best}"\n` + + '```\n\n' + + `Also confirm the \`ocr-bedrock-ci\` IAM inline policy allows invoking the new model ` + + `(the resource is scoped to \`anthropic.claude-opus-*\`), then re-run OCR.\n\n` + + `_Automated by \`.github/workflows/ocr-model-check.yml\`; this issue is upserted._`; + const q = `repo:${context.repo.owner}/${context.repo.repo} in:body "${marker}" state:open`; + const found = await github.rest.search.issuesAndPullRequests({ q, per_page: 1 }); + if (found.data.total_count > 0) { + const n = found.data.items[0].number; + await github.rest.issues.update({ owner: context.repo.owner, repo: context.repo.repo, issue_number: n, title, body }); + } else { + await github.rest.issues.create({ owner: context.repo.owner, repo: context.repo.repo, title, body }); + } diff --git a/.github/workflows/fork-ocr-review.yml b/.github/workflows/fork-ocr-review.yml new file mode 100644 index 0000000000000..0828af429b57c --- /dev/null +++ b/.github/workflows/fork-ocr-review.yml @@ -0,0 +1,427 @@ +# Open Code Review (OCR) — AI PR review backed by AWS Bedrock via a LiteLLM proxy. +# +# Flow: +# PR opened/updated (incl. DRAFTS) ─┐ +# /open-code-review PR comment ─┼─► start LiteLLM (127.0.0.1:4000 → Bedrock) +# manual workflow_dispatch ─┘ └► ocr review --format json +# └► post inline PR review comments +# +# Required (repo settings — all repo *variables*, no secrets; auth is via GitHub OIDC): +# vars.AWS_ROLE_ARN - IAM role to assume via OIDC (granting bedrock:InvokeModel*) +# vars.AWS_REGION - e.g. us-east-1 +# vars.OCR_BEDROCK_MODEL - LiteLLM model string for the Opus inference profile, e.g. +# bedrock/converse/us.anthropic.claude-opus-4-8 +# +# No static AWS keys are stored. GITHUB_TOKEN (auto) posts the review comments. + +name: OCR AI Review + +on: + pull_request: + # Note: no draft filter — drafts are reviewed too. + types: [opened, synchronize, reopened, ready_for_review] + issue_comment: + types: [created] + workflow_dispatch: + inputs: + pr_number: + description: 'PR number to review' + required: true + type: number + +# One review per PR; cancel superseded runs to save Bedrock spend. +concurrency: + group: ocr-review-${{ github.event.pull_request.number || github.event.issue.number || github.event.inputs.pr_number }} + cancel-in-progress: true + +permissions: + id-token: write # required to mint the GitHub OIDC token for AWS role assumption + contents: read + pull-requests: write + +jobs: + ocr-review: + runs-on: ubuntu-latest + # PR events always; comment events only when the comment is on a PR and + # starts with the trigger keyword; manual dispatch always. + if: | + github.event_name == 'pull_request' || + github.event_name == 'workflow_dispatch' || + (github.event_name == 'issue_comment' && github.event.issue.pull_request && + (startsWith(github.event.comment.body, '/open-code-review') || + startsWith(github.event.comment.body, '@open-code-review'))) + + env: + # LiteLLM listens on localhost only; this key never leaves the runner. + LITELLM_MASTER_KEY: sk-ocr-ci-local + OCR_BEDROCK_MODEL: ${{ vars.OCR_BEDROCK_MODEL }} + # Region is a static var (safe at job level). AWS credentials are NOT set + # here — they're minted by the OIDC "Configure AWS credentials" step below + # and exported to the environment for the LiteLLM/boto3 Bedrock calls. + AWS_REGION: ${{ vars.AWS_REGION }} + + steps: + - name: Resolve PR context + id: pr + uses: actions/github-script@v9 + with: + script: | + let prNumber; + if (context.eventName === 'pull_request') { + prNumber = context.payload.pull_request.number; + } else if (context.eventName === 'issue_comment') { + prNumber = context.issue.number; + } else { + prNumber = parseInt('${{ github.event.inputs.pr_number }}', 10); + } + const { data: pr } = await github.rest.pulls.get({ + owner: context.repo.owner, + repo: context.repo.repo, + pull_number: prNumber, + }); + const { data: repo } = await github.rest.repos.get({ + owner: context.repo.owner, + repo: context.repo.repo, + }); + core.setOutput('number', String(prNumber)); + core.setOutput('base_ref', pr.base.ref); + core.setOutput('head_ref', pr.head.ref); + core.setOutput('head_sha', pr.head.sha); + core.setOutput('default_branch', repo.default_branch); + core.setOutput('cross_repo', String(pr.head.repo.full_name !== pr.base.repo.full_name)); + + # NOTE: do NOT checkout the PR head. OCR reads the diff and file contents + # straight from git refs (git diff , git show :path, + # git grep ), so the working tree is irrelevant — but our OCR config + # lives on the default branch, not on the PR branch. We check out the repo + # (default ref), fetch the base/head objects, and materialize the config + # from origin/. + - name: Checkout + uses: actions/checkout@v6 + with: + fetch-depth: 0 + + - name: Prepare git refs and OCR config + env: + BASE_REF: ${{ steps.pr.outputs.base_ref }} + HEAD_REF: ${{ steps.pr.outputs.head_ref }} + HEAD_SHA: ${{ steps.pr.outputs.head_sha }} + DEFAULT_BRANCH: ${{ steps.pr.outputs.default_branch }} + run: | + git fetch --no-tags origin "+refs/heads/${DEFAULT_BRANCH}:refs/remotes/origin/${DEFAULT_BRANCH}" || true + git fetch --no-tags origin "+refs/heads/${BASE_REF}:refs/remotes/origin/${BASE_REF}" || true + git fetch --no-tags origin "+refs/heads/${HEAD_REF}:refs/remotes/origin/${HEAD_REF}" || true + git fetch --no-tags origin "${HEAD_SHA}" || true + + # OCR config lives on the default branch; materialize it independently + # of whatever ref is checked out. + mkdir -p "$RUNNER_TEMP/ocr" + git show "origin/${DEFAULT_BRANCH}:.github/ocr/litellm.yaml" > "$RUNNER_TEMP/ocr/litellm.yaml" + git show "origin/${DEFAULT_BRANCH}:.github/ocr/rule.json" > "$RUNNER_TEMP/ocr/rule.json" + git show "origin/${DEFAULT_BRANCH}:.github/ocr/context.md" > "$RUNNER_TEMP/ocr/context.md" + echo "Config materialized:"; ls -l "$RUNNER_TEMP/ocr" + + - name: Setup Python + uses: actions/setup-python@v6 + with: + python-version: '3.12' + + - name: Setup Node.js + uses: actions/setup-node@v6 + with: + node-version: '20' + + - name: Install LiteLLM proxy + Open Code Review + run: | + python -m pip install --upgrade pip + # Pin LiteLLM to a main commit that supports Claude Opus 4.8 adaptive + # thinking (maps reasoning_effort -> output_config.effort, incl. xhigh). + # Not in any tagged release yet (PyPI latest 1.87.1 lacks the Opus + # normalizer). Bump this SHA once a release ships the feature. + pip install "litellm[proxy] @ git+https://github.com/BerriAI/litellm.git@5be0797d24a2f26eb2123e13788f90055a59d91d" + npm install -g @alibaba-group/open-code-review + + - name: Configure AWS credentials (OIDC) + uses: aws-actions/configure-aws-credentials@v6 + with: + role-to-assume: ${{ vars.AWS_ROLE_ARN }} + aws-region: ${{ vars.AWS_REGION }} + role-session-name: ocr-review-${{ github.run_id }} + + - name: Start LiteLLM proxy (Bedrock bridge) + run: | + if [ -z "$OCR_BEDROCK_MODEL" ]; then + echo "::error::vars.OCR_BEDROCK_MODEL is not set (e.g. bedrock/converse/us.anthropic.claude-opus-4-1-20250805-v1:0)" + exit 1 + fi + nohup litellm --config "$RUNNER_TEMP/ocr/litellm.yaml" --host 127.0.0.1 --port 4000 \ + > /tmp/litellm.log 2>&1 & + echo "Waiting for LiteLLM to become ready..." + for i in $(seq 1 60); do + if curl -sf http://127.0.0.1:4000/health/readiness >/dev/null; then + echo "LiteLLM ready."; exit 0 + fi + sleep 2 + done + echo "::error::LiteLLM did not become ready in time"; cat /tmp/litellm.log; exit 1 + + - name: Configure OCR + run: | + ocr config set llm.url http://127.0.0.1:4000/v1/chat/completions + ocr config set llm.auth_token "$LITELLM_MASTER_KEY" + ocr config set llm.model ocr-bedrock + ocr config set llm.use_anthropic false + ocr config set language English + + - name: Run OCR review + run: | + ocr review \ + --from "origin/${{ steps.pr.outputs.base_ref }}" \ + --to "${{ steps.pr.outputs.head_sha }}" \ + --rule "$RUNNER_TEMP/ocr/rule.json" \ + --background-file "$RUNNER_TEMP/ocr/context.md" \ + --concurrency 3 \ + --timeout 20 \ + --format json \ + > /tmp/ocr-result.json 2>/tmp/ocr-stderr.log || true + echo "----- OCR stdout -----"; cat /tmp/ocr-result.json || true + echo "----- OCR stderr -----"; cat /tmp/ocr-stderr.log || true + echo "----- LiteLLM log (tail) -----"; tail -n 50 /tmp/litellm.log || true + + - name: Post review to PR + uses: actions/github-script@v9 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + const fs = require('fs'); + const prNumber = parseInt('${{ steps.pr.outputs.number }}', 10); + const commitSha = '${{ steps.pr.outputs.head_sha }}'; + + // Opus at high effort can emit dozens of findings. Posting them all + // one-by-one trips GitHub's SECONDARY rate limit (403 "content + // creation"), which is what made every run fail after the review + // was already generated. We (a) cap inline comments and overflow + // the rest into the summary, (b) prefer a single bulk createReview, + // and (c) throttle + back off with Retry-After on any fallback. + const MAX_INLINE = 25; + const sleep = (ms) => new Promise(r => setTimeout(r, ms)); + + async function withRetry(fn, label) { + for (let attempt = 1; attempt <= 5; attempt++) { + try { return await fn(); } + catch (e) { + const status = e.status || (e.response && e.response.status); + const h = (e.response && e.response.headers) || {}; + const isRate = status === 403 || status === 429; + if (!isRate || attempt === 5) throw e; + let waitMs = 0; + if (h['retry-after']) waitMs = parseInt(h['retry-after'], 10) * 1000; + else if (h['x-ratelimit-reset']) waitMs = parseInt(h['x-ratelimit-reset'], 10) * 1000 - Date.now(); + if (!waitMs || Number.isNaN(waitMs) || waitMs < 0) waitMs = 1000 * Math.pow(2, attempt); + waitMs = Math.min(waitMs, 60000) + 500; + core.warning(`${label}: rate-limited (status ${status}); waiting ${Math.round(waitMs / 1000)}s (attempt ${attempt}/5)`); + await sleep(waitMs); + } + } + } + + let result; + try { + result = JSON.parse(fs.readFileSync('/tmp/ocr-result.json', 'utf8')); + } catch (e) { + const stderr = (() => { try { return fs.readFileSync('/tmp/ocr-stderr.log', 'utf8').trim(); } catch { return ''; } })(); + await withRetry(() => github.rest.issues.createComment({ + owner: context.repo.owner, repo: context.repo.repo, issue_number: prNumber, + body: `⚠️ **OCR** could not produce a review.\n\n\`\`\`\n${(stderr || e.message).slice(0, 8000)}\n\`\`\``, + }), 'error-comment'); + return; + } + + const comments = result.comments || []; + const warnings = result.warnings || []; + + const formatComment = (c) => { + let body = c.content || ''; + if (c.suggestion_code && c.existing_code) { + body += '\n\n```suggestion\n' + c.suggestion_code + (c.suggestion_code.endsWith('\n') ? '' : '\n') + '```'; + } + return body; + }; + const formatMarkdown = (c) => { + let md = `### 📄 \`${c.path}\``; + if (c.start_line && c.end_line) md += ` (L${c.start_line}-L${c.end_line})`; + md += '\n\n' + (c.content || ''); + if (c.suggestion_code && c.existing_code) { + md += '\n\n
💡 Suggested change\n\n'; + md += '**Before:**\n```\n' + c.existing_code + '\n```\n\n**After:**\n```\n' + c.suggestion_code + '\n```\n\n
'; + } + return md; + }; + + if (comments.length === 0) { + await withRetry(() => github.rest.issues.createComment({ + owner: context.repo.owner, repo: context.repo.repo, issue_number: prNumber, + body: `✅ **OCR**: ${result.message || 'No issues found.'}`, + }), 'no-issues-comment'); + return; + } + + const inlineAll = []; + const noLine = []; + for (const c of comments) { + const body = formatComment(c); + const hasLine = (c.start_line >= 1) || (c.end_line >= 1); + if (!hasLine) { noLine.push(c); continue; } + const rc = { path: c.path, body, side: 'RIGHT' }; + if (c.start_line >= 1 && c.end_line >= 1 && c.start_line !== c.end_line) { + rc.start_line = c.start_line; rc.line = c.end_line; rc.start_side = 'RIGHT'; + } else { + rc.line = c.end_line >= 1 ? c.end_line : c.start_line; + } + inlineAll.push({ rc, c }); + } + + const inline = inlineAll.slice(0, MAX_INLINE).map(x => x.rc); + const overflow = inlineAll.slice(MAX_INLINE).map(x => x.c); + + let summary = `🔍 **OCR** found **${comments.length}** issue(s).`; + summary += `\n- ${inline.length} inline, ${noLine.length + overflow.length} in summary`; + if (overflow.length) summary += ` (inline capped at ${MAX_INLINE})`; + if (warnings.length) summary += `\n- ⚠️ ${warnings.length} warning(s) during review`; + for (const c of noLine.concat(overflow)) summary += '\n\n---\n\n' + formatMarkdown(c); + + // Preferred path: ONE createReview carrying every inline comment. + try { + await withRetry(() => github.rest.pulls.createReview({ + owner: context.repo.owner, repo: context.repo.repo, pull_number: prNumber, + commit_id: commitSha, body: summary, event: 'COMMENT', comments: inline, + }), 'bulk-review'); + return; + } catch (e) { + core.warning(`bulk createReview failed (${e.status || '?'}: ${e.message}); falling back to throttled per-comment posting`); + } + + // Fallback: an invalid inline position (line not in the diff -> 422) + // rejects the whole bulk review. Post the summary, then each comment + // individually with a delay + backoff, skipping ones GitHub rejects. + let ok = 0; const failed = []; + try { + await withRetry(() => github.rest.pulls.createReview({ + owner: context.repo.owner, repo: context.repo.repo, pull_number: prNumber, + commit_id: commitSha, body: summary, event: 'COMMENT', + }), 'summary-review'); + } catch (err) { failed.push(`summary: ${err.message}`); } + + for (const rc of inline) { + try { + await withRetry(() => github.rest.pulls.createReviewComment({ + owner: context.repo.owner, repo: context.repo.repo, pull_number: prNumber, + commit_id: commitSha, path: rc.path, body: rc.body, + ...(rc.start_line ? { start_line: rc.start_line, start_side: rc.start_side } : {}), + line: rc.line, side: rc.side, + }), `comment ${rc.path}:${rc.line}`); + ok++; + } catch (inner) { + failed.push(`\`${rc.path}\` L${rc.line}: ${inner.message}`); + } + await sleep(1200); // stay under the secondary content-creation limit + } + + if (failed.length) { + await withRetry(() => github.rest.issues.createComment({ + owner: context.repo.owner, repo: context.repo.repo, issue_number: prNumber, + body: `📊 OCR posted ${ok}/${inline.length} inline comment(s).\n\n
${failed.length} could not be posted\n\n${failed.join('\n')}\n
`, + }), 'summary-failures'); + } + + # Companion job: OCR can't call MCP, so this separate agent ties the PR's + # changes to PostgreSQL git + pgsql-hackers history via the Agora MCP server + # (pg.ddx.io) and posts a single, upserted "history & discussion" comment. + pg-history: + runs-on: ubuntu-latest + if: | + github.event_name == 'pull_request' || + github.event_name == 'workflow_dispatch' || + (github.event_name == 'issue_comment' && github.event.issue.pull_request && + (startsWith(github.event.comment.body, '/open-code-review') || + startsWith(github.event.comment.body, '@open-code-review') || + startsWith(github.event.comment.body, '/pg-history'))) + steps: + - name: Resolve PR context + id: pr + uses: actions/github-script@v9 + with: + script: | + let prNumber; + if (context.eventName === 'pull_request') prNumber = context.payload.pull_request.number; + else if (context.eventName === 'issue_comment') prNumber = context.issue.number; + else prNumber = parseInt('${{ github.event.inputs.pr_number }}', 10); + const { data: pr } = await github.rest.pulls.get({ + owner: context.repo.owner, repo: context.repo.repo, pull_number: prNumber }); + core.setOutput('number', String(prNumber)); + core.setOutput('base_ref', pr.base.ref); + core.setOutput('head_sha', pr.head.sha); + core.setOutput('title', pr.title || ''); + + - name: Checkout + uses: actions/checkout@v6 + with: + fetch-depth: 0 + + - name: Make base/head refs available + env: + BASE_REF: ${{ steps.pr.outputs.base_ref }} + HEAD_SHA: ${{ steps.pr.outputs.head_sha }} + run: | + git fetch --no-tags origin "+refs/heads/${BASE_REF}:refs/remotes/origin/${BASE_REF}" || true + git fetch --no-tags origin "${HEAD_SHA}" || true + + - name: Setup Python + uses: actions/setup-python@v6 + with: + python-version: '3.12' + + - name: Configure AWS credentials (OIDC) + uses: aws-actions/configure-aws-credentials@v6 + with: + role-to-assume: ${{ vars.AWS_ROLE_ARN }} + aws-region: ${{ vars.AWS_REGION }} + role-session-name: pg-history-${{ github.run_id }} + + - name: Install deps + run: pip install boto3 + + - name: Run pg-history (Agora MCP) + env: + PG_HISTORY_MODEL: ${{ vars.OCR_BEDROCK_MODEL }} + AWS_REGION: ${{ vars.AWS_REGION }} + BASE_REF: ${{ steps.pr.outputs.base_ref }} + HEAD_SHA: ${{ steps.pr.outputs.head_sha }} + GH_PR_TITLE: ${{ steps.pr.outputs.title }} + PG_HISTORY_OUT: ${{ runner.temp }}/pg-history.md + run: | + python .github/ocr/pg-history.py || true + echo "----- output -----"; cat "${{ runner.temp }}/pg-history.md" 2>/dev/null || echo "(no output)" + + - name: Upsert PR comment + uses: actions/github-script@v9 + with: + script: | + const fs = require('fs'); + const path = process.env.RUNNER_TEMP + '/pg-history.md'; + let body = ''; + try { body = fs.readFileSync(path, 'utf8').trim(); } catch (e) {} + if (!body) { console.log('pg-history: empty output, nothing to post'); return; } + const prNumber = parseInt('${{ steps.pr.outputs.number }}', 10); + const marker = ''; + body = marker + '\n' + body; + const { data: comments } = await github.rest.issues.listComments({ + owner: context.repo.owner, repo: context.repo.repo, issue_number: prNumber, per_page: 100 }); + const mine = comments.find(c => c.user.type === 'Bot' && c.body && c.body.includes(marker)); + if (mine) { + await github.rest.issues.updateComment({ + owner: context.repo.owner, repo: context.repo.repo, comment_id: mine.id, body }); + } else { + await github.rest.issues.createComment({ + owner: context.repo.owner, repo: context.repo.repo, issue_number: prNumber, body }); + } diff --git a/.github/workflows/fork-sync-upstream.yml b/.github/workflows/fork-sync-upstream.yml new file mode 100644 index 0000000000000..e950821239968 --- /dev/null +++ b/.github/workflows/fork-sync-upstream.yml @@ -0,0 +1,47 @@ +# Keep this fork's master rebased on postgres/postgres master, carrying only +# the fork-owned CI files (.github/workflows/fork-*.yml and .github/ocr/). +# +# Requires the SYNC_TOKEN secret: a PAT with repo+workflow scope. The default +# GITHUB_TOKEN is not allowed to push commits that touch .github/workflows/, +# and every commit we carry does. + +name: Sync from upstream + +on: + schedule: + - cron: '17 */3 * * *' + workflow_dispatch: + +concurrency: + group: fork-sync-upstream + +permissions: + contents: read + +jobs: + sync: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + with: + fetch-depth: 0 + token: ${{ secrets.SYNC_TOKEN }} + + - name: Rebase master onto upstream + run: | + set -euo pipefail + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + git remote add upstream https://github.com/postgres/postgres.git + git fetch --no-tags upstream master + + # Master carries fork CI only. Anything else means work landed here + # that belongs on a branch; force-rebasing it on a timer would be wrong. + if git diff --name-only upstream/master...HEAD | + grep -vE '^(\.github/workflows/fork-|\.github/ocr/)'; then + echo "::error::master carries non-CI changes (listed above); move them to a branch" + exit 1 + fi + + git rebase upstream/master + git push --force-with-lease From bbcd98c8b44f8a77794cb9936463aabc4a494027 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Fri, 11 Sep 2026 10:48:09 -0400 Subject: [PATCH 002/338] ci: move fork CI logic to gburd/ci-workflows The OCR review workflows and their config were the last bulk this fork carried on top of upstream: ~970 lines rebased onto postgres/postgres every few hours, editable only by amending a commit that gets force- pushed. They are now reusable (workflow_call) workflows in the sidecar repo gburd/ci-workflows, and master carries two caller stubs instead. Carry after this: 3 files, ~110 lines, one commit. The sidecar is also where the value is: ocr/rule.json and ocr/context.md (the PostgreSQL committer-grade review corpus) become normal reviewable files with normal PRs and history, rather than a blob inside a commit that is rewritten on every sync. Mechanics worth knowing: - The review jobs check ci-workflows out at github.job_workflow_sha, so the rules always come from the same commit as the logic using them, and pinning @v1 (or a sha) in the stub pins both. This replaces the old "git show origin/:.github/ocr/..." dance, which existed only because the config had to be read off a branch other than the PR's. - AWS settings are passed as explicit inputs rather than read from vars inside the reusable workflow. vars does resolve across the boundary, but the contract stays visible at the call site. - A reusable workflow does not change the OIDC subject: it stays repo:gburd/postgres:*, so the ocr-bedrock-ci trust policy is unchanged. - ci-workflows must stay public; callers read it with GITHUB_TOKEN. The sync guard narrows to .github/workflows/fork-* accordingly, and sync-upstream keeps needing SYNC_TOKEN: the stubs are still workflow files, which GITHUB_TOKEN may not push. --- .github/ocr/context.md | 126 ------- .github/ocr/litellm.yaml | 41 --- .github/ocr/pg-history.py | 225 ----------- .github/ocr/rule.json | 65 ---- .github/workflows/fork-ocr-model-check.yml | 91 +---- .github/workflows/fork-ocr-review.yml | 410 +-------------------- .github/workflows/fork-sync-upstream.yml | 5 +- 7 files changed, 26 insertions(+), 937 deletions(-) delete mode 100644 .github/ocr/context.md delete mode 100644 .github/ocr/litellm.yaml delete mode 100644 .github/ocr/pg-history.py delete mode 100644 .github/ocr/rule.json diff --git a/.github/ocr/context.md b/.github/ocr/context.md deleted file mode 100644 index c4a83b85b124e..0000000000000 --- a/.github/ocr/context.md +++ /dev/null @@ -1,126 +0,0 @@ -# OCR review context — PostgreSQL contribution standards - -You are reviewing a change to a **PostgreSQL** fork. Every PR here is destined to -become a patch posted to the **pgsql-hackers** mailing list and tracked in a -**commitfest**. Review with the combined rigor, taste, and attention to detail of -the PostgreSQL committers. This context applies to the *whole* change, on top of -the per-file rules. - -## Review discipline -- Be precise and blunt; lead with the most serious problem. No praise, no - validation of the author, no disclaimers — accuracy is the only metric. -- Verify every claim against the actual diff. Confirm names, signatures, line - numbers, and APIs before asserting. Never invent behavior or cite code not in - the change. If unsure, say so, and tag each finding **high / moderate / low** - confidence. -- Judge the change on its merits regardless of how the PR frames it. A draft PR - is WIP: weight design/approach feedback over style nits. - -## Patch hygiene (top rejection reasons on -hackers) -1. **Minimal diff.** The fastest way to get a patch rejected is unrelated - changes: reformatting untouched lines, rewording unrelated comments, touching - code not required by the change. Flag any hunk not needed for the stated - purpose. After the patch, the code should read as if it had always been - written that way. -2. **Atomic, bisectable commits.** Each commit must build and pass tests on its - own — a broken intermediate commit breaks `git bisect`, revert, and - cherry-pick. Flag a commit that only compiles once a later commit lands. - Prefer one focused patch, or a clearly-ordered series of - independently-committable pieces. -3. **Tests + docs are mandatory.** A user-visible change without regression/TAP - tests **and** documentation is WIP, not commit-ready. New behavior needs - tests that cover edge and error paths, not just the happy path. -4. **DRY / reuse.** Prefer existing infrastructure (`List` in `pg_list.h`, - `StringInfo`, `dynahash`/`simplehash`, `palloc`/`MemoryContext`, `foreach`) - over reinventing it. Flag copy-paste and speculative abstraction alike — the - community wants minimal, targeted changes that fit the subsystem's existing - patterns. -5. **Whitespace.** No trailing whitespace; tabs (width 4) for C indentation; - `git diff --check` must be clean. Whitespace-only churn on untouched lines is - a defect. - -## Committer-owned files — do NOT touch in a patch (flag if present) -These are the committer's job at push time; including them causes needless -merge conflicts and is a mistake: -- **`src/include/catalog/catversion.h`** — the `CATALOG_VERSION_NO` bump is done - by the **committer** when pushing. A catversion bump in the PR is **wrong** — - flag it. (This is the single most common author mistake in catalog patches.) -- **Release notes** (`doc/src/sgml/release-*.sgml`) and version strings - (`configure.ac` `AC_INIT` version, `meson.build` `version`, `PG_VERSION`). - -## Generated files — never hand-edit; edit the source -Flag direct edits to generated output; point the author at the source instead: -- Catalog headers `src/include/catalog/*_d.h`, `postgres.bki`, `schemapg.h`, - `system_constraints.sql` → edit the `pg_*.dat` files. -- `src/backend/nodes/{copy,equal,out,read}funcs.c` and other - `gen_node_support.pl` output → annotate the `Node` struct in its header. -- `fmgroids.h`, `fmgrprotos.h`, `fmgrtab.c` → edit `pg_proc.dat`. -- `utils/errcodes.h` → `errcodes.txt`; wait-event headers → - `wait_event_names.txt`; `lwlocknames.h` → `lwlocknames.txt`. -- `configure` → `configure.ac`; `*.po` translations are handled separately; - generated Unicode tables come from their source scripts. - -## Portability is a hard gate -PostgreSQL runs on Linux, Windows (MSVC), macOS, the BSDs and Solaris, across -**x86_64, ARM64, RISC-V, PPC64, s390x**, both endiannesses and 32/64-bit. Any -change must be portable across all of them: -- No unaligned memory access; no dependence on `char` signedness, integer/pointer - width, endianness, or struct padding for on-disk/wire formats. -- Use `int16/int32/int64`, `Size`, and `INT64_FORMAT`/`UINT64_FORMAT` (never - `%ld` for `int64`). -- Atomics/barriers only via `port/atomics` (`pg_atomic_*`, `pg_read/write_barrier`). -- **Windows/MSVC:** any `extern` variable used from another module or an - extension needs `PGDLLIMPORT` in its header; no VLAs or compiler-specific - extensions beyond the tree's C99 baseline. - -## Backward compatibility — the strongest constraint -Do not break SQL behavior, the libpq wire protocol, the logical-replication -protocol, dump/restore, `pg_upgrade`, or exported/`PGDLLIMPORT` APIs without -extraordinary justification. **ABI** matters for back-branches: changing the -size/layout of an exported struct or the signature of an exported function -breaks installed extensions. - -## Mailing-list context & etiquette -Because each PR becomes a pgsql-hackers email read by a busy, expert, opinionated -audience, also flag what reliably wastes reviewer time or draws rejection: -- A patch that **does more than one thing** or bundles unrelated cleanup — split it. -- **Footguns**: easy-to-misuse APIs, silent data-loss/corruption hazards, unsafe - defaults — name them explicitly. -- **Performance claims without a reproducible benchmark.** -- No reference to the **design discussion / prior -hackers thread** (Message-Id) - for a non-trivial change. -- **Do not bikeshed:** keep style nits proportionate and clearly separated from - substantive correctness findings. - -## Minimalism — the "ponytail" discipline -The best code is the code you never wrote (YAGNI). Before accepting new code, -apply the ladder: (1) Does this need to exist at all? (2) Can existing -code/infrastructure already do it? (3) Is this the simplest thing that works? -Flag: speculative scaffolding and config for a path that isn't wired yet; dead -code and unused "flexibility" (fields, params, abstractions, options with no -caller); premature abstraction (a helper used exactly once); knobs/GUCs/flags -nobody asked for. Minimal, targeted changes that fit the existing patterns beat -clever or general-purpose ones. - -## Comment & identity accuracy -- Comments must describe what the code does **now**. Flag aspirational/ - future-tense comments for behavior that already shipped ("will be", "for now", - "not yet", "future", and stale "TODO/FIXME/XXX/HACK"); comments that drifted - from the code they sit above; and incomplete/trailing comments. Comments - explain **why**, not what. No commented-out code. -- **ASCII only** in source and diffs — no smart quotes, em-dashes, or ellipsis - characters. - -## Commit & versioning discipline -- Conventional-commit style, imperative subject, one logical change per commit, - each commit building on its own. -- Do **not** bump version numbers or generated version stamps (including - `catversion.h`) — that is the maintainer's job at commit/release time. - -Understand common list shorthand so your comments are precise and not -miscommunicated: WIP (work in progress), GUC (config variable), WAL, LSN, OID, -TOAST, FSM, TAM (table access method), RLS, DSM, 2PC, PITR, CIC (concurrent index -creation), SAOP, ABI/API, backpatch (apply to supported back-branches), HEAD -(master tip), catversion (catalog version), pgindent, buildfarm, cfbot, -`s/x/y/` (suggested text substitution), footgun, bikeshedding, POLA (principle of -least astonishment). diff --git a/.github/ocr/litellm.yaml b/.github/ocr/litellm.yaml deleted file mode 100644 index e23cc4eee6fe2..0000000000000 --- a/.github/ocr/litellm.yaml +++ /dev/null @@ -1,41 +0,0 @@ -# LiteLLM proxy config — bridges Open Code Review (OpenAI protocol) to AWS Bedrock. -# -# This proxy is NOT a hosted service. The ocr-review.yml workflow installs it -# (`pip install 'litellm[proxy]'`) and runs it as a background process bound to -# 127.0.0.1:4000 for the duration of a single GitHub Actions job, then it exits. -# -# Auth to Bedrock: LiteLLM uses boto3's default credential chain, which reads -# the temporary AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_SESSION_TOKEN -# minted by the workflow's OIDC "Configure AWS credentials" step; region from -# AWS_REGION. - -model_list: - - model_name: ocr-bedrock - litellm_params: - # Set the repo variable OCR_BEDROCK_MODEL to an Opus inference-profile id - # your account has access to, e.g.: - # bedrock/converse/us.anthropic.claude-opus-4-8 - # The 'converse/' prefix uses Bedrock's Converse API, which is the most - # reliable path for Claude tool-use (what OCR relies on). - model: os.environ/OCR_BEDROCK_MODEL - aws_region_name: os.environ/AWS_REGION - - # "High effort" review. Claude Opus 4.8 on Bedrock uses *adaptive* thinking - # controlled by output_config.effort. Set it DIRECTLY here — NOT via - # reasoning_effort, which LiteLLM still maps to the legacy - # thinking.type.enabled that Opus 4.8 rejects. LiteLLM forwards - # output_config into additionalModelRequestFields for Anthropic models; if - # the build doesn't recognize the effort param it is dropped with a warning - # (no error) and the model reviews at its default effort. - # Valid: low|medium|high|max|xhigh (auto-clamped to the model ceiling). - output_config: - effort: xhigh - max_tokens: 32000 - -litellm_settings: - drop_params: true # silently drop params a model doesn't support - modify_params: true # auto-fix minor request incompatibilities - request_timeout: 600 - -general_settings: - master_key: os.environ/LITELLM_MASTER_KEY diff --git a/.github/ocr/pg-history.py b/.github/ocr/pg-history.py deleted file mode 100644 index 5794f8a920bd7..0000000000000 --- a/.github/ocr/pg-history.py +++ /dev/null @@ -1,225 +0,0 @@ -#!/usr/bin/env python3 -""" -pg-history: tie a PR's changes to PostgreSQL git + pgsql-hackers email history. - -OCR (the code reviewer) cannot call MCP servers, so this is a separate agent: -it runs a Bedrock (Claude Opus) tool-use loop wired to the Agora MCP server at -https://pg.ddx.io/mcp, lets the model search the mailing-list archives / commit -history / commitfest data, and emits a Markdown summary linking the changes to -the relevant threads (https://pg.ddx.io/m/pgsql-hackers/). - -Env: - PG_HISTORY_MCP_URL MCP endpoint (default https://pg.ddx.io/mcp) - PG_HISTORY_MODEL Bedrock model id (e.g. us.anthropic.claude-opus-4-8) - AWS_REGION region (creds come from the OIDC step's env) - BASE_REF, HEAD_SHA PR base ref and head sha (for the git diff context) - GH_PR_TITLE PR title (optional, adds context) - PG_HISTORY_OUT output markdown path (default /tmp/pg-history.md) -Writes the markdown to PG_HISTORY_OUT; exits 0 even on soft failures (writes a note). -""" -import json, os, subprocess, sys, urllib.request - -MCP_URL = os.environ.get("PG_HISTORY_MCP_URL", "https://pg.ddx.io/mcp") -MODEL = os.environ.get("PG_HISTORY_MODEL", "us.anthropic.claude-opus-4-8").replace("bedrock/converse/", "").replace("bedrock/", "") -REGION = os.environ.get("AWS_REGION", "us-east-1") -BASE_REF = os.environ.get("BASE_REF", "") -HEAD_SHA = os.environ.get("HEAD_SHA", "") -PR_TITLE = os.environ.get("GH_PR_TITLE", "") -OUT = os.environ.get("PG_HISTORY_OUT", "/tmp/pg-history.md") -UA = "pg-history/0.1 (+github-actions)" - -# Curated subset of the 108 Agora tools — the ones useful for connecting a -# change to its discussion/commit history. Intersected with what the server -# actually exposes, so unknown names are harmless. -TOOL_WHITELIST = { - "find_related_discussions", "find_similar_messages", "get_thread", - "discussion_links", "get_author_messages", "browse_by_date", - "blame_symbol", "check_upstream_status", "find_related", - "find_entries_for_thread", "find_entries_for_author", "get_commit", - "search", "hybrid_search", "get_callers", "get_callees", "find_pattern", -} -MAX_ROUNDS = 14 -TOOL_RESULT_CAP = 8000 # chars per tool result fed back to the model - - -def _mcp_post(body, sid=None): - headers = {"Content-Type": "application/json", - "Accept": "application/json, text/event-stream", "User-Agent": UA} - if sid: - headers["Mcp-Session-Id"] = sid - req = urllib.request.Request(MCP_URL, data=json.dumps(body).encode(), headers=headers, method="POST") - resp = urllib.request.urlopen(req, timeout=60) - sid_out = resp.headers.get("Mcp-Session-Id") - result = None - for line in resp.read().decode().splitlines(): - line = line.strip() - if line.startswith("data:"): - line = line[5:].strip() - if not line or line.startswith("event:"): - continue - try: - obj = json.loads(line) - except Exception: - continue - if isinstance(obj, dict) and ("result" in obj or "error" in obj): - result = obj - return result, sid_out - - -class MCP: - def __init__(self): - init, self.sid = _mcp_post({"jsonrpc": "2.0", "id": 1, "method": "initialize", - "params": {"protocolVersion": "2025-06-18", "capabilities": {}, - "clientInfo": {"name": "pg-history", "version": "0.1"}}}) - if not init or "result" not in init: - raise RuntimeError(f"MCP initialize failed: {init}") - try: - _mcp_post({"jsonrpc": "2.0", "method": "notifications/initialized", "params": {}}, self.sid) - except Exception: - pass - self._id = 1 - - def list_tools(self): - self._id += 1 - res, _ = _mcp_post({"jsonrpc": "2.0", "id": self._id, "method": "tools/list", "params": {}}, self.sid) - return (res or {}).get("result", {}).get("tools", []) - - def call(self, name, args): - self._id += 1 - res, _ = _mcp_post({"jsonrpc": "2.0", "id": self._id, "method": "tools/call", - "params": {"name": name, "arguments": args or {}}}, self.sid) - if not res: - return "(no response)" - if "error" in res: - return f"ERROR: {json.dumps(res['error'])[:500]}" - parts = [] - for c in res.get("result", {}).get("content", []): - if c.get("type") == "text": - parts.append(c["text"]) - return ("\n".join(parts) or "(empty)")[:TOOL_RESULT_CAP] - - -def git(*args): - try: - return subprocess.check_output(["git", *args], text=True, stderr=subprocess.DEVNULL).strip() - except Exception: - return "" - - -def pr_context(): - base = f"origin/{BASE_REF}" if BASE_REF else "" - rng = f"{base}..{HEAD_SHA}" if base and HEAD_SHA else HEAD_SHA - commits = git("log", "--no-merges", "--format=%h %s", f"{rng}") if rng else "" - stat = git("diff", "--stat", rng) if rng else "" - files = git("diff", "--name-only", rng) if rng else "" - return commits[:4000], stat[:3000], files[:2000] - - -SYSTEM = """You are a PostgreSQL community research assistant. Given a pull request's -commits and changed files, use the available tools (backed by the Agora index of -pgsql-hackers mail, commit history, and commitfest data) to connect the change to -its history. Your goal: - -- Find the mailing-list thread(s) and prior discussion behind this change. -- Identify related/superseded prior commits and any commitfest entry. -- Note relevant prior art, rejected approaches, or design rationale. - -Rules (voice & rigor): -- Be precise and blunt. No praise, no filler, no hedging, no disclaimers. Accuracy is - the only success metric — not the author's approval. Lead with the most important finding. -- NEVER hallucinate. Verify every Message-ID, thread subject, commit hash, author name, - and date against an actual tool result before citing it. If a search returns nothing, - say so plainly — do not guess or fabricate a plausible-looking link. -- Assess the change on its merits, independent of how the PR frames it. -- Tag any inferred (not tool-confirmed) linkage with an explicit confidence level: - high / moderate / low. -- Be decisive and efficient: a handful of targeted tool calls, not exhaustive search. -- Cite every mailing-list message as a Markdown link: [subject](https://pg.ddx.io/m/pgsql-hackers/MESSAGE_ID). -- If you find nothing relevant, say so in one line — do not pad. - -When done, output ONLY Markdown (no preamble) with these sections, omitting any that are empty: -## 🧵 Related discussion -## 🔗 Related commits / prior art -## 📋 Commitfest -## 🧭 Context for reviewers -Keep it tight (use bullets; link generously).""" - - -def to_toolspec(t): - schema = t.get("inputSchema") or {"type": "object", "properties": {}} - return {"toolSpec": {"name": t["name"], - "description": (t.get("description") or "")[:600], - "inputSchema": {"json": schema}}} - - -def main(): - commits, stat, files = pr_context() - if not commits and not files: - open(OUT, "w").write("") # nothing to do - print("No PR diff context; skipping.") - return - user = (f"PR title: {PR_TITLE}\n\n" if PR_TITLE else "") + \ - f"Commits:\n{commits or '(none)'}\n\nChanged files:\n{files or '(none)'}\n\nDiffstat:\n{stat or '(none)'}\n" - - try: - mcp = MCP() - tools = [to_toolspec(t) for t in mcp.list_tools() if t.get("name") in TOOL_WHITELIST] - except Exception as e: - open(OUT, "w").write(f"_pg-history: could not reach the Agora MCP server ({MCP_URL}): {e}_\n") - print(f"MCP unavailable: {e}") - return - if not tools: - open(OUT, "w").write("_pg-history: no usable MCP tools available._\n") - return - - import boto3 - from botocore.config import Config - - # botocore's default read timeout (60s) is too short for a multi-round - # (MAX_ROUNDS) tool-use loop against a large PR diff on a reasoning model; - # each converse() call alone can take several minutes. Bump it well past - # what a single round needs; connect_timeout stays short since a stuck - # TCP handshake is a different (and much cheaper to detect) failure mode. - brt = boto3.client("bedrock-runtime", region_name=REGION, - config=Config(read_timeout=900, connect_timeout=10)) - messages = [{"role": "user", "content": [{"text": user}]}] - final_text = "" - try: - for _ in range(MAX_ROUNDS): - resp = brt.converse( - modelId=MODEL, - system=[{"text": SYSTEM}], - messages=messages, - toolConfig={"tools": tools}, - inferenceConfig={"maxTokens": 4096}, - ) - out = resp["output"]["message"] - messages.append(out) - if resp.get("stopReason") == "tool_use": - results = [] - for blk in out["content"]: - tu = blk.get("toolUse") - if not tu: - continue - res_text = mcp.call(tu["name"], tu.get("input") or {}) - results.append({"toolResult": {"toolUseId": tu["toolUseId"], - "content": [{"text": res_text}]}}) - messages.append({"role": "user", "content": results}) - continue - final_text = "".join(b.get("text", "") for b in out["content"]).strip() - break - except Exception as e: - open(OUT, "w").write(f"_pg-history: Bedrock call failed: {e}_\n") - print(f"Bedrock error: {e}") - return - - if not final_text: - final_text = "_pg-history: no related history found._" - body = "## 📜 Change history & discussion (Agora / pg.ddx.io)\n\n" + final_text + \ - "\n\nGenerated by pg-history via the Agora MCP server (pg.ddx.io).\n" - open(OUT, "w").write(body) - print(body) - - -if __name__ == "__main__": - main() diff --git a/.github/ocr/rule.json b/.github/ocr/rule.json deleted file mode 100644 index 60e13e73dcbe0..0000000000000 --- a/.github/ocr/rule.json +++ /dev/null @@ -1,65 +0,0 @@ -{ - "_comment": "OCR per-file review rules for PostgreSQL core + extensions. Cross-cutting contribution standards & mailing-list etiquette live in .github/ocr/context.md, passed via --background-file. OCR uses FIRST-MATCH-WINS in declaration order, so rules are ordered most-specific first. merge_system_rule:true keeps OCR's built-in fine-tuned checks (thread-safety, injection, NPE) alongside these PostgreSQL-specific rules.", - "rules": [ - { - "path": "src/test/**", - "merge_system_rule": true, - "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL tests. Coverage is mandatory for any behavioral change and must include edge cases (NULL, empty, boundary/overflow) and ERROR paths, not just the happy path. A test that still passes with the feature reverted is worthless — confirm it actually exercises and would catch regressions in the new code. Regression (.sql/expected): deterministic, portable output — ORDER BY where row order matters, no timing/plan-dependent output except intentional EXPLAIN, no absolute paths, locale-independent (C collation or explicit COLLATE), DROP objects the test creates; expected/ output must stay stable across platforms and under the parallel schedule. Concurrency/locking belongs in isolation tests (src/test/isolation, .spec + permutations). End-to-end/crash/replication/CLI behavior belongs in TAP tests (t/*.pl with PostgreSQL::Test::Cluster/Utils) — no hardcoded ports/paths, no sleep as synchronization (use poll_query_until/wait_for), skip cleanly when prerequisites are missing, and clean up nodes." - }, - { - "path": "**/*.{c,h}", - "merge_system_rule": true, - "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL backend/frontend C — review as pgsql-hackers committers do, in priority order.\n\n(1) CORRECTNESS (highest): Memory — every palloc lives in the right MemoryContext; error paths via ereport/elog(ERROR) must not leak memory/buffers/locks/fds (rely on MemoryContext/ResourceOwner reset or PG_TRY/PG_FINALLY); no use-after-free; delete temp contexts. Concurrency — consistent lock ordering (deadlock-free), correct lock levels, balanced LWLockAcquire/Release and START_/END_CRIT_SECTION, no TOCTOU, CHECK_FOR_INTERRUPTS in long loops, async-signal-safe signal handlers (volatile sig_atomic_t). WAL — any change to shared on-disk state must be WAL-logged AND correctly replayed (redo path), crash- and replica-consistent. NULL/edge/overflow handling.\n\n(2) BACKWARD COMPATIBILITY / ABI: don't break behavior, dump/restore, pg_upgrade, libpq wire protocol, logical-replication protocol, or exported/PGDLLIMPORT'd APIs (struct size/layout, function signatures) without extraordinary justification.\n\n(3) CATALOG / GENERATED: new/changed catalog data goes in pg_*.dat, NOT the generated *_d.h/.bki. New Node types: ANNOTATE the struct in its header so gen_node_support.pl regenerates copy/equal/out/read — do NOT hand-edit *funcs.c. New SQL-callable functions: add to pg_proc.dat with an OID from the 8000-9999 developer range (src/include/catalog/unused_oids; check duplicate_oids); committer renumbers at commit. DO NOT bump CATALOG_VERSION_NO in the patch — flag any catversion.h change as a mistake (committer's job).\n\n(4) PERFORMANCE: no regression on hot paths; avoid O(n^2) where better is feasible; minimize work under contended locks; avoid needless palloc churn and large struct copies in hot paths.\n\n(5) SECURITY: bounded string ops (snprintf/strlcpy/strlcat — never strcpy/strcat/sprintf); integer/size-overflow checks before allocation; never user input as a format string; privilege checks via pg_*_aclcheck; beware search_path and SECURITY DEFINER.\n\n(6) PORTABILITY (hard gate): no unaligned access; no dependence on char signedness, int/long/pointer width, endianness, or struct padding for on-disk/wire formats; use int16/int32/int64 + INT64_FORMAT/UINT64_FORMAT (never %ld for int64); align contended shared structs (pg_attribute_aligned/cache-line pad). Atomics/barriers only via port/atomics (pg_atomic_*, pg_read/write_barrier) — never raw intrinsics or volatile-as-barrier. WINDOWS/MSVC: extern vars used cross-module/extension need PGDLLIMPORT; no VLAs or features beyond the C99 baseline the tree targets; use pg_pread/pg_pwrite. Applies across x86_64/ARM64/RISC-V/PPC64/s390x, big/little endian, 32/64-bit.\n\n(7) CONVENTIONS: errmsg starts lowercase, no trailing period, no embedded newlines; errdetail/errhint are complete capitalized sentences; correct ERRCODE_*; wrap user-facing text in _(); errmsg_plural for counts. Assert() only for can't-happen invariants (never user-reachable). Naming: snake_case with subsystem prefix (heap_insert) or CamelCase for major subsystems (ExecInitNode); ALL_CAPS macros. Must pgindent cleanly (tabs, width 4). Comments explain WHY not WHAT; no #ifdef 0 blocks, no commented-out code, no #ifdef fencing your feature. Reuse existing helpers (DRY)." - }, - { - "path": "**/*.dat", - "merge_system_rule": true, - "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL catalog data (pg_proc.dat, pg_type.dat, etc.) — the SOURCE for generated headers. The generated *_d.h, postgres.bki, fmgroids.h, fmgrtab.c must NOT be hand-edited (they regenerate from these files). OIDs: use a value from the developer range 8000-9999 (src/include/catalog/unused_oids; verify with duplicate_oids); committer renumbers to a final contiguous block, so stay in-range and unique but don't over-optimize the exact number. Keep proc entries complete/consistent (prosrc, provolatile, proparallel, prorettype/proargtypes, matching description). DO NOT bump CATALOG_VERSION_NO / catversion.h — committer's job at push time; flag any such change. New catalog columns/views need documentation in doc/src/sgml/catalogs.sgml." - }, - { - "path": "**/*.{sql,pgsql}", - "merge_system_rule": true, - "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL SQL. Valid PostgreSQL dialect (not MySQL/Oracle); correct types (bigint vs int, text vs varchar); sound transaction/isolation and CTE-materialization assumptions. SECURITY: flag SQL injection in dynamic SQL (require quote_identifier/quote_literal or format() with %I/%L), SECURITY DEFINER without a locked-down search_path, inappropriate RLS bypass. Prefer set-based over row-at-a-time/N+1. BACKWARD COMPATIBILITY (a top rejection reason): changing existing SQL behavior, the output of existing functions, or default GUCs needs extraordinary justification. New SQL-callable objects belong in pg_*.dat with OIDs from the 8000-9999 range, not in generated files. Minimal diff; add regression tests + docs." - }, - { - "path": "**/*.{pl,pm}", - "merge_system_rule": true, - "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL Perl (TAP tests and build/catalog tooling). Require 'use strict; use warnings;'. Must be perltidy-clean with the tree's src/tools/pgindent/perltidyrc and pass src/tools/perlcheck/pgperlcritic. Use the framework: PostgreSQL::Test::Cluster, PostgreSQL::Test::Utils, Test::More; no hardcoded ports/paths/PIDs; use safe_psql/poll_query_until, not sleep; skippable without optional prerequisites; clean up nodes. PORTABILITY: run on Windows (no fork-only constructs, use File::Spec, avoid unavailable signals) and the minimum supported Perl. Robustness: avoid two-arg open and string system()/qx with interpolated data (use list forms). Generator scripts (gen_node_support.pl, catalog Perl) must be deterministic and stay in sync with inputs; do not commit their generated output." - }, - { - "path": "**/*.py", - "merge_system_rule": true, - "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL Python (build/test tooling, oauth/pytest tests, src/tools). Follow surrounding style; keep imports to the standard library unless the dependency is already required by the tree (no surprise third-party deps in build/test tooling). PORTABILITY: support the project's minimum Python 3 and run on Windows and the BSDs (use os.path/pathlib, avoid POSIX-only calls and shell=True with interpolated input). Deterministic, self-cleaning tests; no hardcoded ports/paths; skip cleanly without prerequisites. For the Perl->pytest porting effort, confirm behavior parity with the TAP test replaced (same assertions/coverage), not a superficial translation. Minimal diff; match the tree's ruff/black config if present." - }, - { - "path": "**/*.{rs,toml}", - "merge_system_rule": true, - "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. Rust PostgreSQL extension (pgrx) or Rust support crate. Not core C, but it runs inside/alongside the backend, so backend safety applies. SAFETY: in code reachable from an SQL call, a Rust panic aborts the Postgres process — forbid unwrap()/expect()/panic!/unreachable!/todo! and index-panics on reachable paths; use Result and pgrx error reporting (error!/ereport!). Every `unsafe` block needs a comment justifying its invariant; scrutinize raw pointers and FFI across the pg_sys boundary. pgrx: honor #[pg_guard] on extern C fns (correct panic/longjmp handling); never hold Rust references across SPI or anything that can longjmp (skips Rust destructors -> leaks); respect MemoryContext lifetimes for palloc'd data; datum<->Rust conversions must handle NULL. Concurrency uses Postgres shmem/LWLocks (pgrx shmem API), not std::sync alone. Lints: must pass `cargo clippy --all-targets --all-features -- -D warnings` and `cargo fmt --check`; deny unwrap_used/expect_used/panic in libraries; thiserror (libs) / anyhow (bins). Justify every new dependency. Tests: #[pg_test] for in-backend behavior, #[test] for pure logic; cover error and NULL paths. Minimal, idiomatic diff." - }, - { - "path": "**/{configure.ac,*.m4,aclocal.m4}", - "merge_system_rule": true, - "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL Autoconf. Edit configure.ac / the m4 macros — do NOT hand-edit generated 'configure' or pg_config.h.in in the same patch (regeneration is the committer's step; a patch that also rewrites generated configure output is suspect). Feature/header/function probes must be portable and not assume a specific OS/compiler. Every configure knob must be mirrored on the Meson side (meson_options.txt/meson.build) and documented. Minimal diff." - }, - { - "path": "**/{meson.build,meson_options.txt}", - "merge_system_rule": true, - "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL Meson build. Valid syntax; correct subdir()/dependency()/declare_dependency and install paths; new source files must be added here. CRITICAL: PostgreSQL maintains BOTH Meson and Autoconf/Make — any new file, option, or feature check must be mirrored on the configure.ac/Makefile side so the two never drift (a file built by only one system is a common defect). New options need matching docs and sensible defaults. Minimal diff." - }, - { - "path": "**/{Makefile,GNUmakefile,*.mk}", - "merge_system_rule": true, - "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL Makefile (GNU Make). $(VAR) refs; correct .PHONY; accurate dependencies (no parallel -j races); $(MAKE) for recursion; VPATH/out-of-tree build support; no hardcoded paths (use standard PostgreSQL makefile vars and $(top_builddir)); clean/distclean/maintainer-clean must remove new artifacts; extensions use PGXS. Must stay in sync with meson.build. Minimal diff." - }, - { - "path": "doc/**/*.sgml", - "merge_system_rule": true, - "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. PostgreSQL documentation (DocBook SGML). Technically accurate/complete (parameters, limitations, version/compat notes); correct tag usage/nesting (, , , , , /); working cross-references; spell it 'PostgreSQL' in prose; SQL keywords uppercase in examples. Coverage: a new GUC -> config.sgml (and postgresql.conf.sample); new/changed catalogs or views -> catalogs.sgml; new SQL syntax -> the matching ref/*.sgml; new functions -> func.sgml. Do NOT edit release-notes (release-*.sgml) — written by the release team/committers; flag such edits. New user-facing behavior in this PR should ship with matching docs." - }, - { - "path": "**/*.md", - "merge_system_rule": true, - "rule": "REVIEW DISCIPLINE: Precise, blunt, verify against the diff, tag confidence, no praise. Markdown docs. Clear heading hierarchy; fenced code blocks with language hints; accurate instructions/prerequisites; consistent PostgreSQL terminology; no broken relative links or stale claims. Minimal diff." - } - ] -} diff --git a/.github/workflows/fork-ocr-model-check.yml b/.github/workflows/fork-ocr-model-check.yml index 10d250528cf7c..b85a990c66fd7 100644 --- a/.github/workflows/fork-ocr-model-check.yml +++ b/.github/workflows/fork-ocr-model-check.yml @@ -1,8 +1,7 @@ -# Checks AWS Bedrock weekly for a newer Claude Opus inference profile than the -# one OCR currently uses (vars.OCR_BEDROCK_MODEL) and, if found, opens/updates a -# single GitHub issue telling the maintainer to bump the variable. It does NOT -# change the model automatically: GITHUB_TOKEN cannot write Actions *variables* -# (that needs a PAT with admin), so this is a notify-only mechanism by design. +# Weekly check for a newer Claude Opus on Bedrock; opens/updates one issue here +# when OCR_BEDROCK_MODEL is behind. Notify-only (GITHUB_TOKEN cannot write +# Actions variables). Logic lives in gburd/ci-workflows. + name: OCR model self-check on: @@ -10,80 +9,10 @@ on: - cron: '0 12 * * 1' # Mondays 12:00 UTC workflow_dispatch: -permissions: - id-token: write - contents: read - issues: write - jobs: - check-model: - runs-on: ubuntu-latest - steps: - - name: Configure AWS credentials (OIDC) - uses: aws-actions/configure-aws-credentials@v6 - with: - role-to-assume: ${{ vars.AWS_ROLE_ARN }} - aws-region: ${{ vars.AWS_REGION }} - role-session-name: ocr-model-check-${{ github.run_id }} - - - name: Find newest Opus vs configured - id: check - env: - CURRENT: ${{ vars.OCR_BEDROCK_MODEL }} - AWS_REGION: ${{ vars.AWS_REGION }} - run: | - python3 - <<'PY' >> "$GITHUB_OUTPUT" - import os, re, subprocess, json - region = os.environ.get("AWS_REGION", "us-east-1") - current = os.environ.get("CURRENT", "") - out = subprocess.run( - ["aws", "bedrock", "list-inference-profiles", "--region", region, - "--query", "inferenceProfileSummaries[].inferenceProfileId", "--output", "json"], - capture_output=True, text=True) - ids = json.loads(out.stdout or "[]") - # Parse claude-opus-- from any profile id (prefix us./global. ok). - def ver(s): - m = re.search(r"claude-opus-(\d+)-(\d+)", s) - return (int(m.group(1)), int(m.group(2))) if m else None - opus = [(ver(i), i) for i in ids if ver(i) and i.startswith(("us.", "global."))] - if not opus: - print("newer=false"); raise SystemExit(0) - best_ver, best_id = max(opus, key=lambda x: x[0]) - cur = ver(current) - newer = (cur is None) or (best_ver > cur) - print(f"newer={'true' if newer else 'false'}") - print(f"best_id={best_id}") - print(f"best_ver={best_ver[0]}.{best_ver[1]}") - print(f"cur_ver={'unknown' if cur is None else f'{cur[0]}.{cur[1]}'}") - PY - - - name: Open/update issue if a newer model exists - if: steps.check.outputs.newer == 'true' - uses: actions/github-script@v9 - with: - script: | - const best = '${{ steps.check.outputs.best_id }}'; - const bestVer = '${{ steps.check.outputs.best_ver }}'; - const curVer = '${{ steps.check.outputs.cur_ver }}'; - const marker = ''; - const title = `OCR: newer Claude Opus available (${bestVer} > ${curVer})`; - const body = `${marker}\n` + - `A newer Claude Opus inference profile is available on Bedrock.\n\n` + - `- **Configured** (\`vars.OCR_BEDROCK_MODEL\`): Opus ${curVer}\n` + - `- **Newest on Bedrock**: \`${best}\` (Opus ${bestVer})\n\n` + - `To upgrade, set the repo variable:\n\n` + - '```\n' + - `gh variable set OCR_BEDROCK_MODEL -R ${context.repo.owner}/${context.repo.repo} \\\n` + - ` -b "bedrock/converse/${best}"\n` + - '```\n\n' + - `Also confirm the \`ocr-bedrock-ci\` IAM inline policy allows invoking the new model ` + - `(the resource is scoped to \`anthropic.claude-opus-*\`), then re-run OCR.\n\n` + - `_Automated by \`.github/workflows/ocr-model-check.yml\`; this issue is upserted._`; - const q = `repo:${context.repo.owner}/${context.repo.repo} in:body "${marker}" state:open`; - const found = await github.rest.search.issuesAndPullRequests({ q, per_page: 1 }); - if (found.data.total_count > 0) { - const n = found.data.items[0].number; - await github.rest.issues.update({ owner: context.repo.owner, repo: context.repo.repo, issue_number: n, title, body }); - } else { - await github.rest.issues.create({ owner: context.repo.owner, repo: context.repo.repo, title, body }); - } + check: + uses: gburd/ci-workflows/.github/workflows/ocr-model-check.yml@v1 + with: + aws_role_arn: ${{ vars.AWS_ROLE_ARN }} + aws_region: ${{ vars.AWS_REGION }} + bedrock_model: ${{ vars.OCR_BEDROCK_MODEL }} diff --git a/.github/workflows/fork-ocr-review.yml b/.github/workflows/fork-ocr-review.yml index 0828af429b57c..ae938754339b4 100644 --- a/.github/workflows/fork-ocr-review.yml +++ b/.github/workflows/fork-ocr-review.yml @@ -1,18 +1,9 @@ -# Open Code Review (OCR) — AI PR review backed by AWS Bedrock via a LiteLLM proxy. +# AI PR review (Claude Opus on Bedrock). Logic and review rules live in +# gburd/ci-workflows so this fork's master carries a stub instead of ~1000 +# lines that have to be rebased onto upstream forever. # -# Flow: -# PR opened/updated (incl. DRAFTS) ─┐ -# /open-code-review PR comment ─┼─► start LiteLLM (127.0.0.1:4000 → Bedrock) -# manual workflow_dispatch ─┘ └► ocr review --format json -# └► post inline PR review comments -# -# Required (repo settings — all repo *variables*, no secrets; auth is via GitHub OIDC): -# vars.AWS_ROLE_ARN - IAM role to assume via OIDC (granting bedrock:InvokeModel*) -# vars.AWS_REGION - e.g. us-east-1 -# vars.OCR_BEDROCK_MODEL - LiteLLM model string for the Opus inference profile, e.g. -# bedrock/converse/us.anthropic.claude-opus-4-8 -# -# No static AWS keys are stored. GITHUB_TOKEN (auto) posts the review comments. +# Needs repo variables AWS_ROLE_ARN, AWS_REGION, OCR_BEDROCK_MODEL. Auth is +# GitHub OIDC; no static AWS keys. See the ci-workflows README. name: OCR AI Review @@ -34,311 +25,10 @@ concurrency: group: ocr-review-${{ github.event.pull_request.number || github.event.issue.number || github.event.inputs.pr_number }} cancel-in-progress: true -permissions: - id-token: write # required to mint the GitHub OIDC token for AWS role assumption - contents: read - pull-requests: write - jobs: - ocr-review: - runs-on: ubuntu-latest - # PR events always; comment events only when the comment is on a PR and - # starts with the trigger keyword; manual dispatch always. - if: | - github.event_name == 'pull_request' || - github.event_name == 'workflow_dispatch' || - (github.event_name == 'issue_comment' && github.event.issue.pull_request && - (startsWith(github.event.comment.body, '/open-code-review') || - startsWith(github.event.comment.body, '@open-code-review'))) - - env: - # LiteLLM listens on localhost only; this key never leaves the runner. - LITELLM_MASTER_KEY: sk-ocr-ci-local - OCR_BEDROCK_MODEL: ${{ vars.OCR_BEDROCK_MODEL }} - # Region is a static var (safe at job level). AWS credentials are NOT set - # here — they're minted by the OIDC "Configure AWS credentials" step below - # and exported to the environment for the LiteLLM/boto3 Bedrock calls. - AWS_REGION: ${{ vars.AWS_REGION }} - - steps: - - name: Resolve PR context - id: pr - uses: actions/github-script@v9 - with: - script: | - let prNumber; - if (context.eventName === 'pull_request') { - prNumber = context.payload.pull_request.number; - } else if (context.eventName === 'issue_comment') { - prNumber = context.issue.number; - } else { - prNumber = parseInt('${{ github.event.inputs.pr_number }}', 10); - } - const { data: pr } = await github.rest.pulls.get({ - owner: context.repo.owner, - repo: context.repo.repo, - pull_number: prNumber, - }); - const { data: repo } = await github.rest.repos.get({ - owner: context.repo.owner, - repo: context.repo.repo, - }); - core.setOutput('number', String(prNumber)); - core.setOutput('base_ref', pr.base.ref); - core.setOutput('head_ref', pr.head.ref); - core.setOutput('head_sha', pr.head.sha); - core.setOutput('default_branch', repo.default_branch); - core.setOutput('cross_repo', String(pr.head.repo.full_name !== pr.base.repo.full_name)); - - # NOTE: do NOT checkout the PR head. OCR reads the diff and file contents - # straight from git refs (git diff , git show :path, - # git grep ), so the working tree is irrelevant — but our OCR config - # lives on the default branch, not on the PR branch. We check out the repo - # (default ref), fetch the base/head objects, and materialize the config - # from origin/. - - name: Checkout - uses: actions/checkout@v6 - with: - fetch-depth: 0 - - - name: Prepare git refs and OCR config - env: - BASE_REF: ${{ steps.pr.outputs.base_ref }} - HEAD_REF: ${{ steps.pr.outputs.head_ref }} - HEAD_SHA: ${{ steps.pr.outputs.head_sha }} - DEFAULT_BRANCH: ${{ steps.pr.outputs.default_branch }} - run: | - git fetch --no-tags origin "+refs/heads/${DEFAULT_BRANCH}:refs/remotes/origin/${DEFAULT_BRANCH}" || true - git fetch --no-tags origin "+refs/heads/${BASE_REF}:refs/remotes/origin/${BASE_REF}" || true - git fetch --no-tags origin "+refs/heads/${HEAD_REF}:refs/remotes/origin/${HEAD_REF}" || true - git fetch --no-tags origin "${HEAD_SHA}" || true - - # OCR config lives on the default branch; materialize it independently - # of whatever ref is checked out. - mkdir -p "$RUNNER_TEMP/ocr" - git show "origin/${DEFAULT_BRANCH}:.github/ocr/litellm.yaml" > "$RUNNER_TEMP/ocr/litellm.yaml" - git show "origin/${DEFAULT_BRANCH}:.github/ocr/rule.json" > "$RUNNER_TEMP/ocr/rule.json" - git show "origin/${DEFAULT_BRANCH}:.github/ocr/context.md" > "$RUNNER_TEMP/ocr/context.md" - echo "Config materialized:"; ls -l "$RUNNER_TEMP/ocr" - - - name: Setup Python - uses: actions/setup-python@v6 - with: - python-version: '3.12' - - - name: Setup Node.js - uses: actions/setup-node@v6 - with: - node-version: '20' - - - name: Install LiteLLM proxy + Open Code Review - run: | - python -m pip install --upgrade pip - # Pin LiteLLM to a main commit that supports Claude Opus 4.8 adaptive - # thinking (maps reasoning_effort -> output_config.effort, incl. xhigh). - # Not in any tagged release yet (PyPI latest 1.87.1 lacks the Opus - # normalizer). Bump this SHA once a release ships the feature. - pip install "litellm[proxy] @ git+https://github.com/BerriAI/litellm.git@5be0797d24a2f26eb2123e13788f90055a59d91d" - npm install -g @alibaba-group/open-code-review - - - name: Configure AWS credentials (OIDC) - uses: aws-actions/configure-aws-credentials@v6 - with: - role-to-assume: ${{ vars.AWS_ROLE_ARN }} - aws-region: ${{ vars.AWS_REGION }} - role-session-name: ocr-review-${{ github.run_id }} - - - name: Start LiteLLM proxy (Bedrock bridge) - run: | - if [ -z "$OCR_BEDROCK_MODEL" ]; then - echo "::error::vars.OCR_BEDROCK_MODEL is not set (e.g. bedrock/converse/us.anthropic.claude-opus-4-1-20250805-v1:0)" - exit 1 - fi - nohup litellm --config "$RUNNER_TEMP/ocr/litellm.yaml" --host 127.0.0.1 --port 4000 \ - > /tmp/litellm.log 2>&1 & - echo "Waiting for LiteLLM to become ready..." - for i in $(seq 1 60); do - if curl -sf http://127.0.0.1:4000/health/readiness >/dev/null; then - echo "LiteLLM ready."; exit 0 - fi - sleep 2 - done - echo "::error::LiteLLM did not become ready in time"; cat /tmp/litellm.log; exit 1 - - - name: Configure OCR - run: | - ocr config set llm.url http://127.0.0.1:4000/v1/chat/completions - ocr config set llm.auth_token "$LITELLM_MASTER_KEY" - ocr config set llm.model ocr-bedrock - ocr config set llm.use_anthropic false - ocr config set language English - - - name: Run OCR review - run: | - ocr review \ - --from "origin/${{ steps.pr.outputs.base_ref }}" \ - --to "${{ steps.pr.outputs.head_sha }}" \ - --rule "$RUNNER_TEMP/ocr/rule.json" \ - --background-file "$RUNNER_TEMP/ocr/context.md" \ - --concurrency 3 \ - --timeout 20 \ - --format json \ - > /tmp/ocr-result.json 2>/tmp/ocr-stderr.log || true - echo "----- OCR stdout -----"; cat /tmp/ocr-result.json || true - echo "----- OCR stderr -----"; cat /tmp/ocr-stderr.log || true - echo "----- LiteLLM log (tail) -----"; tail -n 50 /tmp/litellm.log || true - - - name: Post review to PR - uses: actions/github-script@v9 - with: - github-token: ${{ secrets.GITHUB_TOKEN }} - script: | - const fs = require('fs'); - const prNumber = parseInt('${{ steps.pr.outputs.number }}', 10); - const commitSha = '${{ steps.pr.outputs.head_sha }}'; - - // Opus at high effort can emit dozens of findings. Posting them all - // one-by-one trips GitHub's SECONDARY rate limit (403 "content - // creation"), which is what made every run fail after the review - // was already generated. We (a) cap inline comments and overflow - // the rest into the summary, (b) prefer a single bulk createReview, - // and (c) throttle + back off with Retry-After on any fallback. - const MAX_INLINE = 25; - const sleep = (ms) => new Promise(r => setTimeout(r, ms)); - - async function withRetry(fn, label) { - for (let attempt = 1; attempt <= 5; attempt++) { - try { return await fn(); } - catch (e) { - const status = e.status || (e.response && e.response.status); - const h = (e.response && e.response.headers) || {}; - const isRate = status === 403 || status === 429; - if (!isRate || attempt === 5) throw e; - let waitMs = 0; - if (h['retry-after']) waitMs = parseInt(h['retry-after'], 10) * 1000; - else if (h['x-ratelimit-reset']) waitMs = parseInt(h['x-ratelimit-reset'], 10) * 1000 - Date.now(); - if (!waitMs || Number.isNaN(waitMs) || waitMs < 0) waitMs = 1000 * Math.pow(2, attempt); - waitMs = Math.min(waitMs, 60000) + 500; - core.warning(`${label}: rate-limited (status ${status}); waiting ${Math.round(waitMs / 1000)}s (attempt ${attempt}/5)`); - await sleep(waitMs); - } - } - } - - let result; - try { - result = JSON.parse(fs.readFileSync('/tmp/ocr-result.json', 'utf8')); - } catch (e) { - const stderr = (() => { try { return fs.readFileSync('/tmp/ocr-stderr.log', 'utf8').trim(); } catch { return ''; } })(); - await withRetry(() => github.rest.issues.createComment({ - owner: context.repo.owner, repo: context.repo.repo, issue_number: prNumber, - body: `⚠️ **OCR** could not produce a review.\n\n\`\`\`\n${(stderr || e.message).slice(0, 8000)}\n\`\`\``, - }), 'error-comment'); - return; - } - - const comments = result.comments || []; - const warnings = result.warnings || []; - - const formatComment = (c) => { - let body = c.content || ''; - if (c.suggestion_code && c.existing_code) { - body += '\n\n```suggestion\n' + c.suggestion_code + (c.suggestion_code.endsWith('\n') ? '' : '\n') + '```'; - } - return body; - }; - const formatMarkdown = (c) => { - let md = `### 📄 \`${c.path}\``; - if (c.start_line && c.end_line) md += ` (L${c.start_line}-L${c.end_line})`; - md += '\n\n' + (c.content || ''); - if (c.suggestion_code && c.existing_code) { - md += '\n\n
💡 Suggested change\n\n'; - md += '**Before:**\n```\n' + c.existing_code + '\n```\n\n**After:**\n```\n' + c.suggestion_code + '\n```\n\n
'; - } - return md; - }; - - if (comments.length === 0) { - await withRetry(() => github.rest.issues.createComment({ - owner: context.repo.owner, repo: context.repo.repo, issue_number: prNumber, - body: `✅ **OCR**: ${result.message || 'No issues found.'}`, - }), 'no-issues-comment'); - return; - } - - const inlineAll = []; - const noLine = []; - for (const c of comments) { - const body = formatComment(c); - const hasLine = (c.start_line >= 1) || (c.end_line >= 1); - if (!hasLine) { noLine.push(c); continue; } - const rc = { path: c.path, body, side: 'RIGHT' }; - if (c.start_line >= 1 && c.end_line >= 1 && c.start_line !== c.end_line) { - rc.start_line = c.start_line; rc.line = c.end_line; rc.start_side = 'RIGHT'; - } else { - rc.line = c.end_line >= 1 ? c.end_line : c.start_line; - } - inlineAll.push({ rc, c }); - } - - const inline = inlineAll.slice(0, MAX_INLINE).map(x => x.rc); - const overflow = inlineAll.slice(MAX_INLINE).map(x => x.c); - - let summary = `🔍 **OCR** found **${comments.length}** issue(s).`; - summary += `\n- ${inline.length} inline, ${noLine.length + overflow.length} in summary`; - if (overflow.length) summary += ` (inline capped at ${MAX_INLINE})`; - if (warnings.length) summary += `\n- ⚠️ ${warnings.length} warning(s) during review`; - for (const c of noLine.concat(overflow)) summary += '\n\n---\n\n' + formatMarkdown(c); - - // Preferred path: ONE createReview carrying every inline comment. - try { - await withRetry(() => github.rest.pulls.createReview({ - owner: context.repo.owner, repo: context.repo.repo, pull_number: prNumber, - commit_id: commitSha, body: summary, event: 'COMMENT', comments: inline, - }), 'bulk-review'); - return; - } catch (e) { - core.warning(`bulk createReview failed (${e.status || '?'}: ${e.message}); falling back to throttled per-comment posting`); - } - - // Fallback: an invalid inline position (line not in the diff -> 422) - // rejects the whole bulk review. Post the summary, then each comment - // individually with a delay + backoff, skipping ones GitHub rejects. - let ok = 0; const failed = []; - try { - await withRetry(() => github.rest.pulls.createReview({ - owner: context.repo.owner, repo: context.repo.repo, pull_number: prNumber, - commit_id: commitSha, body: summary, event: 'COMMENT', - }), 'summary-review'); - } catch (err) { failed.push(`summary: ${err.message}`); } - - for (const rc of inline) { - try { - await withRetry(() => github.rest.pulls.createReviewComment({ - owner: context.repo.owner, repo: context.repo.repo, pull_number: prNumber, - commit_id: commitSha, path: rc.path, body: rc.body, - ...(rc.start_line ? { start_line: rc.start_line, start_side: rc.start_side } : {}), - line: rc.line, side: rc.side, - }), `comment ${rc.path}:${rc.line}`); - ok++; - } catch (inner) { - failed.push(`\`${rc.path}\` L${rc.line}: ${inner.message}`); - } - await sleep(1200); // stay under the secondary content-creation limit - } - - if (failed.length) { - await withRetry(() => github.rest.issues.createComment({ - owner: context.repo.owner, repo: context.repo.repo, issue_number: prNumber, - body: `📊 OCR posted ${ok}/${inline.length} inline comment(s).\n\n
${failed.length} could not be posted\n\n${failed.join('\n')}\n
`, - }), 'summary-failures'); - } - - # Companion job: OCR can't call MCP, so this separate agent ties the PR's - # changes to PostgreSQL git + pgsql-hackers history via the Agora MCP server - # (pg.ddx.io) and posts a single, upserted "history & discussion" comment. - pg-history: - runs-on: ubuntu-latest + review: + # Comment events only when the comment is on a PR and starts with the + # trigger keyword; PR events and manual dispatch always. if: | github.event_name == 'pull_request' || github.event_name == 'workflow_dispatch' || @@ -346,82 +36,8 @@ jobs: (startsWith(github.event.comment.body, '/open-code-review') || startsWith(github.event.comment.body, '@open-code-review') || startsWith(github.event.comment.body, '/pg-history'))) - steps: - - name: Resolve PR context - id: pr - uses: actions/github-script@v9 - with: - script: | - let prNumber; - if (context.eventName === 'pull_request') prNumber = context.payload.pull_request.number; - else if (context.eventName === 'issue_comment') prNumber = context.issue.number; - else prNumber = parseInt('${{ github.event.inputs.pr_number }}', 10); - const { data: pr } = await github.rest.pulls.get({ - owner: context.repo.owner, repo: context.repo.repo, pull_number: prNumber }); - core.setOutput('number', String(prNumber)); - core.setOutput('base_ref', pr.base.ref); - core.setOutput('head_sha', pr.head.sha); - core.setOutput('title', pr.title || ''); - - - name: Checkout - uses: actions/checkout@v6 - with: - fetch-depth: 0 - - - name: Make base/head refs available - env: - BASE_REF: ${{ steps.pr.outputs.base_ref }} - HEAD_SHA: ${{ steps.pr.outputs.head_sha }} - run: | - git fetch --no-tags origin "+refs/heads/${BASE_REF}:refs/remotes/origin/${BASE_REF}" || true - git fetch --no-tags origin "${HEAD_SHA}" || true - - - name: Setup Python - uses: actions/setup-python@v6 - with: - python-version: '3.12' - - - name: Configure AWS credentials (OIDC) - uses: aws-actions/configure-aws-credentials@v6 - with: - role-to-assume: ${{ vars.AWS_ROLE_ARN }} - aws-region: ${{ vars.AWS_REGION }} - role-session-name: pg-history-${{ github.run_id }} - - - name: Install deps - run: pip install boto3 - - - name: Run pg-history (Agora MCP) - env: - PG_HISTORY_MODEL: ${{ vars.OCR_BEDROCK_MODEL }} - AWS_REGION: ${{ vars.AWS_REGION }} - BASE_REF: ${{ steps.pr.outputs.base_ref }} - HEAD_SHA: ${{ steps.pr.outputs.head_sha }} - GH_PR_TITLE: ${{ steps.pr.outputs.title }} - PG_HISTORY_OUT: ${{ runner.temp }}/pg-history.md - run: | - python .github/ocr/pg-history.py || true - echo "----- output -----"; cat "${{ runner.temp }}/pg-history.md" 2>/dev/null || echo "(no output)" - - - name: Upsert PR comment - uses: actions/github-script@v9 - with: - script: | - const fs = require('fs'); - const path = process.env.RUNNER_TEMP + '/pg-history.md'; - let body = ''; - try { body = fs.readFileSync(path, 'utf8').trim(); } catch (e) {} - if (!body) { console.log('pg-history: empty output, nothing to post'); return; } - const prNumber = parseInt('${{ steps.pr.outputs.number }}', 10); - const marker = ''; - body = marker + '\n' + body; - const { data: comments } = await github.rest.issues.listComments({ - owner: context.repo.owner, repo: context.repo.repo, issue_number: prNumber, per_page: 100 }); - const mine = comments.find(c => c.user.type === 'Bot' && c.body && c.body.includes(marker)); - if (mine) { - await github.rest.issues.updateComment({ - owner: context.repo.owner, repo: context.repo.repo, comment_id: mine.id, body }); - } else { - await github.rest.issues.createComment({ - owner: context.repo.owner, repo: context.repo.repo, issue_number: prNumber, body }); - } + uses: gburd/ci-workflows/.github/workflows/ocr-review.yml@v1 + with: + aws_role_arn: ${{ vars.AWS_ROLE_ARN }} + aws_region: ${{ vars.AWS_REGION }} + bedrock_model: ${{ vars.OCR_BEDROCK_MODEL }} diff --git a/.github/workflows/fork-sync-upstream.yml b/.github/workflows/fork-sync-upstream.yml index e950821239968..022b3101b5a9d 100644 --- a/.github/workflows/fork-sync-upstream.yml +++ b/.github/workflows/fork-sync-upstream.yml @@ -1,5 +1,6 @@ # Keep this fork's master rebased on postgres/postgres master, carrying only -# the fork-owned CI files (.github/workflows/fork-*.yml and .github/ocr/). +# the fork-owned CI stubs (.github/workflows/fork-*.yml). The workflows they +# call, and the OCR review rules, live in gburd/ci-workflows. # # Requires the SYNC_TOKEN secret: a PAT with repo+workflow scope. The default # GITHUB_TOKEN is not allowed to push commits that touch .github/workflows/, @@ -38,7 +39,7 @@ jobs: # Master carries fork CI only. Anything else means work landed here # that belongs on a branch; force-rebasing it on a timer would be wrong. if git diff --name-only upstream/master...HEAD | - grep -vE '^(\.github/workflows/fork-|\.github/ocr/)'; then + grep -v '^\.github/workflows/fork-'; then echo "::error::master carries non-CI changes (listed above); move them to a branch" exit 1 fi From e488b412fd57a4d56b479468cdb975444c31e24c Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:40 -0400 Subject: [PATCH 003/338] Add property-based testing harness (src/test/prop) Introduce an optional, opt-in property-based testing layer built on the Hegel engine (libhegel), off by default and gated on an explicit configure flag plus the engine's presence plus TAP tests being enabled. Adds the harness (pg_prop driver, CLI, TAP output, the PG_PROP registry, the libpq oracle helpers), the two-tier TAP drivers, the autoconf and meson plumbing, and the README and DESIGN documents. Areas are added in following commits. --- configure.ac | 29 ++ meson.build | 44 ++ meson_options.txt | 3 + src/Makefile.global.in | 2 + src/test/Makefile | 5 +- src/test/meson.build | 4 + src/test/prop/DESIGN | 380 +++++++++++++++ src/test/prop/Makefile | 46 ++ src/test/prop/README | 209 ++++++++ src/test/prop/meson.build | 43 ++ src/test/prop/pg_prop.c | 811 +++++++++++++++++++++++++++++++ src/test/prop/pg_prop.h | 238 +++++++++ src/test/prop/pg_prop_sql.c | 195 ++++++++ src/test/prop/t/001_prop.pl | 93 ++++ src/test/prop/t/002_sql_props.pl | 97 ++++ src/tools/pgindent/typedefs.list | 5 + 16 files changed, 2203 insertions(+), 1 deletion(-) create mode 100644 src/test/prop/DESIGN create mode 100644 src/test/prop/Makefile create mode 100644 src/test/prop/README create mode 100644 src/test/prop/meson.build create mode 100644 src/test/prop/pg_prop.c create mode 100644 src/test/prop/pg_prop.h create mode 100644 src/test/prop/pg_prop_sql.c create mode 100644 src/test/prop/t/001_prop.pl create mode 100644 src/test/prop/t/002_sql_props.pl diff --git a/configure.ac b/configure.ac index 6864268eaa071..f1f7b2e4ab8c7 100644 --- a/configure.ac +++ b/configure.ac @@ -241,6 +241,23 @@ PGAC_ARG_BOOL(enable, injection-points, no, [enable injection points (for testin [AC_DEFINE([USE_INJECTION_POINTS], 1, [Define to 1 to build with injection points. (--enable-injection-points)])]) AC_SUBST(enable_injection_points) +# +# Property-based tests (src/test/prop), built on libhegel. Requires +# --enable-tap-tests as well, since the suite is driven through the TAP +# harness; the actual presence of libhegel's header and library is checked +# further below, once --with-includes / --with-libraries are in effect. +# +PGAC_ARG_BOOL(with, hegel, no, + [build property-based tests (requires libhegel and --enable-tap-tests)]) +if test "$with_hegel" = yes && test "$enable_tap_tests" != yes; then + AC_MSG_ERROR([--with-hegel requires --enable-tap-tests]) +fi +AC_SUBST(with_hegel) +# Export the extra library directories so the property-test Makefile can add +# them to pg_prop's rpath; libhegel is typically installed outside the system +# loader path and reached via --with-libraries. +AC_SUBST(LIBRARY_DIRS) + # # Block size # @@ -1416,6 +1433,18 @@ failure. It is possible the compiler isn't looking in the proper directory. Use --without-zlib to disable zlib support.])]) fi +# libhegel for property-based tests. The C ABI header has no PostgreSQL +# dependencies; the engine ships as a plain shared library, libhegel_c. Both +# must be reachable via --with-includes / --with-libraries (or a system path). +if test "$with_hegel" = yes; then + AC_CHECK_HEADER(hegel.h, [], + [AC_MSG_ERROR([header file is required for --with-hegel +Point --with-includes at libhegel's include directory.])]) + AC_CHECK_LIB(hegel_c, hegel_version, [], + [AC_MSG_ERROR([library 'hegel_c' (libhegel) is required for --with-hegel +Point --with-libraries at libhegel's library directory.])]) +fi + if test "$with_libcurl" = yes ; then PGAC_CHECK_LIBCURL fi diff --git a/meson.build b/meson.build index d05634fef0d8c..35820c667b10d 100644 --- a/meson.build +++ b/meson.build @@ -1827,6 +1827,49 @@ endif +############################################################### +# Library: libhegel (property-based testing engine) +############################################################### + +# Property-based tests (src/test/prop) require all three of: +# 1. -Dprop_tests=enabled (explicit opt-in; default disabled), +# 2. libhegel's header and shared library present (found via +# extra_include_dirs / extra_lib_dirs or a system path), and +# 3. TAP tests enabled (the prop suite is driven through the TAP wrapper). +# Any missing piece disables the suite; with -Dprop_tests=enabled a missing +# piece is a hard error so the opt-in cannot silently do nothing. +prop_tests_enabled = false +libhegel = not_found_dep +propopt = get_option('prop_tests') +if not propopt.disabled() + if not tap_tests_enabled + if propopt.enabled() + error('prop_tests requires tap_tests to be enabled') + else + message('property-based tests disabled: TAP tests are not enabled') + endif + else + # The C ABI header has no PostgreSQL dependencies; the engine ships as a + # plain shared library named libhegel_c. Look for both. + libhegel_h = cc.has_header('hegel.h', + args: test_c_args, include_directories: postgres_inc, + required: propopt) + libhegel_lib = cc.find_library('hegel_c', + dirs: postgres_lib_d, required: propopt) + if libhegel_h and libhegel_lib.found() + libhegel = declare_dependency(dependencies: libhegel_lib) + prop_tests_enabled = true + elif propopt.enabled() + error('prop_tests requires libhegel (hegel.h and libhegel_c); ' + + 'point --extra-include-dirs / --extra-lib-dirs at its install') + else + message('property-based tests disabled: libhegel not found') + endif + endif +endif + + + ############################################################### # Library: zstd ############################################################### @@ -4322,6 +4365,7 @@ summary( 'icu': icu, 'ldap': ldap, 'libcurl': libcurl, + 'libhegel (prop_tests)': libhegel, 'libnuma': libnuma, 'liburing': liburing, 'libxml': libxml, diff --git a/meson_options.txt b/meson_options.txt index 6a793f3e47943..98b5537e901e9 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -46,6 +46,9 @@ option('tap_tests', type: 'feature', value: 'auto', option('injection_points', type: 'boolean', value: false, description: 'Enable injection points') +option('prop_tests', type: 'feature', value: 'disabled', + description: 'Enable property-based tests (requires libhegel and TAP tests)') + option('PG_TEST_EXTRA', type: 'string', value: '', description: 'Enable selected extra tests. Overridden by PG_TEST_EXTRA environment variable.') diff --git a/src/Makefile.global.in b/src/Makefile.global.in index 38914773b837b..67ff783635971 100644 --- a/src/Makefile.global.in +++ b/src/Makefile.global.in @@ -211,6 +211,8 @@ enable_dtrace = @enable_dtrace@ enable_coverage = @enable_coverage@ enable_injection_points = @enable_injection_points@ enable_tap_tests = @enable_tap_tests@ +with_hegel = @with_hegel@ +LIBRARY_DIRS = @LIBRARY_DIRS@ python_includespec = @python_includespec@ python_libdir = @python_libdir@ diff --git a/src/test/Makefile b/src/test/Makefile index 3eb0a06abb46e..43b568438bc54 100644 --- a/src/test/Makefile +++ b/src/test/Makefile @@ -34,12 +34,15 @@ endif ifeq ($(with_ssl),openssl) SUBDIRS += ssl endif +ifeq ($(with_hegel),yes) +SUBDIRS += prop +endif # We don't build or execute these by default, but we do want "make # clean" etc to recurse into them. (We must filter out those that we # have conditionally included into SUBDIRS above, else there will be # make confusion.) -ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos icu ldap ssl) +ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos icu ldap ssl prop) # We want to recurse to all subdirs for all standard targets, except that # installcheck and install should not recurse into the subdirectory "modules". diff --git a/src/test/meson.build b/src/test/meson.build index cd45cbf57fb0f..a0026b27fa69b 100644 --- a/src/test/meson.build +++ b/src/test/meson.build @@ -26,3 +26,7 @@ if icu.found() endif subdir('perl') + +if prop_tests_enabled + subdir('prop') +endif \ No newline at end of file diff --git a/src/test/prop/DESIGN b/src/test/prop/DESIGN new file mode 100644 index 0000000000000..65bb33d22b4a9 --- /dev/null +++ b/src/test/prop/DESIGN @@ -0,0 +1,380 @@ +src/test/prop/DESIGN + +Design of the property-based test harness +========================================= + +This document explains how the property-based test (PBT) harness is put +together and why. README covers how to run it and when to use it; this is the +under-the-covers companion. + + +Goals and constraints +====================== + +1. Optional and opt-in. PostgreSQL must build and test exactly as before when + the feature is off. The engine is an external dependency with prebuilt + binaries for a limited set of platforms, so it cannot be a mandatory part of + the tree. + +2. C, not C++. PostgreSQL is a C project maintained by C programmers. The + engine, libhegel, exposes a stable C ABI; everything in this directory is + plain C that calls it. No C++ appears in the tree. + +3. Fits the existing test world. When enabled, the suite is just more TAP + tests, discovered and run by "meson test" / "make check-world" like any + other. + +4. Reproducible. Every run has a seed (printed if not supplied); every failure + prints a replay blob and shrinks to a minimal counterexample. + + +What Hegel provides +=================== + +libhegel is the engine behind Hegel, shipped as a shared library (libhegel_c) +with a C ABI declared in hegel.h. It owns the hard parts of property-based +testing: input generation, the shrinking search, and the example database. The +harness drives it through a small slice of that ABI: + +* hegel_settings_* -- configure a run (test-case count, seed, verbosity, + database path and key). + +* hegel_run_start / hegel_next_test_case / hegel_run_result -- the run loop: the + engine hands us one test case at a time; we execute the property body against + it and report the outcome with hegel_mark_complete. + +* hegel_generate_integer / _float / _boolean / _bytes, and + hegel_string_generator_text + hegel_generate_string -- the generators the + property body draws from. + +* hegel_run_result_failure / hegel_failure_reproduction_blob -- the minimal + counterexample and its replay blob after a failing run. + +* hegel_test_case_from_blob -- replay a counterexample for reporting. + +The engine runs in-process: there is no separate server process and it starts no +threads of its own. One property run is single-threaded. + + +Shape of the tree +================= + + pg_prop.h public harness API: the PG_PROP() macro, draws, + assume/note/fail, isolate(), and the Tier-B SQL helpers. + pg_prop.c the libhegel driver, generators, CLI, TAP output; main(). + pg_prop_sql.c libpq oracle helpers (Tier B); isolates the libpq + dependency. + prop_.c one file per area; a set of PG_PROP() properties. + sql/ query-level areas that generate whole SQL statements + (query_gen.c plus the TLP, NoREC, and plan-advice areas). + t/001_prop.pl TAP driver for the in-process (Tier A) areas. + t/002_sql_props.pl TAP driver for the SQL-oracle (Tier B) areas; starts a + node. + Makefile autoconf build (one program, pg_prop). + meson.build meson build (same program) plus test registration. + +All areas compile into one program, pg_prop. The first command-line argument +selects the area; the rest select individual properties. One binary (rather +than one per area) keeps the build small and gives the "pg_prop " dispatch +the user-facing CLI wants. + + +The PG_PROP registry +-------------------- + +Each prop_.c does: + + #define PG_PROP_AREA "path" + #include "pg_prop.h" + + PG_PROP(canonicalize_idempotent) + { + ... + } + +PG_PROP(name) expands to the property function plus a function-pointer +registration run before main() via __attribute__((constructor)). Each +registration records the area (from PG_PROP_AREA), the name, and the function. +main() then looks up the area named on the command line and runs its properties. +Because registration is a constructor, simply linking an area file into pg_prop +makes its properties available -- there is no central table to edit. + + +The run loop and failure reporting +---------------------------------- + +For each selected property, run_property(): + +1. builds a hegel_settings_t from the CLI (test_cases, seed, verbosity, and a + database_key of "area/name" so persisted counterexamples are scoped per + property); + +2. starts a run and pulls test cases until the engine signals completion; + +3. for each case, run_one_case() sets up a pg_prop_tc and calls the property body + inside sigsetjmp. The body reports its outcome by not returning normally: + + * returning normally -> HEGEL_STATUS_VALID (property held), + * pg_prop_assume(false) -> siglongjmp(..., 2) -> HEGEL_STATUS_INVALID + (case discarded), + * pg_prop_fail(...) -> siglongjmp(..., 1) -> HEGEL_STATUS_INTERESTING + (counterexample), + * a draw past the budget -> siglongjmp -> HEGEL_STATUS_OVERRUN. + + setjmp/longjmp lets pg_prop_fail be called from arbitrarily deep in a property + body and still be pg_noreturn; + +4. after the run, if it failed, replays each counterexample's blob through + hegel_test_case_from_blob to re-derive the failing message, and prints the + "falsified:" line plus the "rerun with" blob as TAP diagnostics. + +Output is TAP by default (one ok/not ok per property), so the surrounding "meson +test" / "prove" consumes it directly; --no-tap switches to plain text for +interactive use. + + +Seeds and the example database +------------------------------ + +If -s is not given, main() picks a seed from the PID and clock and prints +"# seed: N". A buildfarm failure is therefore always reproducible with -s N. +The example database (where Hegel persists counterexamples so a known failure +replays first) is placed under TESTDATADIR when the TAP wrapper sets it, and +disabled otherwise, so nothing is written into the source tree. + + +Two tiers +========= + +Backend code -- the planner, executor, type I/O -- cannot be linked into a +frontend program, so the harness has two tiers that differ only in how the code +under test is reached. + + +Tier A: in-process +------------------ + +src/common and src/port code (path, jsonapi, encoding, unicode_case, +unicode_norm, pglz, base64, prng, crypto, scram, binaryheap, bitutils) is linked +directly into pg_prop and called in the property body. Each case is a function +call: microseconds. t/001_prop.pl runs these; no server is needed. + +Crash isolation. Frontend PostgreSQL code reports fatal problems by abort() (a +failed Assert under cassert) or exit(). In-process, either would take down the +whole pg_prop process and lose the counterexample. pg_prop_run_isolated() forks +a child to run code that might do this and turns a crash, signal, or non-zero +exit into an ordinary value the property can test and pg_prop_fail on -- with +the drawn inputs, generated in the parent, still in hand for shrinking. (The +current area files mostly do not need it, since their functions return error +codes rather than aborting, but it is there for code that does and for running +under a sanitizer.) + + +Tier B: SQL oracle +------------------ + +Backend behavior is tested differentially over libpq against a live node: the +property generates SQL and compares the answer to an oracle that must agree. +pg_prop_sql.c wraps just enough of libpq -- connect, run a scalar/rowset/DDL +query, render the result as a comparable string -- and keeps the libpq +dependency out of the rest of the harness. t/002_sql_props.pl starts one node, +exports PGHOST/PGPORT, and runs these areas against it. + +The oracles are chosen so a disagreement is unambiguously a bug, never a +difference of opinion: + +* partprune: a partitioned table with pruning on, the same table with pruning + off, and an unpartitioned table holding identical rows must all return the + same result for any query. Pruning is a performance optimization that must + never change results. + +* scan and join differentials (nbtree, indexscan, indexdiff, joinmethods, + outerjoin, ...): the same query run under forced scan methods, or forced join + methods, or a tiny versus ample work_mem, must return identical rows. + +* type algebra (interval, numeric, numfmt, float, cash, date, timestamptz, and + the rest): algebraic identities evaluated in SQL -- round-trips, a = q*b + r, + distributivity, a total order, text output that re-parses, add/subtract + inverses -- that the type is required to satisfy for all finite inputs (with + legitimate overflow skipped, and never silently wrapped). + +* structured types (jsonb, jsonpath, array, range, multirange, record, + tsvector, and friends): round-trips through text and the set/algebra laws of + the type's operators (containment, union, intersection, key dedup, + aggregate/unnest inverse, builder/accessor inverse, lexicographic ordering), + exact because these types normalize on input. + +* parsing and formatting (like, tsquery, tochar, bool, char): matching, + formatting, and parsing cross-checked against independent builtins + (starts_with, length, an explicit unnest membership scan) and the boolean + structure of the operators. + +Connection hygiene. A property that fails or is assumed away unwinds by +longjmp, jumping over its own pg_prop_disconnect(). The driver therefore calls +pg_prop_sql_cleanup() after every case, which closes any connection a case left +open; otherwise a long run would exhaust the server's connection slots. (It is +a no-op for Tier A.) + + +Query-level areas (sql/) +======================== + +The highest-value Tier-B areas do not test one function; they test the query +processor. They live in sql/ and share query_gen.c, which builds a random table +and a random three-valued boolean predicate as SQL text. Each area wraps that +in a metamorphic oracle -- a relationship that must hold with no independent +reference answer, so a disagreement is unambiguously a server bug: + +* query_tlp -- Ternary Logic Partitioning (Rigger and Su, OOPSLA 2020). For + any predicate p every row falls in exactly one of WHERE p, WHERE NOT p, WHERE + p IS NULL, so the three partitions UNION ALL'd must reconstruct the unfiltered + relation. + +* query_norec -- compare an optimizable count(*) ... WHERE p against the + un-optimizable twin that pulls p into a bare per-row scan; the counts must + match. + +* query_planadvice -- run a generated query with the planner free and again + steered by contrib/pg_plan_advice (SEQ_SCAN, NESTED_LOOP_PLAIN, and so on, via + the pg_plan_advice.advice GUC), plus a tiny-work_mem spill variant; the result + must not depend on the plan the planner chose. Advice is used in preference + to the blunt enable_* GUCs because it names the exact relation or join to + re-shape and the extension rejects advice it cannot honor, so the alternate + plan is a targeted legal variant of the same query rather than a global cost + distortion. + + +Build-system integration +======================== + +meson +----- + +* meson_options.txt adds a feature option prop_tests (default disabled). + +* The top-level meson.build, right after the TAP-tests block, resolves + prop_tests_enabled: it requires tap_tests_enabled, then looks for hegel.h (via + postgres_inc, which includes --extra-include-dirs) and the hegel_c library + (via postgres_lib_d, i.e. --extra-lib-dirs). All three present -> enabled; + -Dprop_tests=enabled with a piece missing -> hard error; otherwise silently + off. A summary() line reports the result. + +* src/test/meson.build descends into prop/ only when prop_tests_enabled. + src/test/prop/meson.build builds pg_prop (linking frontend_code, libpq, and + libhegel) and registers the two TAP drivers as the "prop" suite. + +autoconf +-------- + +* configure.ac adds --with-hegel (default no); it errors immediately if + --enable-tap-tests was not also given. Down with the other library probes + (where --with-includes/--with-libraries are in effect) it runs + AC_CHECK_HEADER(hegel.h) and AC_CHECK_LIB(hegel_c, hegel_version), both hard + errors under --with-hegel. The AC_CHECK_LIB deliberately does not add + -lhegel_c to the global LIBS; only pg_prop links it. + +* with_hegel flows through Makefile.global.in. src/test/Makefile adds prop to + SUBDIRS when with_hegel = yes (and always to ALWAYS_SUBDIRS so "make clean" + recurses). src/test/prop/Makefile builds the one program and declares + TAP_TESTS = 1, so "make check" runs t/*.pl. + + +Measuring the value: coverage and input diversity +================================================== + +Two things were measured on an instrumented build (meson setup ... +-Db_coverage=true), PostgreSQL src/ only, branch coverage off, comparing the PBT +suite against "test-world" (the regress and isolation suites). + +Line coverage, the number that undersells it: + + configuration line coverage function coverage + test-world (regress + isolation) 72.1% (315,641) 77.6% + PBT suite only 32.6% (130,227) 42.4% + union (test-world + PBT) 72.8% (327,606) 79.0% + +The PBTs add 1,293 server lines test-world never reaches (+0.41% relative). +That is small, and honestly so: test-world already touches almost every adt/*.c +line at least once. + +Execution depth per line, what we hoped for and did not find. It is tempting to +argue the PBTs at least execute the shared lines far more often -- more inputs +per line. Measured directly with gcov execution counts, the honest result is +no, not in raw count. Run for run, the dense regression corpus executes the hot +type-I/O and planner lines more than a 100-iteration PBT pass, and even a single +area at 8,000 iterations executes src/common/jsonapi.c only about 0.2x as many +times total as the json/jsonb regression tests do. Per hot file the PBT/world +execution-count ratios ran from about 0.1x (formatting.c, selfuncs.c) to about +2.7x (nodeAgg.c); there is no "orders of magnitude" effect and none is claimed. + +Input diversity via differential coverage, the real measured gain. The gain is +on a different axis: the generators reach different lines, not the same lines +more. Measured on the file each area targets, the jsonapi area at 8,000 +iterations covers 527 lines of src/common/jsonapi.c, of which 242 are lines the +json and jsonb regression tests never reach (test-world covers 464). Those 242 +lines are overwhelmingly the incremental-parser state machine, which only +executes when a value is delivered one byte at a time -- precisely what the +generator does and what no hand-written case enumerates. This is the same +mechanism that caught the reverted jsonapi "0.0" bug. + +To reproduce the measurement, build with -Db_coverage=true, zero the counters +(find ... -name '*.gcda' -delete), run one suite, capture with lcov (adding +--ignore-errors inconsistent,corrupt), extract the PostgreSQL src/ paths, repeat +for the other suite, and compare the per-line DA: counts and the covered-line +sets between the two tracefiles. + +The takeaway for a reviewer: judge this suite by which code it reaches and by +the standing differential net, not by a coverage-percentage delta or a +depth-per-line multiplier -- the latter does not favor PBT and we do not claim +it does. + + +Why this shape +============== + +* One binary, area-dispatched. Matches the "pg_prop " CLI, keeps the + build to a single target, and lets a developer run or debug exactly one + property with a fixed seed. + +* C ABI, not the C++ frontend. Keeps the tree C-only; the only non-C artifact + is the prebuilt engine library itself. + +* setjmp for fail/assume. Lets assertions sit naturally deep in a property body + and keeps pg_prop_fail pg_noreturn. + +* Differential oracles for backend code. Avoids running the engine inside a + backend (which would need PG_TRY shims to keep longjmp out of the engine's + frames -- not needed here, and not worth the complexity until a round-trip + through SQL becomes a measured bottleneck). + +* Default 100 iterations. Keeps check-world cheap; the iteration count is a knob + (-n / PG_PROP_ITERATIONS) you turn up when hunting. + + +Limitations and future work +=========================== + +* Platform coverage is whatever libhegel ships a prebuilt library for (Linux + x86_64/arm64, macOS arm64, Windows at the time of writing); other platforms + would need a source build of the engine. The opt-in default keeps this from + affecting anyone who does not ask for it. + +* Beta engine. Hegel is pre-1.0; its API can change and replay blobs are only + guaranteed to reproduce on the same engine version. The configure checks pin + to the header and library actually installed. + +* In-backend properties (running the engine inside a running backend, for + cheaper backend coverage than round-tripping SQL) are deliberately not + attempted yet; they would require PG_TRY-based shims so a longjmp cannot cross + the engine's frames. Worth revisiting only if the SQL round-trip cost becomes + the bottleneck. + +* Stateful and model testing (sequences of operations checked against a + reference model) is supported by Hegel but not yet used here; the data + structures in src/backend/lib (for example the radix tree and the TID store) + are the natural first customers. + +* More query-generation oracles are the obvious growth area: TLP for aggregates + and DISTINCT, Pivoted Query Synthesis, join-order metamorphic testing via + pg_plan_advice, and cardinality-monotonicity checks. These are where a live + wrong-results bug is most likely to turn up. diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile new file mode 100644 index 0000000000000..656173886bb11 --- /dev/null +++ b/src/test/prop/Makefile @@ -0,0 +1,46 @@ +# src/test/prop/Makefile +# +# Property-based tests built on libhegel. Only reachable when configure was +# run with --with-hegel (which also requires --enable-tap-tests); see +# src/test/Makefile. +# +# Everything links into one program, pg_prop, whose first argument selects the +# area: pg_prop [options] [property ...]. The TAP driver runs each +# area; a developer debugging a failure runs one area or one property by hand +# with --iterations / --seed. See README and DESIGN. + +PGFILEDESC = "pg_prop - property-based tests" +PGAPPICON = win32 + +PROGRAM = pg_prop +# One object per area file, plus the shared harness (pg_prop.o) and the SQL +# oracle helper (pg_prop_sql.o). Add prop_.o here as each area lands +# (keep in step with the area lists in t/001_prop.pl and t/002_sql_props.pl). +OBJS = \ + $(WIN32RES) \ + pg_prop.o \ + pg_prop_sql.o + +PG_CPPFLAGS = -I$(libpq_srcdir) +PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) +# libhegel's C engine. Its include/lib dirs reach us through --with-includes +# / --with-libraries (CPPFLAGS / LDFLAGS); here we just name the library. +PG_LIBS_INTERNAL += -lhegel_c + +NO_INSTALL = 1 +TAP_TESTS = 1 + +subdir = src/test/prop +top_builddir = ../../.. +include $(top_builddir)/src/Makefile.global +include $(top_srcdir)/contrib/contrib-global.mk + +# libhegel is usually installed outside the system loader path (reached via +# --with-libraries), so record each configured extra library directory in +# pg_prop's rpath; otherwise the dynamic loader cannot find libhegel_c.so at +# run time. Must come after Makefile.global, which defines LIBRARY_DIRS. +# Harmless when LIBRARY_DIRS is empty. +LDFLAGS += $(foreach dir,$(LIBRARY_DIRS),-Wl,-rpath,$(dir)) + +pg_prop$(X): | submake-libpgfeutils +check: pg_prop$(X) diff --git a/src/test/prop/README b/src/test/prop/README new file mode 100644 index 0000000000000..44d7ae58623f9 --- /dev/null +++ b/src/test/prop/README @@ -0,0 +1,209 @@ +src/test/prop/README + +Property-based tests +==================== + +This directory holds PostgreSQL's optional property-based tests (PBTs), built +on the Hegel engine (libhegel; see https://hegel.dev). A property-based test +states a property that must hold for every input in some domain -- a +round-trip, an invariant, an algebraic identity, or agreement between two ways +of computing the same answer -- and the engine generates many inputs trying to +falsify it. When it finds a counterexample it shrinks it to a minimal failing +input and prints a blob that replays it exactly. + +These tests are entirely optional and are off by default. They are built and +run only when all of the following hold: + +1. you explicitly enable them at configure time (--with-hegel for autoconf, + -Dprop_tests=enabled for meson); + +2. libhegel (its hegel.h header and libhegel_c shared library) is present and + reachable via --with-includes/--with-libraries (autoconf) or + --extra-include-dirs/--extra-lib-dirs (meson); and + +3. TAP tests are enabled (--enable-tap-tests / -Dtap_tests=enabled). + +If any piece is missing the suite is silently skipped. If the opt-in flag is +set but a piece is missing, configure fails with a clear message rather than +quietly doing nothing. + +See DESIGN in this directory for how it all works under the covers. + + +When to use a property-based test (and when not to) +=================================================== + +Reach for a PBT when the code has a property that should hold across a large +input space: + +* Round-trips: parse(format(x)) = x, decompress(compress(x)) = x. + +* Idempotence: f(f(x)) = f(x) -- normalization, case folding, canonical path + cleanup. + +* Invariants and contracts: a verifier stops on a character boundary; a + function's output stays within documented bounds; a comparator is a total + order. + +* Differential oracles: two ways of computing the same answer must agree -- an + index scan versus a sequential scan, a query with an optimization on versus + off, a partitioned table versus an unpartitioned one with the same rows. + +* No-crash and robustness: a parser handles arbitrary bytes without crashing. + +Prefer an ordinary regression or TAP test when: + +* The test checks exact output. "render(x) = ..." depends on a specific + format; there is no general property, just a fixed expectation. + +* The test checks a specific error message. PBTs are good at "an error is + raised", poor at "the error says exactly this". Test the error class or + SQLSTATE, never the wording (it is reworded and translated between releases). + +* The output is deliberately nondeterministic: random(), unseeded + gen_random_uuid(), clock_timestamp(), pg_stat_*, timing, memory figures. + There is no stable oracle. + +* The target is plan shape or cost, not plan results. EXPLAIN output is + cost-model dependent and is intended to change across releases and with + statistics. (Plan results are a prime PBT target -- see the query_* areas; + plan shape is not.) + +* The only oracle is a second copy of the same algorithm. If the test + reimplements the function to check the function, it finds bugs in the test, + not the server. This is the single most common way a property goes wrong; + prefer a round-trip or a differential oracle (two independent ways that must + agree) over an invented formula. + +* The target is floating-point textual edge output (extra_float_digits, + last-ULP rendering): that is platform and libc behavior, not PostgreSQL's. + +* The target is performance or resource use. PBT asserts correctness, not + speed; that is a benchmark. + +* Setup dominates. If each case needs elaborate fixtures, the generator is + mostly scaffolding and a few hand-written cases are clearer. + +* No real property is apparent. Don't force a tautology ("x = x"); a good + unit test beats a contrived PBT. + +The rule of thumb: a PBT belongs only where a disagreement is unambiguously a +bug. That is why the SQL-oracle (Tier B) areas lean on differential oracles +and round-trips, not invented formulas. + +A PBT complements, rather than replaces, the existing regression and TAP +suites. Its value is exercising the combinations and boundary inputs a human +does not think to write by hand -- the three-byte "0.0" fed one byte at a time, +the two-partition prefix-equality query -- not raising line coverage. Measured, +the suite adds only about 0.4% line coverage over the regression and isolation +suites, and it does not execute shared lines more often than they do; its +measurable gain is reaching different lines (for example, 242 lines of +src/common/jsonapi.c that the json and jsonb regression tests never reach) plus +the standing differential net over the planner and executor. See DESIGN for +the measurement method and figures. + + +Running +======= + +The tests are organized into areas, each targeting one piece of code. There +are two kinds. In-process (Tier A) areas link the code under test directly +into the test program and need no server. SQL-oracle (Tier B) areas drive SQL +against a running cluster and compare the answer to an oracle. All areas are +compiled into one program, pg_prop, whose first argument selects the area: + + pg_prop [options] [property ...] + +Through the test harness: + + # meson: run the whole prop suite (both TAP drivers) + meson test --suite prop + + # meson: one driver (001 = in-process areas, 002 = SQL-oracle areas) + meson test prop/001_prop + meson test prop/002_sql_props + + # autoconf + make -C src/test/prop check + +Directly, for development and debugging. An in-process area needs no server: + + pg_prop --list # list every area/property + pg_prop jsonapi --list # any area filters the list + pg_prop jsonapi # all jsonapi properties + pg_prop jsonapi chunking_agrees_with_oneshot # one property + pg_prop jsonapi chunking_agrees_with_oneshot -n 100000 -s 42 + +A SQL-oracle area reads PGHOST/PGPORT/PGDATABASE from the environment, so point +it at any cluster: + + PGHOST=/tmp PGPORT=5432 PGDATABASE=postgres \ + pg_prop partprune pruning_preserves_rows -n 100000 -s 42 + +Controlling iterations and the seed: + +* -n, --iterations N -- number of test cases per property (default 100, kept + low so check-world stays cheap; raise it when hunting a bug). + +* -s, --seed N -- fix the RNG seed for a reproducible run. When you do not give + one, pg_prop picks a seed and prints it ("# seed: NNN"), so a failure seen in + CI can be replayed with -s NNN. + +The same three controls are available as the environment variables +PG_PROP_ITERATIONS, PG_PROP_SEED, and PG_PROP_TESTS (a space-separated list of +property names), which the TAP drivers forward -- so you can raise the +iteration count of a meson test run without editing anything: + + PG_PROP_ITERATIONS=10000 PG_PROP_SEED=42 meson test --suite prop + PG_PROP_TESTS="pruning_preserves_rows" meson test prop/002_sql_props + + +Debugging a failure +=================== + +1. Read the counterexample. On failure pg_prop prints a one-line "falsified:" + message that includes the minimal inputs Hegel shrank to, plus a "rerun + with" blob. The TAP drivers echo this as "#" diagnostics. + +2. Reproduce it deterministically. Re-run the single property with the seed + from the run's "# seed:" line: + + pg_prop -s -n + + (For a SQL area, set PGHOST/PGPORT first.) You can also raise verbosity with + -v to see every drawn value, not just the final counterexample. + +3. Decide what the failure means. There are three possibilities: + + * A real bug in the code under test -- the code violated its own contract. + Capture the reproducer. + + * An unsound property -- the test asserted something the code never promised + (for example, that a string verifier and a character verifier agree on a + NUL byte -- they deliberately do not). Fix the property. + + * An over-broad generator -- the failing input is genuinely outside the + function's domain. Only then narrow the generator, and say why in a + comment. + +4. Step through it. The reproducing command is an ordinary process; run it + under gdb or lldb with a breakpoint in the code under test. Because the seed + is fixed, the same inputs are generated every time. + + +Adding a property +================= + +To add a property to an existing area, add a PG_PROP(name) { ... } function to +that area's prop_.c. It is registered automatically. + +To add a new area, create prop_.c beginning with + + #define PG_PROP_AREA "" + +before including pg_prop.h, then add the file to the source list in meson.build, +the OBJS list in Makefile, and the area list in the matching TAP driver +(t/001_prop.pl for in-process areas, t/002_sql_props.pl for SQL-oracle areas). + +See pg_prop.h for the full generator and assertion API, and DESIGN for the +rationale and internals. diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build new file mode 100644 index 0000000000000..d8d64fbd1ea07 --- /dev/null +++ b/src/test/prop/meson.build @@ -0,0 +1,43 @@ +# Copyright (c) 2026, PostgreSQL Global Development Group +# +# Property-based tests built on libhegel. Reached only when +# prop_tests_enabled (see the top-level meson.build): -Dprop_tests=enabled, +# libhegel present, and TAP tests enabled. +# +# All areas link into one binary, pg_prop, whose first argument selects the +# area: pg_prop [options] [property ...]. Two TAP drivers run it: +# 001_prop.pl runs the in-process areas (Tier A: src/common, src/port code +# linked directly); 002_sql_props.pl starts a node and runs the SQL-oracle +# areas (Tier B: planner / executor / type code, reached over libpq) against +# it. A developer debugging a failure runs one area (or one property) by hand +# with --iterations / --seed. See README and DESIGN. + +# Area source files. Add one line as each target lands; keep the area lists +# in t/001_prop.pl (Tier A) and t/002_sql_props.pl (Tier B) in step. +prop_area_sources = files( + # Tier A: in-process + # Tier B: SQL oracle (needs a running node) +) + + +pg_prop = executable('pg_prop', + files('pg_prop.c', 'pg_prop_sql.c') + prop_area_sources, + dependencies: [frontend_code, libpq, libhegel], + kwargs: default_bin_args + { + 'install': false, + }, +) +testprep_targets += pg_prop + +tests += { + 'name': 'prop', + 'sd': meson.current_source_dir(), + 'bd': meson.current_build_dir(), + 'tap': { + 'tests': [ + 't/001_prop.pl', + 't/002_sql_props.pl', + ], + 'deps': [pg_prop], + }, +} diff --git a/src/test/prop/pg_prop.c b/src/test/prop/pg_prop.c new file mode 100644 index 0000000000000..9b089777e7855 --- /dev/null +++ b/src/test/prop/pg_prop.c @@ -0,0 +1,811 @@ +/*------------------------------------------------------------------------- + * + * pg_prop.c + * Property-based testing harness: libhegel driver, generators, CLI, TAP. + * + * See pg_prop.h for the API contract. This file is linked into every test + * binary under src/test/prop and provides main(). + * + * The CLI shape is: + * + * pg_prop_ [options] [PROPERTY ...] + * + * --list list registered properties and exit + * -n, --iterations N number of test cases per property (default 100) + * -s, --seed N fixed RNG seed (default: random, printed as a note) + * -v, --verbose raise engine verbosity + * -q, --quiet lower engine verbosity + * --no-tap plain output instead of TAP (for interactive use) + * -h, --help usage + * + * With no PROPERTY arguments every registered property runs. Environment + * variables PG_PROP_ITERATIONS, PG_PROP_SEED and PG_PROP_TESTS supply the same + * three controls (so they can be driven through the TAP wrapper and meson); + * explicit command-line options win over the environment. + * + * Output is TAP by default so meson/prove consume it directly: one "ok"/"not + * ok" line per property, the counterexample and replay blob as "#" diagnostic + * lines under a failure. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/pg_prop.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#include +#include +#include +#include +#include +#include +#include + +#include "common/logging.h" +#include "getopt_long.h" +#include "lib/stringinfo.h" +#include "pg_prop.h" + +/* + * libhegel's C ABI. Installed under a prefix given to configure via + * --with-includes / extra_include_dirs; the header has no PostgreSQL + * dependencies. + */ +#include "hegel.h" + +/* ---- registry -------------------------------------------------------- */ + +typedef struct PropEntry +{ + const char *area; + const char *name; + pg_prop_fn fn; + void *arg; +} PropEntry; + +#define MAX_PROPS 4096 +static PropEntry props[MAX_PROPS]; +static int nprops = 0; + +void +pg_prop_register(const char *area, const char *name, pg_prop_fn fn, void *arg) +{ + if (nprops >= MAX_PROPS) + { + fprintf(stderr, "pg_prop: too many properties (max %d)\n", MAX_PROPS); + exit(2); + } + props[nprops].area = area; + props[nprops].name = name; + props[nprops].fn = fn; + props[nprops].arg = arg; + nprops++; +} + +/* ---- per-case handle ------------------------------------------------- */ + +/* + * One pg_prop_tc wraps the libhegel context and the current test case. A + * failure is reported by longjmp back to the driver, which marks the case + * INTERESTING; the engine then shrinks and replays. We use setjmp rather + * than returning a status so pg_prop_fail can be called from arbitrarily deep + * in a property body (and so it can be pg_attribute_noreturn, which keeps the + * compiler from warning about missing returns after it). + */ +struct pg_prop_tc +{ + hegel_context_t *ctx; + hegel_test_case_t *tc; + StringInfoData failmsg; /* set when the body called pg_prop_fail */ + bool failed; + sigjmp_buf jmp; /* longjmp target: body -> driver */ + const char *origin; /* stable failure origin for this property */ +}; + +/* ---- engine error helper --------------------------------------------- */ + +static void +check_rc(hegel_context_t *ctx, hegel_result_t rc, const char *what) +{ + /* + * HEGEL_E_STOP_TEST and HEGEL_E_ASSUME are not errors: they are the + * engine telling the body to stop (budget exhausted) or that the case was + * assumed away. Both are handled by the callers that can see them; any + * other negative code is a real engine fault and is fatal. + */ + if (rc == HEGEL_OK || rc == HEGEL_E_STOP_TEST || rc == HEGEL_E_ASSUME) + return; + fprintf(stderr, "pg_prop: libhegel %s failed (%d): %s\n", + what, (int) rc, hegel_context_last_error(ctx)); + exit(2); +} + +/* + * A draw that returns HEGEL_E_STOP_TEST means the engine wants the body to + * stop immediately (it ran out of choice budget for this case). We model + * that as a failure-free longjmp out of the body: the driver then marks the + * case OVERRUN. pg_prop_fail reuses the same jump with failed=true. + */ +static pg_prop_tc *current_tc; /* for the stop/overrun jump */ +static bool overrun; + +static void +stop_test(pg_prop_tc *tc) +{ + overrun = true; + siglongjmp(tc->jmp, 1); +} + +/* ---- generators ------------------------------------------------------ */ + +int64 +pg_prop_draw_int(pg_prop_tc *tc, int64 min, int64 max) +{ + int64 out = 0; + hegel_result_t rc = hegel_generate_integer(tc->ctx, tc->tc, min, max, &out); + + if (rc == HEGEL_E_STOP_TEST) + stop_test(tc); + check_rc(tc->ctx, rc, "generate_integer"); + return out; +} + +double +pg_prop_draw_float(pg_prop_tc *tc, double min, double max) +{ + double out = 0; + hegel_result_t rc; + + /* + * Full doubles including NaN and the infinities only when the range is + * unbounded on the matching side, matching the engine's own defaults. + */ + rc = hegel_generate_float(tc->ctx, tc->tc, 64, min, max, + /* allow_nan */ isinf(min) || isinf(max) ? false : false, + /* allow_infinity */ false, + false, false, 5e-324, &out); + if (rc == HEGEL_E_STOP_TEST) + stop_test(tc); + check_rc(tc->ctx, rc, "generate_float"); + return out; +} + +bool +pg_prop_draw_bool(pg_prop_tc *tc) +{ + bool out = false; + hegel_result_t rc = hegel_generate_boolean(tc->ctx, tc->tc, 0.5, + false, false, &out); + + if (rc == HEGEL_E_STOP_TEST) + stop_test(tc); + check_rc(tc->ctx, rc, "generate_boolean"); + return out; +} + +unsigned char * +pg_prop_draw_bytes(pg_prop_tc *tc, size_t min_size, size_t max_size, size_t *len) +{ + hegel_generate_bytes_result_t res = {NULL, 0}; + hegel_result_t rc = hegel_generate_bytes(tc->ctx, tc->tc, min_size, + max_size, &res); + unsigned char *out; + + if (rc == HEGEL_E_STOP_TEST) + stop_test(tc); + check_rc(tc->ctx, rc, "generate_bytes"); + + out = pg_malloc(res.len + 1); + if (res.len > 0) + memcpy(out, res.data, res.len); + out[res.len] = '\0'; + *len = res.len; + hegel_generate_bytes_result_free(tc->ctx, &res); + return out; +} + +/* + * Shared text-draw path: build a one-shot string generator, draw from it, + * copy the result out, free both. Building a generator per draw is cheap + * relative to a round trip through the engine and keeps the API stateless. + */ +static char * +draw_string(pg_prop_tc *tc, hegel_string_generator_t *gen, size_t *len) +{ + hegel_generate_string_result_t res = {NULL, 0}; + hegel_result_t rc = hegel_generate_string(tc->ctx, tc->tc, gen, &res); + char *out; + + if (rc == HEGEL_E_STOP_TEST) + { + hegel_string_generator_free(tc->ctx, gen); + stop_test(tc); + } + if (rc == HEGEL_E_ASSUME) + { + /* generator rejected its own draw (e.g. length cap); discard case */ + hegel_string_generator_free(tc->ctx, gen); + siglongjmp(tc->jmp, 2); /* 2 == assume/invalid */ + } + check_rc(tc->ctx, rc, "generate_string"); + + out = pg_malloc(res.len + 1); + if (res.len > 0) + memcpy(out, res.data, res.len); + out[res.len] = '\0'; + *len = res.len; + hegel_generate_string_result_free(tc->ctx, &res); + hegel_string_generator_free(tc->ctx, gen); + return out; +} + +char * +pg_prop_draw_utf8(pg_prop_tc *tc, size_t min_size, size_t max_size, size_t *len) +{ + hegel_string_generator_t *gen = NULL; + hegel_result_t rc; + + rc = hegel_string_generator_text(tc->ctx, min_size, max_size, + "utf-8", 0, UINT32_MAX, + NULL, 0, NULL, 0, NULL, 0, NULL, 0, &gen); + check_rc(tc->ctx, rc, "string_generator_text"); + return draw_string(tc, gen, len); +} + +char * +pg_prop_draw_from(pg_prop_tc *tc, const char *alphabet, + size_t min_size, size_t max_size, size_t *len) +{ + hegel_string_generator_t *gen = NULL; + hegel_result_t rc; + + /* + * There is no "alphabet" field in the C ABI's text generator; emulate it + * with include_characters plus an empty category set, which restricts the + * draw to exactly those characters. + */ + rc = hegel_string_generator_text(tc->ctx, min_size, max_size, + "ascii", 0, UINT32_MAX, + /* categories: non-NULL empty list -> empty base alphabet */ + (const char *const *) &alphabet, 0, + NULL, 0, + (const unsigned char *) alphabet, strlen(alphabet), + NULL, 0, &gen); + check_rc(tc->ctx, rc, "string_generator_text(alphabet)"); + return draw_string(tc, gen, len); +} + +/* ---- assume / note / fail -------------------------------------------- */ + +void +pg_prop_assume(pg_prop_tc *tc, bool condition) +{ + if (!condition) + siglongjmp(tc->jmp, 2); /* 2 == assume/invalid */ +} + +void +pg_prop_note(pg_prop_tc *tc, const char *fmt,...) +{ + va_list ap; + StringInfoData s; + + initStringInfo(&s); + va_start(ap, fmt); + appendStringInfoVA(&s, fmt, ap); + va_end(ap); + hegel_note(tc->ctx, tc->tc, (const unsigned char *) s.data, s.len); + pfree(s.data); +} + +void +pg_prop_fail(pg_prop_tc *tc, const char *fmt,...) +{ + va_list ap; + + resetStringInfo(&tc->failmsg); + va_start(ap, fmt); + appendStringInfoVA(&tc->failmsg, fmt, ap); + va_end(ap); + tc->failed = true; + siglongjmp(tc->jmp, 1); /* 1 == interesting/failure */ +} + +/* ---- fork isolation -------------------------------------------------- */ + +int +pg_prop_run_isolated(pg_prop_isolated_fn body, void *arg, char **detail) +{ + pid_t pid; + int status; + + if (detail) + *detail = NULL; + + fflush(stdout); + fflush(stderr); + pid = fork(); + if (pid < 0) + { + if (detail) + *detail = pg_strdup("fork failed"); + return -1; + } + if (pid == 0) + { + /* child: run the code under test, then _exit so no atexit/flush runs */ + body(arg); + fflush(stdout); + fflush(stderr); + _exit(0); + } + + /* parent: wait and classify */ + while (waitpid(pid, &status, 0) < 0) + { + if (errno != EINTR) + { + if (detail) + *detail = pg_strdup("waitpid failed"); + return -1; + } + } + + if (WIFEXITED(status)) + { + int code = WEXITSTATUS(status); + + if (code == 0) + return 0; + if (detail) + { + char buf[64]; + + snprintf(buf, sizeof(buf), "child exited with status %d", code); + *detail = pg_strdup(buf); + } + return code ? code : 1; + } + if (WIFSIGNALED(status)) + { + int sig = WTERMSIG(status); + + if (detail) + { + char buf[96]; + + snprintf(buf, sizeof(buf), "child killed by signal %d (%s)", + sig, strsignal(sig)); + *detail = pg_strdup(buf); + } + /* encode as a distinct non-zero code */ + return 1000 + sig; + } + if (detail) + *detail = pg_strdup("child stopped or unknown status"); + return -1; +} + +/* ---- driver ---------------------------------------------------------- */ + +/* + * Per-property run state passed from run_property() through the body-runner. + * libhegel's run loop hands us a test case; we build a pg_prop_tc, run the + * body under setjmp, and mark the case. + */ +typedef struct RunState +{ + PropEntry *prop; + StringInfoData last_fail; /* message from the last failing body */ +} RunState; + +static hegel_verbosity_t verbosity = HEGEL_VERBOSITY_NORMAL; + +/* + * Run the body for one test case handle; returns the hegel_status_t to pass + * to hegel_mark_complete. Captures the failure message into rs->last_fail. + */ +static uint32_t +run_one_case(RunState *rs, hegel_context_t *ctx, hegel_test_case_t *handle) +{ + pg_prop_tc t; + int jmpval; + uint32_t status; + + t.ctx = ctx; + t.tc = handle; + t.failed = false; + t.origin = rs->prop->name; + initStringInfo(&t.failmsg); + current_tc = &t; + overrun = false; + + jmpval = sigsetjmp(t.jmp, 1); + if (jmpval == 0) + { + rs->prop->fn(&t, rs->prop->arg); + /* body returned normally: property held for this case */ + status = HEGEL_STATUS_VALID; + } + else if (jmpval == 2) + { + /* assume/invalid */ + status = HEGEL_STATUS_INVALID; + } + else if (overrun) + { + status = HEGEL_STATUS_OVERRUN; + } + else + { + /* failure: keep the message for the report */ + resetStringInfo(&rs->last_fail); + appendBinaryStringInfo(&rs->last_fail, t.failmsg.data, t.failmsg.len); + status = HEGEL_STATUS_INTERESTING; + } + + pfree(t.failmsg.data); + + /* + * Close any SQL connection the body opened and did not close -- a body + * that failed or assumed away longjmp'd over its own pg_prop_disconnect(). + * This must run on every exit path, or a long run exhausts the server's + * connection slots. A no-op for in-process (Tier A) areas. + */ + pg_prop_sql_cleanup(); + + return status; +} + +/* + * Replay a counterexample blob to re-derive the failing message and drawn + * values for the report. Returns the (malloc'd) failure message. + */ +static char * +replay_blob(RunState *rs, hegel_context_t *ctx, const hegel_settings_t *s, + const char *blob) +{ + hegel_test_case_t *handle = NULL; + hegel_result_t rc; + uint32_t status; + char *msg; + + rc = hegel_test_case_from_blob(ctx, s, blob, NULL, NULL, &handle); + if (rc != HEGEL_OK || handle == NULL) + return pg_strdup("(could not replay counterexample)"); + + status = run_one_case(rs, ctx, handle); + hegel_mark_complete(ctx, handle, status, + status == HEGEL_STATUS_INTERESTING ? rs->prop->name : NULL); + hegel_test_case_free(ctx, handle); + + if (status == HEGEL_STATUS_INTERESTING && rs->last_fail.len > 0) + msg = pg_strdup(rs->last_fail.data); + else + msg = pg_strdup("(counterexample did not reproduce)"); + return msg; +} + +/* + * Run one property to completion. Returns true if it held, false if a + * counterexample was found. On failure, prints the message, the minimal + * counterexample's drawn values (via replay), and the replay blob as TAP + * diagnostics. "testno" is the 1-based TAP point number. + */ +static bool +run_property(PropEntry *prop, uint64 iterations, bool have_seed, uint64 seed, + const char *dbpath, int testno, bool tap) +{ + hegel_context_t *ctx = hegel_context_new(); + hegel_settings_t *s = NULL; + hegel_run_t *run = NULL; + hegel_run_result_t *result = NULL; + hegel_run_status_t rstatus; + RunState rs; + bool ok; + + char dbkey[256]; + + rs.prop = prop; + initStringInfo(&rs.last_fail); + + check_rc(ctx, hegel_settings_new(ctx, &s), "settings_new"); + check_rc(ctx, hegel_settings_set_test_cases(ctx, s, iterations), "set_test_cases"); + check_rc(ctx, hegel_settings_set_verbosity(ctx, s, verbosity), "set_verbosity"); + check_rc(ctx, hegel_settings_set_seed(ctx, s, seed, have_seed), "set_seed"); + snprintf(dbkey, sizeof(dbkey), "%s/%s", prop->area, prop->name); + check_rc(ctx, hegel_settings_set_database_key(ctx, s, dbkey), "set_db_key"); + check_rc(ctx, hegel_settings_set_database(ctx, s, dbpath ? dbpath : ""), + "set_database"); + + check_rc(ctx, hegel_run_start(ctx, s, NULL, NULL, &run), "run_start"); + + for (;;) + { + hegel_test_case_t *handle = NULL; + uint32_t status; + + check_rc(ctx, hegel_next_test_case(ctx, run, &handle), "next_test_case"); + if (handle == NULL) + break; + status = run_one_case(&rs, ctx, handle); + hegel_mark_complete(ctx, handle, status, + status == HEGEL_STATUS_INTERESTING ? prop->name : NULL); + hegel_test_case_free(ctx, handle); + } + + check_rc(ctx, hegel_run_result(ctx, run, &result), "run_result"); + check_rc(ctx, hegel_run_result_status(ctx, result, &rstatus), "run_result_status"); + + ok = (rstatus == HEGEL_RUN_STATUS_PASSED); + + if (tap) + printf("%s %d - %s/%s\n", ok ? "ok" : "not ok", testno, + prop->area, prop->name); + else + printf("%s/%s: %s\n", prop->area, prop->name, + ok ? "PASSED" : "FAILED"); + + if (!ok) + { + const char *pfx = tap ? "# " : " "; + + if (rstatus == HEGEL_RUN_STATUS_ERROR) + { + const char *err = NULL; + + hegel_run_result_error(ctx, result, &err); + printf("%srun error: %s\n", pfx, err ? err : "(unknown)"); + } + else + { + size_t count = 0; + + hegel_run_result_failure_count(ctx, result, &count); + for (size_t i = 0; i < count; i++) + { + hegel_failure_t *f = NULL; + const char *blob = NULL; + char *msg; + + hegel_run_result_failure(ctx, result, i, &f); + hegel_failure_reproduction_blob(ctx, f, &blob); + msg = replay_blob(&rs, ctx, s, blob); + + printf("%sfalsified: %s\n", pfx, msg); + if (blob) + printf("%sreplay: PG_PROP_SEED unavailable; rerun with " + "--blob to reproduce: %s\n", pfx, blob); + pg_free(msg); + hegel_failure_free(ctx, f); + } + } + } + fflush(stdout); + + pfree(rs.last_fail.data); + hegel_run_result_free(ctx, result); + hegel_run_free(ctx, run); + hegel_settings_free(ctx, s); + hegel_context_free(ctx); + return ok; +} + +/* ---- CLI ------------------------------------------------------------- */ + +static void +usage(const char *argv0) +{ + printf("Usage: %s AREA [options] [PROPERTY ...]\n\n", argv0); + printf(" AREA which group of properties to run\n"); + printf(" (run with --list to see the areas)\n"); + printf(" --list list AREA/PROPERTY names and exit\n"); + printf(" -n, --iterations N test cases per property (default 100)\n"); + printf(" -s, --seed N fixed RNG seed (default: random)\n"); + printf(" -v, --verbose raise engine verbosity\n"); + printf(" -q, --quiet lower engine verbosity\n"); + printf(" --no-tap plain output instead of TAP\n"); + printf(" -h, --help this help\n\n"); + printf("With no PROPERTY, every property in AREA runs. Examples:\n"); + printf(" %s jsonapi --list\n", argv0); + printf(" %s path canonicalize_idempotent -n 100000 -s 42\n\n", argv0); + printf("Environment overrides: PG_PROP_ITERATIONS, PG_PROP_SEED, PG_PROP_TESTS.\n"); +} + +/* Does name match any of the selectors? */ +static bool +selected(const char *name, char **sel, int nsel) +{ + if (nsel == 0) + return true; + for (int i = 0; i < nsel; i++) + if (strcmp(name, sel[i]) == 0) + return true; + return false; +} + +int +main(int argc, char **argv) +{ + static struct option long_options[] = { + {"list", no_argument, NULL, 'L'}, + {"iterations", required_argument, NULL, 'n'}, + {"seed", required_argument, NULL, 's'}, + {"verbose", no_argument, NULL, 'v'}, + {"quiet", no_argument, NULL, 'q'}, + {"no-tap", no_argument, NULL, 'T'}, + {"help", no_argument, NULL, 'h'}, + {NULL, 0, NULL, 0} + }; + uint64 iterations = 100; + bool have_seed = false; + uint64 seed = 0; + bool do_list = false; + bool tap = true; + const char *env; + const char *dbpath; + const char *area = NULL; + int c; + int npass = 0; + int nrun = 0; + int nsel; + char **sel; + + pg_logging_init(argv[0]); + + /* environment defaults (command line wins) */ + if ((env = getenv("PG_PROP_ITERATIONS")) != NULL && env[0]) + iterations = strtou64(env, NULL, 10); + if ((env = getenv("PG_PROP_SEED")) != NULL && env[0]) + { + seed = strtou64(env, NULL, 10); + have_seed = true; + } + + while ((c = getopt_long(argc, argv, "n:s:vqhL", long_options, NULL)) != -1) + { + switch (c) + { + case 'L': + do_list = true; + break; + case 'n': + iterations = strtou64(optarg, NULL, 10); + break; + case 's': + seed = strtou64(optarg, NULL, 10); + have_seed = true; + break; + case 'v': + verbosity = HEGEL_VERBOSITY_VERBOSE; + break; + case 'q': + verbosity = HEGEL_VERBOSITY_QUIET; + break; + case 'T': + tap = false; + break; + case 'h': + usage(argv[0]); + return 0; + default: + usage(argv[0]); + return 2; + } + } + + if (do_list) + { + for (int i = 0; i < nprops; i++) + printf("%s/%s\n", props[i].area, props[i].name); + return 0; + } + + /* + * First positional argument selects the area (the target group). Running + * every area at once is intentionally not supported from one invocation: + * the TAP driver runs one area per binary subcommand, and a developer + * debugging a failure wants one area in front of them. + */ + if (optind >= argc) + { + fprintf(stderr, "pg_prop: missing AREA argument " + "(try --list, or -h for help)\n"); + return 2; + } + area = argv[optind++]; + + /* remaining positionals, else PG_PROP_TESTS, select properties by name */ + sel = &argv[optind]; + nsel = argc - optind; + if (nsel == 0 && (env = getenv("PG_PROP_TESTS")) != NULL && env[0]) + { + /* split PG_PROP_TESTS on whitespace into a fresh argv-like array */ + static char *envsel[MAX_PROPS]; + char *dup = pg_strdup(env); + char *tok = strtok(dup, " \t\n"); + + nsel = 0; + while (tok && nsel < MAX_PROPS) + { + envsel[nsel++] = tok; + tok = strtok(NULL, " \t\n"); + } + sel = envsel; + } + + /* + * Pick and announce a seed when none was given, so a failure seen on a + * buildfarm animal can be reproduced exactly. derandomize is left off: + * we want fresh randomness unless the caller pins a seed. + */ + if (!have_seed) + { + srandom((unsigned int) (getpid() ^ (unsigned int) time(NULL))); + seed = ((uint64) random() << 32) ^ (uint64) random(); + have_seed = true; + if (tap) + printf("# seed: %llu (rerun with -s %llu)\n", + (unsigned long long) seed, (unsigned long long) seed); + else + printf("seed: %llu\n", (unsigned long long) seed); + } + + /* + * The example database persists counterexamples between runs so a known + * failure replays first. Keep it out of the source tree: honour + * TESTDATADIR (set by the TAP wrapper) when present, else disable it. + */ + dbpath = getenv("PG_PROP_DB"); + if (dbpath == NULL) + { + const char *td = getenv("TESTDATADIR"); + + dbpath = td ? NULL : ""; /* NULL under TESTDATADIR -> default .hegel + * there; handled below */ + if (td) + { + static char dbbuf[MAXPGPATH]; + + snprintf(dbbuf, sizeof(dbbuf), "%s/hegel-db", td); + dbpath = dbbuf; + } + } + + /* count the selected properties for the TAP plan */ + if (tap) + { + int plan = 0; + + for (int i = 0; i < nprops; i++) + if (strcmp(props[i].area, area) == 0 && + selected(props[i].name, sel, nsel)) + plan++; + printf("1..%d\n", plan); + } + + for (int i = 0; i < nprops; i++) + { + if (strcmp(props[i].area, area) != 0) + continue; + if (!selected(props[i].name, sel, nsel)) + continue; + nrun++; + if (run_property(&props[i], iterations, have_seed, seed, dbpath, + nrun, tap)) + npass++; + } + + if (nrun == 0) + { + fprintf(stderr, "pg_prop: no matching properties in area \"%s\"\n", area); + return 2; + } + if (!tap) + printf("%d/%d properties passed\n", npass, nrun); + + return (npass == nrun) ? 0 : 1; +} diff --git a/src/test/prop/pg_prop.h b/src/test/prop/pg_prop.h new file mode 100644 index 0000000000000..91669e023622c --- /dev/null +++ b/src/test/prop/pg_prop.h @@ -0,0 +1,238 @@ +/*------------------------------------------------------------------------- + * + * pg_prop.h + * Property-based testing harness for PostgreSQL, built on libhegel. + * + * This is the single entry point for every property-based test binary under + * src/test/prop. It wraps the libhegel C ABI (hegel.h) in a small, plain-C + * API so individual test files never touch the engine handles directly, and + * so a test file reads like an ordinary C unit test. + * + * A property is a function: + * + * static void prop_my_thing(pg_prop_tc *tc, void *arg) + * { + * int64 n = pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + * if (!my_invariant(n)) + * pg_prop_fail(tc, "invariant violated for %lld", + * (long long) n); + * } + * + * registered with PG_PROP() and run from the generated main(). The engine + * generates many inputs, and on failure shrinks to a minimal counterexample + * and prints a base64 blob that replays it exactly. + * + * Why a C API and not hegel-cpp: PostgreSQL is a C project maintained by C + * programmers. libhegel ships a stable C ABI; we bind to it directly. The + * only C++-ism anywhere in this tree is that the engine's own shared library + * is built in Rust; nothing we write is C++. + * + * The isolate() facility (pg_prop_run_isolated) forks a child to run code + * that may crash, abort() on a failed Assert, or exit() on a fatal error -- + * all common in frontend PostgreSQL code -- and turns any of those into an + * ordinary property failure the engine can shrink. Without it, a crash in + * the code under test would take the whole test process down and lose the + * counterexample. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/pg_prop.h + * + *------------------------------------------------------------------------- + */ +#ifndef PG_PROP_H +#define PG_PROP_H + +#include "postgres_fe.h" + +/* + * Opaque per-test-case handle. One is handed to each property invocation; + * it must not be stored past the end of that invocation. + */ +typedef struct pg_prop_tc pg_prop_tc; + +/* A property function: draws inputs from tc, checks, calls pg_prop_fail. */ +typedef void (*pg_prop_fn) (pg_prop_tc *tc, void *arg); + +/* + * Register a property. Use the PG_PROP() macro rather than calling this + * directly. "area" groups properties that share a target (and a binary + * subcommand); "name" selects one on the command line and names it in the + * example database as "area/name"; "fn" is the property body. + */ +extern void pg_prop_register(const char *area, const char *name, + pg_prop_fn fn, void *arg); + +/* + * Each prop_.c defines PG_PROP_AREA to its area name before including + * this header: + * + * #define PG_PROP_AREA "path" + * #include "pg_prop.h" + * + * PG_PROP(ident) then defines and registers a property body under that area. + * Run it as: pg_prop [options] [name ...] + * + * The harness translation unit (pg_prop.c) includes this header without + * defining PG_PROP_AREA, which is fine -- it never uses PG_PROP(). Only a + * use of PG_PROP() without an area is an error, enforced below. + */ +#ifndef PG_PROP_AREA +#define PG_PROP_AREA pg_prop_area_not_defined +#endif + +#define PG_PROP(ident) \ + static void ident(pg_prop_tc *tc, void *arg); \ + static void pg_prop_reg_##ident(void) pg_attribute_constructor; \ + static void pg_prop_reg_##ident(void) \ + { \ + pg_prop_register(PG_PROP_AREA, CppAsString(ident), ident, NULL); \ + } \ + static void ident(pg_prop_tc *tc, void *arg) + +/* + * pg_attribute_constructor runs a function before main(). c.h does not define + * one, so define it here (GCC/Clang only, which is all the platforms libhegel + * ships a prebuilt engine for). + */ +#ifndef pg_attribute_constructor +#define pg_attribute_constructor __attribute__((constructor)) +#endif + +/* + * Draw functions. Each pulls one value from the engine's choice stream for + * this test case. Bounds are inclusive. The engine deliberately favours + * boundary values (min, max, 0, -1, 1), so pass the widest bounds the code + * under test is supposed to accept -- narrowing them hides exactly the bugs + * this tool exists to find. + */ +extern int64 pg_prop_draw_int(pg_prop_tc *tc, int64 min, int64 max); +extern double pg_prop_draw_float(pg_prop_tc *tc, double min, double max); +extern bool pg_prop_draw_bool(pg_prop_tc *tc); + +/* + * Draw a byte buffer of length in [min_size, max_size]. The result is + * malloc'd and owned by the caller; free it. *len receives the length. The + * buffer is NUL-terminated for convenience (one extra byte past *len) but may + * contain interior NULs, so always use *len for the real content. + */ +extern unsigned char *pg_prop_draw_bytes(pg_prop_tc *tc, size_t min_size, + size_t max_size, size_t *len); + +/* + * Draw a UTF-8 string of length in [min_size, max_size] code points, drawn + * from the full assigned Unicode range (not ASCII-only). Caller frees. *len + * receives the byte length. NUL-terminated; may contain interior NULs. + */ +extern char *pg_prop_draw_utf8(pg_prop_tc *tc, size_t min_size, + size_t max_size, size_t *len); + +/* + * Draw a string restricted to the given alphabet (a NUL-terminated C string + * of allowed ASCII characters). Useful when the code under test is only + * meaningful on a restricted charset (identifiers, numeric text, ...). + * Caller frees. + */ +extern char *pg_prop_draw_from(pg_prop_tc *tc, const char *alphabet, + size_t min_size, size_t max_size, size_t *len); + +/* + * Reject the current test case: its generated inputs do not satisfy a + * precondition, so it should be discarded rather than counted. Use sparingly + * -- a high rejection rate trips a health check. Prefer constructing valid + * inputs directly over drawing-then-rejecting. + */ +extern void pg_prop_assume(pg_prop_tc *tc, bool condition); + +/* + * Attach a diagnostic note to the current case. It is printed only when this + * case is the final shrunk counterexample, so use it to record context the + * drawn values alone don't show (e.g. an intermediate the property computed). + */ +extern void pg_prop_note(pg_prop_tc *tc, const char *fmt,...) pg_attribute_printf(2, 3); + +/* + * Fail the current test case. This is how a property reports that its + * invariant did not hold. The message is the human-readable description of + * the failure; it is printed with the counterexample. Does not return. + */ +extern pg_noreturn void pg_prop_fail(pg_prop_tc *tc, const char *fmt,...) pg_attribute_printf(2, 3); + +/* + * Run "body" in a forked child. Returns 0 if the child exited 0, otherwise a + * non-zero code describing how it died. *detail (if non-NULL) receives a + * malloc'd human-readable description ("exited N", "signal N (SIGSEGV)", ...). + * + * Use this to wrap any call into code that may crash, abort() on a failed + * assertion, or exit() -- so the failure becomes a value you can test and + * pg_prop_fail on, with the drawn inputs still in hand to shrink. Draw your + * inputs in the parent (before calling this) so the engine sees them; the + * child only runs the code under test. + */ +typedef void (*pg_prop_isolated_fn) (void *arg); +extern int pg_prop_run_isolated(pg_prop_isolated_fn body, void *arg, + char **detail); + +/* + * SQL oracle support (Tier B). + * + * Backend code -- the planner, executor, type I/O -- cannot be linked into a + * frontend binary, so properties that exercise it instead drive SQL against a + * live cluster and compare the answer to an oracle (a reference query, or the + * same query with an optimisation disabled). The TAP driver t/002_sql_props.pl + * starts one node and exports PGHOST/PGPORT; these helpers connect to it. + * + * A SQL area opens one connection in its property body (connections are cheap + * against a local socket and keeping them per-case keeps state from leaking + * between cases). The handle is an opaque PGconn-wrapper so pg_prop.h need + * not include libpq-fe.h. + */ +typedef struct pg_prop_conn pg_prop_conn; + +/* + * Connect to the test cluster named by PGHOST/PGPORT/PGDATABASE in the + * environment. Returns NULL and sets *err (malloc'd) on failure; the caller + * typically pg_prop_fail()s on NULL. Close with pg_prop_disconnect. + */ +extern pg_prop_conn *pg_prop_connect(char **err); +extern void pg_prop_disconnect(pg_prop_conn *conn); + +/* + * Run "sql" and return its single-row, single-column result as a malloc'd + * string (caller frees), or NULL for SQL NULL. On a SQL error, *errmsg (if + * non-NULL) receives a malloc'd message and the function returns NULL with + * *failed set true; callers distinguish a genuine NULL result from an error + * via *failed. Use pg_prop_sql_scalar for the common one-value oracle. + */ +extern char *pg_prop_sql_scalar(pg_prop_conn *conn, const char *sql, + bool *failed, char **errmsg); + +/* + * Run "sql" expecting a result set, and return the whole thing as one newline- + * joined, row-major string (every value rendered, NULLs as "\\N"), suitable for + * comparing two queries that should return identical rows in identical order. + * Caller frees. *failed / *errmsg as above. The caller is responsible for + * ordering (add ORDER BY) when it wants order-insensitive comparison use a + * sorted projection. + */ +extern char *pg_prop_sql_rows(pg_prop_conn *conn, const char *sql, + bool *failed, char **errmsg); + +/* + * Run "sql" for side effects (DDL/DML), returning true on success. On error + * returns false and sets *errmsg (malloc'd). + */ +extern bool pg_prop_sql_exec(pg_prop_conn *conn, const char *sql, char **errmsg); + +/* + * Close any connections still open at the end of a test case. A property that + * fails or is assumed away unwinds by longjmp, jumping over its own + * pg_prop_disconnect(); the driver calls this after every case so a leaked + * connection cannot accumulate across the thousands of cases in a run (which + * would otherwise exhaust the server's connection slots). Defined in + * pg_prop_sql.c; a no-op if no SQL area ever connected. + */ +extern void pg_prop_sql_cleanup(void); + +#endif /* PG_PROP_H */ diff --git a/src/test/prop/pg_prop_sql.c b/src/test/prop/pg_prop_sql.c new file mode 100644 index 0000000000000..e259b490405b0 --- /dev/null +++ b/src/test/prop/pg_prop_sql.c @@ -0,0 +1,195 @@ +/*------------------------------------------------------------------------- + * + * pg_prop_sql.c + * libpq oracle helpers for Tier-B property tests. + * + * Backend code (planner, executor, type I/O) cannot be linked into a frontend + * binary, so properties that exercise it drive SQL against a live cluster and + * compare the result to an oracle. These helpers wrap just enough of libpq + * for that: connect, run a scalar/rowset/DDL query, render the result as a + * string the property can compare. See pg_prop.h for the contract. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/pg_prop_sql.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#include "lib/stringinfo.h" +#include "libpq-fe.h" +#include "pg_prop.h" + +struct pg_prop_conn +{ + PGconn *conn; + struct pg_prop_conn *next; /* link in the live-connection registry */ +}; + +/* + * Live connections, so the driver can close any a failing/assumed-away case + * leaked by longjmp'ing over its pg_prop_disconnect(). One run is single- + * threaded, so a plain list needs no locking. + */ +static pg_prop_conn *live_conns = NULL; + +pg_prop_conn * +pg_prop_connect(char **err) +{ + pg_prop_conn *c; + PGconn *conn; + + /* + * Empty connection string: everything comes from the environment + * (PGHOST, PGPORT, PGDATABASE), which the TAP driver points at the test + * node. dbname defaults to "postgres" if PGDATABASE is unset. + */ + const char *keywords[] = {"dbname", NULL}; + const char *values[] = {getenv("PGDATABASE") ? getenv("PGDATABASE") : "postgres", NULL}; + + conn = PQconnectdbParams(keywords, values, true); + if (PQstatus(conn) != CONNECTION_OK) + { + if (err) + *err = pg_strdup(PQerrorMessage(conn)); + PQfinish(conn); + return NULL; + } + c = pg_malloc(sizeof(pg_prop_conn)); + c->conn = conn; + c->next = live_conns; + live_conns = c; + + /* Quieten DROP ... IF EXISTS notices and similar chatter. */ + PQclear(PQexec(conn, "SET client_min_messages = warning")); + + return c; +} + +/* Unlink c from the live list, if present. */ +static void +unlink_conn(pg_prop_conn *c) +{ + pg_prop_conn **pp = &live_conns; + + while (*pp != NULL) + { + if (*pp == c) + { + *pp = c->next; + return; + } + pp = &(*pp)->next; + } +} + +void +pg_prop_disconnect(pg_prop_conn *conn) +{ + if (conn == NULL) + return; + unlink_conn(conn); + PQfinish(conn->conn); + pg_free(conn); +} + +void +pg_prop_sql_cleanup(void) +{ + while (live_conns != NULL) + { + pg_prop_conn *c = live_conns; + + live_conns = c->next; + PQfinish(c->conn); + pg_free(c); + } +} + +char * +pg_prop_sql_scalar(pg_prop_conn *conn, const char *sql, bool *failed, char **errmsg) +{ + PGresult *res = PQexec(conn->conn, sql); + ExecStatusType st = PQresultStatus(res); + char *out; + + if (failed) + *failed = false; + if (st != PGRES_TUPLES_OK) + { + if (failed) + *failed = true; + if (errmsg) + *errmsg = pg_strdup(PQresultErrorMessage(res)); + PQclear(res); + return NULL; + } + if (PQntuples(res) == 0 || PQnfields(res) == 0 || PQgetisnull(res, 0, 0)) + { + PQclear(res); + return NULL; /* genuine NULL / empty, *failed stays false */ + } + out = pg_strdup(PQgetvalue(res, 0, 0)); + PQclear(res); + return out; +} + +char * +pg_prop_sql_rows(pg_prop_conn *conn, const char *sql, bool *failed, char **errmsg) +{ + PGresult *res = PQexec(conn->conn, sql); + ExecStatusType st = PQresultStatus(res); + StringInfoData s; + int nrows, + ncols; + + if (failed) + *failed = false; + if (st != PGRES_TUPLES_OK) + { + if (failed) + *failed = true; + if (errmsg) + *errmsg = pg_strdup(PQresultErrorMessage(res)); + PQclear(res); + return NULL; + } + + initStringInfo(&s); + nrows = PQntuples(res); + ncols = PQnfields(res); + for (int r = 0; r < nrows; r++) + { + for (int col = 0; col < ncols; col++) + { + if (col > 0) + appendStringInfoChar(&s, '\t'); + if (PQgetisnull(res, r, col)) + appendStringInfoString(&s, "\\N"); + else + appendStringInfoString(&s, PQgetvalue(res, r, col)); + } + appendStringInfoChar(&s, '\n'); + } + PQclear(res); + return s.data; /* caller frees */ +} + +bool +pg_prop_sql_exec(pg_prop_conn *conn, const char *sql, char **errmsg) +{ + PGresult *res = PQexec(conn->conn, sql); + ExecStatusType st = PQresultStatus(res); + + if (st != PGRES_COMMAND_OK && st != PGRES_TUPLES_OK) + { + if (errmsg) + *errmsg = pg_strdup(PQresultErrorMessage(res)); + PQclear(res); + return false; + } + PQclear(res); + return true; +} diff --git a/src/test/prop/t/001_prop.pl b/src/test/prop/t/001_prop.pl new file mode 100644 index 0000000000000..4667bf8c66123 --- /dev/null +++ b/src/test/prop/t/001_prop.pl @@ -0,0 +1,93 @@ +# Copyright (c) 2026, PostgreSQL Global Development Group +# +# TAP driver for the property-based tests. Runs each area of pg_prop and +# reports one test point per area; pg_prop's own per-property output is echoed +# as TAP diagnostics so a failure shows the counterexample and replay blob. +# +# Controls (all optional) come from the environment and are forwarded, so one +# `meson test` run is reproducible and a buildfarm failure can be replayed: +# PG_PROP_ITERATIONS test cases per property (default: pg_prop's own 100) +# PG_PROP_SEED fixed RNG seed (default: pg_prop picks and prints one) +# PG_PROP_TESTS whitespace-separated property names to run +# +# To debug a single property, run pg_prop directly, e.g. +# pg_prop path canonicalize_idempotent --iterations 100000 --seed 42 +# See README and DESIGN. + +use strict; +use warnings FATAL => 'all'; + +use PostgreSQL::Test::Utils; +use Test::More; + +# TAP driver for the in-process (Tier A) property tests. These exercise +# src/common and src/port code linked directly into pg_prop, so they need no +# cluster. The SQL-oracle (Tier B) areas run from t/002_sql_props.pl instead. +# +# Keep in step with the Tier-A entries in prop_area_sources (meson.build). +my @areas = qw(); + +# No Tier-A areas wired in yet (e.g. the initial harness-only commit): nothing +# to run, which is a pass, not a failure. +if (!@areas) +{ + plan skip_all => 'no in-process property areas registered'; +} + +# Shared arguments from the environment. +my @common_args = ('--no-tap'); +push @common_args, ('--iterations', $ENV{PG_PROP_ITERATIONS}) + if defined $ENV{PG_PROP_ITERATIONS} && $ENV{PG_PROP_ITERATIONS} ne ''; +push @common_args, ('--seed', $ENV{PG_PROP_SEED}) + if defined $ENV{PG_PROP_SEED} && $ENV{PG_PROP_SEED} ne ''; + +# PG_PROP_TESTS selects individual properties by name (unqualified). +my @select; +@select = split /\s+/, $ENV{PG_PROP_TESTS} + if defined $ENV{PG_PROP_TESTS} && $ENV{PG_PROP_TESTS} ne ''; + +# All areas are known to one binary; list once. +my $list = ''; +run_log([ 'pg_prop', '--list' ], '>', \$list); +my %known_in_area; +for my $qualified (split /\n/, $list) +{ + my ($a, $n) = split m{/}, $qualified, 2; + push @{ $known_in_area{$a} }, $n if defined $n; +} + +foreach my $area (@areas) +{ + my @args = @common_args; + if (@select) + { + my %have = map { $_ => 1 } @{ $known_in_area{$area} // [] }; + my @mine = grep { $have{$_} } @select; + # A selection that this area has none of: skip it with a passing TODO + # so the plan still accounts for every area. + SKIP: + { + skip "no selected properties in area $area", 1 unless @mine; + run_area($area, [ @args, @mine ]); + } + next; + } + run_area($area, \@args); +} + +done_testing(); + +# Run one area of pg_prop; emit one test point plus diagnostics. +sub run_area +{ + my ($area, $args) = @_; + my ($stdout, $stderr) = ('', ''); + my $ok = run_log([ 'pg_prop', $area, @$args ], + '>', \$stdout, '2>', \$stderr); + + # Echo pg_prop's plain output as TAP diagnostics (note() prefixes "# "). + note($stdout) if $stdout ne ''; + note($stderr) if $stderr ne ''; + + ok($ok, "pg_prop $area: all properties held"); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl new file mode 100644 index 0000000000000..6ff6a0740baef --- /dev/null +++ b/src/test/prop/t/002_sql_props.pl @@ -0,0 +1,97 @@ +# Copyright (c) 2026, PostgreSQL Global Development Group +# +# TAP driver for the SQL-oracle (Tier B) property tests. These exercise +# backend code -- the planner, executor, and type machinery -- which cannot be +# linked into a frontend binary, so each property drives SQL against a live +# node and compares the answer to an oracle (a reference query, or the same +# query with an optimisation disabled). This script starts one node, exports +# its PGHOST/PGPORT, and runs each Tier-B area of pg_prop against it. +# +# Controls (optional) are read from the environment and forwarded, so a run is +# reproducible and a buildfarm failure can be replayed: +# PG_PROP_ITERATIONS test cases per property (default: pg_prop's own 100) +# PG_PROP_SEED fixed RNG seed (default: pg_prop picks and prints one) +# PG_PROP_TESTS whitespace-separated property names to run +# +# To debug one failure interactively, start a node, point PGHOST/PGPORT at it, +# then run e.g. pg_prop partprune pruning_preserves_rows -n 100000 -s 42 +# See README and DESIGN. + +use strict; +use warnings FATAL => 'all'; + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More; + +# Keep in step with the Tier-B entries in prop_area_sources (meson.build). +my @areas = qw(); + +# No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing +# to run, which is a pass. Skip before starting a node, so we don't pay for a +# cluster we won't use. +if (!@areas) +{ + plan skip_all => 'no SQL-oracle property areas registered'; +} + +my $node = PostgreSQL::Test::Cluster->new('prop'); +$node->init; +$node->start; + +# Point libpq (used by pg_prop's Tier-B helpers) at this node. +$ENV{PGHOST} = $node->host; +$ENV{PGPORT} = $node->port; +$ENV{PGDATABASE} = 'postgres'; + +my @common_args = ('--no-tap'); +push @common_args, ('--iterations', $ENV{PG_PROP_ITERATIONS}) + if defined $ENV{PG_PROP_ITERATIONS} && $ENV{PG_PROP_ITERATIONS} ne ''; +push @common_args, ('--seed', $ENV{PG_PROP_SEED}) + if defined $ENV{PG_PROP_SEED} && $ENV{PG_PROP_SEED} ne ''; + +my @select; +@select = split /\s+/, $ENV{PG_PROP_TESTS} + if defined $ENV{PG_PROP_TESTS} && $ENV{PG_PROP_TESTS} ne ''; + +my $list = ''; +run_log([ 'pg_prop', '--list' ], '>', \$list); +my %known_in_area; +for my $qualified (split /\n/, $list) +{ + my ($a, $n) = split m{/}, $qualified, 2; + push @{ $known_in_area{$a} }, $n if defined $n; +} + +foreach my $area (@areas) +{ + my @args = @common_args; + if (@select) + { + my %have = map { $_ => 1 } @{ $known_in_area{$area} // [] }; + my @mine = grep { $have{$_} } @select; + SKIP: + { + skip "no selected properties in area $area", 1 unless @mine; + run_area($area, [ @args, @mine ]); + } + next; + } + run_area($area, \@args); +} + +$node->stop; +done_testing(); + +sub run_area +{ + my ($area, $args) = @_; + my ($stdout, $stderr) = ('', ''); + my $ok = run_log([ 'pg_prop', $area, @$args ], + '>', \$stdout, '2>', \$stderr); + + note($stdout) if $stdout ne ''; + note($stderr) if $stderr ne ''; + + ok($ok, "pg_prop $area: all properties held"); +} diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index a7a8dbdd23f79..08bc6ee247320 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2428,6 +2428,7 @@ ProjectSetState ProjectionInfo ProjectionPath PromptInterruptContext +PropEntry ProtocolVersion PrsStorage PruneFreezeParams @@ -4039,6 +4040,10 @@ pg_md5_ctx pg_on_exit_callback pg_plan_advice_advisor_hook pg_prng_state +pg_prop_conn +pg_prop_fn +pg_prop_isolated_fn +pg_prop_tc pg_re_flags pg_regex_t pg_regmatch_t From 0d9a580b7caeee69be1bce980ff429f36e957122 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:40 -0400 Subject: [PATCH 004/338] prop: add path property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_path.c | 125 ++++++++++++++++++++++++++++++++++++ src/test/prop/t/001_prop.pl | 2 +- 4 files changed, 129 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_path.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 656173886bb11..000d88b784e29 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -19,7 +19,8 @@ PROGRAM = pg_prop OBJS = \ $(WIN32RES) \ pg_prop.o \ - pg_prop_sql.o + pg_prop_sql.o \ + prop_path.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index d8d64fbd1ea07..9cb4dd4d154dd 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -17,6 +17,7 @@ prop_area_sources = files( # Tier A: in-process # Tier B: SQL oracle (needs a running node) + 'prop_path.c', ) diff --git a/src/test/prop/prop_path.c b/src/test/prop/prop_path.c new file mode 100644 index 0000000000000..7b37ab2e0735e --- /dev/null +++ b/src/test/prop/prop_path.c @@ -0,0 +1,125 @@ +/*------------------------------------------------------------------------- + * + * prop_path.c + * Property-based tests for src/port/path.c path manipulation. + * + * Target #10 in the design. canonicalize_path() and friends are the trust + * boundary for archive extraction (pg_basebackup, pg_rewind): a recent fix + * (a1063eecedf, "Prevent path traversal in pg_basebackup and pg_rewind") and + * the SJIS de-backslashing fix (6cddecdfb00) both live here. Security- + * relevant, pure, and cheap to exercise. + * + * Properties: + * - canonicalize_path is idempotent: canon(canon(p)) == canon(p). + * - canonicalize never lengthens a path (the code relies on this to rewrite + * in place; prove it externally). + * - path_is_prefix_of_path(a,a) holds for any path (reflexive). + * - a path judged safe for extraction, once canonicalized, truly contains no + * ".." component that could escape the extraction root. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_path.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "path" +#include "pg_prop.h" +#include "port.h" + +/* + * Draw a path-shaped string: slashes, dots, and a few letters, so most draws + * are actual paths with components, "." and ".." in them rather than random + * noise. Keep it bounded well under MAXPGPATH. + */ +static char * +draw_path(pg_prop_tc *tc, size_t *len) +{ + return pg_prop_draw_from(tc, "/.ab", 0, 200, len); +} + +PG_PROP(canonicalize_idempotent) +{ + size_t len; + char *p = draw_path(tc, &len); + char a[MAXPGPATH]; + char b[MAXPGPATH]; + + /* canonicalize_path works on NUL-terminated input within MAXPGPATH */ + strlcpy(a, p, sizeof(a)); + canonicalize_path(a); + + strlcpy(b, a, sizeof(b)); + canonicalize_path(b); + + if (strcmp(a, b) != 0) + pg_prop_fail(tc, "not idempotent: canon(%s)=\"%s\" but canon again=\"%s\"", + p, a, b); + pg_free(p); +} + +PG_PROP(canonicalize_never_grows) +{ + size_t len; + char *p = draw_path(tc, &len); + char a[MAXPGPATH]; + size_t before, + after; + + strlcpy(a, p, sizeof(a)); + before = strlen(a); + canonicalize_path(a); + after = strlen(a); + + /* + * canonicalize_path rewrites in place and documents that it never makes + * the path longer. If that is ever violated it is a buffer-overflow + * waiting to happen. + */ + if (after > before) + pg_prop_fail(tc, "canon grew \"%s\" (%zu) to \"%s\" (%zu)", + p, before, a, after); + pg_free(p); +} + +PG_PROP(prefix_reflexive) +{ + size_t len; + char *p = draw_path(tc, &len); + + /* every path is a prefix of itself */ + if (!path_is_prefix_of_path(p, p)) + pg_prop_fail(tc, "path is not a prefix of itself: \"%s\"", p); + pg_free(p); +} + +PG_PROP(safe_extraction_has_no_parent_ref) +{ + size_t len; + char *p = draw_path(tc, &len); + char buf[MAXPGPATH]; + + if (path_is_safe_for_extraction(p)) + { + /* + * If the path is declared safe to extract, then after canonicalizing + * it there must be no leading ".." component -- otherwise extraction + * could write outside the target directory. This re-derives the + * check independently of path_is_relative_and_below_cwd(). + */ + strlcpy(buf, p, sizeof(buf)); + canonicalize_path(buf); + + if (is_absolute_path(buf)) + pg_prop_fail(tc, "safe-for-extraction but absolute: \"%s\" -> \"%s\"", + p, buf); + if (buf[0] == '.' && buf[1] == '.' && + (buf[2] == '\0' || buf[2] == '/')) + pg_prop_fail(tc, "safe-for-extraction but escapes: \"%s\" -> \"%s\"", + p, buf); + } + pg_free(p); +} diff --git a/src/test/prop/t/001_prop.pl b/src/test/prop/t/001_prop.pl index 4667bf8c66123..5536d759e8e48 100644 --- a/src/test/prop/t/001_prop.pl +++ b/src/test/prop/t/001_prop.pl @@ -25,7 +25,7 @@ # cluster. The SQL-oracle (Tier B) areas run from t/002_sql_props.pl instead. # # Keep in step with the Tier-A entries in prop_area_sources (meson.build). -my @areas = qw(); +my @areas = qw(path); # No Tier-A areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass, not a failure. From a56f94ceeb91a90644ac89a07b82ce4b72f8042d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:40 -0400 Subject: [PATCH 005/338] prop: add jsonapi property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonapi.c | 229 +++++++++++++++++++++++++++++++++++ src/test/prop/t/001_prop.pl | 2 +- 4 files changed, 233 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonapi.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 000d88b784e29..e310bbd7bf337 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -20,7 +20,8 @@ OBJS = \ $(WIN32RES) \ pg_prop.o \ pg_prop_sql.o \ - prop_path.o + prop_path.o \ + prop_jsonapi.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 9cb4dd4d154dd..da04fe8d49f14 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -18,6 +18,7 @@ prop_area_sources = files( # Tier A: in-process # Tier B: SQL oracle (needs a running node) 'prop_path.c', + 'prop_jsonapi.c', ) diff --git a/src/test/prop/prop_jsonapi.c b/src/test/prop/prop_jsonapi.c new file mode 100644 index 0000000000000..1a669be628fc8 --- /dev/null +++ b/src/test/prop/prop_jsonapi.c @@ -0,0 +1,229 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonapi.c + * Property-based tests for src/common/jsonapi.c incremental JSON parser. + * + * The JSON parser has two front ends over one grammar: a one-shot recursive + * descent parser (pg_parse_json) and an incremental, chunk-at-a-time parser + * (pg_parse_json_incremental) used by COPY, the JSON datum input path, and + * libpq. They are supposed to accept and reject exactly the same inputs. + * The incremental lexer carries a "partial token" across chunk boundaries, so + * a token split across a chunk edge (a number cut as "4" then "-", a string + * cut mid-escape, "tr" then "ue") is where cross-chunk reassembly bugs live. + * Drawing a json-ish restricted alphabet and tiny chunk sizes aims straight + * at those boundaries. + * + * Properties: + * - chunking_agrees_with_oneshot: one-shot and incremental parses of the + * same bytes reach the same final outcome (both succeed, or both fail). + * - oneshot_no_crash: one-shot parsing of arbitrary bytes always returns a + * JsonParseErrorType and never crashes. + * - incremental_chunk_size_invariant: two incremental parses of the same + * bytes with different chunk sizes reach the same final outcome. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonapi.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonapi" +#include "pg_prop.h" + +#include "common/jsonapi.h" +#include "lib/stringinfo.h" +#include "mb/pg_wchar.h" + +/* + * A json-ish alphabet: structural punctuation, digits, number syntax, and the + * letters of true/false/null, plus a little whitespace. Most draws from this + * set are structurally interesting (partial numbers, half-finished keywords, + * unbalanced brackets) rather than random noise that fails at the first byte. + */ +#define JSON_ALPHABET "{}[]\":,0123456789-+.eEtruefalsn \t" + +/* + * Format a possibly-non-printable, possibly-NUL-containing buffer into an + * escaped, NUL-terminated C string for diagnostics. Non-printable bytes + * become \xNN. The result is malloc'd; caller frees. + */ +static char * +escape_for_note(const unsigned char *buf, size_t len) +{ + char *out; + size_t i; + size_t o; + + /* worst case every byte expands to "\xNN" (4 chars), plus terminator */ + out = pg_malloc(len * 4 + 1); + o = 0; + for (i = 0; i < len; i++) + { + unsigned char c = buf[i]; + + if (c >= 0x20 && c < 0x7f && c != '\\') + out[o++] = (char) c; + else + { + snprintf(out + o, 5, "\\x%02x", c); + o += 4; + } + } + out[o] = '\0'; + return out; +} + +/* + * Parse the whole buffer at once with the recursive-descent parser. Uses a + * heap-allocated lex (NULL first argument) so freeJsonLexContext() frees the + * struct too. Always drives the parser by explicit length, never strlen, so + * interior NUL bytes in the input are passed through faithfully. + */ +static JsonParseErrorType +parse_oneshot(const char *buf, size_t len) +{ + JsonLexContext *lex; + JsonParseErrorType result; + + lex = makeJsonLexContextCstringLen(NULL, buf, len, PG_UTF8, false); + result = pg_parse_json(lex, &nullSemAction); + freeJsonLexContext(lex); + + return result; +} + +/* + * Parse the whole buffer with the incremental parser, feeding it in slices of + * at most chunk_size bytes. is_last is set only on the final slice (including + * the empty-input case, where a single is_last slice of length zero is fed). + * + * A non-final chunk must return JSON_INCOMPLETE; anything else from a non-final + * chunk is already a definitive result (success is impossible mid-stream, and + * any error is final), so we return it immediately. The final chunk yields the + * authoritative outcome. We feed successive slices of the original buffer + * directly, mirroring test_json_parser_incremental.c but without the staging + * StringInfo (the parser only ever reads the length we pass). + */ +static JsonParseErrorType +parse_incremental(const char *buf, size_t len, int64 chunk_size) +{ + JsonLexContext *lex; + JsonParseErrorType result; + size_t offset; + + lex = makeJsonLexContextIncremental(NULL, PG_UTF8, false); + + offset = 0; + for (;;) + { + size_t remaining = len - offset; + size_t this_chunk; + bool is_last; + + this_chunk = (remaining < (size_t) chunk_size) ? remaining : (size_t) chunk_size; + is_last = (offset + this_chunk >= len); + + result = pg_parse_json_incremental(lex, &nullSemAction, + buf + offset, this_chunk, + is_last); + + if (is_last) + break; + + /* + * A non-final chunk that is not merely incomplete has produced a + * definitive answer (an error that will not change by feeding more + * input). Report it as the outcome. + */ + if (result != JSON_INCOMPLETE) + break; + + offset += this_chunk; + } + + freeJsonLexContext(lex); + + return result; +} + +/* + * The key property: the one-shot and incremental parsers must agree on whether + * a given byte sequence is valid JSON. We only compare success-vs-failure, + * not the specific error code (the two front ends legitimately report + * different error codes for the same bad input). + */ +PG_PROP(chunking_agrees_with_oneshot) +{ + size_t len; + char *s = pg_prop_draw_from(tc, JSON_ALPHABET, 0, 60, &len); + int64 chunk_size = pg_prop_draw_int(tc, 1, 8); + JsonParseErrorType one; + JsonParseErrorType inc; + + one = parse_oneshot(s, len); + inc = parse_incremental(s, len, chunk_size); + + if ((one == JSON_SUCCESS) != (inc == JSON_SUCCESS)) + { + char *shown = escape_for_note((const unsigned char *) s, len); + + pg_prop_fail(tc, + "oneshot/incremental disagree: oneshot=%d incremental=%d " + "(chunk_size=%lld input=\"%s\")", + (int) one, (int) inc, (long long) chunk_size, shown); + } + + pg_free(s); +} + +/* + * Robustness: the one-shot parser must return some JsonParseErrorType for any + * byte sequence at all, never crash. Because properties run in-process, a + * crash would abort the whole run and surface as a hard failure -- which is + * exactly the signal we want for a no-crash property; we do not fork here. If + * the call returns, the parser survived and the property passes. + */ +PG_PROP(oneshot_no_crash) +{ + size_t len; + unsigned char *buf = pg_prop_draw_bytes(tc, 0, 256, &len); + + /* parse by explicit length; interior NULs are fine for the parser */ + (void) parse_oneshot((const char *) buf, len); + + pg_free(buf); +} + +/* + * Chunk-boundary invariant: parsing the same bytes incrementally with two + * different small chunk sizes must reach the same final outcome. This + * isolates cross-chunk reassembly bugs from any one-shot/incremental + * difference -- both sides here use the incremental front end. + */ +PG_PROP(incremental_chunk_size_invariant) +{ + size_t len; + char *s = pg_prop_draw_from(tc, JSON_ALPHABET, 0, 60, &len); + int64 size_a = pg_prop_draw_int(tc, 1, 8); + int64 size_b = pg_prop_draw_int(tc, 1, 8); + JsonParseErrorType res_a; + JsonParseErrorType res_b; + + res_a = parse_incremental(s, len, size_a); + res_b = parse_incremental(s, len, size_b); + + if ((res_a == JSON_SUCCESS) != (res_b == JSON_SUCCESS)) + { + char *shown = escape_for_note((const unsigned char *) s, len); + + pg_prop_fail(tc, + "chunk sizes %lld and %lld disagree: %d vs %d (input=\"%s\")", + (long long) size_a, (long long) size_b, + (int) res_a, (int) res_b, shown); + } + + pg_free(s); +} diff --git a/src/test/prop/t/001_prop.pl b/src/test/prop/t/001_prop.pl index 5536d759e8e48..e0a4093050ccd 100644 --- a/src/test/prop/t/001_prop.pl +++ b/src/test/prop/t/001_prop.pl @@ -25,7 +25,7 @@ # cluster. The SQL-oracle (Tier B) areas run from t/002_sql_props.pl instead. # # Keep in step with the Tier-A entries in prop_area_sources (meson.build). -my @areas = qw(path); +my @areas = qw(path jsonapi); # No Tier-A areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass, not a failure. From 4fbd2c41f393a965cad3328e027f106d73b312ce Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:40 -0400 Subject: [PATCH 006/338] prop: add encoding property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_encoding.c | 228 ++++++++++++++++++++++++++++++++++ src/test/prop/t/001_prop.pl | 2 +- 4 files changed, 232 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_encoding.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index e310bbd7bf337..fde8cc3ee9fc3 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -21,7 +21,8 @@ OBJS = \ pg_prop.o \ pg_prop_sql.o \ prop_path.o \ - prop_jsonapi.o + prop_jsonapi.o \ + prop_encoding.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index da04fe8d49f14..d304aace43676 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -19,6 +19,7 @@ prop_area_sources = files( # Tier B: SQL oracle (needs a running node) 'prop_path.c', 'prop_jsonapi.c', + 'prop_encoding.c', ) diff --git a/src/test/prop/prop_encoding.c b/src/test/prop/prop_encoding.c new file mode 100644 index 0000000000000..1d258e4809931 --- /dev/null +++ b/src/test/prop/prop_encoding.c @@ -0,0 +1,228 @@ +/*------------------------------------------------------------------------- + * + * prop_encoding.c + * Property-based tests for src/common/wchar.c multibyte verification. + * + * The multibyte verifiers are the trust boundary for every byte that enters + * the server claiming to be text in some encoding. pg_encoding_verifymbstr() + * decides how much of an input buffer is a valid string; pg_encoding_mblen() + * and its bounded cousin decide how far to advance. A short input or a + * malformed lead byte that makes one of these over-read is a classic class of + * bug (see commit 74ee636cc93, which fixed exactly such an over-read), so we + * exercise them with arbitrary byte buffers across a spread of encodings with + * interesting multibyte rules. + * + * Properties: + * - verifymbstr returns a count in [0, len] and, when short, stops exactly on + * a byte that does not begin a complete valid character. + * - verifymbstr agrees with an independent char-by-char re-walk. + * - mblen_bounded always returns a length in [1, max_length], even when the + * input is shorter than max_length (the over-read class). + * - verifymbstr is idempotent on its own accepted UTF-8 prefix. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_encoding.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "encoding" +#include "pg_prop.h" +#include "mb/pg_wchar.h" + +/* + * Draw one encoding from a small fixed set that is server-safe and has + * interesting multibyte rules (variable-length lead bytes, 4-byte GB18030, + * EUC multibyte, and plain single-byte baselines). + */ +static int +draw_encoding(pg_prop_tc *tc) +{ + static const int encs[] = { + PG_SQL_ASCII, PG_UTF8, PG_LATIN1, PG_EUC_JP, + PG_EUC_KR, PG_GB18030, PG_BIG5 + }; + + return encs[pg_prop_draw_int(tc, 0, (int) (lengthof(encs) - 1))]; +} + +/* + * Append the hex of a byte buffer to a note so a shrunk counterexample shows + * the exact bytes. The buffer is small (<= 64 bytes) so a stack buffer is + * plenty. + */ +static void +note_hex(pg_prop_tc *tc, const unsigned char *buf, size_t len) +{ + char hex[64 * 3 + 1]; + size_t i; + size_t off = 0; + + for (i = 0; i < len && off + 3 < sizeof(hex); i++) + off += sprintf(hex + off, "%02x ", buf[i]); + hex[off] = '\0'; + pg_prop_note(tc, "bytes: %s", hex); +} + +PG_PROP(verifymbstr_prefix_is_whole) +{ + int enc = draw_encoding(tc); + size_t len; + unsigned char *buf = pg_prop_draw_bytes(tc, 0, 64, &len); + int n; + + n = pg_encoding_verifymbstr(enc, (char *) buf, (int) len); + + /* n is the number of leading bytes that form a valid string */ + if (!(n >= 0 && n <= (int) len)) + { + note_hex(tc, buf, len); + pg_prop_fail(tc, "verifymbstr out of range: enc=%d len=%zu n=%d", + enc, len, n); + } + + /* + * If the verifier stopped short, the byte at offset n must be a legitimate + * stop: either it does not begin a complete valid character (verifymbchar + * returns <= 0, i.e. invalid or incomplete), or it is a NUL byte. The + * string verifiers deliberately stop at an embedded NUL even though the + * per-character verifier (e.g. SQL_ASCII's, which accepts any byte) treats + * NUL as a valid 1-byte character -- a text value must not contain NUL. + * Anything else means verifymbstr stopped on a boundary it should have + * consumed. + */ + if (n < (int) len) + { + int c = pg_encoding_verifymbchar(enc, (char *) buf + n, + (int) len - n); + + if (c > 0 && buf[n] != '\0') + { + note_hex(tc, buf, len); + pg_prop_fail(tc, "verifymbstr stopped at a valid char boundary: " + "enc=%d len=%zu n=%d verifymbchar=%d", + enc, len, n, c); + } + } + + pg_free(buf); +} + +PG_PROP(verifymbstr_consistent_with_charwalk) +{ + int enc = draw_encoding(tc); + size_t len; + unsigned char *buf = pg_prop_draw_bytes(tc, 0, 64, &len); + int n; + int off = 0; + + /* + * Independently re-walk the buffer one character at a time; the sum of the + * valid char lengths must equal what the whole-string verifier reports. + * The string verifier also stops at an embedded NUL (a text value cannot + * contain one), which the per-character verifier does not model, so stop + * the walk at a NUL as well to compare like with like. + */ + for (;;) + { + int c; + + if (off >= (int) len || buf[off] == '\0') + break; + c = pg_encoding_verifymbchar(enc, (char *) buf + off, + (int) len - off); + if (c <= 0) + break; + off += c; + } + + n = pg_encoding_verifymbstr(enc, (char *) buf, (int) len); + + if (off != n) + { + note_hex(tc, buf, len); + pg_prop_fail(tc, "verifymbstr disagrees with char-walk: " + "enc=%d len=%zu verifymbstr=%d charwalk=%d", + enc, len, n, off); + } + + pg_free(buf); +} + +PG_PROP(mblen_bounded_within_limit) +{ + int enc = draw_encoding(tc); + size_t len; + unsigned char *buf = pg_prop_draw_bytes(tc, 1, 64, &len); + int maxlen = pg_encoding_max_length(enc); + unsigned char *padded; + int m; + + /* + * pg_encoding_mblen_bounded() takes only (enc, mbstr) and may read up to + * pg_encoding_mblen() bytes before its strnlen() stops. For GB18030 that + * is up to two bytes examined to decide the length, so a short drawn + * buffer could over-read. Pad with max_length trailing zero bytes: any + * over-read then reads our own zeros, and strnlen() bounds the result at + * the first zero -- exactly the "short input over-read" guard this + * property exists to defend (commit 74ee636cc93). + */ + padded = pg_malloc(len + maxlen + 1); + memcpy(padded, buf, len); + memset(padded + len, 0, maxlen + 1); + + m = pg_encoding_mblen_bounded(enc, (char *) padded); + + /* + * mblen_bounded is strnlen(mbstr, mblen), so it returns 0 for a buffer + * that begins with a NUL (it must never report length past a NUL), and + * otherwise a positive length no larger than the encoding's maximum. + */ + if (padded[0] == '\0') + { + if (m != 0) + { + note_hex(tc, buf, len); + pg_prop_fail(tc, "mblen_bounded of leading NUL not 0: enc=%d m=%d", + enc, m); + } + } + else if (!(m >= 1 && m <= maxlen)) + { + note_hex(tc, buf, len); + pg_prop_fail(tc, "mblen_bounded out of range: enc=%d m=%d max_length=%d len=%zu", + enc, m, maxlen, len); + } + + pg_free(padded); + pg_free(buf); +} + +PG_PROP(utf8_valid_string_fully_accepted) +{ + int enc = PG_UTF8; + size_t len; + unsigned char *buf = pg_prop_draw_bytes(tc, 0, 64, &len); + int n; + int n2; + + /* + * Whatever prefix the UTF-8 verifier accepts, re-verifying just that + * prefix must accept all of it -- verifymbstr is idempotent on its own + * accepted output. + */ + n = pg_encoding_verifymbstr(enc, (char *) buf, (int) len); + n2 = pg_encoding_verifymbstr(enc, (char *) buf, n); + + if (n2 != n) + { + note_hex(tc, buf, len); + pg_prop_fail(tc, "verifymbstr not idempotent on accepted prefix: " + "len=%zu n=%d reverify=%d", len, n, n2); + } + + pg_free(buf); +} diff --git a/src/test/prop/t/001_prop.pl b/src/test/prop/t/001_prop.pl index e0a4093050ccd..1a7793a9a16c0 100644 --- a/src/test/prop/t/001_prop.pl +++ b/src/test/prop/t/001_prop.pl @@ -25,7 +25,7 @@ # cluster. The SQL-oracle (Tier B) areas run from t/002_sql_props.pl instead. # # Keep in step with the Tier-A entries in prop_area_sources (meson.build). -my @areas = qw(path jsonapi); +my @areas = qw(path jsonapi encoding); # No Tier-A areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass, not a failure. From e611955ddf70232c19afc98dfa536bdf16a825a4 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:40 -0400 Subject: [PATCH 007/338] prop: add unicode_case property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_unicode_case.c | 184 ++++++++++++++++++++++++++++++ src/test/prop/t/001_prop.pl | 2 +- 4 files changed, 188 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_unicode_case.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index fde8cc3ee9fc3..7a062b3159e38 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -22,7 +22,8 @@ OBJS = \ pg_prop_sql.o \ prop_path.o \ prop_jsonapi.o \ - prop_encoding.o + prop_encoding.o \ + prop_unicode_case.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index d304aace43676..77b3e3f6ea971 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -20,6 +20,7 @@ prop_area_sources = files( 'prop_path.c', 'prop_jsonapi.c', 'prop_encoding.c', + 'prop_unicode_case.c', ) diff --git a/src/test/prop/prop_unicode_case.c b/src/test/prop/prop_unicode_case.c new file mode 100644 index 0000000000000..8af5b14f05d92 --- /dev/null +++ b/src/test/prop/prop_unicode_case.c @@ -0,0 +1,184 @@ +/*------------------------------------------------------------------------- + * + * prop_unicode_case.c + * Property-based tests for src/common/unicode_case.c case mapping. + * + * Unicode case conversion (unicode_strlower/strupper/strfold) is pure, + * length-based, and a classic bug site: special mappings (ß, final sigma), + * measure-mode buffer sizing, and truncated-UTF-8 inputs. Recent fixes + * defending against truncated UTF-8 (21ffc271d47, 07211f64ace) live here. + * + * Properties: + * - lowercase is idempotent: lower(lower(x)) == lower(x) (full mappings). + * - casefold is idempotent: fold(fold(x)) == fold(x) -- a documented + * contract and the classic fold bug site. + * - measure mode (dstsize 0, dst NULL) returns the same length as the real + * conversion, and consumes all of valid src both times. + * - truncated UTF-8 input never consumes past the given srclen (no overrun). + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_unicode_case.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "unicode_case" +#include "pg_prop.h" +#include "common/unicode_case.h" +#include "mb/pg_wchar.h" + +/* signature shared by unicode_strlower/strupper/strfold */ +typedef size_t (*case_fn) (char *dst, size_t dstsize, const char *src, + size_t srclen, size_t *pconsumed, bool full); + +/* + * Convert src (length srclen) with the given case function into a freshly + * malloc'd, NUL-terminated buffer. Returns the buffer and stores the result + * byte length in *outlen. src must be valid UTF-8; if a decoding error is + * detected (consumed < srclen) the current test case is discarded. + */ +static char * +convert_dup(pg_prop_tc *tc, case_fn fn, const char *src, size_t srclen, + bool full, size_t *outlen) +{ + size_t need; + size_t consumed; + size_t got; + char *buf; + + /* measure mode: dst may be NULL when dstsize is 0 */ + need = fn(NULL, 0, src, srclen, &consumed, full); + + /* harness draws valid UTF-8, so the whole input must be consumed */ + Assert(consumed == srclen); + pg_prop_assume(tc, consumed == srclen); + + buf = pg_malloc(need + 1); + got = fn(buf, need + 1, src, srclen, &consumed, full); + + /* dstsize > result length guarantees NUL termination */ + Assert(got == need); + Assert(consumed == srclen); + buf[need] = '\0'; + *outlen = got; + return buf; +} + +/* Append src bytes to a note-friendly hex string; caller frees. */ +static char * +to_hex(const char *src, size_t len) +{ + char *out = pg_malloc(len * 2 + 1); + size_t i; + + for (i = 0; i < len; i++) + snprintf(out + i * 2, 3, "%02x", (unsigned char) src[i]); + out[len * 2] = '\0'; + return out; +} + +PG_PROP(lower_idempotent) +{ + size_t len; + char *s = pg_prop_draw_utf8(tc, 0, 60, &len); + size_t len1, + len2; + char *once = convert_dup(tc, unicode_strlower, s, len, true, &len1); + char *twice = convert_dup(tc, unicode_strlower, once, len1, true, &len2); + + if (len1 != len2 || memcmp(once, twice, len1) != 0) + { + char *hex = to_hex(s, len); + + pg_prop_fail(tc, "lower not idempotent for input (hex): %s", hex); + } + + pg_free(twice); + pg_free(once); + pg_free(s); +} + +PG_PROP(fold_idempotent) +{ + size_t len; + char *s = pg_prop_draw_utf8(tc, 0, 60, &len); + size_t len1, + len2; + char *once = convert_dup(tc, unicode_strfold, s, len, true, &len1); + char *twice = convert_dup(tc, unicode_strfold, once, len1, true, &len2); + + if (len1 != len2 || memcmp(once, twice, len1) != 0) + { + char *hex = to_hex(s, len); + + pg_prop_fail(tc, "fold not idempotent for input (hex): %s", hex); + } + + pg_free(twice); + pg_free(once); + pg_free(s); +} + +PG_PROP(dstsize_zero_computes_true_length) +{ + size_t len; + char *s = pg_prop_draw_utf8(tc, 0, 60, &len); + bool full = pg_prop_draw_bool(tc); + size_t need; + size_t consumed; + size_t got; + size_t consumed2; + char *buf; + + /* measure mode: dst NULL, dstsize 0 */ + need = unicode_strupper(NULL, 0, s, len, &consumed, full); + pg_prop_assume(tc, consumed == len); + + buf = pg_malloc(need + 1); + got = unicode_strupper(buf, need + 1, s, len, &consumed2, full); + + if (got != need) + pg_prop_fail(tc, + "measure length %zu != real length %zu (full=%d)", + need, got, (int) full); + if (consumed != len || consumed2 != len) + pg_prop_fail(tc, + "consumed mismatch: measure=%zu real=%zu srclen=%zu", + consumed, consumed2, len); + + pg_free(buf); + pg_free(s); +} + +PG_PROP(truncated_utf8_no_overrun) +{ + size_t len; + char *s = pg_prop_draw_utf8(tc, 1, 60, &len); + size_t tlen; + size_t consumed; + bool full = pg_prop_draw_bool(tc); + + /* + * Truncate to a shorter byte length, possibly cutting a multibyte + * character in half. len >= 1 so [0, len-1] is a valid range. + */ + tlen = (size_t) pg_prop_draw_int(tc, 0, (int64) len - 1); + + /* + * Measure mode on the truncated input. The function must return without + * reading past src + tlen (verified under ASan) and must never report + * consuming more than it was given. A trailing truncated char should + * leave consumed < tlen (decoding error); a clean cut consumes all tlen. + */ + (void) unicode_strlower(NULL, 0, s, tlen, &consumed, full); + + if (consumed > tlen) + pg_prop_fail(tc, + "consumed %zu > srclen %zu on truncated input", + consumed, tlen); + + pg_free(s); +} diff --git a/src/test/prop/t/001_prop.pl b/src/test/prop/t/001_prop.pl index 1a7793a9a16c0..9afd1b86da03f 100644 --- a/src/test/prop/t/001_prop.pl +++ b/src/test/prop/t/001_prop.pl @@ -25,7 +25,7 @@ # cluster. The SQL-oracle (Tier B) areas run from t/002_sql_props.pl instead. # # Keep in step with the Tier-A entries in prop_area_sources (meson.build). -my @areas = qw(path jsonapi encoding); +my @areas = qw(path jsonapi encoding unicode_case); # No Tier-A areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass, not a failure. From 97459d15abf85db25fc893ecc76308862d047a46 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:40 -0400 Subject: [PATCH 008/338] prop: add unicode_norm property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_unicode_norm.c | 199 ++++++++++++++++++++++++++++++ src/test/prop/t/001_prop.pl | 2 +- 4 files changed, 203 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_unicode_norm.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 7a062b3159e38..3c0a0974309f2 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -23,7 +23,8 @@ OBJS = \ prop_path.o \ prop_jsonapi.o \ prop_encoding.o \ - prop_unicode_case.o + prop_unicode_case.o \ + prop_unicode_norm.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 77b3e3f6ea971..e05d210d85392 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -21,6 +21,7 @@ prop_area_sources = files( 'prop_jsonapi.c', 'prop_encoding.c', 'prop_unicode_case.c', + 'prop_unicode_norm.c', ) diff --git a/src/test/prop/prop_unicode_norm.c b/src/test/prop/prop_unicode_norm.c new file mode 100644 index 0000000000000..a0fd77bcb40a2 --- /dev/null +++ b/src/test/prop/prop_unicode_norm.c @@ -0,0 +1,199 @@ +/*------------------------------------------------------------------------- + * + * prop_unicode_norm.c + * Property-based tests for src/common/unicode_norm.c normalization. + * + * unicode_normalize() implements the UAX #15 normalization forms (NFC, NFD, + * NFKC, NFKD) over NUL-terminated char32_t code point arrays. It is pure, + * shared by frontend and backend, and has a history of subtle composition + * bugs: f2ff15e4c37 fixed an off-by-one in NFC recomposition for Hangul, and + * 066b7b144f3 fixed a buffer overrun -- both in exactly the recompose path + * these properties drive. + * + * This binary is a frontend program, so unicode_normalize() allocates its + * results with malloc (freed here with pg_free) and may return NULL on OOM. + * We deliberately do NOT call unicode_is_normalized_quickcheck(): it is + * compiled only in the backend (#ifndef FRONTEND). + * + * Properties: + * - NFC is idempotent: NFC(NFC(x)) == NFC(x). + * - NFD is idempotent: NFD(NFD(x)) == NFD(x). + * - Canonical equivalence: NFC(x) == NFC(NFD(x)) for all x (the strong + * property that catches the Hangul recomposition bugs above). + * - NFKD is idempotent: NFKD(NFKD(x)) == NFKD(x) (exercises the compat path). + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_unicode_norm.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "unicode_norm" +#include "pg_prop.h" +#include "common/unicode_norm.h" +#include "lib/stringinfo.h" + +/* + * Draw a NUL-terminated char32_t code point array. Returns a pg_malloc'd + * array with cps[n] == 0; *out_n receives the code point count (which may be + * 0 -- the empty string is a valid and interesting input). + */ +static char32_t * +draw_codepoints(pg_prop_tc *tc, int *out_n) +{ + int n = (int) pg_prop_draw_int(tc, 0, 24); /* small strings */ + char32_t *cps = pg_malloc(sizeof(char32_t) * (n + 1)); + int i; + + for (i = 0; i < n; i++) + { + /* + * Draw an assigned, non-surrogate scalar value. Bias toward the + * ranges where normalization actually does something: Latin + + * combining marks (U+0041..U+024F, U+0300..U+036F), Hangul + * (U+AC00..U+D7A3 precomposed and U+1100..U+11FF jamo), and a few + * compatibility chars. Draw a bucket, then a codepoint in it. + */ + int bucket = (int) pg_prop_draw_int(tc, 0, 4); + char32_t c; + + switch (bucket) + { + case 0: + c = (char32_t) pg_prop_draw_int(tc, 0x0041, 0x024F); /* Latin + ext */ + break; + case 1: + c = (char32_t) pg_prop_draw_int(tc, 0x0300, 0x036F); /* combining marks */ + break; + case 2: + c = (char32_t) pg_prop_draw_int(tc, 0xAC00, 0xD7A3); /* Hangul syllables */ + break; + case 3: + c = (char32_t) pg_prop_draw_int(tc, 0x1100, 0x11FF); /* Hangul jamo */ + break; + default: + c = (char32_t) pg_prop_draw_int(tc, 0x00A0, 0x2FFF); /* mixed BMP */ + break; + } + cps[i] = c; + } + cps[n] = 0; + *out_n = n; + return cps; +} + +/* Equality of two NUL-terminated char32_t arrays (same length, same values). */ +static bool +cp_equal(const char32_t *a, const char32_t *b) +{ + int i; + + for (i = 0; a[i] != 0 || b[i] != 0; i++) + { + if (a[i] != b[i]) + return false; + } + return true; +} + +/* Format a code point array as "U+0041 U+0301 ..." into a palloc'd buffer. */ +static char * +cp_show(const char32_t *s) +{ + StringInfoData buf; + int i; + + initStringInfo(&buf); + if (s[0] == 0) + appendStringInfoString(&buf, "(empty)"); + for (i = 0; s[i] != 0; i++) + appendStringInfo(&buf, "%sU+%04X", i == 0 ? "" : " ", (unsigned) s[i]); + return buf.data; +} + +PG_PROP(nfc_idempotent) +{ + int n; + char32_t *cps = draw_codepoints(tc, &n); + char32_t *a = unicode_normalize(UNICODE_NFC, cps); + char32_t *b = unicode_normalize(UNICODE_NFC, a); + + if (!cp_equal(a, b)) + { + char *in = cp_show(cps); + + pg_prop_fail(tc, "NFC not idempotent for input [%s]: NFC(x)=[%s] NFC(NFC(x))=[%s]", + in, cp_show(a), cp_show(b)); + } + pg_free(a); + pg_free(b); + pg_free(cps); +} + +PG_PROP(nfd_idempotent) +{ + int n; + char32_t *cps = draw_codepoints(tc, &n); + char32_t *a = unicode_normalize(UNICODE_NFD, cps); + char32_t *b = unicode_normalize(UNICODE_NFD, a); + + if (!cp_equal(a, b)) + { + char *in = cp_show(cps); + + pg_prop_fail(tc, "NFD not idempotent for input [%s]: NFD(x)=[%s] NFD(NFD(x))=[%s]", + in, cp_show(a), cp_show(b)); + } + pg_free(a); + pg_free(b); + pg_free(cps); +} + +PG_PROP(nfc_of_nfd_equals_nfc) +{ + int n; + char32_t *cps = draw_codepoints(tc, &n); + char32_t *direct = unicode_normalize(UNICODE_NFC, cps); + char32_t *nfd = unicode_normalize(UNICODE_NFD, cps); + char32_t *viaD = unicode_normalize(UNICODE_NFC, nfd); + + /* + * Canonical equivalence: decomposing fully and recomposing must land on + * the same NFC form as composing directly. This is where the Hangul + * recomposition off-by-one (f2ff15e4c37) and buffer overrun (066b7b144f3) + * showed up. + */ + if (!cp_equal(direct, viaD)) + { + char *in = cp_show(cps); + + pg_prop_fail(tc, "NFC(x) != NFC(NFD(x)) for input [%s]: NFC(x)=[%s] NFC(NFD(x))=[%s] (NFD(x)=[%s])", + in, cp_show(direct), cp_show(viaD), cp_show(nfd)); + } + pg_free(direct); + pg_free(nfd); + pg_free(viaD); + pg_free(cps); +} + +PG_PROP(nfkd_idempotent) +{ + int n; + char32_t *cps = draw_codepoints(tc, &n); + char32_t *a = unicode_normalize(UNICODE_NFKD, cps); + char32_t *b = unicode_normalize(UNICODE_NFKD, a); + + if (!cp_equal(a, b)) + { + char *in = cp_show(cps); + + pg_prop_fail(tc, "NFKD not idempotent for input [%s]: NFKD(x)=[%s] NFKD(NFKD(x))=[%s]", + in, cp_show(a), cp_show(b)); + } + pg_free(a); + pg_free(b); + pg_free(cps); +} diff --git a/src/test/prop/t/001_prop.pl b/src/test/prop/t/001_prop.pl index 9afd1b86da03f..58a8dbb17915a 100644 --- a/src/test/prop/t/001_prop.pl +++ b/src/test/prop/t/001_prop.pl @@ -25,7 +25,7 @@ # cluster. The SQL-oracle (Tier B) areas run from t/002_sql_props.pl instead. # # Keep in step with the Tier-A entries in prop_area_sources (meson.build). -my @areas = qw(path jsonapi encoding unicode_case); +my @areas = qw(path jsonapi encoding unicode_case unicode_norm); # No Tier-A areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass, not a failure. From d454da53667d7eb6e8dac336831b05679a7a25a4 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:41 -0400 Subject: [PATCH 009/338] prop: add pglz property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_pglz.c | 193 ++++++++++++++++++++++++++++++++++++ src/test/prop/t/001_prop.pl | 2 +- 4 files changed, 197 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_pglz.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 3c0a0974309f2..1e1573c40db89 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -24,7 +24,8 @@ OBJS = \ prop_jsonapi.o \ prop_encoding.o \ prop_unicode_case.o \ - prop_unicode_norm.o + prop_unicode_norm.o \ + prop_pglz.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index e05d210d85392..c94582f87c2c9 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -22,6 +22,7 @@ prop_area_sources = files( 'prop_encoding.c', 'prop_unicode_case.c', 'prop_unicode_norm.c', + 'prop_pglz.c', ) diff --git a/src/test/prop/prop_pglz.c b/src/test/prop/prop_pglz.c new file mode 100644 index 0000000000000..5896636f4cb2d --- /dev/null +++ b/src/test/prop/prop_pglz.c @@ -0,0 +1,193 @@ +/*------------------------------------------------------------------------- + * + * prop_pglz.c + * Property-based tests for src/common/pg_lzcompress.c, the builtin pglz + * compressor used by TOAST. + * + * pglz_compress() returns the compressed length, or -1 when the input is + * refused (too small for the strategy, or the result would not be smaller + * than the input). A -1 is an ordinary outcome, not a bug, so the roundtrip + * properties treat it as "nothing to check" rather than a failure, and lean + * on repetitive inputs drawn from a tiny alphabet to keep compression (and + * thus the interesting path) exercised most of the time. + * + * Properties: + * - roundtrip: decompress(compress(x)) == x for every x that compresses. + * - prefix_decompress_matches: decompressing only the first + * pglz_maximum_compressed_size(prefix_len, clen) compressed bytes with + * check_complete=false yields exactly the first prefix_len raw bytes. + * This is the TOAST slice-decompress path (detoast.c); see the heap- + * overflow fix 2b5ba2a0a14 and the "+2" match-tag corner case. + * - decompress_corrupt_no_crash: pglz_decompress() on arbitrary bytes never + * returns more than rawsize (which would mean it wrote past the output + * buffer) and never reads/writes out of bounds. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_pglz.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "pglz" +#include "pg_prop.h" +#include "common/pg_lzcompress.h" + +/* + * Keep every length well under 1<<16 so all casts into the int32-typed pglz + * API are safe and PGLZ_MAX_OUTPUT (+4) cannot overflow. + */ +#define PGLZ_PROP_MAX_LEN 65536 + +/* + * Format up to the first "n" bytes of "buf" as hex into "out", for failure + * messages. "out" must hold at least prefix*2 + 1 bytes. + */ +static void +hex_prefix(const unsigned char *buf, size_t len, size_t prefix, char *out) +{ + static const char hexdig[] = "0123456789abcdef"; + size_t i; + size_t n = Min(len, prefix); + + for (i = 0; i < n; i++) + { + out[i * 2] = hexdig[(buf[i] >> 4) & 0x0f]; + out[i * 2 + 1] = hexdig[buf[i] & 0x0f]; + } + out[n * 2] = '\0'; +} + +/* + * Draw a compressible source: a repetitive string over a tiny alphabet, long + * enough that pglz_compress almost always finds a match and succeeds. + */ +static unsigned char * +draw_compressible(pg_prop_tc *tc, size_t min_len, size_t *len) +{ + return (unsigned char *) pg_prop_draw_from(tc, "ab", min_len, + PGLZ_PROP_MAX_LEN, len); +} + +PG_PROP(roundtrip) +{ + size_t slen; + unsigned char *src = draw_compressible(tc, 16, &slen); + char *cbuf = pg_malloc(PGLZ_MAX_OUTPUT(slen)); + char *dbuf; + int32 clen; + int32 dlen; + char hex[65]; + + /* + * Use PGLZ_strategy_always so compression is attempted on inputs the + * default strategy's min-length gate would reject outright. + */ + clen = pglz_compress((const char *) src, (int32) slen, cbuf, + PGLZ_strategy_always); + + /* -1 means the input was deemed incompressible: nothing to roundtrip. */ + if (clen < 0) + { + pg_free(src); + pg_free(cbuf); + return; + } + + dbuf = pg_malloc(slen); + dlen = pglz_decompress(cbuf, clen, dbuf, (int32) slen, true); + + if (dlen != (int32) slen || memcmp(dbuf, src, slen) != 0) + { + hex_prefix(src, slen, 32, hex); + pg_prop_fail(tc, + "roundtrip failed: slen=%zu clen=%d dlen=%d src[0..]=%s", + slen, clen, dlen, hex); + } + + pg_free(src); + pg_free(cbuf); + pg_free(dbuf); +} + +PG_PROP(prefix_decompress_matches) +{ + size_t slen; + unsigned char *src = draw_compressible(tc, 32, &slen); + char *cbuf = pg_malloc(PGLZ_MAX_OUTPUT(slen)); + char *dbuf; + int32 clen; + int32 plen; + int32 need; + int32 got; + char hex[65]; + + clen = pglz_compress((const char *) src, (int32) slen, cbuf, + PGLZ_strategy_always); + + if (clen < 0) + { + pg_free(src); + pg_free(cbuf); + return; + } + + /* Choose a raw prefix length to reproduce from the compressed data. */ + plen = (int32) pg_prop_draw_int(tc, 1, (int64) slen); + + /* + * Maximum compressed bytes needed to produce "plen" raw bytes. The + * function is documented to cap this at total_compressed_size (clen); the + * clamp below enforces that defensively in case that contract changes. + */ + need = pglz_maximum_compressed_size(plen, clen); + if (need > clen) + need = clen; + + dbuf = pg_malloc(plen); + got = pglz_decompress(cbuf, need, dbuf, plen, false); + + if (got != plen || memcmp(dbuf, src, plen) != 0) + { + hex_prefix(src, slen, 32, hex); + pg_prop_fail(tc, + "prefix mismatch: slen=%zu clen=%d plen=%d need=%d got=%d src[0..]=%s", + slen, clen, plen, need, got, hex); + } + + pg_free(src); + pg_free(cbuf); + pg_free(dbuf); +} + +PG_PROP(decompress_corrupt_no_crash) +{ + size_t slen; + unsigned char *src = pg_prop_draw_bytes(tc, 0, 256, &slen); + int32 rawsize = (int32) pg_prop_draw_int(tc, 0, 4096); + char *dbuf = pg_malloc(rawsize > 0 ? (size_t) rawsize : 1); + int32 got; + char hex[65]; + + /* + * Treat arbitrary bytes as compressed input. pglz_decompress must return + * at most rawsize (a larger return would mean it wrote past dbuf) or -1, + * and must never read or write out of bounds (under ASan the latter would + * fault; here we assert the return bound directly). + */ + got = pglz_decompress((const char *) src, (int32) slen, dbuf, rawsize, + false); + + if (got > rawsize) + { + hex_prefix(src, slen, 32, hex); + pg_prop_fail(tc, + "decompress overran output: slen=%zu rawsize=%d got=%d src[0..]=%s", + slen, rawsize, got, hex); + } + + pg_free(src); + pg_free(dbuf); +} diff --git a/src/test/prop/t/001_prop.pl b/src/test/prop/t/001_prop.pl index 58a8dbb17915a..cfa13c30a517d 100644 --- a/src/test/prop/t/001_prop.pl +++ b/src/test/prop/t/001_prop.pl @@ -25,7 +25,7 @@ # cluster. The SQL-oracle (Tier B) areas run from t/002_sql_props.pl instead. # # Keep in step with the Tier-A entries in prop_area_sources (meson.build). -my @areas = qw(path jsonapi encoding unicode_case unicode_norm); +my @areas = qw(path jsonapi encoding unicode_case unicode_norm pglz); # No Tier-A areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass, not a failure. From 6a04fa48129811c668b4477a945297d70c0ecb6a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:41 -0400 Subject: [PATCH 010/338] prop: add base64 property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_base64.c | 99 +++++++++++++++++++++++++++++++++++++ src/test/prop/t/001_prop.pl | 2 +- 4 files changed, 103 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_base64.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 1e1573c40db89..336e63e14763b 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -25,7 +25,8 @@ OBJS = \ prop_encoding.o \ prop_unicode_case.o \ prop_unicode_norm.o \ - prop_pglz.o + prop_pglz.o \ + prop_base64.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index c94582f87c2c9..11070fc876f4d 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -23,6 +23,7 @@ prop_area_sources = files( 'prop_unicode_case.c', 'prop_unicode_norm.c', 'prop_pglz.c', + 'prop_base64.c', ) diff --git a/src/test/prop/prop_base64.c b/src/test/prop/prop_base64.c new file mode 100644 index 0000000000000..47b99166ba0eb --- /dev/null +++ b/src/test/prop/prop_base64.c @@ -0,0 +1,99 @@ +/*------------------------------------------------------------------------- + * + * prop_base64.c + * Property-based tests for src/common/base64.c. + * + * base64 encode/decode is a pure codec with a clean round-trip property and + * explicit length contracts (pg_b64_enc_len / pg_b64_dec_len), used for SCRAM, + * the COPY and replication protocols, and more. A codec is the textbook case + * for property testing: encode-then-decode must reproduce the input for every + * byte string, and the length helpers must bound the real output. + * + * Tier A: in-process, no server needed. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_base64.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "base64" +#include "pg_prop.h" + +#include "common/base64.h" + +PG_PROP(roundtrip) +{ + size_t len; + unsigned char *src = pg_prop_draw_bytes(tc, 0, 256, &len); + int enclen; + int declen; + char *enc; + unsigned char *dec; + + /* encode into a buffer sized by the library's own length helper */ + enc = pg_malloc(pg_b64_enc_len((int) len) + 1); + enclen = pg_b64_encode(src, (int) len, enc, pg_b64_enc_len((int) len)); + if (enclen < 0) + pg_prop_fail(tc, "encode failed for len=%zu", len); + + /* decode back; the result must be byte-identical to the input */ + dec = pg_malloc(pg_b64_dec_len(enclen) + 1); + declen = pg_b64_decode(enc, enclen, dec, pg_b64_dec_len(enclen)); + if (declen < 0) + pg_prop_fail(tc, "decode failed for len=%zu enclen=%d", len, enclen); + + if (declen != (int) len || memcmp(dec, src, len) != 0) + pg_prop_fail(tc, "roundtrip mismatch: len=%zu declen=%d", len, declen); + + pg_free(dec); + pg_free(enc); + pg_free(src); +} + +PG_PROP(enc_len_bounds_output) +{ + size_t len; + unsigned char *src = pg_prop_draw_bytes(tc, 0, 256, &len); + int bound = pg_b64_enc_len((int) len); + char *enc = pg_malloc(bound + 1); + int enclen = pg_b64_encode(src, (int) len, enc, bound); + + /* + * pg_b64_enc_len() is what every caller uses to size the destination, so + * the actual encoded length must never exceed it. + */ + if (enclen < 0 || enclen > bound) + pg_prop_fail(tc, "encoded length %d exceeds pg_b64_enc_len bound %d (input len=%zu)", + enclen, bound, len); + + pg_free(enc); + pg_free(src); +} + +PG_PROP(decode_garbage_no_overrun) +{ + size_t len; + char *src = pg_prop_draw_from(tc, + "ABCDEFGHIJKLMNOPQRSTUVWXYZ" + "abcdefghijklmnopqrstuvwxyz0123456789+/= \n", + 0, 256, &len); + int bound = pg_b64_dec_len((int) len); + unsigned char *dst = pg_malloc(bound > 0 ? bound + 1 : 1); + int declen = pg_b64_decode(src, (int) len, dst, bound); + + /* + * Arbitrary base64-alphabet-ish input either decodes to at most the + * length pg_b64_dec_len() promised, or is rejected (< 0); it must never + * report writing more than the buffer bound. + */ + if (declen > bound) + pg_prop_fail(tc, "decoded length %d exceeds pg_b64_dec_len bound %d (input len=%zu)", + declen, bound, len); + + pg_free(dst); + pg_free(src); +} diff --git a/src/test/prop/t/001_prop.pl b/src/test/prop/t/001_prop.pl index cfa13c30a517d..a15e475889295 100644 --- a/src/test/prop/t/001_prop.pl +++ b/src/test/prop/t/001_prop.pl @@ -25,7 +25,7 @@ # cluster. The SQL-oracle (Tier B) areas run from t/002_sql_props.pl instead. # # Keep in step with the Tier-A entries in prop_area_sources (meson.build). -my @areas = qw(path jsonapi encoding unicode_case unicode_norm pglz); +my @areas = qw(path jsonapi encoding unicode_case unicode_norm pglz base64); # No Tier-A areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass, not a failure. From e0e8b413a6c7b6ac13aaf35be71ec90954c3ba32 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:41 -0400 Subject: [PATCH 011/338] prop: add prng property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_prng.c | 132 ++++++++++++++++++++++++++++++++++++ src/test/prop/t/001_prop.pl | 2 +- 4 files changed, 136 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_prng.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 336e63e14763b..1adcb259e1dcc 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -26,7 +26,8 @@ OBJS = \ prop_unicode_case.o \ prop_unicode_norm.o \ prop_pglz.o \ - prop_base64.o + prop_base64.o \ + prop_prng.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 11070fc876f4d..252b19b002782 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -24,6 +24,7 @@ prop_area_sources = files( 'prop_unicode_norm.c', 'prop_pglz.c', 'prop_base64.c', + 'prop_prng.c', ) diff --git a/src/test/prop/prop_prng.c b/src/test/prop/prop_prng.c new file mode 100644 index 0000000000000..647c3aab8224c --- /dev/null +++ b/src/test/prop/prop_prng.c @@ -0,0 +1,132 @@ +/*------------------------------------------------------------------------- + * + * prop_prng.c + * Property-based tests for src/common/pg_prng.c. + * + * pg_prng is the shared pseudo-random generator (xoroshiro128**) behind + * random(), sampling, and backend jitter. Its range helpers carry explicit + * contracts -- a value in [rmin, rmax], a double in [0, 1) -- that must hold + * for every seed and every bound, including the degenerate rmin == rmax case + * and the full-width range. Those contracts are exactly what a property test + * pins down. + * + * Tier A: in-process, no server needed. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_prng.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "prng" +#include "pg_prop.h" + +#include "common/pg_prng.h" + +/* + * Seed a fresh generator from a drawn 64-bit value. Seeding the PRNG from a + * hegel-drawn integer is the right pattern here (unlike feeding a seed to code + * that then makes its own random decisions): the property is about the range + * helpers' output bounds, which must hold for every internal state, so the + * seed is itself the input we want hegel to explore and shrink. + */ +static void +seed_from_draw(pg_prop_tc *tc, pg_prng_state *state) +{ + uint64 s = (uint64) pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + + pg_prng_seed(state, s); +} + +PG_PROP(uint64_range_within_bounds) +{ + pg_prng_state state; + uint64 lo = (uint64) pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + uint64 hi = (uint64) pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + uint64 r; + + if (lo > hi) + { + uint64 tmp = lo; + + lo = hi; + hi = tmp; + } + + seed_from_draw(tc, &state); + + /* draw several values so one seed exercises more of the stream */ + for (int i = 0; i < 20; i++) + { + r = pg_prng_uint64_range(&state, lo, hi); + if (r < lo || r > hi) + pg_prop_fail(tc, + "uint64_range out of bounds: lo=%llu hi=%llu got=%llu", + (unsigned long long) lo, (unsigned long long) hi, + (unsigned long long) r); + } +} + +PG_PROP(int64_range_within_bounds) +{ + pg_prng_state state; + int64 lo = pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + int64 hi = pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + int64 r; + + if (lo > hi) + { + int64 tmp = lo; + + lo = hi; + hi = tmp; + } + + seed_from_draw(tc, &state); + + for (int i = 0; i < 20; i++) + { + r = pg_prng_int64_range(&state, lo, hi); + if (r < lo || r > hi) + pg_prop_fail(tc, + "int64_range out of bounds: lo=%lld hi=%lld got=%lld", + (long long) lo, (long long) hi, (long long) r); + } +} + +PG_PROP(singleton_range_is_exact) +{ + pg_prng_state state; + int64 v = pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + + seed_from_draw(tc, &state); + + /* a range of one value must always return that value */ + for (int i = 0; i < 8; i++) + { + int64 r = pg_prng_int64_range(&state, v, v); + + if (r != v) + pg_prop_fail(tc, "singleton range [%lld,%lld] returned %lld", + (long long) v, (long long) v, (long long) r); + } +} + +PG_PROP(double_in_unit_interval) +{ + pg_prng_state state; + + seed_from_draw(tc, &state); + + /* pg_prng_double() is documented to return a value in [0.0, 1.0) */ + for (int i = 0; i < 50; i++) + { + double d = pg_prng_double(&state); + + if (!(d >= 0.0 && d < 1.0)) + pg_prop_fail(tc, "double not in [0,1): got %.17g", d); + } +} diff --git a/src/test/prop/t/001_prop.pl b/src/test/prop/t/001_prop.pl index a15e475889295..2cadd5b0d34de 100644 --- a/src/test/prop/t/001_prop.pl +++ b/src/test/prop/t/001_prop.pl @@ -25,7 +25,7 @@ # cluster. The SQL-oracle (Tier B) areas run from t/002_sql_props.pl instead. # # Keep in step with the Tier-A entries in prop_area_sources (meson.build). -my @areas = qw(path jsonapi encoding unicode_case unicode_norm pglz base64); +my @areas = qw(path jsonapi encoding unicode_case unicode_norm pglz base64 prng); # No Tier-A areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass, not a failure. From 7bd3ccab612f042def24129e35757ac827613612 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:41 -0400 Subject: [PATCH 012/338] prop: add crypto property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_crypto.c | 133 ++++++++++++++++++++++++++++++++++++ src/test/prop/t/001_prop.pl | 2 +- 4 files changed, 137 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_crypto.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 1adcb259e1dcc..5b38f92ffcf8c 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -27,7 +27,8 @@ OBJS = \ prop_unicode_norm.o \ prop_pglz.o \ prop_base64.o \ - prop_prng.o + prop_prng.o \ + prop_crypto.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 252b19b002782..e4e13604715e3 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -25,6 +25,7 @@ prop_area_sources = files( 'prop_pglz.c', 'prop_base64.c', 'prop_prng.c', + 'prop_crypto.c', ) diff --git a/src/test/prop/prop_crypto.c b/src/test/prop/prop_crypto.c new file mode 100644 index 0000000000000..8216f071db1da --- /dev/null +++ b/src/test/prop/prop_crypto.c @@ -0,0 +1,133 @@ +/*------------------------------------------------------------------------- + * + * prop_crypto.c + * Property-based tests for src/common cryptohash and HMAC. + * + * The hash and HMAC wrappers (pg_cryptohash_*, pg_hmac_*) back SCRAM, MD5 + * auth, checksums, and the backup manifest. They are deterministic functions + * with fixed-size output: the same input must always produce the same digest, + * a one-byte change must change it, and the digest is exactly the type's + * length. These are the properties every caller silently relies on. + * + * Tier A: in-process, no server needed. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_crypto.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "crypto" +#include "pg_prop.h" + +#include "common/cryptohash.h" +#include "common/hmac.h" +#include "common/sha2.h" + +/* + * Compute a SHA-256 digest of src[0..len) into dest (PG_SHA256_DIGEST_LENGTH + * bytes). Fails the test case on any wrapper error (these should never fail + * on valid input). + */ +static void +sha256(pg_prop_tc *tc, const uint8 *src, size_t len, uint8 *dest) +{ + pg_cryptohash_ctx *ctx = pg_cryptohash_create(PG_SHA256); + + if (ctx == NULL) + pg_prop_fail(tc, "pg_cryptohash_create failed"); + if (pg_cryptohash_init(ctx) < 0 || + pg_cryptohash_update(ctx, src, len) < 0 || + pg_cryptohash_final(ctx, dest, PG_SHA256_DIGEST_LENGTH) < 0) + { + pg_cryptohash_free(ctx); + pg_prop_fail(tc, "sha256 computation failed for len=%zu", len); + } + pg_cryptohash_free(ctx); +} + +PG_PROP(sha256_deterministic) +{ + size_t len; + unsigned char *src = pg_prop_draw_bytes(tc, 0, 256, &len); + uint8 a[PG_SHA256_DIGEST_LENGTH]; + uint8 b[PG_SHA256_DIGEST_LENGTH]; + + /* hashing the same bytes twice must give the same digest */ + sha256(tc, src, len, a); + sha256(tc, src, len, b); + + if (memcmp(a, b, PG_SHA256_DIGEST_LENGTH) != 0) + pg_prop_fail(tc, "sha256 not deterministic for len=%zu", len); + + pg_free(src); +} + +PG_PROP(sha256_sensitive_to_single_bit) +{ + size_t len; + unsigned char *src; + uint8 a[PG_SHA256_DIGEST_LENGTH]; + uint8 b[PG_SHA256_DIGEST_LENGTH]; + size_t pos; + int bit; + + /* need at least one byte to flip */ + src = pg_prop_draw_bytes(tc, 1, 256, &len); + pos = (size_t) pg_prop_draw_int(tc, 0, (int) len - 1); + bit = (int) pg_prop_draw_int(tc, 0, 7); + + sha256(tc, src, len, a); + src[pos] ^= (unsigned char) (1 << bit); + sha256(tc, src, len, b); + + /* + * Flipping any single bit of the input must change the digest. A + * collision here would be a catastrophic hash failure. + */ + if (memcmp(a, b, PG_SHA256_DIGEST_LENGTH) == 0) + pg_prop_fail(tc, "sha256 unchanged after flipping bit %d of byte %zu (len=%zu)", + bit, pos, len); + + pg_free(src); +} + +PG_PROP(hmac_sha256_deterministic_and_sized) +{ + size_t klen; + size_t dlen; + unsigned char *key = pg_prop_draw_bytes(tc, 0, 64, &klen); + unsigned char *data = pg_prop_draw_bytes(tc, 0, 256, &dlen); + uint8 a[PG_SHA256_DIGEST_LENGTH]; + uint8 b[PG_SHA256_DIGEST_LENGTH]; + pg_hmac_ctx *ctx; + + /* compute the HMAC twice over the same key and data */ + for (int pass = 0; pass < 2; pass++) + { + uint8 *dest = (pass == 0) ? a : b; + + ctx = pg_hmac_create(PG_SHA256); + if (ctx == NULL) + pg_prop_fail(tc, "pg_hmac_create failed"); + if (pg_hmac_init(ctx, key, klen) < 0 || + pg_hmac_update(ctx, data, dlen) < 0 || + pg_hmac_final(ctx, dest, PG_SHA256_DIGEST_LENGTH) < 0) + { + pg_hmac_free(ctx); + pg_prop_fail(tc, "hmac computation failed (klen=%zu dlen=%zu)", + klen, dlen); + } + pg_hmac_free(ctx); + } + + if (memcmp(a, b, PG_SHA256_DIGEST_LENGTH) != 0) + pg_prop_fail(tc, "hmac-sha256 not deterministic (klen=%zu dlen=%zu)", + klen, dlen); + + pg_free(data); + pg_free(key); +} diff --git a/src/test/prop/t/001_prop.pl b/src/test/prop/t/001_prop.pl index 2cadd5b0d34de..051c4552c03e1 100644 --- a/src/test/prop/t/001_prop.pl +++ b/src/test/prop/t/001_prop.pl @@ -25,7 +25,7 @@ # cluster. The SQL-oracle (Tier B) areas run from t/002_sql_props.pl instead. # # Keep in step with the Tier-A entries in prop_area_sources (meson.build). -my @areas = qw(path jsonapi encoding unicode_case unicode_norm pglz base64 prng); +my @areas = qw(path jsonapi encoding unicode_case unicode_norm pglz base64 prng crypto); # No Tier-A areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass, not a failure. From 2be79968a9ca3b6d84e4cbe5fa87b85ecce8cea7 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:41 -0400 Subject: [PATCH 013/338] prop: add scram property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_scram.c | 172 ++++++++++++++++++++++++++++++++++++ src/test/prop/t/001_prop.pl | 2 +- 4 files changed, 176 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_scram.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 5b38f92ffcf8c..203f3ef49fba9 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -28,7 +28,8 @@ OBJS = \ prop_pglz.o \ prop_base64.o \ prop_prng.o \ - prop_crypto.o + prop_crypto.o \ + prop_scram.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index e4e13604715e3..c96d684d032b4 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -26,6 +26,7 @@ prop_area_sources = files( 'prop_base64.c', 'prop_prng.c', 'prop_crypto.c', + 'prop_scram.c', ) diff --git a/src/test/prop/prop_scram.c b/src/test/prop/prop_scram.c new file mode 100644 index 0000000000000..627cd3e1d8e0c --- /dev/null +++ b/src/test/prop/prop_scram.c @@ -0,0 +1,172 @@ +/*------------------------------------------------------------------------- + * + * prop_scram.c + * Property-based tests for src/common/scram-common.c. + * + * SCRAM secret derivation (scram_SaltedPassword, scram_ClientKey, + * scram_ServerKey, scram_build_secret) is the heart of SCRAM-SHA-256 + * authentication. It is deterministic in all of its inputs -- password, + * salt, iteration count -- so the same inputs must yield the same keys, and a + * change in any input must change them. Getting this wrong is an + * authentication correctness and security problem, which makes it worth + * pinning down with properties. + * + * Tier A: in-process, no server needed. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_scram.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "scram" +#include "pg_prop.h" + +#include "common/cryptohash.h" +#include "common/scram-common.h" + +#define KEYLEN SCRAM_SHA_256_KEY_LEN + +/* + * Draw an ASCII password (SCRAM input is a UTF-8 string; keep it to printable + * ASCII so the SQL-free literal is unambiguous and SASLprep is a no-op). + */ +static char * +draw_password(pg_prop_tc *tc, size_t *len) +{ + return pg_prop_draw_from(tc, + "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789 ", + 0, 40, len); +} + +/* Compute the salted password for the given inputs; fail on any error. */ +static void +salted(pg_prop_tc *tc, const char *pw, const uint8 *salt, int saltlen, + int iters, uint8 *out) +{ + const char *errstr = NULL; + + if (scram_SaltedPassword(pw, PG_SHA256, KEYLEN, salt, saltlen, iters, + out, &errstr) < 0) + pg_prop_fail(tc, "scram_SaltedPassword failed: %s", + errstr ? errstr : "(no message)"); +} + +PG_PROP(salted_password_deterministic) +{ + size_t pwlen; + size_t saltlen; + char *pw = draw_password(tc, &pwlen); + unsigned char *salt = pg_prop_draw_bytes(tc, 1, 32, &saltlen); + int iters = (int) pg_prop_draw_int(tc, 1, 50); + uint8 a[KEYLEN]; + uint8 b[KEYLEN]; + + /* same password, salt, and iteration count -> same salted password */ + salted(tc, pw, salt, (int) saltlen, iters, a); + salted(tc, pw, salt, (int) saltlen, iters, b); + + if (memcmp(a, b, KEYLEN) != 0) + pg_prop_fail(tc, "salted password not deterministic (pwlen=%zu iters=%d)", + pwlen, iters); + + pg_free(salt); + pg_free(pw); +} + +PG_PROP(client_server_keys_deterministic_and_differ) +{ + size_t pwlen; + size_t saltlen; + char *pw = draw_password(tc, &pwlen); + unsigned char *salt = pg_prop_draw_bytes(tc, 1, 32, &saltlen); + int iters = (int) pg_prop_draw_int(tc, 1, 50); + uint8 sp[KEYLEN]; + uint8 ck[KEYLEN]; + uint8 sk[KEYLEN]; + uint8 ck2[KEYLEN]; + const char *errstr = NULL; + + salted(tc, pw, salt, (int) saltlen, iters, sp); + + if (scram_ClientKey(sp, PG_SHA256, KEYLEN, ck, &errstr) < 0 || + scram_ServerKey(sp, PG_SHA256, KEYLEN, sk, &errstr) < 0 || + scram_ClientKey(sp, PG_SHA256, KEYLEN, ck2, &errstr) < 0) + pg_prop_fail(tc, "scram key derivation failed: %s", + errstr ? errstr : "(no message)"); + + /* ClientKey is deterministic */ + if (memcmp(ck, ck2, KEYLEN) != 0) + pg_prop_fail(tc, "ClientKey not deterministic"); + + /* + * ClientKey and ServerKey are derived from the same salted password by + * different HMAC messages ("Client Key" vs "Server Key"), so they must + * differ -- if they were equal the protocol's separation would be broken. + */ + if (memcmp(ck, sk, KEYLEN) == 0) + pg_prop_fail(tc, "ClientKey equals ServerKey"); + + pg_free(salt); + pg_free(pw); +} + +PG_PROP(salt_change_changes_salted_password) +{ + size_t pwlen; + size_t saltlen; + char *pw = draw_password(tc, &pwlen); + unsigned char *salt = pg_prop_draw_bytes(tc, 1, 32, &saltlen); + int iters = (int) pg_prop_draw_int(tc, 1, 50); + size_t pos = 0; + uint8 a[KEYLEN]; + uint8 b[KEYLEN]; + + salted(tc, pw, salt, (int) saltlen, iters, a); + + /* flip one bit of the salt; the salted password must change */ + pos = (size_t) pg_prop_draw_int(tc, 0, (int) saltlen - 1); + salt[pos] ^= 0x01; + salted(tc, pw, salt, (int) saltlen, iters, b); + + if (memcmp(a, b, KEYLEN) == 0) + pg_prop_fail(tc, "salted password unchanged after altering salt byte %zu", + pos); + + pg_free(salt); + pg_free(pw); +} + +PG_PROP(build_secret_nonempty_and_stable_shape) +{ + size_t pwlen; + size_t saltlen; + char *pw = draw_password(tc, &pwlen); + unsigned char *salt = pg_prop_draw_bytes(tc, 1, 32, &saltlen); + int iters = (int) pg_prop_draw_int(tc, 1, 50); + const char *errstr = NULL; + char *s1; + char *s2; + + /* + * scram_build_secret is deterministic in (hash, key_length, salt, + * iterations, password): two calls with identical inputs yield identical + * SCRAM verifier strings. + */ + s1 = scram_build_secret(PG_SHA256, KEYLEN, salt, (int) saltlen, iters, pw, &errstr); + s2 = scram_build_secret(PG_SHA256, KEYLEN, salt, (int) saltlen, iters, pw, &errstr); + if (s1 == NULL || s2 == NULL) + pg_prop_fail(tc, "scram_build_secret failed: %s", + errstr ? errstr : "(no message)"); + + if (strcmp(s1, s2) != 0) + pg_prop_fail(tc, "scram_build_secret not deterministic:\n %s\n %s", s1, s2); + + free(s1); + free(s2); + pg_free(salt); + pg_free(pw); +} diff --git a/src/test/prop/t/001_prop.pl b/src/test/prop/t/001_prop.pl index 051c4552c03e1..e14ba539c1ba6 100644 --- a/src/test/prop/t/001_prop.pl +++ b/src/test/prop/t/001_prop.pl @@ -25,7 +25,7 @@ # cluster. The SQL-oracle (Tier B) areas run from t/002_sql_props.pl instead. # # Keep in step with the Tier-A entries in prop_area_sources (meson.build). -my @areas = qw(path jsonapi encoding unicode_case unicode_norm pglz base64 prng crypto); +my @areas = qw(path jsonapi encoding unicode_case unicode_norm pglz base64 prng crypto scram); # No Tier-A areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass, not a failure. From db4d292370a9f53c54ae36715bbb03becd69bd05 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:42 -0400 Subject: [PATCH 014/338] prop: add binaryheap property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_binaryheap.c | 165 ++++++++++++++++++++++++++++++++ src/test/prop/t/001_prop.pl | 2 +- 4 files changed, 169 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_binaryheap.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 203f3ef49fba9..8065a8af6fb89 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -29,7 +29,8 @@ OBJS = \ prop_base64.o \ prop_prng.o \ prop_crypto.o \ - prop_scram.o + prop_scram.o \ + prop_binaryheap.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index c96d684d032b4..04f9c1f8cdaf5 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -27,6 +27,7 @@ prop_area_sources = files( 'prop_prng.c', 'prop_crypto.c', 'prop_scram.c', + 'prop_binaryheap.c', ) diff --git a/src/test/prop/prop_binaryheap.c b/src/test/prop/prop_binaryheap.c new file mode 100644 index 0000000000000..af3d83d5b51e5 --- /dev/null +++ b/src/test/prop/prop_binaryheap.c @@ -0,0 +1,165 @@ +/*------------------------------------------------------------------------- + * + * prop_binaryheap.c + * Property-based tests for src/common/binaryheap.c. + * + * binaryheap is the shared binary heap used by merge append, the executor's + * bounded sorts, and more. A heap has a crisp model: a priority queue. The + * highest-value first test for any data structure is a model test -- drive + * the structure and a trusted reference (here, a sorted array) through the + * same operations and require they agree. These tests do exactly that for + * the add / first / remove_first / build operations. + * + * Tier A: in-process, no server needed. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_binaryheap.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "binaryheap" +#include "pg_prop.h" + +#include "lib/binaryheap.h" + +/* + * In frontend builds bh_node_type is void *; we store int64 values cast + * through intptr_t. A min-heap comparator: returns >0 when a should come out + * before b (binaryheap is a max-heap on the comparator, so to pop ascending + * we make "smaller is greater"). + */ +static int +cmp_min(bh_node_type a, bh_node_type b, void *arg) +{ + int64 x = (int64) (intptr_t) a; + int64 y = (int64) (intptr_t) b; + + /* max-heap pops the comparator-largest first; invert for ascending pops */ + if (x < y) + return 1; + if (x > y) + return -1; + return 0; +} + +static int +cmp_i64(const void *a, const void *b) +{ + int64 x = *(const int64 *) a; + int64 y = *(const int64 *) b; + + return (x > y) - (x < y); +} + +/* + * Draw n values into a caller-provided array, bounded so they fit in intptr_t + * on any platform (use int32 range, widened to int64). + */ +static int +draw_values(pg_prop_tc *tc, int64 *vals, int maxn) +{ + int n = (int) pg_prop_draw_int(tc, 0, maxn); + + for (int i = 0; i < n; i++) + vals[i] = pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + return n; +} + +PG_PROP(pop_order_is_sorted) +{ + int64 vals[256]; + int64 sorted[256]; + int n = draw_values(tc, vals, 200); + binaryheap *heap = binaryheap_allocate(n > 0 ? n : 1, cmp_min, NULL); + + /* add every value, then pop them all: the pops must come out ascending */ + for (int i = 0; i < n; i++) + binaryheap_add(heap, (bh_node_type) (intptr_t) vals[i]); + + memcpy(sorted, vals, sizeof(int64) * n); + qsort(sorted, n, sizeof(int64), cmp_i64); + + for (int i = 0; i < n; i++) + { + int64 got; + + if (binaryheap_empty(heap)) + pg_prop_fail(tc, "heap empty after %d of %d pops", i, n); + got = (int64) (intptr_t) binaryheap_remove_first(heap); + if (got != sorted[i]) + pg_prop_fail(tc, "pop %d: got %lld, expected %lld (n=%d)", + i, (long long) got, (long long) sorted[i], n); + } + if (!binaryheap_empty(heap)) + pg_prop_fail(tc, "heap not empty after %d pops", n); + + binaryheap_free(heap); +} + +PG_PROP(first_is_minimum) +{ + int64 vals[256]; + int n; + binaryheap *heap; + int64 expected_min = 0; + + /* need at least one element for binaryheap_first */ + n = (int) pg_prop_draw_int(tc, 1, 200); + vals[0] = 0; /* silence -Wmaybe-uninitialized; n >= 1 fills it */ + for (int i = 0; i < n; i++) + vals[i] = pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + heap = binaryheap_allocate(n, cmp_min, NULL); + for (int i = 0; i < n; i++) + binaryheap_add(heap, (bh_node_type) (intptr_t) vals[i]); + + expected_min = vals[0]; + for (int i = 1; i < n; i++) + if (vals[i] < expected_min) + expected_min = vals[i]; + + { + int64 first = (int64) (intptr_t) binaryheap_first(heap); + + if (first != expected_min) + pg_prop_fail(tc, "first = %lld, expected min %lld (n=%d)", + (long long) first, (long long) expected_min, n); + } + binaryheap_free(heap); +} + +PG_PROP(build_matches_incremental_add) +{ + int64 vals[256]; + int n = draw_values(tc, vals, 200); + binaryheap *h1 = binaryheap_allocate(n > 0 ? n : 1, cmp_min, NULL); + binaryheap *h2 = binaryheap_allocate(n > 0 ? n : 1, cmp_min, NULL); + + /* + * Two ways to construct the same heap: incremental add (sift-up each), and + * add_unordered followed by binaryheap_build (Floyd's bulk heapify). They + * must pop in the same (sorted) order. + */ + for (int i = 0; i < n; i++) + binaryheap_add(h1, (bh_node_type) (intptr_t) vals[i]); + + for (int i = 0; i < n; i++) + binaryheap_add_unordered(h2, (bh_node_type) (intptr_t) vals[i]); + binaryheap_build(h2); + + for (int i = 0; i < n; i++) + { + int64 a = (int64) (intptr_t) binaryheap_remove_first(h1); + int64 b = (int64) (intptr_t) binaryheap_remove_first(h2); + + if (a != b) + pg_prop_fail(tc, "pop %d: incremental=%lld build=%lld (n=%d)", + i, (long long) a, (long long) b, n); + } + binaryheap_free(h1); + binaryheap_free(h2); +} diff --git a/src/test/prop/t/001_prop.pl b/src/test/prop/t/001_prop.pl index e14ba539c1ba6..497359932133a 100644 --- a/src/test/prop/t/001_prop.pl +++ b/src/test/prop/t/001_prop.pl @@ -25,7 +25,7 @@ # cluster. The SQL-oracle (Tier B) areas run from t/002_sql_props.pl instead. # # Keep in step with the Tier-A entries in prop_area_sources (meson.build). -my @areas = qw(path jsonapi encoding unicode_case unicode_norm pglz base64 prng crypto scram); +my @areas = qw(path jsonapi encoding unicode_case unicode_norm pglz base64 prng crypto scram binaryheap); # No Tier-A areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass, not a failure. From 0fff3d4b25c380d6799b03e6e65dd61b50dc0ef5 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:42 -0400 Subject: [PATCH 015/338] prop: add bitutils property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_bitutils.c | 129 ++++++++++++++++++++++++++++++++++ src/test/prop/t/001_prop.pl | 2 +- 4 files changed, 133 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_bitutils.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 8065a8af6fb89..b311d457a01b1 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -30,7 +30,8 @@ OBJS = \ prop_prng.o \ prop_crypto.o \ prop_scram.o \ - prop_binaryheap.o + prop_binaryheap.o \ + prop_bitutils.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 04f9c1f8cdaf5..50e92fdfa50ca 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -28,6 +28,7 @@ prop_area_sources = files( 'prop_crypto.c', 'prop_scram.c', 'prop_binaryheap.c', + 'prop_bitutils.c', ) diff --git a/src/test/prop/prop_bitutils.c b/src/test/prop/prop_bitutils.c new file mode 100644 index 0000000000000..24b316df1224b --- /dev/null +++ b/src/test/prop/prop_bitutils.c @@ -0,0 +1,129 @@ +/*------------------------------------------------------------------------- + * + * prop_bitutils.c + * Property-based tests for src/port/pg_bitutils.c. + * + * The bit utilities -- population count, leftmost/rightmost set-bit position -- + * back the buffer manager, bitmapsets, visibility maps, and more. Each has a + * dispatch path (lookup table, intrinsic, or runtime-selected SIMD) that must + * agree with a trivial reference loop for every input. SIMD-vs-scalar + * divergence is a classic bug class, which makes these worth pinning down + * against an independent oracle. + * + * Tier A: in-process, no server needed. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_bitutils.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "bitutils" +#include "pg_prop.h" + +#include "port/pg_bitutils.h" + +/* Reference popcount: count set bits one at a time. */ +static int +ref_popcount64(uint64 w) +{ + int n = 0; + + while (w) + { + n += (int) (w & 1); + w >>= 1; + } + return n; +} + +PG_PROP(popcount32_matches_reference) +{ + uint32 w = (uint32) pg_prop_draw_int(tc, 0, UINT32_MAX); + + if ((int) pg_popcount32(w) != ref_popcount64(w)) + pg_prop_fail(tc, "pg_popcount32(%u) = %d, reference = %d", + w, (int) pg_popcount32(w), ref_popcount64(w)); +} + +PG_PROP(popcount64_matches_reference) +{ + uint64 w = (uint64) pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + + if ((int) pg_popcount64(w) != ref_popcount64(w)) + pg_prop_fail(tc, "pg_popcount64(%llu) = %d, reference = %d", + (unsigned long long) w, (int) pg_popcount64(w), + ref_popcount64(w)); +} + +PG_PROP(popcount_buffer_matches_reference) +{ + size_t len; + unsigned char *buf = pg_prop_draw_bytes(tc, 0, 256, &len); + uint64 got; + int ref = 0; + + /* exercise the multi-byte dispatch path (table / intrinsic / SIMD) */ + got = pg_popcount((const char *) buf, (int) len); + for (size_t i = 0; i < len; i++) + ref += ref_popcount64(buf[i]); + + if ((int) got != ref) + pg_prop_fail(tc, "pg_popcount(len=%zu) = %llu, reference = %d", + len, (unsigned long long) got, ref); + + pg_free(buf); +} + +PG_PROP(leftmost_one_pos32_matches_reference) +{ + uint32 w = (uint32) pg_prop_draw_int(tc, 1, UINT32_MAX); + int ref; + int got; + + /* pg_leftmost_one_pos32 is undefined for 0, so draw >= 1 */ + pg_prop_assume(tc, w != 0); + + /* reference: index of the highest set bit */ + ref = 0; + for (int i = 31; i >= 0; i--) + { + if (w & (1u << i)) + { + ref = i; + break; + } + } + got = pg_leftmost_one_pos32(w); + if (got != ref) + pg_prop_fail(tc, "pg_leftmost_one_pos32(%u) = %d, reference = %d", + w, got, ref); +} + +PG_PROP(rightmost_one_pos64_matches_reference) +{ + uint64 w = (uint64) pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + int ref; + int got; + + /* pg_rightmost_one_pos64 is undefined for 0 */ + pg_prop_assume(tc, w != 0); + + /* reference: index of the lowest set bit */ + ref = 0; + for (int i = 0; i < 64; i++) + { + if (w & (UINT64CONST(1) << i)) + { + ref = i; + break; + } + } + got = pg_rightmost_one_pos64(w); + if (got != ref) + pg_prop_fail(tc, "pg_rightmost_one_pos64(%llu) = %d, reference = %d", + (unsigned long long) w, got, ref); +} diff --git a/src/test/prop/t/001_prop.pl b/src/test/prop/t/001_prop.pl index 497359932133a..5a4b787c43129 100644 --- a/src/test/prop/t/001_prop.pl +++ b/src/test/prop/t/001_prop.pl @@ -25,7 +25,7 @@ # cluster. The SQL-oracle (Tier B) areas run from t/002_sql_props.pl instead. # # Keep in step with the Tier-A entries in prop_area_sources (meson.build). -my @areas = qw(path jsonapi encoding unicode_case unicode_norm pglz base64 prng crypto scram binaryheap); +my @areas = qw(path jsonapi encoding unicode_case unicode_norm pglz base64 prng crypto scram binaryheap bitutils); # No Tier-A areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass, not a failure. From fac00f3bd9bd19279c13df0886c0eb16304d074c Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:42 -0400 Subject: [PATCH 016/338] prop: add numeric property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_numeric.c | 326 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 330 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_numeric.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index b311d457a01b1..f4a2d3e94d6b4 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -31,7 +31,8 @@ OBJS = \ prop_crypto.o \ prop_scram.o \ prop_binaryheap.o \ - prop_bitutils.o + prop_bitutils.o \ + prop_numeric.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 50e92fdfa50ca..868b3f3acb519 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -29,6 +29,7 @@ prop_area_sources = files( 'prop_scram.c', 'prop_binaryheap.c', 'prop_bitutils.c', + 'prop_numeric.c', ) diff --git a/src/test/prop/prop_numeric.c b/src/test/prop/prop_numeric.c new file mode 100644 index 0000000000000..6a3b810be01b4 --- /dev/null +++ b/src/test/prop/prop_numeric.c @@ -0,0 +1,326 @@ +/*------------------------------------------------------------------------- + * + * prop_numeric.c + * Property-based tests for arbitrary-precision NUMERIC arithmetic. + * + * Target #9 in the design. The 2024 numeric multiplication/division rewrite + * (ca481d3c9ab, c4e44224cf6, 8dc28d7eb86, 9428c001f67) reworked carry + * handling in mul_var, mul_var_short and div_var -- the base-NBASE^2 + * accumulation path and the "short" multiply paths that handle 1..6 NBASE + * digits. Carry/length bugs in those routines show up as broken algebraic + * identities, so these properties check exact identities (commutativity, + * distributivity, the a = q*b + r division identity, additive inverse) in SQL + * against the numeric type itself, plus a sound bound on sqrt. + * + * The oracle here is the type's own algebra: numeric add/subtract/multiply + * and trunc-division (div/mod) are exact at the operands' combined scale, so + * these identities must hold with no tolerance. Any "f" result is a bug in + * the arithmetic, not a rounding artefact. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_numeric.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "numeric" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a decimal NUMERIC literal as text into buf, parenthesized so a leading + * sign never misparses when the literal is substituted into a larger + * expression. + * + * Digit counts straddle the boundaries the mul rewrite cares about: one NBASE + * digit is 4 decimal digits, and mul_var_short handles 1..6 NBASE digits + * (~24 integer decimal digits), so the integer part is drawn up to 24 digits + * to cross both the NBASE and NBASE^2 length thresholds. Products of two + * 24-digit values are ~48 digits, far below the numeric limit (10^131072), so + * there is no overflow to gate on. + */ +static void +draw_numeric(pg_prop_tc *tc, StringInfo buf) +{ + int nint = (int) pg_prop_draw_int(tc, 0, 24); + int nfrac = (int) pg_prop_draw_int(tc, 0, 8); + bool neg = pg_prop_draw_bool(tc); + int ndigits = 0; + + appendStringInfoString(buf, "("); + if (neg) + appendStringInfoChar(buf, '-'); + + /* integer part */ + for (int i = 0; i < nint; i++) + { + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + + /* fractional part */ + if (nfrac > 0) + { + appendStringInfoChar(buf, '.'); + for (int i = 0; i < nfrac; i++) + { + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + } + + /* ensure a valid numeric: at least one digit */ + if (ndigits == 0) + appendStringInfoChar(buf, '0'); + + appendStringInfoString(buf, ")::numeric"); +} + +/* + * Draw a NUMERIC literal forced non-negative (positive sign), for sqrt(). + */ +static void +draw_numeric_nonneg(pg_prop_tc *tc, StringInfo buf) +{ + int nint = (int) pg_prop_draw_int(tc, 0, 24); + int nfrac = (int) pg_prop_draw_int(tc, 0, 8); + int ndigits = 0; + + appendStringInfoString(buf, "("); + for (int i = 0; i < nint; i++) + { + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + if (nfrac > 0) + { + appendStringInfoChar(buf, '.'); + for (int i = 0; i < nfrac; i++) + { + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + } + if (ndigits == 0) + appendStringInfoChar(buf, '0'); + appendStringInfoString(buf, ")::numeric"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the arithmetic errored or produced NULL on finite inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Multiplication commutes: X * Y = Y * X. The base-NBASE^2 rewrite reorders + * accumulation, so a carry bug would surface as asymmetry here. + */ +PG_PROP(mul_commutes) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + y, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&y); + draw_numeric(tc, &x); + draw_numeric(tc, &y); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s * %s) = (%s * %s)", + x.data, y.data, y.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(y.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The fundamental trunc-division identity: div(X, Y) * Y + mod(X, Y) = X. + * div() is integer/trunc division for numeric and mod() its matching + * remainder; this directly exercises div_var. The CASE guards Y = 0 so no + * division-by-zero error is expected -- if one still occurs, require_true + * fails on it. + */ +PG_PROP(div_mod_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + y, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&y); + draw_numeric(tc, &x); + draw_numeric(tc, &y); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN %s = 0 THEN true " + "ELSE div(%s, %s) * %s + mod(%s, %s) = %s END", + y.data, + x.data, y.data, y.data, x.data, y.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(y.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Additive inverse: (X + Y) - Y = X. Numeric add/subtract are exact at the + * operands' scale, so this holds with no rounding. + */ +PG_PROP(add_sub_inverse) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + y, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&y); + draw_numeric(tc, &x); + draw_numeric(tc, &y); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s + %s) - %s) = %s", + x.data, y.data, y.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(y.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Distributivity: X * (Y + Z) = X*Y + X*Z. A carry/length bug in mul_var + * often breaks this when one operand crosses a digit-count boundary while the + * other does not, so the two sides accumulate carries differently. + */ +PG_PROP(mul_distributes_over_add) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + y, + z, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&y); + initStringInfo(&z); + draw_numeric(tc, &x); + draw_numeric(tc, &y); + draw_numeric(tc, &z); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s * (%s + %s)) = (%s * %s + %s * %s)", + x.data, y.data, z.data, + x.data, y.data, x.data, z.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(y.data); + pfree(z.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * sqrt bound: S = sqrt(abs(X)) is non-negative and, since numeric sqrt rounds + * to a scale, S*S must not exceed abs(X) by more than a small relative-plus- + * absolute tolerance. This is a sound one-sided bound: an exact sqrt gives + * S*S = X, and the only slack is the rounding of the last digit, well within + * abs(X) * 1e-10 + 1e-10. (The symmetric lower bound is scale-dependent and + * fiddly to state soundly, so we assert only the clean upper side plus the + * sign.) + */ +PG_PROP(sqrt_bounds) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric_nonneg(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT sqrt(abs(%s)) >= 0 " + "AND sqrt(abs(%s)) * sqrt(abs(%s)) " + "<= abs(%s) * (1 + 1e-10::numeric) + 1e-10::numeric", + x.data, x.data, x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 6ff6a0740baef..93cb247313c89 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(); +my @areas = qw(numeric); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 8e9b3536702a0955daf6968ae20870e8ac0e60e5 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:43 -0400 Subject: [PATCH 017/338] prop: add numeric cross-type cast round-trips and widening property tests (numcast) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_numcast.c | 415 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 419 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_numcast.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index f4a2d3e94d6b4..7da53099d9562 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -32,7 +32,8 @@ OBJS = \ prop_scram.o \ prop_binaryheap.o \ prop_bitutils.o \ - prop_numeric.o + prop_numeric.o \ + prop_numcast.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 868b3f3acb519..9b5da0a05e767 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -30,6 +30,7 @@ prop_area_sources = files( 'prop_binaryheap.c', 'prop_bitutils.c', 'prop_numeric.c', + 'prop_numcast.c', ) diff --git a/src/test/prop/prop_numcast.c b/src/test/prop/prop_numcast.c new file mode 100644 index 0000000000000..22b9ee833f416 --- /dev/null +++ b/src/test/prop/prop_numcast.c @@ -0,0 +1,415 @@ +/*------------------------------------------------------------------------- + * + * prop_numcast.c + * Property-based tests for numeric cross-type casts + * (int2/int4/int8 <-> numeric <-> float8, and text). + * + * Casts between the numeric types must preserve value where the target can + * represent it and round/overflow predictably otherwise; these check the + * lossless round-trips and the documented rounding. + * + * numeric exactly represents every integer; float8 exactly represents the + * integers with |x| < 2^53 (so an int4 is always safe, but an int8 is not -- + * only int4 is used for the float round-trip). The numeric->integer cast + * rounds to nearest, half away from zero, identically to round(numeric), so + * round() is the oracle for the rounding property. Narrowing casts + * (int8->int2, etc.) raise an out-of-range error rather than silently + * wrapping, so an overflow there is a legitimate skip, not a wrong result. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_numcast.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "numcast" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * True when a SQL error is a legitimate out-of-range/overflow condition rather + * than a wrong-results bug. A narrowing cast that does not fit the target + * type raises such an error (never wraps), so those cases are discarded. + */ +static bool +is_overflow(const char *errmsg) +{ + if (errmsg == NULL) + return false; + return strstr(errmsg, "out of range") != NULL || + strstr(errmsg, "overflow") != NULL; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the cast errored or produced NULL on finite inputs, which + * is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Like require_true, but a SQL error that is a legitimate overflow/out-of-range + * condition discards the case (pg_prop_assume(false)) instead of failing. Used + * where a narrowing cast may legally not fit: the property is that it either + * fits exactly or errors -- it must never silently wrap. + */ +static void +require_true_or_skip(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + if (is_overflow(err)) + { + pg_prop_note(tc, "overflow (skipped): %s", err); + if (err) + pg_free(err); + pg_prop_assume(tc, false); + return; /* unreachable: assume(false) longjmps */ + } + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Emit an integer literal safe for immediate casting. A bare negative like + * -2147483648::int4 parses as -(2147483648::int4), and the positive magnitude + * overflows the target type before negation; the same bites INT16_MIN::int2 + * and INT32_MIN::int4. Wrapping any negative in parentheses makes the cast + * apply to the already-negative value. INT64_MIN additionally cannot be + * written as a positive literal at all, so it is emitted via subtraction. + */ +static void +append_int_literal(StringInfo buf, int64 n) +{ + if (n == INT64_MIN) + appendStringInfoString(buf, "(-9223372036854775807 - 1)"); + else if (n < 0) + appendStringInfo(buf, "(%lld)", (long long) n); + else + appendStringInfo(buf, "%lld", (long long) n); +} + +/* + * 1. int_numeric_roundtrip + * + * numeric exactly represents every int8, so n::numeric::int8 = n for the full + * int64 range. + */ +PG_PROP(int_numeric_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_int_literal(&q, n); + appendStringInfoString(&q, "::int8::numeric::int8) = "); + append_int_literal(&q, n); + appendStringInfoString(&q, "::int8"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. int4_numeric_float_roundtrip + * + * Every int4 has |n| < 2^53, so it is exactly representable in float8, and + * n::float8::int4 = n with no loss. + */ +PG_PROP(int4_numeric_float_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_int_literal(&q, n); + appendStringInfoString(&q, "::int4::float8::int4) = "); + append_int_literal(&q, n); + appendStringInfoString(&q, "::int4"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. numeric_to_int_rounds_half_away + * + * The numeric->integer cast rounds to nearest, half away from zero, which is + * exactly what round(numeric) does. For a numeric X with a fractional part, + * X::bigint = round(X)::bigint. X is built with a modest integer part so no + * overflow is possible. + */ +PG_PROP(numeric_to_int_rounds_half_away) +{ + pg_prop_conn *conn; + char *err = NULL; + int nint; + int nfrac; + bool neg; + int ndigits = 0; + StringInfoData x; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* integer part up to 9 digits stays well inside int8; fractional 1..8 */ + nint = (int) pg_prop_draw_int(tc, 0, 9); + nfrac = (int) pg_prop_draw_int(tc, 1, 8); + neg = pg_prop_draw_bool(tc); + + initStringInfo(&x); + appendStringInfoChar(&x, '('); + if (neg) + appendStringInfoChar(&x, '-'); + for (int i = 0; i < nint; i++) + { + appendStringInfoChar(&x, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + if (ndigits == 0) + appendStringInfoChar(&x, '0'); + appendStringInfoChar(&x, '.'); + for (int i = 0; i < nfrac; i++) + appendStringInfoChar(&x, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + appendStringInfoChar(&x, ')'); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s)::numeric::bigint = round(%s::numeric)::bigint", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. float_to_numeric_roundtrip_small + * + * A float8 that is an exact integer within int4 range round-trips through + * numeric: n::float8::numeric::float8 = n::float8. + */ +PG_PROP(float_to_numeric_roundtrip_small) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_int_literal(&q, n); + appendStringInfoString(&q, "::float8::numeric::float8) = "); + append_int_literal(&q, n); + appendStringInfoString(&q, "::float8"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. int_widening_lossless + * + * int2 -> int4 -> int8 -> numeric widening never changes the value: for an + * int2 m, that chain equals m::int2::numeric directly. + */ +PG_PROP(int_widening_lossless) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 m; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + m = pg_prop_draw_int(tc, INT16_MIN, INT16_MAX); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_int_literal(&q, m); + appendStringInfoString(&q, "::int2::int4::int8::numeric) = "); + append_int_literal(&q, m); + appendStringInfoString(&q, "::int2::numeric"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. numeric_text_roundtrip + * + * A numeric formatted to text and parsed back is unchanged: + * X::text::numeric = X. + */ +PG_PROP(numeric_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int nint; + int nfrac; + bool neg; + int ndigits = 0; + StringInfoData x; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + nint = (int) pg_prop_draw_int(tc, 0, 24); + nfrac = (int) pg_prop_draw_int(tc, 0, 8); + neg = pg_prop_draw_bool(tc); + + initStringInfo(&x); + appendStringInfoChar(&x, '('); + if (neg) + appendStringInfoChar(&x, '-'); + for (int i = 0; i < nint; i++) + { + appendStringInfoChar(&x, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + if (nfrac > 0) + { + appendStringInfoChar(&x, '.'); + for (int i = 0; i < nfrac; i++) + { + appendStringInfoChar(&x, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + } + if (ndigits == 0) + appendStringInfoChar(&x, '0'); + appendStringInfoChar(&x, ')'); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s)::numeric::text::numeric = (%s)::numeric", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. int_overflow_on_narrowing_detected + * + * Narrowing a large int8 to int2 either fits exactly or raises an out-of-range + * error -- it must never silently wrap. When it fits the identity is trivially + * true; when it overflows the cast errors and the case is skipped via + * require_true_or_skip. + */ +PG_PROP(int_overflow_on_narrowing_detected) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_int_literal(&q, n); + appendStringInfoString(&q, "::int8::int2) = "); + append_int_literal(&q, n); + appendStringInfoString(&q, "::int8::int2"); + + require_true_or_skip(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 93cb247313c89..5131078f9f0f7 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric); +my @areas = qw(numeric numcast); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 1fe79a80b9da4133b8f567b19a7b7cb780ae0cf0 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:43 -0400 Subject: [PATCH 018/338] prop: add numeric scale, precision, and typmod rounding property tests (numscale) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_numscale.c | 367 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 371 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_numscale.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 7da53099d9562..64106763398f1 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -33,7 +33,8 @@ OBJS = \ prop_binaryheap.o \ prop_bitutils.o \ prop_numeric.o \ - prop_numcast.o + prop_numcast.o \ + prop_numscale.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 9b5da0a05e767..486b3fe9b6a60 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -31,6 +31,7 @@ prop_area_sources = files( 'prop_bitutils.c', 'prop_numeric.c', 'prop_numcast.c', + 'prop_numscale.c', ) diff --git a/src/test/prop/prop_numscale.c b/src/test/prop/prop_numscale.c new file mode 100644 index 0000000000000..581acc1457844 --- /dev/null +++ b/src/test/prop/prop_numscale.c @@ -0,0 +1,367 @@ +/*------------------------------------------------------------------------- + * + * prop_numscale.c + * Property-based tests for NUMERIC scale/precision/typmod behaviour. + * + * Distinct from prop_numeric.c (arithmetic algebra): this file targets the + * scale and precision machinery -- rounding a value when it is cast to + * numeric(p,s), the scale() introspection function, trunc() at a given + * scale, and the idempotence/round-trip of a typmod-constrained value. The + * oracle is each operation's own contract: a cast to numeric(p,s) must equal + * round(x,s), truncation toward zero must never increase magnitude, and a + * value already at scale s must survive a second cast and a text round-trip + * unchanged. A false result is a bug in apply_typmod / scale handling. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_numscale.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "numscale" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a decimal NUMERIC literal as ".", parenthesized so a + * leading sign never misparses when substituted into a larger expression. + * + * The integer part is kept modest (up to 6 digits, so < 10^6) and the + * fractional part up to 6 digits, so the value always fits comfortably in + * numeric(20,s) for s <= 5 (14 integer digits of headroom). This keeps every + * cast in these properties free of precision overflow, which is what we want: + * we are testing rounding/scale, not the overflow error path. + */ +static void +draw_numeric(pg_prop_tc *tc, StringInfo buf) +{ + int nint = (int) pg_prop_draw_int(tc, 0, 6); + int nfrac = (int) pg_prop_draw_int(tc, 0, 6); + bool neg = pg_prop_draw_bool(tc); + int ndigits = 0; + + appendStringInfoString(buf, "("); + if (neg) + appendStringInfoChar(buf, '-'); + + for (int i = 0; i < nint; i++) + { + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + + if (nfrac > 0) + { + appendStringInfoChar(buf, '.'); + for (int i = 0; i < nfrac; i++) + { + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + } + + if (ndigits == 0) + appendStringInfoChar(buf, '0'); + + appendStringInfoString(buf, ")::numeric"); +} + +/* + * Draw a NUMERIC literal forced non-negative (no sign), for properties whose + * invariant is stated only for X >= 0. + */ +static void +draw_numeric_nonneg(pg_prop_tc *tc, StringInfo buf) +{ + int nint = (int) pg_prop_draw_int(tc, 0, 6); + int nfrac = (int) pg_prop_draw_int(tc, 0, 6); + int ndigits = 0; + + appendStringInfoString(buf, "("); + for (int i = 0; i < nint; i++) + { + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + if (nfrac > 0) + { + appendStringInfoChar(buf, '.'); + for (int i = 0; i < nfrac; i++) + { + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + } + if (ndigits == 0) + appendStringInfoChar(buf, '0'); + appendStringInfoString(buf, ")::numeric"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operation errored or produced NULL on finite inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Casting to numeric(20,2) rounds to 2 fractional digits: X::numeric(20,2) + * must equal round(X, 2). Precision 20 is far larger than any drawn value + * needs, so the cast cannot overflow and only the scale rounding is exercised + * -- it must agree with the round() function to the digit. + */ +PG_PROP(cast_to_scale_rounds) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)::numeric(20,2) = round(%s, 2)", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Rounding to s fractional digits yields a value whose scale is at most s: + * scale(round(X, s)) <= s. (It can be less -- trailing zeros are dropped -- + * so the bound is one-sided.) s is drawn in 0..5. + */ +PG_PROP(scale_of_rounded) +{ + pg_prop_conn *conn; + char *err = NULL; + int s; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + s = (int) pg_prop_draw_int(tc, 0, 5); + initStringInfo(&x); + draw_numeric(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT scale(round(%s, %d)) <= %d", + x.data, s, s); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Truncation toward zero never increases magnitude: abs(trunc(X, s)) <= + * abs(X). trunc() drops digits beyond scale s toward zero, so the result's + * absolute value can only shrink or stay equal. s is drawn in 0..5. + */ +PG_PROP(trunc_le_abs) +{ + pg_prop_conn *conn; + char *err = NULL; + int s; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + s = (int) pg_prop_draw_int(tc, 0, 5); + initStringInfo(&x); + draw_numeric(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT abs(trunc(%s, %d)) <= abs(%s)", + x.data, s, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Casting to a fixed scale is idempotent: once a value is at scale 3, casting + * it to numeric(20,3) again changes nothing. (X::numeric(20,3))::numeric(20,3) + * must equal X::numeric(20,3). + */ +PG_PROP(cast_scale_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s)::numeric(20,3))::numeric(20,3) " + "= (%s)::numeric(20,3)", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Round brackets truncation for X >= 0: trunc(X,s) <= round(X,s) <= + * trunc(X,s) + 10^(-s). For non-negative X, truncation drops the tail and + * rounding can only push up by less than one unit in the last place, so the + * rounded value lies in [trunc, trunc + ulp]. X is drawn non-negative; s in + * 0..5. + */ +PG_PROP(round_trunc_relation) +{ + pg_prop_conn *conn; + char *err = NULL; + int s; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + s = (int) pg_prop_draw_int(tc, 0, 5); + initStringInfo(&x); + draw_numeric_nonneg(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT trunc(%s, %d) <= round(%s, %d) " + "AND round(%s, %d) <= trunc(%s, %d) " + "+ (10::numeric ^ (-%d))", + x.data, s, x.data, s, + x.data, s, x.data, s, s); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A scale-constrained value survives a text round-trip: rendering + * X::numeric(20,4) to text and parsing it back as numeric must reproduce the + * same value. (X::numeric(20,4))::text::numeric = X::numeric(20,4). + */ +PG_PROP(precision_preserved_text) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s)::numeric(20,4))::text::numeric " + "= (%s)::numeric(20,4)", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The scale of a value cast to numeric(20,s) is bounded by its typmod: + * scale(X::numeric(20,s)) is between 0 and s. The cast normalises to at most + * s fractional digits and scale() is never negative, so the result sits in + * [0, s]. s is drawn in 0..5. + */ +PG_PROP(scale_nonneg) +{ + pg_prop_conn *conn; + char *err = NULL; + int s; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + s = (int) pg_prop_draw_int(tc, 0, 5); + initStringInfo(&x); + draw_numeric(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT scale((%s)::numeric(20,%d)) BETWEEN 0 AND %d", + x.data, s, s); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 5131078f9f0f7..105a0c3026e2a 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast); +my @areas = qw(numeric numcast numscale); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 0c2aecdb93e12ea4606d89547415265cc6e7dd43 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:43 -0400 Subject: [PATCH 019/338] prop: add numround property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_numround.c | 367 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 371 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_numround.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 64106763398f1..8e30b3162415f 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -34,7 +34,8 @@ OBJS = \ prop_bitutils.o \ prop_numeric.o \ prop_numcast.o \ - prop_numscale.o + prop_numscale.o \ + prop_numround.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 486b3fe9b6a60..d7965bf642c4c 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -32,6 +32,7 @@ prop_area_sources = files( 'prop_numeric.c', 'prop_numcast.c', 'prop_numscale.c', + 'prop_numround.c', ) diff --git a/src/test/prop/prop_numround.c b/src/test/prop/prop_numround.c new file mode 100644 index 0000000000000..0a90e0f0df737 --- /dev/null +++ b/src/test/prop/prop_numround.c @@ -0,0 +1,367 @@ +/*------------------------------------------------------------------------- + * + * prop_numround.c + * Property-based tests for NUMERIC rounding and scaling. + * + * NUMERIC round/trunc/ceil/floor and scale clamping (fixed in 1ff39f4ff2b) + * have exact bound and ordering properties; these pin them down, distinct + * from the arithmetic-identity numeric area. + * + * These exercise the rounding/scaling family -- round, trunc, ceil, floor, + * scale -- with and without a scale argument. NUMERIC is exact arbitrary + * precision, so every property is stated as an exact equality or an exact + * bound (e.g. half an ULP, 0.5 * power(10, -s)); no float tolerance is used. + * This is distinct from the arithmetic-identity "numeric" area (+, -, *, /) + * and from the float8 "mathfn" area. + * + * The oracle here is the exact decimal semantics of the type: floor/ceil + * bracket a value and differ by 0 or 1, trunc moves toward zero, round(x, s) + * lands within half an ULP of x at scale s, and all four are the identity on + * integer-valued inputs. Any "f" result is a bug, not a rounding artefact. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_numround.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "numround" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a decimal NUMERIC literal as text into buf, parenthesized so a leading + * sign never misparses when the literal is substituted into a larger + * expression. + * + * Magnitudes are kept modest -- 0..12 integer digits and 0..8 fractional + * digits with a random sign -- so counterexamples stay legible. numeric has + * no overflow concern at these sizes. When nfrac is 0 the value is an exact + * integer, which the integer-fixpoint property relies on. + */ +static void +draw_numeric(pg_prop_tc *tc, StringInfo buf, int nfrac) +{ + int nint = (int) pg_prop_draw_int(tc, 0, 12); + bool neg = pg_prop_draw_bool(tc); + int ndigits = 0; + + appendStringInfoString(buf, "("); + if (neg) + appendStringInfoChar(buf, '-'); + + /* integer part */ + for (int i = 0; i < nint; i++) + { + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + + /* fractional part */ + if (nfrac > 0) + { + appendStringInfoChar(buf, '.'); + for (int i = 0; i < nfrac; i++) + { + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + } + + /* ensure a valid numeric: at least one digit */ + if (ndigits == 0) + appendStringInfoChar(buf, '0'); + + appendStringInfoString(buf, ")::numeric"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the rounding errored or produced NULL on a finite numeric, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * floor and ceil bracket X and differ by 0 or 1: + * floor(X) <= X <= ceil(X), and ceil(X) - floor(X) IN (0, 1). + */ +PG_PROP(floor_le_x_le_ceil) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x, (int) pg_prop_draw_int(tc, 0, 8)); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT floor(%s) <= %s AND %s <= ceil(%s) " + "AND (ceil(%s) - floor(%s)) IN (0, 1)", + x.data, x.data, x.data, x.data, + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * trunc toward zero never increases magnitude and yields an integer: + * abs(trunc(X)) <= abs(X) and scale(trunc(X)) = 0. + */ +PG_PROP(trunc_abs_le_abs) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x, (int) pg_prop_draw_int(tc, 0, 8)); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(trunc(%s)) <= abs(%s) " + "AND scale(trunc(%s)) = 0", + x.data, x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * round(X, s) lands within half an ULP of X at scale s: + * abs(round(X, s) - X) <= 0.5 * power(10, -s). + */ +PG_PROP(round_scale_bounds) +{ + pg_prop_conn *conn; + char *err = NULL; + int s = (int) pg_prop_draw_int(tc, 0, 6); + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x, (int) pg_prop_draw_int(tc, 0, 8)); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(round(%s, %d) - %s) " + "<= 0.5 * power(10::numeric, -%d)", + x.data, s, x.data, s); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * round(X, s) has scale <= s: it rounds to s fractional digits, and trailing + * zeros may be dropped so the resulting scale can be smaller. + */ +PG_PROP(round_scale_is_scale) +{ + pg_prop_conn *conn; + char *err = NULL; + int s = (int) pg_prop_draw_int(tc, 0, 6); + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x, (int) pg_prop_draw_int(tc, 0, 8)); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT scale(round(%s, %d)) <= %d", + x.data, s, s); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * trunc(X, s) toward zero at scale s never increases magnitude and discards + * strictly less than one unit in the last place: + * abs(trunc(X, s)) <= abs(X) and abs(X - trunc(X, s)) < power(10, -s). + */ +PG_PROP(trunc_scale_bounds) +{ + pg_prop_conn *conn; + char *err = NULL; + int s = (int) pg_prop_draw_int(tc, 0, 6); + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x, (int) pg_prop_draw_int(tc, 0, 8)); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(trunc(%s, %d)) <= abs(%s) " + "AND abs(%s - trunc(%s, %d)) < power(10::numeric, -%d)", + x.data, s, x.data, + x.data, x.data, s, s); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Negation symmetry of floor and ceil: floor(-X) = -ceil(X). A clean exact + * identity that any sign-handling bug in the two routines would break. + */ +PG_PROP(floor_ceil_negation_symmetry) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x, (int) pg_prop_draw_int(tc, 0, 8)); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT floor(-(%s)) = -ceil(%s)", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Rounding to scale s is idempotent: round(round(X, s), s) = round(X, s). + * Once a value is at scale s, rounding again at the same scale changes + * nothing. + */ +PG_PROP(round_idempotent_at_scale) +{ + pg_prop_conn *conn; + char *err = NULL; + int s = (int) pg_prop_draw_int(tc, 0, 6); + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x, (int) pg_prop_draw_int(tc, 0, 8)); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT round(round(%s, %d), %d) = round(%s, %d)", + x.data, s, s, x.data, s); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * On an integer-valued numeric (drawn with no fractional part), floor, ceil, + * round and trunc are all the identity: + * floor(N) = N AND ceil(N) = N AND round(N) = N AND trunc(N) = N. + */ +PG_PROP(ceil_floor_integer_fixpoint) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + draw_numeric(tc, &n, 0); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT floor(%s) = %s AND ceil(%s) = %s " + "AND round(%s) = %s AND trunc(%s) = %s", + n.data, n.data, n.data, n.data, + n.data, n.data, n.data, n.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 105a0c3026e2a..5dc804049fed1 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale); +my @areas = qw(numeric numcast numscale numround); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 14fdc94314159f41dca42d92fc96be17af3fd736 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:44 -0400 Subject: [PATCH 020/338] prop: add numfmt property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_numfmt.c | 443 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 447 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_numfmt.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 8e30b3162415f..2028b6d496b2f 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -35,7 +35,8 @@ OBJS = \ prop_numeric.o \ prop_numcast.o \ prop_numscale.o \ - prop_numround.o + prop_numround.o \ + prop_numfmt.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index d7965bf642c4c..958b8ac2483d4 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -33,6 +33,7 @@ prop_area_sources = files( 'prop_numcast.c', 'prop_numscale.c', 'prop_numround.c', + 'prop_numfmt.c', ) diff --git a/src/test/prop/prop_numfmt.c b/src/test/prop/prop_numfmt.c new file mode 100644 index 0000000000000..923f163c9ae1e --- /dev/null +++ b/src/test/prop/prop_numfmt.c @@ -0,0 +1,443 @@ +/*------------------------------------------------------------------------- + * + * prop_numfmt.c + * Property-based tests for integer type I/O and overflow handling + * for int2/int4/int8 (src/backend/utils/adt/int.c, int8.c, numutils.c). + * + * This is the "numfmt" area (integer parsing/formatting/overflow), distinct + * from the "numeric" area (arbitrary-precision NUMERIC). Integer input + * (pg_strtoint16/32/64 in numutils.c) and arithmetic overflow checks (int.c, + * int8.c) have had fixes; these properties check text round-trip, that + * overflow is detected rather than wrapping, and non-decimal literal parsing. + * + * The oracle is the type's own text I/O and the exact NUMERIC algebra: an + * integer formatted to text and parsed back must equal the original (across + * the full width including INT64_MIN/MAX), and integer add/multiply must + * either equal the exact numeric result or raise an overflow error -- never + * wrap silently. An "out of range" error for inputs whose true result + * exceeds the type is the correct behavior and is skipped, not reported. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_numfmt.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "numfmt" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * True when a SQL error is a legitimate out-of-range/overflow condition rather + * than a wrong-results bug. Integer add/multiply of random operands near the + * type's range ends can legally overflow; those cases are the correct result + * and are discarded, not reported. + */ +static bool +is_overflow(const char *errmsg) +{ + if (errmsg == NULL) + return false; + return strstr(errmsg, "out of range") != NULL || + strstr(errmsg, "overflow") != NULL; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the query errored or produced NULL on valid inputs, which + * is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Run a SELECT that is REQUIRED to raise an out-of-range/overflow error. If + * the query instead returns a value, the overflow check that should have + * fired did not -- the arithmetic wrapped -- and that is a bug. Used for the + * fixed boundary check (negating INT64_MIN). + */ +static void +require_overflow(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + if (is_overflow(err)) + { + /* correct: the overflow was detected */ + if (err) + pg_free(err); + return; + } + pg_prop_fail(tc, "expected overflow error but got a different error " + "for: %s\n error: %s", sql, err ? err : "(unknown)"); + } + + /* no error -- the result was produced, i.e. the overflow wrapped: a bug */ + pg_prop_fail(tc, "expected overflow error but query returned %s for: %s", + res ? res : "NULL", sql); +} + +/* + * Draw a full-width int64 and format it with "%lld". The engine favours the + * boundary values (INT64_MIN/MAX, 0, -1, 1); drawing the full range is the + * point of these tests, so do not narrow it. + */ +static void +append_int64(pg_prop_tc *tc, StringInfo buf) +{ + int64 n = pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + + appendStringInfo(buf, "%lld", (long long) n); +} + +/* + * Draw an int32-range value and format it. + */ +static void +append_int32(pg_prop_tc *tc, StringInfo buf) +{ + int64 n = pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + appendStringInfo(buf, "%lld", (long long) n); +} + +/* + * Draw an int16-range value and format it. + */ +static void +append_int16(pg_prop_tc *tc, StringInfo buf) +{ + int64 n = pg_prop_draw_int(tc, -32768, 32767); + + appendStringInfo(buf, "%lld", (long long) n); +} + +/* + * 1. int8_text_roundtrip + * + * An int8 formatted to text and parsed back must equal the original, across + * the full int64 width including INT64_MIN/MAX. Exercises int8out and the + * pg_strtoint64 input path (numutils.c). + */ +PG_PROP(int8_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + append_int64(tc, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s)::int8)::text::int8 = (%s)::int8", + n.data, n.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. int4_text_roundtrip + * + * An int4 formatted to text and parsed back must equal the original, across + * the full int32 width including INT32_MIN/MAX. Exercises int4out and the + * pg_strtoint32 input path (numutils.c). + */ +PG_PROP(int4_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + append_int32(tc, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s)::int4)::text::int4 = (%s)::int4", + n.data, n.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. int2_text_roundtrip + * + * An int2 formatted to text and parsed back must equal the original, across + * the full int16 width including -32768/32767. Exercises int2out and the + * pg_strtoint16 input path (numutils.c). + */ +PG_PROP(int2_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + append_int16(tc, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s)::int2)::text::int2 = (%s)::int2", + n.data, n.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. int8_add_matches_numeric_or_overflows + * + * int8 addition of A and B must either equal the exact numeric sum or raise an + * out-of-range error -- it must never wrap silently. NUMERIC is the exact + * oracle: (A::numeric + B::numeric) is the true sum, and casting it back to + * int8 only succeeds when the true sum fits. If the int8 add raises + * "out of range", the true sum is outside int8 and that is the correct result, + * so the case is skipped. Targets the int8 addition overflow check (int8.c). + */ +PG_PROP(int8_add_matches_numeric_or_overflows) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a; + StringInfoData b; + StringInfoData q; + char *result; + bool failed; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + append_int64(tc, &a); + append_int64(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s)::int8 + (%s)::int8) " + "= ((%s)::numeric + (%s)::numeric)::int8", + a.data, b.data, a.data, b.data); + + result = pg_prop_sql_scalar(conn, q.data, &failed, &err); + + if (failed) + { + if (is_overflow(err)) + { + /* true sum exceeds int8: overflow is the correct result, skip */ + pg_prop_note(tc, "overflow (skipped): %s", err); + if (err) + pg_free(err); + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); + pg_prop_assume(tc, false); + return; /* unreachable: assume(false) longjmps */ + } + pg_prop_fail(tc, + "unexpected error on int8 add vs numeric\n" + " A: %s\n B: %s\n error: %s", + a.data, b.data, err ? err : "(unknown)"); + } + + if (result == NULL || strcmp(result, "t") != 0) + pg_prop_fail(tc, + "int8 addition did not match the exact numeric sum\n" + " A: %s\n B: %s\n result: %s", + a.data, b.data, result ? result : "NULL"); + + pg_free(result); + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. int8_mul_matches_numeric_or_overflows + * + * As for addition: int8 multiplication of A and B must either equal the exact + * numeric product or raise an out-of-range error, never wrapping. An + * "out of range" error means the true product is outside int8, which is the + * correct result, so the case is skipped. Targets the int8 multiply overflow + * check (int8.c). + */ +PG_PROP(int8_mul_matches_numeric_or_overflows) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a; + StringInfoData b; + StringInfoData q; + char *result; + bool failed; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + append_int64(tc, &a); + append_int64(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s)::int8 * (%s)::int8) " + "= ((%s)::numeric * (%s)::numeric)::int8", + a.data, b.data, a.data, b.data); + + result = pg_prop_sql_scalar(conn, q.data, &failed, &err); + + if (failed) + { + if (is_overflow(err)) + { + /* true product exceeds int8: overflow is correct, skip */ + pg_prop_note(tc, "overflow (skipped): %s", err); + if (err) + pg_free(err); + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); + pg_prop_assume(tc, false); + return; /* unreachable: assume(false) longjmps */ + } + pg_prop_fail(tc, + "unexpected error on int8 mul vs numeric\n" + " A: %s\n B: %s\n error: %s", + a.data, b.data, err ? err : "(unknown)"); + } + + if (result == NULL || strcmp(result, "t") != 0) + pg_prop_fail(tc, + "int8 multiplication did not match the exact numeric product\n" + " A: %s\n B: %s\n result: %s", + a.data, b.data, result ? result : "NULL"); + + pg_free(result); + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. negate_min_overflows + * + * A fixed boundary check with no draws: negating INT64_MIN has no int8 + * representation (its magnitude is one past INT64_MAX), so it must raise an + * out-of-range error, not wrap back to INT64_MIN. INT64_MIN is written as + * (-9223372036854775807 - 1) so the literal is unambiguously parsed as int8 + * first; negating it must then overflow. This is the classic overflow site + * in int8um (int8.c). If the query returns a value instead of erroring, the + * negation wrapped -- a bug. + */ +PG_PROP(negate_min_overflows) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_overflow(tc, conn, + "SELECT (-((-9223372036854775807 - 1)::int8))"); + + pg_prop_disconnect(conn); +} + +/* + * 7. nondecimal_roundtrip + * + * PostgreSQL accepts hex/octal/binary integer literals (0x, 0o, 0b) since v16. + * For a non-negative n, prefixing the hex digits of n (from to_hex) with "0x" + * and casting to int8 must reparse to n. Exercises the non-decimal parsing + * path of pg_strtoint (numutils.c). + */ +PG_PROP(nondecimal_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, 0, 1000000); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('0x' || to_hex(%lld))::int8 = (%lld)::int8", + (long long) n, (long long) n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 5dc804049fed1..de0bfc924816f 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround); +my @areas = qw(numeric numcast numscale numround numfmt); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 6c9fa55ac4b86b62798be9d777fe02e658a277d8 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:44 -0400 Subject: [PATCH 021/338] prop: add numeric pow/log/mod/gcd/lcm identities property tests (numericalg) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_numericalg.c | 339 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 343 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_numericalg.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 2028b6d496b2f..27b7123498b01 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -36,7 +36,8 @@ OBJS = \ prop_numcast.o \ prop_numscale.o \ prop_numround.o \ - prop_numfmt.o + prop_numfmt.o \ + prop_numericalg.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 958b8ac2483d4..f44aa03e34680 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -34,6 +34,7 @@ prop_area_sources = files( 'prop_numscale.c', 'prop_numround.c', 'prop_numfmt.c', + 'prop_numericalg.c', ) diff --git a/src/test/prop/prop_numericalg.c b/src/test/prop/prop_numericalg.c new file mode 100644 index 0000000000000..7c2c2f9bd850f --- /dev/null +++ b/src/test/prop/prop_numericalg.c @@ -0,0 +1,339 @@ +/*------------------------------------------------------------------------- + * + * prop_numericalg.c + * Property-based tests for NUMERIC transcendental and integer algebra. + * + * Covers numeric algebra not exercised by the numeric/numround/numfmt/numcast + * areas: the transcendental functions (sqrt, exp, ln, log, power) and the + * integer helpers (mod, gcd, lcm) together with the scale/trunc/round + * interplay. The oracle is each function's defining identity -- sqrt(x)^2=x, + * ln(exp(x))=x, log(b,b^e)=e, a=trunc(a/b)*b+mod(a,b), round-to-scale bound, + * and gcd/lcm divisibility. Transcendental identities carry a small tolerance + * (results are rounded to a working scale); mod/gcd/lcm on integers are exact. + * A violation is a bug in numeric.c's series/Newton routines or in the + * gcd/lcm/mod paths, not a rounding artefact. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_numericalg.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "numericalg" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a small decimal NUMERIC literal as ".", parenthesized so a + * leading sign never misparses when substituted into a larger expression. + * "small" (<=4 integer digits, <=4 fractional) keeps every transcendental and + * integer identity far from overflow and keeps counterexamples legible. + */ +static void +draw_numeric(pg_prop_tc *tc, StringInfo buf) +{ + int nint = (int) pg_prop_draw_int(tc, 0, 4); + int nfrac = (int) pg_prop_draw_int(tc, 0, 4); + bool neg = pg_prop_draw_bool(tc); + int ndigits = 0; + + appendStringInfoString(buf, "("); + if (neg) + appendStringInfoChar(buf, '-'); + + for (int i = 0; i < nint; i++) + { + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + + if (nfrac > 0) + { + appendStringInfoChar(buf, '.'); + for (int i = 0; i < nfrac; i++) + { + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + } + + /* ensure a valid numeric: at least one digit */ + if (ndigits == 0) + appendStringInfoChar(buf, '0'); + + appendStringInfoString(buf, ")::numeric"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on in-range inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * sqrt_sq: for x = abs(drawn numeric) >= 0, sqrt(x) squared returns x within a + * relative-plus-absolute tolerance. numeric sqrt rounds to a working scale, + * so the only slack is the last-digit rounding, well inside 1e-9*(abs(x)+1). + */ +PG_PROP(sqrt_sq) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_numeric(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(sqrt(abs(%s)) * sqrt(abs(%s)) - abs(%s)) " + "< 1e-9::numeric * (abs(%s) + 1)", + a.data, a.data, a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ln_exp_inverse: ln(exp(x)) = x for x in [1, 20]. exp of larger x grows past + * the comfortable working range, so x is kept <= 20; both are rounded to a + * scale, hence the 1e-6*(x+1) tolerance. + */ +PG_PROP(ln_exp_inverse) +{ + pg_prop_conn *conn; + char *err = NULL; + int x; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x = (int) pg_prop_draw_int(tc, 1, 20); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(ln(exp(%d::numeric)) - %d) " + "< 1e-6::numeric * (%d + 1)", + x, x, x); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * log_pow_inverse: log(b, b^e) = e for integer base b in [2, 9] and exponent e + * in [0, 6]. b^e <= 9^6 = 531441, far from overflow; log rounds to a scale, + * hence the 1e-6 tolerance. + */ +PG_PROP(log_pow_inverse) +{ + pg_prop_conn *conn; + char *err = NULL; + int b, + e; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + b = (int) pg_prop_draw_int(tc, 2, 9); + e = (int) pg_prop_draw_int(tc, 0, 6); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(log(%d::numeric, (%d::numeric) ^ %d) - %d) " + "< 1e-6::numeric", + b, b, e, e); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * mod_identity: the trunc-division identity a = trunc(a/b)*b + mod(a,b) for a + * drawn numeric and a non-zero integer divisor b in [-20, -1] U [1, 20]. Exact + * in principle, but a/b carries a fractional scale, so a 1e-9 tolerance absorbs + * the display rounding of the quotient. + */ +PG_PROP(mod_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + int b; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_numeric(tc, &a); + + /* b in 1..20 with a drawn sign; never zero */ + b = (int) pg_prop_draw_int(tc, 1, 20); + if (pg_prop_draw_bool(tc)) + b = -b; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(%s - (trunc(%s / (%d)) * (%d) " + "+ mod(%s, (%d)))) < 1e-9::numeric", + a.data, a.data, b, b, a.data, b); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * round_scale: round(x, s) for s in [0, 4] has scale <= s and differs from x by + * at most half a unit in the last kept place, i.e. 0.5 * 10^(-s). Both parts + * are exact for numeric, so this is an exact (no tolerance) check. scale() of + * the rounded value is compared to s. + */ +PG_PROP(round_scale) +{ + pg_prop_conn *conn; + char *err = NULL; + int s; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x); + + s = (int) pg_prop_draw_int(tc, 0, 4); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT scale(round(%s, %d)) <= %d " + "AND abs(round(%s, %d) - %s) " + "<= 0.5::numeric * (10::numeric ^ (-%d))", + x.data, s, s, + x.data, s, x.data, s); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * gcd_divides_both: gcd(a, b) divides both a and b for positive integers a, b + * in [1, 1000]. Exact integer property: a % gcd = 0 AND b % gcd = 0. + */ +PG_PROP(gcd_divides_both) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (int) pg_prop_draw_int(tc, 1, 1000); + b = (int) pg_prop_draw_int(tc, 1, 1000); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%d::numeric %% gcd(%d::numeric, %d::numeric)) = 0 " + "AND (%d::numeric %% gcd(%d::numeric, %d::numeric)) = 0", + a, a, b, + b, a, b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * lcm_multiple: lcm(a, b) is a multiple of both a and b for positive integers + * a, b in [1, 100]. Exact integer property: lcm %% a = 0 AND lcm %% b = 0. + * lcm(a,b) <= 100*100 = 10000, far from overflow. + */ +PG_PROP(lcm_multiple) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (int) pg_prop_draw_int(tc, 1, 100); + b = (int) pg_prop_draw_int(tc, 1, 100); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (lcm(%d::numeric, %d::numeric) %% %d::numeric) = 0 " + "AND (lcm(%d::numeric, %d::numeric) %% %d::numeric) = 0", + a, b, a, + a, b, b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index de0bfc924816f..2d452aa86156b 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt); +my @areas = qw(numeric numcast numscale numround numfmt numericalg); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 5cc6c4856813312fe925b910b1a65dc4d03064e6 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:45 -0400 Subject: [PATCH 022/338] prop: add numeric comparison, NaN ordering, and min/max property tests (numericcmp) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_numericcmp.c | 359 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 363 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_numericcmp.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 27b7123498b01..d366574a3ea7c 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -37,7 +37,8 @@ OBJS = \ prop_numscale.o \ prop_numround.o \ prop_numfmt.o \ - prop_numericalg.o + prop_numericalg.o \ + prop_numericcmp.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index f44aa03e34680..4633c5e69740c 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -35,6 +35,7 @@ prop_area_sources = files( 'prop_numround.c', 'prop_numfmt.c', 'prop_numericalg.c', + 'prop_numericcmp.c', ) diff --git a/src/test/prop/prop_numericcmp.c b/src/test/prop/prop_numericcmp.c new file mode 100644 index 0000000000000..3a6d0161c1976 --- /dev/null +++ b/src/test/prop/prop_numericcmp.c @@ -0,0 +1,359 @@ +/*------------------------------------------------------------------------- + * + * prop_numericcmp.c + * Property-based tests for NUMERIC comparison, ordering and special values. + * + * Distinct from prop_numeric (arithmetic identities): this file pins down the + * comparison semantics of the numeric type -- the total order over finite + * values, the special rule that NaN sorts as the largest value and compares + * equal to itself, scale-insensitive equality (0 = -0 = 0.00), and that + * numeric-vs-int comparison agrees with int-vs-int. A broken numeric_cmp, + * least/greatest, abs or sign shows up as an "f" here. + * + * The oracle is the type's own documented comparison contract: these are + * exact boolean facts true for every finite (and, where stated, NaN) input, + * so any "f" or NULL result is a bug, not a rounding artefact. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_numericcmp.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "numericcmp" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a finite decimal NUMERIC literal as the text '.'::numeric + * -- the sign lives inside the single-quoted string, so there is no leading- + * minus misparse and no bare-literal overflow (the value is parsed by + * numeric_in from text, never as an integer literal). Scales straddle the + * usual boundaries but stay far below the numeric limit, so no overflow. + */ +static void +draw_numeric(pg_prop_tc *tc, StringInfo buf) +{ + int nint = (int) pg_prop_draw_int(tc, 0, 12); + int nfrac = (int) pg_prop_draw_int(tc, 0, 6); + bool neg = pg_prop_draw_bool(tc); + int ndigits = 0; + + appendStringInfoChar(buf, '\''); + if (neg) + appendStringInfoChar(buf, '-'); + + for (int i = 0; i < nint; i++) + { + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + + if (nfrac > 0) + { + appendStringInfoChar(buf, '.'); + for (int i = 0; i < nfrac; i++) + { + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + } + + if (ndigits == 0) + appendStringInfoChar(buf, '0'); + + appendStringInfoString(buf, "'::numeric"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a comparison errored or produced NULL on inputs where the + * contract is total, which is itself a bug -- so either is a failure. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * NaN sorts largest: 'NaN'::numeric > x for every finite x. Numeric NaN is + * defined to order above all finite values (mirroring the b-tree opclass used + * for sorts and indexes), so this strict inequality must hold for any drawn + * finite x. + */ +PG_PROP(numeric_nan_largest) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT 'NaN'::numeric > %s", x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * NaN equals NaN: 'NaN'::numeric = 'NaN'::numeric is TRUE. Unlike float, the + * numeric type defines NaN = NaN so NaN has a well-defined place in the total + * order; this must be true (a constant check, but it guards that rule). + */ +PG_PROP(numeric_nan_eq_nan) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, "SELECT 'NaN'::numeric = 'NaN'::numeric"); + + pg_prop_disconnect(conn); +} + +/* + * Total order (trichotomy): for finite A, B exactly one of A < B, A = B, + * A > B holds. Summing the three booleans as ints must equal 1 for any pair. + */ +PG_PROP(numeric_total_order) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_numeric(tc, &a); + draw_numeric(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s < %s)::int + (%s = %s)::int " + "+ (%s > %s)::int = 1", + a.data, b.data, a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * int-vs-numeric agrees with int-vs-int: for int n and numeric x, + * (n::numeric < x) = (n < x). The second comparison promotes n to numeric + * via the implicit cast; both must agree, so the numeric comparison operator + * selected for a mixed int/numeric comparison is consistent with the pure + * numeric one. n is drawn across the full int4 range; the negative literal + * is cast as (%d) so INT_MIN does not misparse. + */ +PG_PROP(numeric_int_consistent) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, -2147483648, 2147483647); + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%d)::numeric < %s) = ((%d)::int4 < %s)", + n, x.data, n, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * LEAST/GREATEST bound every argument: LEAST(a,b,c) <= each of a,b,c and + * each <= GREATEST(a,b,c). Holds for any finite numerics (no NULLs drawn), + * so LEAST/GREATEST must straddle the inputs exactly. + */ +PG_PROP(greatest_least_numeric) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + initStringInfo(&c); + draw_numeric(tc, &a); + draw_numeric(tc, &b); + draw_numeric(tc, &c); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT LEAST(%s, %s, %s) <= %s " + "AND LEAST(%s, %s, %s) <= %s " + "AND LEAST(%s, %s, %s) <= %s " + "AND %s <= GREATEST(%s, %s, %s) " + "AND %s <= GREATEST(%s, %s, %s) " + "AND %s <= GREATEST(%s, %s, %s)", + a.data, b.data, c.data, a.data, + a.data, b.data, c.data, b.data, + a.data, b.data, c.data, c.data, + a.data, a.data, b.data, c.data, + b.data, a.data, b.data, c.data, + c.data, a.data, b.data, c.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * abs/sign contract: abs(x) >= 0 and x = sign(x) * abs(x) for any finite x. + * sign() returns -1, 0 or 1 for numeric; the product reconstructs x exactly + * (abs and sign are exact), so this must hold with no tolerance. + */ +PG_PROP(abs_sign_numeric) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(%s) >= 0 AND %s = sign(%s) * abs(%s)", + x.data, x.data, x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Zero and scale do not affect equality: 0 = -0 and 0.00 = 0. Numeric + * equality ignores both the sign of zero and the display scale, so these + * constant facts must be TRUE (guards against a scale- or sign-sensitive + * comparison path). + */ +PG_PROP(numeric_zero_signs) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT '0'::numeric = '-0'::numeric " + "AND '0.00'::numeric = '0'::numeric"); + + pg_prop_disconnect(conn); +} + +/* + * least(a,b) <= greatest(a,b) for a drawn pair of finite numerics. A weaker, + * two-argument companion to greatest_least_numeric that still catches an + * inverted comparison inside the min/max machinery. + */ +PG_PROP(min_max_order) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_numeric(tc, &a); + draw_numeric(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT least(%s, %s) <= greatest(%s, %s)", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 2d452aa86156b..55cd72311ea9f 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From a671fb268e32526fda5b7194f77d4b4688495f59 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:45 -0400 Subject: [PATCH 023/338] prop: add numeric division and multiplication precision property tests (numericdiv) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_numericdiv.c | 376 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 380 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_numericdiv.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index d366574a3ea7c..3e4d8d8e11ac0 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -38,7 +38,8 @@ OBJS = \ prop_numround.o \ prop_numfmt.o \ prop_numericalg.o \ - prop_numericcmp.o + prop_numericcmp.o \ + prop_numericdiv.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 4633c5e69740c..eb56577af99b5 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -36,6 +36,7 @@ prop_area_sources = files( 'prop_numfmt.c', 'prop_numericalg.c', 'prop_numericcmp.c', + 'prop_numericdiv.c', ) diff --git a/src/test/prop/prop_numericdiv.c b/src/test/prop/prop_numericdiv.c new file mode 100644 index 0000000000000..bd19c0ec7b483 --- /dev/null +++ b/src/test/prop/prop_numericdiv.c @@ -0,0 +1,376 @@ +/*------------------------------------------------------------------------- + * + * prop_numericdiv.c + * Property-based tests for NUMERIC division/multiplication scale. + * + * Numeric multiplication is exact at the operands' combined scale, but + * division (/) rounds to a selected display scale, so the division oracle + * here is a rounded identity: a round-trip through multiply-then-divide, and + * algebraic identities (commutativity, associativity, distributivity, + * division by 1 and by self, sign of a quotient) compared at a common scale. + * A broken carry in mul_var/div_var surfaces as one of these identities + * failing even after rounding to a modest common scale, which is a bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_numericdiv.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "numericdiv" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a small-magnitude NUMERIC literal as '.', parenthesized and + * cast, so products and quotients stay comfortably representable and the + * division rounding is dominated by a modest common scale. The integer part + * is 0..20 and the fractional part is 1..3 digits. A random sign is applied. + */ +static void +draw_numeric(pg_prop_tc *tc, StringInfo buf) +{ + int ipart = (int) pg_prop_draw_int(tc, 0, 20); + int nfrac = (int) pg_prop_draw_int(tc, 1, 3); + bool neg = pg_prop_draw_bool(tc); + + appendStringInfoString(buf, "("); + if (neg) + appendStringInfoChar(buf, '-'); + appendStringInfo(buf, "%d.", ipart); + for (int i = 0; i < nfrac; i++) + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + appendStringInfoString(buf, ")::numeric"); +} + +/* + * Draw a non-zero NUMERIC literal as '<1..20>.' with a random sign. + * The integer part starts at 1 so the magnitude is at least 1.0, keeping it + * well away from zero for use as a divisor. + */ +static void +draw_numeric_nonzero(pg_prop_tc *tc, StringInfo buf) +{ + int ipart = (int) pg_prop_draw_int(tc, 1, 20); + int nfrac = (int) pg_prop_draw_int(tc, 1, 3); + bool neg = pg_prop_draw_bool(tc); + + appendStringInfoString(buf, "("); + if (neg) + appendStringInfoChar(buf, '-'); + appendStringInfo(buf, "%d.", ipart); + for (int i = 0; i < nfrac; i++) + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + appendStringInfoString(buf, ")::numeric"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the arithmetic errored or produced NULL on finite inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Multiply/divide round-trip: round((X*Y)/Y, 6) = round(X, 6) for Y <> 0. + * Multiplication is exact; the only slack is division's rounding, so rounding + * both sides to a common scale (6) must recover X. + */ +PG_PROP(mul_div_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + y, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&y); + draw_numeric(tc, &x); + draw_numeric_nonzero(tc, &y); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT round((%s * %s) / %s, 6) = round(%s, 6)", + x.data, y.data, y.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(y.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Division scale is sufficient: Y * (X / Y) reconstructs X to within a tiny + * tolerance. numeric division selects a display scale big enough that the + * residual abs(Y*(X/Y) - X) is far below 1e-6. + */ +PG_PROP(div_scale_sufficient) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + y, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&y); + draw_numeric(tc, &x); + draw_numeric_nonzero(tc, &y); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(%s * (%s / %s) - %s) < 1e-6::numeric", + y.data, x.data, y.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(y.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Multiplication commutes exactly: X * Y = Y * X. No tolerance, since + * numeric multiply is exact at the combined scale. + */ +PG_PROP(mul_commutative) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + y, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&y); + draw_numeric(tc, &x); + draw_numeric(tc, &y); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s * %s) = (%s * %s)", + x.data, y.data, y.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(y.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Multiplication associates: (X*Y)*Z = X*(Y*Z) within a common scale. + * Multiply is exact, so this holds exactly, but we round both sides to 6 to + * match the area's division-rounding convention and stay robust on small + * magnitudes. + */ +PG_PROP(mul_associative) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + y, + z, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&y); + initStringInfo(&z); + draw_numeric(tc, &x); + draw_numeric(tc, &y); + draw_numeric(tc, &z); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT round((%s * %s) * %s, 6) = round(%s * (%s * %s), 6)", + x.data, y.data, z.data, x.data, y.data, z.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(y.data); + pfree(z.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Division by one is identity: X / 1 = X, exactly. + */ +PG_PROP(div_by_one) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s / 1) = %s", x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Division by self is one: X / X = 1 for X <> 0. The divisor is drawn + * non-zero so no division-by-zero error is expected. + */ +PG_PROP(div_self_one) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric_nonzero(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s / %s) = 1", x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Multiplication distributes over addition: X*(Y+Z) = X*Y + X*Z within a + * common scale. Both sides are exact for numeric, but we round to 6 to match + * the area convention. + */ +PG_PROP(mul_distributes) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + y, + z, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&y); + initStringInfo(&z); + draw_numeric(tc, &x); + draw_numeric(tc, &y); + draw_numeric(tc, &z); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT round(%s * (%s + %s), 6) = round(%s * %s + %s * %s, 6)", + x.data, y.data, z.data, + x.data, y.data, x.data, z.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(y.data); + pfree(z.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Sign of a quotient: (-X) / Y = -(X / Y) within a common scale, for Y <> 0. + * numeric division truncates toward even rounding of the display scale + * symmetrically about zero, so negating the dividend negates the rounded + * quotient; comparing at scale 6 absorbs any last-digit rounding. + */ +PG_PROP(negative_div) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + y, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&y); + draw_numeric(tc, &x); + draw_numeric_nonzero(tc, &y); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT round((-(%s)) / %s, 6) = round(-((%s) / %s), 6)", + x.data, y.data, x.data, y.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(y.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 55cd72311ea9f..92130f04ecfeb 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 9046a5be2b2bc3f2414ee7afdaae3b08d0261c2d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:46 -0400 Subject: [PATCH 024/338] prop: add numeric text I/O and trailing-zero handling property tests (decimalfmt) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_decimalfmt.c | 300 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 304 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_decimalfmt.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 3e4d8d8e11ac0..35fece4487890 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -39,7 +39,8 @@ OBJS = \ prop_numfmt.o \ prop_numericalg.o \ prop_numericcmp.o \ - prop_numericdiv.o + prop_numericdiv.o \ + prop_decimalfmt.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index eb56577af99b5..d5b12f7a86dc6 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -37,6 +37,7 @@ prop_area_sources = files( 'prop_numericalg.c', 'prop_numericcmp.c', 'prop_numericdiv.c', + 'prop_decimalfmt.c', ) diff --git a/src/test/prop/prop_decimalfmt.c b/src/test/prop/prop_decimalfmt.c new file mode 100644 index 0000000000000..09d7d5b10f6fc --- /dev/null +++ b/src/test/prop/prop_decimalfmt.c @@ -0,0 +1,300 @@ +/*------------------------------------------------------------------------- + * + * prop_decimalfmt.c + * Property-based tests for NUMERIC decimal text I/O (input/output). + * + * This exercises the textual edges of numeric_in/numeric_out: parsing of + * trailing zeros, leading zeros, signs and scientific-notation exponents, and + * the round-trip through ::text. The oracle is parse/print identity: a value + * written out and read back must equal itself, and the display scale implied + * by the input text must be preserved. A failure means numeric's text codec + * dropped or added a digit, mis-set the display scale, or mis-parsed an + * exponent -- distinct from the arithmetic covered by numeric/numscale/ + * numround/numericcmp/numfmt. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_decimalfmt.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "decimalfmt" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a non-negative decimal magnitude as the raw text "." into + * buf (no sign, no cast). Sign is handled by the caller so a property can + * build both "X" and "-X" from the same magnitude. Digit counts stay small + * (0..12 integer, 0..8 fractional) -- the point here is text formatting, not + * carry handling -- and at least one digit is always emitted so the result is + * a valid numeric literal. + */ +static void +draw_magnitude(pg_prop_tc *tc, StringInfo buf) +{ + int nint = (int) pg_prop_draw_int(tc, 0, 12); + int nfrac = (int) pg_prop_draw_int(tc, 0, 8); + int ndigits = 0; + + for (int i = 0; i < nint; i++) + { + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + if (nfrac > 0) + { + appendStringInfoChar(buf, '.'); + for (int i = 0; i < nfrac; i++) + { + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + ndigits++; + } + } + if (ndigits == 0) + appendStringInfoChar(buf, '0'); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the text codec errored or produced NULL on a valid + * literal, which is itself a bug -- so either is a failure, reported with the + * SQL. (Same shape as prop_numeric.c's require_true.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * text_roundtrip: a drawn magnitude cast to numeric, out to text, and back to + * numeric equals itself. Exercises numeric_out followed by numeric_in on a + * value with an arbitrary trailing-fraction scale. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_magnitude(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (('%s'::numeric)::text)::numeric = '%s'::numeric", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * trailing_zeros_equal: trailing fractional zeros do not change value -- + * '1.50' = '1.5' and '1.500' = '1.5'. numeric_in must parse them to the same + * value even though their display scales differ. + */ +PG_PROP(trailing_zeros_equal) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT '1.50'::numeric = '1.5'::numeric " + "AND '1.500'::numeric = '1.5'::numeric"); + + pg_prop_disconnect(conn); +} + +/* + * scale_from_text: the display scale is taken from the literal's fractional + * digit count -- scale('1.50') = 2 and scale('1.5') = 1 -- so trailing zeros + * in the text are preserved as scale even though they do not change value. + */ +PG_PROP(scale_from_text) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT scale('1.50'::numeric) = 2 " + "AND scale('1.5'::numeric) = 1"); + + pg_prop_disconnect(conn); +} + +/* + * exponent_roundtrip: scientific notation parses to the same value as the + * expanded form -- '1.5e3' = 1500 and '2.5E2' = 250 -- covering both exponent + * marker cases. + */ +PG_PROP(exponent_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT '1.5e3'::numeric = 1500::numeric " + "AND '2.5E2'::numeric = 250::numeric"); + + pg_prop_disconnect(conn); +} + +/* + * leading_zeros_equal: leading integer zeros do not change value -- + * '007.5' = '7.5'. numeric_in must strip them on parse. + */ +PG_PROP(leading_zeros_equal) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, "SELECT '007.5'::numeric = '7.5'::numeric"); + + pg_prop_disconnect(conn); +} + +/* + * negative_roundtrip: for a drawn non-negative magnitude X, prefixing a '-' + * on the literal yields the arithmetic negation -- ('-' || X) = -(X). The + * magnitude is non-negative by construction, so the sign is unambiguous. + */ +PG_PROP(negative_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_magnitude(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT '-%s'::numeric = -('%s'::numeric)", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * normalize_text: feeding a value's own text representation back in is + * idempotent -- (X::text)::numeric = X::numeric -- so numeric_out emits no + * trailing-zero noise beyond the value's scale that would read back differently. + */ +PG_PROP(normalize_text) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_magnitude(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (('%s'::numeric)::text)::numeric = '%s'::numeric", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * plus_zero_identity: adding '0.0' leaves the value unchanged -- + * X + '0.0' = X. Addition is exact, so the only way this fails is a text + * parse that misreads the drawn literal. + */ +PG_PROP(plus_zero_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_magnitude(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('%s'::numeric + '0.0'::numeric) = '%s'::numeric", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 92130f04ecfeb..ce497619163ea 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From cdee16e1ef4fb301544fbb9aa274ec4613398afe Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:46 -0400 Subject: [PATCH 025/338] prop: add mathfn property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_mathfn.c | 406 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 410 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_mathfn.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 35fece4487890..f322c69246f08 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -40,7 +40,8 @@ OBJS = \ prop_numericalg.o \ prop_numericcmp.o \ prop_numericdiv.o \ - prop_decimalfmt.o + prop_decimalfmt.o \ + prop_mathfn.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index d5b12f7a86dc6..f48516a141be1 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -38,6 +38,7 @@ prop_area_sources = files( 'prop_numericcmp.c', 'prop_numericdiv.c', 'prop_decimalfmt.c', + 'prop_mathfn.c', ) diff --git a/src/test/prop/prop_mathfn.c b/src/test/prop/prop_mathfn.c new file mode 100644 index 0000000000000..480c173b50ccb --- /dev/null +++ b/src/test/prop/prop_mathfn.c @@ -0,0 +1,406 @@ +/*------------------------------------------------------------------------- + * + * prop_mathfn.c + * Property-based tests for the float8 math functions (abs, mod, power, + * sqrt, exp, ln, floor, ceil, round, trunc, sign). + * + * The float8 math functions must satisfy their defining identities and + * bounds; these check them, skipping domain errors and non-finite inputs + * where appropriate. + * + * The oracle here is each function's defining identity -- sqrt squared + * recovers its argument, floor/ceil bracket the value and differ by at most + * one, ln and exp are inverses, trunc-division reconstructs the dividend, and + * power agrees with repeated multiplication. Float arithmetic is inexact, so + * every inexact identity is checked against a tolerance scaled by magnitude; + * the exact ones (abs, floor/ceil bracketing) are checked exactly. Inputs + * are drawn in a moderate range (-1000..1000) so exp/power do not overflow, + * and NaN/Inf are skipped in C before any SQL is built since those have + * special-case behaviour out of scope here. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_mathfn.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "mathfn" +#include "pg_prop.h" + +#include + +#include "lib/stringinfo.h" + +/* + * Draw one finite double in [min, max] and format it as a bare "%.17g" + * literal into buf (parenthesized by the caller as needed). NaN and Inf are + * discarded in C before any SQL is built -- the math functions have + * special-case behaviour on those which is out of scope for these identity + * checks. "%.17g" round-trips any IEEE double exactly, so the literal names + * the drawn value precisely. The drawn value is returned so a property can + * inspect it in C. + */ +static double +draw_f8(pg_prop_tc *tc, StringInfo buf, double min, double max) +{ + double v = pg_prop_draw_float(tc, min, max); + char num[64]; + + pg_prop_assume(tc, !isnan(v) && !isinf(v)); + + snprintf(num, sizeof(num), "%.17g", v); + appendStringInfoString(buf, num); + return v; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on inputs the + * property is supposed to handle, which is itself a failure, reported with + * the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Like require_true, but a SQL error that is a legitimate domain or overflow + * error (out of range, overflow, "cannot", "invalid", or a domain error) + * discards the case rather than failing it. Used for exp/ln/power, which can + * overflow or hit a domain boundary even inside a moderate input range. + */ +static void +require_true_or_skip(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + bool benign = (err != NULL && + (strstr(err, "out of range") != NULL || + strstr(err, "overflow") != NULL || + strstr(err, "cannot") != NULL || + strstr(err, "invalid") != NULL || + strstr(err, "domain") != NULL)); + + if (err) + pg_free(err); + if (benign) + pg_prop_assume(tc, false); /* legitimate domain/overflow: skip */ + pg_prop_fail(tc, "SQL error for: %s", sql); + } + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); +} + +/* + * abs is non-negative and idempotent: abs(V) >= 0 and abs(abs(V)) = abs(V). + * Both are exact, so no tolerance is used. + */ +PG_PROP(abs_nonneg_and_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_f8(tc, &v, -1000.0, 1000.0); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(%s::float8) >= 0 " + "AND abs(abs(%s::float8)) = abs(%s::float8)", + v.data, v.data, v.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * sqrt squared recovers its argument: sqrt(X)^2 ~= X for X >= 0. X is forced + * non-negative with abs() so sqrt is always defined. sqrt rounds to the + * nearest representable value, so the identity is checked against a relative + * tolerance (scaled by magnitude, floored at 1 so small X still has slack). + */ +PG_PROP(sqrt_squared_approx) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_f8(tc, &v, -1000.0, 1000.0); + + /* x = abs(v) so sqrt is defined on a non-negative argument */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(sqrt(abs(%s::float8)) * sqrt(abs(%s::float8)) " + "- abs(%s::float8)) " + "<= 1e-9 * greatest(abs(%s::float8), 1)", + v.data, v.data, v.data, v.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * floor and ceil bracket V and differ by 0 or 1: floor(V) <= V <= ceil(V) and + * ceil(V) - floor(V) is 0 (V integral) or 1. All exact, no tolerance. + */ +PG_PROP(floor_le_x_le_ceil) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_f8(tc, &v, -1000.0, 1000.0); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT floor(%s::float8) <= %s::float8 " + "AND %s::float8 <= ceil(%s::float8) " + "AND (ceil(%s::float8) - floor(%s::float8)) IN (0, 1)", + v.data, v.data, v.data, v.data, v.data, v.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * round is within half a unit of V: abs(round(V) - V) <= 0.5 (plus a tiny + * absolute slack for the representation of the half). + */ +PG_PROP(round_within_half) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_f8(tc, &v, -1000.0, 1000.0); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(round(%s::float8) - %s::float8) <= 0.5 + 1e-9", + v.data, v.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * trunc truncates toward zero: abs(trunc(V)) <= abs(V) and trunc(V) has the + * same sign as V or is zero. The magnitude side gets a tiny absolute slack. + */ +PG_PROP(trunc_toward_zero) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_f8(tc, &v, -1000.0, 1000.0); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(trunc(%s::float8)) <= abs(%s::float8) + 1e-9 " + "AND (sign(trunc(%s::float8)) = sign(%s::float8) " + "OR trunc(%s::float8) = 0)", + v.data, v.data, v.data, v.data, v.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ln and exp are inverses on positive X: exp(ln(X)) ~= X. X is drawn in a + * small positive-to-bounded range (0.01 .. 100] so ln is defined and exp + * stays finite; the identity is checked against a relative tolerance. A + * domain/overflow error is skipped rather than failed. + */ +PG_PROP(power_ln_exp_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_f8(tc, &x, 0.01, 100.0); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(exp(ln(%s::float8)) - %s::float8) " + "<= 1e-6 * %s::float8", + x.data, x.data, x.data); + + require_true_or_skip(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * degrees() and radians() are inverse conversions, so radians(degrees(x)) ~= x + * within a relative tolerance. (There is no float8 mod() in PostgreSQL -- mod + * is numeric/integer only -- so the trunc-division identity is tested in the + * numeric areas, not here.) + */ +PG_PROP(degrees_radians_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_f8(tc, &x, -1000.0, 1000.0); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(radians(degrees(%s::float8)) - %s::float8) " + "<= 1e-9 * greatest(abs(%s::float8), 1)", + x.data, x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * power cross-checked against repeated multiplication for a small integer + * exponent: power(X, K) ~= X*X*...*X (K factors), using the explicit K-fold + * product as an independent oracle (1 when K=0). X is drawn in -10..10 and K + * in 0..5 so the product stays well within range. Checked against a relative + * tolerance; a domain/overflow error is skipped. + */ +PG_PROP(power_matches_repeated_mult_small_exp) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + prod, + q; + int k; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_f8(tc, &x, -10.0, 10.0); + k = (int) pg_prop_draw_int(tc, 0, 5); + + /* build the explicit K-fold product: "1" for K=0, else x*x*...*x */ + initStringInfo(&prod); + if (k == 0) + appendStringInfoString(&prod, "1::float8"); + else + { + for (int i = 0; i < k; i++) + { + if (i > 0) + appendStringInfoString(&prod, " * "); + appendStringInfo(&prod, "(%s::float8)", x.data); + } + } + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(power(%s::float8, %d) - (%s)) " + "<= 1e-6 * greatest(abs(power(%s::float8, %d)), 1)", + x.data, k, prod.data, x.data, k); + + require_true_or_skip(tc, conn, q.data); + + pfree(x.data); + pfree(prod.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index ce497619163ea..b6ae8c53dee8d 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From e56b514bcf7323e4c60b1a11b712c778b9cc84a4 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:46 -0400 Subject: [PATCH 026/338] prop: add round/trunc/ceil/floor/sign relationships property tests (mathround) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_mathround.c | 338 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 342 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_mathround.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index f322c69246f08..a6c6b1fbc18ab 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -41,7 +41,8 @@ OBJS = \ prop_numericcmp.o \ prop_numericdiv.o \ prop_decimalfmt.o \ - prop_mathfn.o + prop_mathfn.o \ + prop_mathround.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index f48516a141be1..1b151469d4da3 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -39,6 +39,7 @@ prop_area_sources = files( 'prop_numericdiv.c', 'prop_decimalfmt.c', 'prop_mathfn.c', + 'prop_mathround.c', ) diff --git a/src/test/prop/prop_mathround.c b/src/test/prop/prop_mathround.c new file mode 100644 index 0000000000000..de64277d58eaf --- /dev/null +++ b/src/test/prop/prop_mathround.c @@ -0,0 +1,338 @@ +/*------------------------------------------------------------------------- + * + * prop_mathround.c + * Property-based tests for the rounding-family functions and their + * algebraic relationships: floor, ceil, round, trunc and sign. + * + * These check the ordering and integrality relationships that tie the + * rounding family together -- floor(x) <= x <= ceil(x), round(x) landing on + * one of the two bounding integers, trunc rounding toward zero, sign matching + * the comparison trichotomy, and NUMERIC round-half-away-from-zero. The + * oracle is each relationship stated as a boolean SELECT that must be true for + * every finite input; a false (or NULL/error) result is a bug in dtrunc/dround + * or numeric_round/numeric_trunc, not a rounding artefact, because every + * assertion here is exact at the type's own scale. This area is deliberately + * distinct from float/floatspecial (I/O and non-finite values), numround + * (scale-parameterised numeric rounding bounds) and mathfn (transcendental + * float8 functions): it is about the cross-function round/trunc/ceil/floor/ + * sign algebra over both float8 and numeric. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_mathround.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "mathround" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a finite decimal value as a single-quoted "." literal body + * (e.g. '12.34'), both signs, with up to 12 integer and 6 fractional digits. + * The caller appends the cast (::float8 or ::numeric). Single-quoting plus a + * trailing cast keeps a leading '-' from misparsing when substituted into a + * larger expression, and the literal is identical whichever type it feeds, so + * a float8 and numeric check see the same value. + */ +static void +draw_val(pg_prop_tc *tc, StringInfo buf) +{ + int nint = (int) pg_prop_draw_int(tc, 1, 12); + int nfrac = (int) pg_prop_draw_int(tc, 0, 6); + bool neg = pg_prop_draw_bool(tc); + + appendStringInfoChar(buf, '\''); + if (neg) + appendStringInfoChar(buf, '-'); + + for (int i = 0; i < nint; i++) + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + + if (nfrac > 0) + { + appendStringInfoChar(buf, '.'); + for (int i = 0; i < nfrac; i++) + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + } + + appendStringInfoChar(buf, '\''); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on a finite input, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * floor(x) <= x <= ceil(x), and ceil(x) - floor(x) is 0 (x integral) or 1. + * The float8 oracle: floor never overshoots, ceil never undershoots, and the + * two bounding integers are adjacent (or equal). + */ +PG_PROP(floor_le_ceil) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_val(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT floor(%s::float8) <= %s::float8 " + "AND %s::float8 <= ceil(%s::float8) " + "AND (ceil(%s::float8) - floor(%s::float8)) IN (0, 1)", + x.data, x.data, x.data, x.data, x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * round(x) lands on one of the two bounding integers: round(x) is floor(x) or + * ceil(x). True regardless of the half-rounding rule the platform uses. + */ +PG_PROP(round_between) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_val(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT round(%s::float8) IN (floor(%s::float8), ceil(%s::float8))", + x.data, x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * trunc rounds toward zero: for x >= 0 trunc(x) = floor(x); for x < 0 + * trunc(x) = ceil(x). draw_val yields both signs, so this covers both + * branches of the toward-zero rule. + */ +PG_PROP(trunc_toward_zero) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_val(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN %s::float8 >= 0 " + "THEN trunc(%s::float8) = floor(%s::float8) " + "ELSE trunc(%s::float8) = ceil(%s::float8) END", + x.data, x.data, x.data, x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * sign(x) agrees with the comparison trichotomy: 1 if x > 0, -1 if x < 0, + * else 0. float8 sign. + */ +PG_PROP(sign_consistency) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_val(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT sign(%s::float8) = " + "CASE WHEN %s::float8 > 0 THEN 1 " + "WHEN %s::float8 < 0 THEN -1 ELSE 0 END", + x.data, x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * floor(x) and ceil(x) are already integers, so truncating them is a no-op: + * floor(x) = trunc(floor(x)) AND ceil(x) = trunc(ceil(x)). + */ +PG_PROP(floor_ceil_integer) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_val(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT floor(%s::float8) = trunc(floor(%s::float8)) " + "AND ceil(%s::float8) = trunc(ceil(%s::float8))", + x.data, x.data, x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * NUMERIC round rounds half away from zero: round(0.5) = 1, round(1.5) = 2, + * round(2.5) = 3, round(-0.5) = -1. float8 round is libc-dependent (some + * platforms round half to even), so this is asserted only for ::numeric, + * where the rule is defined by PostgreSQL. Constant checks, no drawn input. + */ +PG_PROP(round_half_away) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT round('0.5'::numeric) = 1 " + "AND round('1.5'::numeric) = 2 " + "AND round('2.5'::numeric) = 3 " + "AND round('-0.5'::numeric) = -1"); + + pg_prop_disconnect(conn); +} + +/* + * The fractional part x - floor(x) lies in [0, 1): non-negative and strictly + * below one. float8; strict upper bound because floor removes exactly the + * whole part. + */ +PG_PROP(floor_plus_frac) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_val(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s::float8 - floor(%s::float8)) >= 0 " + "AND (%s::float8 - floor(%s::float8)) < 1", + x.data, x.data, x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Rounding to a fixed scale is idempotent: round(round(x, 2), 2) = + * round(x, 2). Once rounded to 2 fractional digits, rounding again changes + * nothing. numeric, so the scale is exact and the equality has no tolerance. + */ +PG_PROP(double_round_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_val(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT round(round(%s::numeric, 2), 2) = round(%s::numeric, 2)", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index b6ae8c53dee8d..46e60887db606 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 9d5f505f102ad291ad5407c39cf2c97e8da6d7ff Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:47 -0400 Subject: [PATCH 027/338] prop: add power/exp/log/factorial laws property tests (powexp) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_powexp.c | 303 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 307 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_powexp.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index a6c6b1fbc18ab..4a1f34124f54b 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -42,7 +42,8 @@ OBJS = \ prop_numericdiv.o \ prop_decimalfmt.o \ prop_mathfn.o \ - prop_mathround.o + prop_mathround.o \ + prop_powexp.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 1b151469d4da3..f187fa092228d 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -40,6 +40,7 @@ prop_area_sources = files( 'prop_decimalfmt.c', 'prop_mathfn.c', 'prop_mathround.c', + 'prop_powexp.c', ) diff --git a/src/test/prop/prop_powexp.c b/src/test/prop/prop_powexp.c new file mode 100644 index 0000000000000..d75caaf40eecf --- /dev/null +++ b/src/test/prop/prop_powexp.c @@ -0,0 +1,303 @@ +/*------------------------------------------------------------------------- + * + * prop_powexp.c + * Property-based tests for power, exponential, logarithm and factorial. + * + * The oracle here is the algebra of these functions themselves: exponent + * addition (b^(e1+e2) = b^e1 * b^e2), the exp/ln and log10/pow10 inverse + * pairs, the factorial recurrence (n! = n*(n-1)!), and monotonicity of pow + * and ln. Integer power and factorial are exact so those identities use no + * tolerance; the transcendental ones (sqrt, exp/ln, log) are checked within + * a relative-plus-absolute tolerance. A broken identity is a bug in the + * numeric power/exp/ln/log/factorial implementation, not a rounding + * artefact, because the bases/exponents are kept small enough that no + * overflow or scale loss can occur (b<=5, e<=6, factorial n<=12). + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_powexp.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "powexp" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on finite inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Exponent addition: b^(e1+e2) = b^e1 * b^e2 for integer base and exponents. + * Numeric integer power is exact at these small magnitudes (b<=5, e<=5, so + * the largest value is 5^10 ~= 9.8e6), so this holds with no tolerance. + */ +PG_PROP(pow_add_exponents) +{ + pg_prop_conn *conn; + char *err = NULL; + int b, + e1, + e2; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + b = (int) pg_prop_draw_int(tc, 2, 5); + e1 = (int) pg_prop_draw_int(tc, 0, 5); + e2 = (int) pg_prop_draw_int(tc, 0, 5); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%d::numeric ^ %d) = (%d::numeric ^ %d) * (%d::numeric ^ %d)", + b, e1 + e2, b, e1, b, e2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Identity exponents: b^0 = 1 and b^1 = b for every base. Exact. + */ +PG_PROP(pow_zero_one) +{ + pg_prop_conn *conn; + char *err = NULL; + int b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + b = (int) pg_prop_draw_int(tc, 2, 9); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%d::numeric ^ 0) = 1 AND (%d::numeric ^ 1) = %d", + b, b, b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Half power equals square root: x^0.5 == sqrt(x) within tolerance. Both are + * computed by numeric and rounded to a scale, so compare with a relative- + * plus-absolute tolerance scaled by (x+1). + */ +PG_PROP(sqrt_pow_half) +{ + pg_prop_conn *conn; + char *err = NULL; + int x; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x = (int) pg_prop_draw_int(tc, 1, 100); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs((%d::numeric ^ 0.5) - sqrt(%d::numeric)) " + "< 1e-6::numeric * (%d + 1)", + x, x, x); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * exp/ln inverse: exp(ln(x)) == x within tolerance. ln and exp are rounded + * to a scale, so compare with a relative-plus-absolute tolerance scaled by + * (x+1). x is kept in 1..20 so exp(ln(x)) stays well within range. + */ +PG_PROP(exp_ln_inverse) +{ + pg_prop_conn *conn; + char *err = NULL; + int x; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x = (int) pg_prop_draw_int(tc, 1, 20); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(exp(ln(%d::numeric)) - %d) < 1e-6::numeric * (%d + 1)", + x, x, x); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * log10 of a power of ten: log(10^e) == e. log(numeric) is base 10; 10^e for + * e<=6 is at most 1e6, exactly representable, so the result is e exactly. + */ +PG_PROP(log10_pow10) +{ + pg_prop_conn *conn; + char *err = NULL; + int e; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + e = (int) pg_prop_draw_int(tc, 0, 6); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT log(10::numeric ^ %d) = %d", + e, e); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Factorial recurrence: factorial(n) = n * factorial(n-1) for 1<=n<=12. 12! + * is 479001600, inside int range; factorial(int) returns numeric and is + * exact, so this holds with no tolerance. + */ +PG_PROP(factorial_recursive) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 1, 12); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT factorial(%d) = %d * factorial(%d)", + n, n, n - 1); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Power is strictly monotone in the exponent: for e1 < e2 and base >= 2, + * b^e1 < b^e2. Exact integer power, so no tolerance. e1 is drawn in 0..5 + * and e2 strictly above it up to 6. + */ +PG_PROP(pow_monotone) +{ + pg_prop_conn *conn; + char *err = NULL; + int b, + e1, + e2; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + b = (int) pg_prop_draw_int(tc, 2, 5); + e1 = (int) pg_prop_draw_int(tc, 0, 5); + e2 = (int) pg_prop_draw_int(tc, e1 + 1, 6); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%d::numeric ^ %d) < (%d::numeric ^ %d)", + b, e1, b, e2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ln is strictly monotone: for x1 < x2, ln(x1) < ln(x2). Both are rounded to + * the same scale, and for integers 1..100 the gap between successive ln values + * (>= ln(100)-ln(99) ~= 0.01) far exceeds the rounding of the final digit, so + * the strict inequality is sound. x1 is drawn in 1..99 and x2 strictly above. + */ +PG_PROP(ln_monotone) +{ + pg_prop_conn *conn; + char *err = NULL; + int x1, + x2; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x1 = (int) pg_prop_draw_int(tc, 1, 99); + x2 = (int) pg_prop_draw_int(tc, x1 + 1, 100); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ln(%d::numeric) < ln(%d::numeric)", + x1, x2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 46e60887db606..cbc8dee9fac6e 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 2b41eea52ece1cc795dd550cbab66213181c4e4e Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:47 -0400 Subject: [PATCH 028/338] prop: add float property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_float.c | 333 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 337 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_float.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 4a1f34124f54b..39b9c55c38cfe 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -43,7 +43,8 @@ OBJS = \ prop_decimalfmt.o \ prop_mathfn.o \ prop_mathround.o \ - prop_powexp.o + prop_powexp.o \ + prop_float.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index f187fa092228d..01d3de70cc1d5 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -41,6 +41,7 @@ prop_area_sources = files( 'prop_mathfn.c', 'prop_mathround.c', 'prop_powexp.c', + 'prop_float.c', ) diff --git a/src/test/prop/prop_float.c b/src/test/prop/prop_float.c new file mode 100644 index 0000000000000..59dc1383b660d --- /dev/null +++ b/src/test/prop/prop_float.c @@ -0,0 +1,333 @@ +/*------------------------------------------------------------------------- + * + * prop_float.c + * Property-based tests for float4/float8 output/input (the Ryu formatter). + * + * float8out/float4out use Ryu to produce the shortest decimal string that + * reads back as the identical IEEE value. The headline property is therefore + * an exact text round-trip: float8 -> text -> float8 is the identity for every + * value, including subnormals, +/-0, +/-Inf and NaN. Alongside the round-trip + * these properties check that the type's ordering and comparison operators are + * consistent (a total order on non-NaN values) and that negation and abs + * behave algebraically -- all of which must hold with no tolerance. + * + * The oracle here is the type's own text I/O and algebra: the server formats a + * value, we feed its own text back in, and require the result to equal the + * value we started from. Because the property is about the SERVER's output + * round-tripping, seeding the comparison from a "%.17g" literal (which itself + * round-trips any IEEE double) is fine -- we then compare the server's own + * ::text::float8 against that parsed literal. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_float.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "float" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +#include + +/* + * Draw one double across the widest possible range and format it as a SQL + * float8 literal into buf. With unbounded range the harness's float generator + * yields the special values too (NaN, +/-Inf, subnormals, +/-0); those are the + * point, so the range is deliberately not narrowed. + * + * Finite values are emitted with "%.17g", which round-trips any IEEE double + * exactly, so the literal names the drawn value precisely before it ever + * reaches the server. The specials have no decimal form and are emitted as + * their canonical PostgreSQL spellings. The returned double is the value + * passed in SQL, so a property can inspect it in C (e.g. to skip NaN). + */ +static double +draw_f8_literal(pg_prop_tc *tc, StringInfo buf) +{ + double v = pg_prop_draw_float(tc, -INFINITY, INFINITY); + + if (isnan(v)) + appendStringInfoString(buf, "'NaN'::float8"); + else if (isinf(v)) + appendStringInfoString(buf, v < 0 ? "'-Infinity'::float8" + : "'Infinity'::float8"); + else + { + char num[64]; + + snprintf(num, sizeof(num), "%.17g", v); + appendStringInfo(buf, "'%s'::float8", num); + } + return v; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the comparison produced NULL or errored on inputs the + * property is supposed to handle (almost always NaN leaking into a comparison, + * which a sound property must special-case rather than blame on the server) -- + * so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Like require_true, but a SQL error that is a legitimate out-of-range + * overflow (e.g. casting a float8 beyond the float4 range) discards the case + * rather than failing it. + */ +static void +require_true_or_skip(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + bool overflow = (err != NULL && + (strstr(err, "out of range") != NULL || + strstr(err, "overflow") != NULL)); + + if (err) + pg_free(err); + if (overflow) + pg_prop_assume(tc, false); /* legitimate overflow: skip */ + pg_prop_fail(tc, "SQL error for: %s", sql); + } + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); +} + +/* + * Core Ryu property: float8 -> text -> float8 is the identity. Ryu emits the + * shortest decimal that reads back as the identical value, so re-parsing the + * server's own output must recover the value exactly. NaN needs special + * handling because NaN <> NaN; for it we check that the text is 'NaN' instead. + */ +PG_PROP(text_roundtrip_f8) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_f8_literal(tc, &v); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN %s = 'NaN'::float8 " + "THEN (%s::text::float8) = 'NaN'::float8 OR (%s::text = 'NaN') " + "ELSE %s::text::float8 = %s END", + v.data, v.data, v.data, v.data, v.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Same round-trip for float4. The drawn double is cast to float4 first so it + * lands on a real float4 value, then that value's own text must read back as + * the identical float4. NaN special-cased as above. + */ +PG_PROP(text_roundtrip_f4) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_f8_literal(tc, &v); + + /* f is the drawn value landed on a float4 */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN (%s)::float4 = 'NaN'::float4 " + "THEN (%s)::float4::text = 'NaN' " + "ELSE (%s)::float4::text::float4 = (%s)::float4 END", + v.data, v.data, v.data, v.data); + + /* a float8 beyond the float4 range overflows on the cast; skip those */ + require_true_or_skip(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Non-NaN floats form a total order: of (V1 < V2), (V1 = V2), (V1 > V2) + * exactly one holds, and (V1 < V2) is the negation of (V1 >= V2). NaN is + * unordered, so cases drawing a NaN are discarded (checked in C before any SQL + * is built, since NaN would make every comparison NULL). + */ +PG_PROP(order_consistent_with_double) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v1, + v2, + q; + double d1, + d2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v1); + initStringInfo(&v2); + d1 = draw_f8_literal(tc, &v1); + d2 = draw_f8_literal(tc, &v2); + pg_prop_assume(tc, !isnan(d1) && !isnan(d2)); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s < %s) = NOT (%s >= %s)) " + "AND (((%s < %s)::int + (%s = %s)::int + (%s > %s)::int) = 1)", + v1.data, v2.data, v1.data, v2.data, + v1.data, v2.data, v1.data, v2.data, v1.data, v2.data); + + require_true(tc, conn, q.data); + + pfree(v1.data); + pfree(v2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Double negation is the identity: -(-V) = V. NaN <> NaN would make the + * equality NULL even though -NaN is still NaN, so the OR covers the NaN case + * explicitly. + */ +PG_PROP(negation_involution) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_f8_literal(tc, &v); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (- (- %s)) = %s OR (%s = 'NaN'::float8)", + v.data, v.data, v.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * abs is non-negative and equals either V or -V. NaN has no sign-magnitude + * meaning here (and would make the comparisons NULL), so NaN cases are + * discarded in C before building SQL. + */ +PG_PROP(abs_nonnegative) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + q; + double d; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + d = draw_f8_literal(tc, &v); + pg_prop_assume(tc, !isnan(d)); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(%s) >= 0 AND (abs(%s) = %s OR abs(%s) = -%s)", + v.data, v.data, v.data, v.data, v.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Targeted +/-0 check (no draw): +0 and -0 compare equal, and the text of -0 + * is well-defined -- Ryu's sign-of-zero handling must render it as '-0' (or + * '0'). A fixed-literal property, but valuable: signed zero is a classic + * formatter corner. + */ +PG_PROP(zero_signs_equal_but_text_distinct) +{ + pg_prop_conn *conn; + char *err = NULL; + + (void) tc; + (void) arg; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT ('0'::float8 = '-0'::float8) " + "AND (('-0'::float8)::text IN ('-0','0'))"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index cbc8dee9fac6e..5d932c5839a42 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 79d5009cdf87867ac64ecb2db63fe9c0e0583e15 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:47 -0400 Subject: [PATCH 029/338] prop: add float inf/nan ordering and arithmetic property tests (floatspecial) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_floatspecial.c | 286 ++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 290 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_floatspecial.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 39b9c55c38cfe..f8b0912af1850 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -44,7 +44,8 @@ OBJS = \ prop_mathfn.o \ prop_mathround.o \ prop_powexp.o \ - prop_float.o + prop_float.o \ + prop_floatspecial.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 01d3de70cc1d5..ae22e8c20b455 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -42,6 +42,7 @@ prop_area_sources = files( 'prop_mathround.c', 'prop_powexp.c', 'prop_float.c', + 'prop_floatspecial.c', ) diff --git a/src/test/prop/prop_floatspecial.c b/src/test/prop/prop_floatspecial.c new file mode 100644 index 0000000000000..8c3b8c7657485 --- /dev/null +++ b/src/test/prop/prop_floatspecial.c @@ -0,0 +1,286 @@ +/*------------------------------------------------------------------------- + * + * prop_floatspecial.c + * Property-based tests for float4/float8 special values and widening. + * + * This area targets the IEEE special values (+/-Infinity, NaN, signed zero) + * and the float4<->float8 relationship, not the Ryu round-trip the "float" + * area already covers. PostgreSQL gives floats a total order that differs + * from raw IEEE: NaN compares equal to NaN and sorts as the largest value, + * and -0 compares equal to +0. These properties pin those documented + * semantics plus infinity arithmetic and the precision loss of a float8 + * narrowed to float4 and widened back. A failure means a comparison or + * arithmetic routine disagrees with the type's defined total order, which is + * a bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_floatspecial.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "floatspecial" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a small finite float8 literal as "%d.%d" (whole.frac), parenthesized + * and cast, so a leading sign never misparses when substituted into a larger + * expression. Small magnitudes keep the float4<->float8 widening bound tight + * and avoid any infinity/overflow surprise -- the specials are supplied as + * literal casts by the properties that need them. + */ +static void +draw_finite_f8(pg_prop_tc *tc, StringInfo buf) +{ + int whole = (int) pg_prop_draw_int(tc, -1000, 1000); + int frac = (int) pg_prop_draw_int(tc, 0, 999); + + appendStringInfo(buf, "(%d.%d)::float8", whole, frac); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the oracle expression errored or produced NULL, which is + * itself a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * NaN equals NaN under PostgreSQL's total-order semantics: both the = operator + * and IS NOT DISTINCT FROM report NaN = NaN as true (unlike raw IEEE, where + * NaN compares unequal to everything including itself). + */ +PG_PROP(nan_not_equal_itself_but_is_not_distinct) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT ('NaN'::float8 = 'NaN'::float8) " + "AND ('NaN'::float8 IS NOT DISTINCT FROM 'NaN'::float8)"); + + pg_prop_disconnect(conn); +} + +/* + * NaN sorts as the largest value: it is greater than any finite value and also + * greater than +Infinity. + */ +PG_PROP(nan_greater_than_all) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_finite_f8(tc, &v); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('NaN'::float8 > %s) " + "AND ('NaN'::float8 > 'Infinity'::float8)", + v.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Infinity arithmetic: Infinity + finite = Infinity, and Infinity - Infinity + * is NaN. The second half relies on NaN = NaN being true to compare the + * result. + */ +PG_PROP(inf_arithmetic) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_finite_f8(tc, &v); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (('Infinity'::float8 + %s) = 'Infinity'::float8) " + "AND (('Infinity'::float8 - 'Infinity'::float8) " + "= 'NaN'::float8)", + v.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Infinity bounds the finite line: -Infinity < v < Infinity for any finite v. + */ +PG_PROP(inf_order) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_finite_f8(tc, &v); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('-Infinity'::float8 < %s) " + "AND (%s < 'Infinity'::float8)", + v.data, v.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * float4<->float8 widening: a finite v narrowed to float4 and widened back to + * float8 stays within float4's ~7-digit relative precision of v. The bound is + * a relative term (float4 carries ~1e-6 relative error) plus a small absolute + * term to cover values near zero. + */ +PG_PROP(float4_float8_widening) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_finite_f8(tc, &v); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(%s::float4::float8 - %s) " + "<= abs(%s) * 1e-6::float8 + 1e-6::float8", + v.data, v.data, v.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Signed zeros compare equal: +0 = -0. + */ +PG_PROP(float8_sign_zero) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT '0'::float8 = '-0'::float8"); + + pg_prop_disconnect(conn); +} + +/* + * The full defined total order is strict at each step: + * -Infinity < finite < Infinity < NaN. + */ +PG_PROP(special_total_order) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT ('-Infinity'::float8 < 0::float8) " + "AND (0::float8 < 'Infinity'::float8) " + "AND ('Infinity'::float8 < 'NaN'::float8)"); + + pg_prop_disconnect(conn); +} + +/* + * Specials round-trip through text: NaN and Infinity each format and reparse + * back to themselves. The NaN comparison relies on NaN = NaN being true. + */ +PG_PROP(nan_roundtrip_text) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT (('NaN'::float8)::text::float8 = 'NaN'::float8) " + "AND (('Infinity'::float8)::text::float8 " + "= 'Infinity'::float8)"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 5d932c5839a42..f263215cd0965 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From a6a7dbc5d310ffe37e4fb65e33e6ca206bafda42 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:48 -0400 Subject: [PATCH 030/338] prop: add float4/float8/numeric cast relationships property tests (floatcast) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_floatcast.c | 344 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 348 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_floatcast.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index f8b0912af1850..e62c4434080e1 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -45,7 +45,8 @@ OBJS = \ prop_mathround.o \ prop_powexp.o \ prop_float.o \ - prop_floatspecial.o + prop_floatspecial.o \ + prop_floatcast.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index ae22e8c20b455..799c3e1bd9036 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -43,6 +43,7 @@ prop_area_sources = files( 'prop_powexp.c', 'prop_float.c', 'prop_floatspecial.c', + 'prop_floatcast.c', ) diff --git a/src/test/prop/prop_floatcast.c b/src/test/prop/prop_floatcast.c new file mode 100644 index 0000000000000..58e0330bac845 --- /dev/null +++ b/src/test/prop/prop_floatcast.c @@ -0,0 +1,344 @@ +/*------------------------------------------------------------------------- + * + * prop_floatcast.c + * Property-based tests for float4/float8/numeric cast relationships. + * + * These check the IEEE-754 representability facts that PostgreSQL's binary + * float casts rely on: float4 widens to float8 exactly, every int4 (and every + * int8 below 2^52) is exact in float8, small decimals survive a float8<-> + * numeric round-trip within tolerance, numeric->float8 is monotone, and the + * float4 narrowing cast stays within its ~7-significant-digit precision. A + * failure is a wrong-results bug in the float/numeric cast code, not a + * rounding artefact: exact facts are asserted exactly, lossy ones only within + * a stated tolerance. + * + * Distinct from prop_float/floatspecial/numcast/numericcmp: this file is + * purely about the width/precision relationships *between* float4, float8 and + * numeric, not arithmetic, specials, integer-numeric casts, or comparison. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_floatcast.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "floatcast" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the cast errored or produced NULL on finite inputs, which + * is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Emit an integer literal safe for immediate casting. A bare negative like + * -2147483648::int4 parses as -(2147483648::int4), whose positive magnitude + * overflows the target type before negation; wrapping any negative in + * parentheses makes the cast apply to the already-negative value. INT64_MIN + * cannot be written as a positive literal at all (unused here, as the drawn + * ranges exclude it, but handled for safety). + */ +static void +append_int_literal(StringInfo buf, int64 n) +{ + if (n == INT64_MIN) + appendStringInfoString(buf, "(-9223372036854775807 - 1)"); + else if (n < 0) + appendStringInfo(buf, "(%lld)", (long long) n); + else + appendStringInfo(buf, "%lld", (long long) n); +} + +/* + * Draw a small exact decimal literal "." with up to 4 fractional + * digits, parenthesized (with a leading sign when negative) so it substitutes + * cleanly into a larger expression. The integer part is kept to a few digits + * so the value stays small enough that float8 and float4 round-trips behave as + * the properties expect. + */ +static void +draw_small_decimal(pg_prop_tc *tc, StringInfo buf) +{ + int nint = (int) pg_prop_draw_int(tc, 1, 4); + int nfrac = (int) pg_prop_draw_int(tc, 0, 4); + bool neg = pg_prop_draw_bool(tc); + + appendStringInfoChar(buf, '('); + if (neg) + appendStringInfoChar(buf, '-'); + for (int i = 0; i < nint; i++) + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + if (nfrac > 0) + { + appendStringInfoChar(buf, '.'); + for (int i = 0; i < nfrac; i++) + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + } + appendStringInfoChar(buf, ')'); +} + +/* + * 1. float4_to_float8_exact + * + * Widening a float4 value to float8 and narrowing it back is lossless: float8 + * exactly represents every float4, so v::float4::float8::float4 = v::float4 + * for a drawn finite v. v is drawn as a small decimal literal. + */ +PG_PROP(float4_to_float8_exact) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_small_decimal(tc, &v); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT %s::float4::float8::float4 = %s::float4", + v.data, v.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. int_to_float8_exact + * + * Every int4 has |n| < 2^31 < 2^53, so it is exactly representable in float8 + * and n::float8::int4 = n with no loss. + */ +PG_PROP(int_to_float8_exact) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_int_literal(&q, n); + appendStringInfoString(&q, "::int4::float8::int4) = "); + append_int_literal(&q, n); + appendStringInfoString(&q, "::int4"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. float8_numeric_roundtrip_small + * + * A small exact decimal x, once cast to float8, may not be exactly + * representable, so a float8->numeric->float8 round-trip is checked only + * within tolerance: abs(x::float8::numeric::float8 - x::float8) < 1e-9. The + * comparison is anchored to x::float8 (not x) so the inexactness of the + * decimal->binary step is not counted against the round-trip. + */ +PG_PROP(float8_numeric_roundtrip_small) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_small_decimal(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(%s::float8::numeric::float8 - %s::float8) " + "< 1e-9", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. numeric_to_float8_monotone + * + * The numeric->float8 cast is monotone: for two small numerics with a <= b, + * a::float8 <= b::float8. Two small decimals are drawn and ordered with + * least()/greatest() so the <= always holds on the inputs; the property is + * that casting preserves that order. + */ +PG_PROP(numeric_to_float8_monotone) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_small_decimal(tc, &a); + draw_small_decimal(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT least(%s::numeric, %s::numeric)::float8 " + "<= greatest(%s::numeric, %s::numeric)::float8", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. float4_precision_bound + * + * The float4 narrowing cast keeps ~7 significant decimal digits, so for a + * small drawn float8 v, abs(v::float4::float8 - v) <= abs(v) * 1e-6 + 1e-6. + * The relative term covers the ~2^-24 mantissa step for larger magnitudes and + * the absolute term covers values near zero. + */ +PG_PROP(float4_precision_bound) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_small_decimal(tc, &v); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(%s::float8::float4::float8 - %s::float8) " + "<= abs(%s::float8) * 1e-6 + 1e-6", + v.data, v.data, v.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. zero_casts + * + * Zero maps to zero across every width and type: 0::float4 = 0::float8, the + * numeric zero casts to float8 zero, and a float8 zero narrows to int4 zero. + * No inputs are drawn; this is a fixed sanity anchor for the zero element. + */ +PG_PROP(zero_casts) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT 0::float4 = 0::float8 " + "AND 0::numeric::float8 = 0::float8 " + "AND 0.0::float8::int4 = 0"); + + pg_prop_disconnect(conn); +} + +/* + * 7. int8_to_float8_large_bound + * + * float8 exactly represents every integer with |n| <= 2^53; drawing n in + * 0..2^52 stays well inside that, so n::float8::int8 = n with no loss even for + * values far beyond the int4 range checked in int_to_float8_exact. + */ +PG_PROP(int8_to_float8_large_bound) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, 0, INT64CONST(4503599627370496)); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_int_literal(&q, n); + appendStringInfoString(&q, "::int8::float8::int8) = "); + append_int_literal(&q, n); + appendStringInfoString(&q, "::int8"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index f263215cd0965..e474f82ba09de 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 7cce6cf7c3bd33848b0cbbc5da36aaf9a7fa2aa3 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:48 -0400 Subject: [PATCH 031/338] prop: add integer arithmetic laws and cast round-trips property tests (intops) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_intops.c | 387 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 391 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_intops.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index e62c4434080e1..3a0499f968b3f 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -46,7 +46,8 @@ OBJS = \ prop_powexp.o \ prop_float.o \ prop_floatspecial.o \ - prop_floatcast.o + prop_floatcast.o \ + prop_intops.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 799c3e1bd9036..d15f53f84496d 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -44,6 +44,7 @@ prop_area_sources = files( 'prop_float.c', 'prop_floatspecial.c', 'prop_floatcast.c', + 'prop_intops.c', ) diff --git a/src/test/prop/prop_intops.c b/src/test/prop/prop_intops.c new file mode 100644 index 0000000000000..79c24224f12da --- /dev/null +++ b/src/test/prop/prop_intops.c @@ -0,0 +1,387 @@ +/*------------------------------------------------------------------------- + * + * prop_intops.c + * Property-based tests for integer (int2/int4/int8) arithmetic algebra. + * + * The oracle here is integer arithmetic's own algebra: addition commutes and + * associates, multiplication distributes over addition, trunc-division obeys + * (a/b)*b + a%b = a, and abs/sign agree with their definitions. Operands are + * drawn in sub-ranges that keep every intermediate within int8, so the laws + * must hold with no overflow error -- any "f" result (or an unexpected error) + * is a bug in the integer arithmetic, not a legitimate out-of-range. One + * property instead checks that widening to bigint avoids an int4 overflow. + * + * This area is arithmetic algebra and overflow detection: distinct from + * "intbit" (bitwise operators) and "numcast" (cross-type cast round-trips). + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_intops.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "intops" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the arithmetic errored or produced NULL on inputs chosen + * to be in range, which is itself a bug -- so either is a failure. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Emit an integer literal safe for immediate casting. A bare negative like + * -2147483648::int4 parses as -(2147483648::int4), and the positive magnitude + * overflows the target type before negation; the same bites INT16_MIN::int2 + * and INT32_MIN::int4. Wrapping any negative in parentheses makes the cast + * apply to the already-negative value. The per-width minimum additionally + * cannot be written as a positive literal in its own width, so each is emitted + * via subtraction (INT64_MIN as "(-9223372036854775807 - 1)"). + * + * "width" is the number of bytes of the target type (2, 4 or 8); it selects + * the right MIN rendering so a drawn value equal to that width's minimum is + * never written as a bare over-wide positive. + */ +static void +append_int_literal(StringInfo buf, int64 n, int width) +{ + if (width == 8 && n == INT64_MIN) + appendStringInfoString(buf, "(-9223372036854775807 - 1)"); + else if (width == 4 && n == INT32_MIN) + appendStringInfoString(buf, "(-2147483647 - 1)"); + else if (width == 2 && n == INT16_MIN) + appendStringInfoString(buf, "(-32767 - 1)"); + else if (n < 0) + appendStringInfo(buf, "(%lld)", (long long) n); + else + appendStringInfo(buf, "%lld", (long long) n); +} + +/* + * 1. add_commutative + * + * a + b = b + a for int8 a, b in a safe range (-1e9..1e9) where every sum + * stays well within int8. + */ +PG_PROP(add_commutative) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a, + b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = pg_prop_draw_int(tc, -100000, 100000); + b = pg_prop_draw_int(tc, -100000, 100000); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_int_literal(&q, a, 8); + appendStringInfoString(&q, "::int8 + "); + append_int_literal(&q, b, 8); + appendStringInfoString(&q, "::int8) = ("); + append_int_literal(&q, b, 8); + appendStringInfoString(&q, "::int8 + "); + append_int_literal(&q, a, 8); + appendStringInfoString(&q, "::int8)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. add_associative + * + * (a + b) + c = a + (b + c) for int8 a, b, c in -1e6..1e6, so the total sum + * (<= 3e6 in magnitude) fits int8 with room to spare. + */ +PG_PROP(add_associative) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a, + b, + c; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = pg_prop_draw_int(tc, -10000, 10000); + b = pg_prop_draw_int(tc, -10000, 10000); + c = pg_prop_draw_int(tc, -10000, 10000); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT (("); + append_int_literal(&q, a, 8); + appendStringInfoString(&q, "::int8 + "); + append_int_literal(&q, b, 8); + appendStringInfoString(&q, "::int8) + "); + append_int_literal(&q, c, 8); + appendStringInfoString(&q, "::int8) = ("); + append_int_literal(&q, a, 8); + appendStringInfoString(&q, "::int8 + ("); + append_int_literal(&q, b, 8); + appendStringInfoString(&q, "::int8 + "); + append_int_literal(&q, c, 8); + appendStringInfoString(&q, "::int8))"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. mul_distributes_over_add + * + * a * (b + c) = a*b + a*c for int8 a, b, c in -1000..1000. Each product is at + * most ~2e6 and the sum ~4e6, far inside int8, so no overflow is possible. + */ +PG_PROP(mul_distributes_over_add) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a, + b, + c; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = pg_prop_draw_int(tc, -1000, 1000); + b = pg_prop_draw_int(tc, -1000, 1000); + c = pg_prop_draw_int(tc, -1000, 1000); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_int_literal(&q, a, 8); + appendStringInfoString(&q, "::int8 * ("); + append_int_literal(&q, b, 8); + appendStringInfoString(&q, "::int8 + "); + append_int_literal(&q, c, 8); + appendStringInfoString(&q, "::int8)) = ("); + append_int_literal(&q, a, 8); + appendStringInfoString(&q, "::int8 * "); + append_int_literal(&q, b, 8); + appendStringInfoString(&q, "::int8 + "); + append_int_literal(&q, a, 8); + appendStringInfoString(&q, "::int8 * "); + append_int_literal(&q, c, 8); + appendStringInfoString(&q, "::int8)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. div_mod_identity + * + * For b <> 0, (a/b)*b + (a%b) = a. SQL integer division truncates toward + * zero, and % is the matching remainder, so this identity is exact. b is + * drawn in -1000..1000; the loop re-draws until b <> 0 so no division by zero + * is attempted. a in -1e6..1e6. Every intermediate fits int8. + */ +PG_PROP(div_mod_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a, + b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = pg_prop_draw_int(tc, -10000, 10000); + /* draw a non-zero divisor without a reject loop: 1..1000 with a sign */ + b = pg_prop_draw_int(tc, 1, 1000); + if (pg_prop_draw_bool(tc)) + b = -b; + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT (("); + append_int_literal(&q, a, 8); + appendStringInfoString(&q, "::int8 / "); + append_int_literal(&q, b, 8); + appendStringInfoString(&q, "::int8) * "); + append_int_literal(&q, b, 8); + appendStringInfoString(&q, "::int8 + ("); + append_int_literal(&q, a, 8); + appendStringInfoString(&q, "::int8 % "); + append_int_literal(&q, b, 8); + appendStringInfoString(&q, "::int8)) = "); + append_int_literal(&q, a, 8); + appendStringInfoString(&q, "::int8"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. abs_sign + * + * abs(a) >= 0, sign(a) in (-1, 0, 1), and a = sign(a) * abs(a). Drawn over + * the full int8 range EXCLUDING INT64_MIN, whose absolute value overflows + * int8; INT64_MIN + 1 is the lowest value drawn so abs() never errors. + */ +PG_PROP(abs_sign) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = pg_prop_draw_int(tc, INT64_MIN + 1, INT64_MAX); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT abs("); + append_int_literal(&q, a, 8); + appendStringInfoString(&q, "::int8) >= 0 AND sign("); + append_int_literal(&q, a, 8); + appendStringInfoString(&q, "::int8) IN (-1, 0, 1) AND "); + append_int_literal(&q, a, 8); + appendStringInfoString(&q, "::int8 = sign("); + append_int_literal(&q, a, 8); + appendStringInfoString(&q, "::int8) * abs("); + append_int_literal(&q, a, 8); + appendStringInfoString(&q, "::int8)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. int4_overflow_detected (widening avoids overflow) + * + * 2147483647 + k overflows int4 for any k in 1..1000, so the int4 addition is + * the correct error path -- but asserting an error is awkward. Instead assert + * the sound positive property: widening the left operand to bigint lets the + * same addition succeed and give the true sum, 2147483647 + k. This checks + * that widening is what avoids the int4 overflow. (The int4 add itself is not + * evaluated here; only the widened bigint add, which cannot overflow for these + * operands.) + */ +PG_PROP(int4_overflow_detected) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 k; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + k = pg_prop_draw_int(tc, 1, 1000); + + /* + * Adding past INT32_MAX in bigint must give the exact mathematical result + * (no int4 wraparound). Both sides are evaluated in bigint -- the right + * side crosses the int4 boundary a different way (2147483648 + (k-1)) so + * it is an independent computation, not a copy of the left. + */ + initStringInfo(&q); + appendStringInfoString(&q, "SELECT (2147483647::bigint + "); + append_int_literal(&q, k, 8); + appendStringInfoString(&q, "::bigint) = (2147483648::bigint + "); + append_int_literal(&q, k - 1, 8); + appendStringInfoString(&q, "::bigint)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. cast_narrow_roundtrip_when_in_range + * + * For int8 n within the int2 range (-32768..32767), n::int2::int8 = n; and + * for n within the int4 range, n::int4::int8 = n. Both narrowing casts are + * lossless when the value fits, so the round-trip is the identity. n is drawn + * in the int2 range so BOTH halves of the AND are in range at once (every + * int2 value is also an int4 value). + */ +PG_PROP(cast_narrow_roundtrip_when_in_range) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, INT16_MIN, INT16_MAX); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + append_int_literal(&q, n, 2); + appendStringInfoString(&q, "::int2::int8 = "); + append_int_literal(&q, n, 8); + appendStringInfoString(&q, "::int8 AND "); + append_int_literal(&q, n, 4); + appendStringInfoString(&q, "::int4::int8 = "); + append_int_literal(&q, n, 8); + appendStringInfoString(&q, "::int8"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index e474f82ba09de..0348e5c1de85f 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From bd4da6c4dc860f71f1f17eeba86810364bc3b7a1 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:49 -0400 Subject: [PATCH 032/338] prop: add intbit property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_intbit.c | 315 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 319 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_intbit.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 3a0499f968b3f..b3dd323070f12 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -47,7 +47,8 @@ OBJS = \ prop_float.o \ prop_floatspecial.o \ prop_floatcast.o \ - prop_intops.o + prop_intops.o \ + prop_intbit.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index d15f53f84496d..0efe89541c664 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -45,6 +45,7 @@ prop_area_sources = files( 'prop_floatspecial.c', 'prop_floatcast.c', 'prop_intops.c', + 'prop_intbit.c', ) diff --git a/src/test/prop/prop_intbit.c b/src/test/prop/prop_intbit.c new file mode 100644 index 0000000000000..0ab54222fef3a --- /dev/null +++ b/src/test/prop/prop_intbit.c @@ -0,0 +1,315 @@ +/*------------------------------------------------------------------------- + * + * prop_intbit.c + * Property-based tests for integer bitwise operators and shifts. + * + * Integer bitwise operators and shifts have well-defined algebraic laws; + * these check them on int4 and int8 across the full value range. + * + * The operators under test are the integer bitwise AND (&), OR (|), XOR (#), + * NOT (~) and the arithmetic shifts (<<, >>) on int4 and int8. Unlike + * floating point, these are exact: De Morgan's laws, XOR self-inverse, + * absorption, double complement and the shift/multiply correspondence all + * hold with no tolerance, so any "f" result is a bug, not rounding. + * + * The pure-algebra properties (De Morgan, XOR, absorption, NOT, identity/ + * annihilator) draw the FULL int4/int8 range -- boundary values such as + * INT32_MIN, INT64_MIN, 0 and -1 are exactly where two's-complement bit + * tricks go wrong, so they are the point. Only the shift properties bound + * their inputs, so a << s cannot overflow int8 and turn a legitimate + * algebraic identity into an overflow error. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_intbit.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "intbit" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the bitwise op errored or produced NULL on finite inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * De Morgan over bitwise ops on int4: ~(a & b) = (~a) | (~b). Drawn across + * the full int4 range so INT32_MIN and -1 (all-ones) are exercised. + */ +PG_PROP(and_or_demorgan_int4) +{ + pg_prop_conn *conn; + char *err = NULL; + int a; + int b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + b = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (~(((%d)::int4) & ((%d)::int4))) " + "= ((~((%d)::int4)) | (~((%d)::int4)))", + a, b, a, b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * XOR is its own inverse: a # a = 0. Drawn across the full int4 range. + */ +PG_PROP(xor_self_zero_int4) +{ + pg_prop_conn *conn; + char *err = NULL; + int a; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (((%d)::int4) # ((%d)::int4)) = 0::int4", + a, a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * XOR by the same value twice is identity: (a # b) # b = a. Drawn across the + * full int8 range so INT64_MIN is exercised. + */ +PG_PROP(xor_twice_identity_int8) +{ + pg_prop_conn *conn; + char *err = NULL; + long long a; + long long b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (long long) pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + b = (long long) pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((((%lld)::int8) # ((%lld)::int8)) # ((%lld)::int8)) " + "= ((%lld)::int8)", + a, b, b, a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Absorption laws on int4: a & (a | b) = a and a | (a & b) = a. Drawn across + * the full int4 range. + */ +PG_PROP(and_or_absorption_int4) +{ + pg_prop_conn *conn; + char *err = NULL; + int a; + int b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + b = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (((%d)::int4) & (((%d)::int4) | ((%d)::int4))) " + "= ((%d)::int4) " + "AND (((%d)::int4) | (((%d)::int4) & ((%d)::int4))) " + "= ((%d)::int4)", + a, a, b, a, + a, a, b, a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Double complement is identity: ~(~a) = a. Drawn across the full int8 + * range. + */ +PG_PROP(not_involution_int8) +{ + pg_prop_conn *conn; + char *err = NULL; + long long a; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (long long) pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (~(~((%lld)::int8))) = ((%lld)::int8)", + a, a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Left shift is multiply by a power of two: a << s = a * 2^s. The 2^s + * multiplier is built in C as (1LL << s) and both a (0..1000000) and s (0..10) + * are bounded so a * (1 << s) stays well within int8 -- no overflow, so the + * identity holds with no error. + */ +PG_PROP(shift_left_is_multiply_small) +{ + pg_prop_conn *conn; + char *err = NULL; + long long a; + int s; + long long m; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (long long) pg_prop_draw_int(tc, 0, 1000000); + s = (int) pg_prop_draw_int(tc, 0, 10); + m = 1LL << s; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (((%lld)::int8) << %d) " + "= (((%lld)::int8) * ((%lld)::int8))", + a, s, a, m); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Left then right shift by the same amount restores a: (a << s) >> s = a, when + * no bits are lost to overflow. Bounding a to -1000000..1000000 and s to + * 0..10 guarantees a << s does not overflow int8, so the floor-division of the + * right shift exactly undoes the left shift. + */ +PG_PROP(shift_right_left_floor_int8) +{ + pg_prop_conn *conn; + char *err = NULL; + long long a; + int s; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (long long) pg_prop_draw_int(tc, -1000000, 1000000); + s = (int) pg_prop_draw_int(tc, 0, 10); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((((%lld)::int8) << %d) >> %d) = ((%lld)::int8)", + a, s, s, a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Identity and annihilator for AND on int4: a & -1 = a (since -1 is all-ones + * in two's complement) and a & 0 = 0. Drawn across the full int4 range. + */ +PG_PROP(and_identity_annihilator_int4) +{ + pg_prop_conn *conn; + char *err = NULL; + int a; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (((%d)::int4) & ((-1)::int4)) = ((%d)::int4) " + "AND (((%d)::int4) & (0::int4)) = (0::int4)", + a, a, a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 0348e5c1de85f..103be3cbbb7aa 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From a12088f8c4fc27823249a8758e8b67b96ef9aefc Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:49 -0400 Subject: [PATCH 033/338] prop: add bit-string and integer cast identities property tests (bitint) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_bitint.c | 315 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 319 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_bitint.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index b3dd323070f12..943e83c36170f 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -48,7 +48,8 @@ OBJS = \ prop_floatspecial.o \ prop_floatcast.o \ prop_intops.o \ - prop_intbit.o + prop_intbit.o \ + prop_bitint.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 0efe89541c664..1a8250c9033a5 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -46,6 +46,7 @@ prop_area_sources = files( 'prop_floatcast.c', 'prop_intops.c', 'prop_intbit.c', + 'prop_bitint.c', ) diff --git a/src/test/prop/prop_bitint.c b/src/test/prop/prop_bitint.c new file mode 100644 index 0000000000000..30f3ce68bd95a --- /dev/null +++ b/src/test/prop/prop_bitint.c @@ -0,0 +1,315 @@ +/*------------------------------------------------------------------------- + * + * prop_bitint.c + * Property-based tests for bit-string <-> integer casts and identities. + * + * This area targets the int4 <-> bit(32) cast path and the bit-level + * identities that must hold on the bit string produced from an integer: the + * cast is a two's-complement reinterpretation, so a round trip must recover + * the original int4 across the full range (including INT32_MIN and -1 where + * sign handling goes wrong), length is always the declared width, and the + * boolean algebra (AND/OR/NOT) plus the shift/multiply correspondence and a + * popcount cross-check all hold exactly. Any "f" is a cast or bit-op bug. + * + * Distinct from prop_bit (bit-string ops in isolation), prop_bitstringops + * (set/get/shift on literal bit strings) and prop_intbit (integer bitwise + * operators): here the subject is specifically the cast between int and + * bit(N) and the equalities that bridge the two representations. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_bitint.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "bitint" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the cast errored or produced NULL on a finite integer, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Round trip: n::bit(32)::int = n for any int4 n across the full range. The + * int->bit cast lays down the two's-complement bits and the bit->int cast + * reads them back, so the composition is the identity; INT32_MIN and -1 are + * exactly where sign handling can go wrong, so the full range is drawn. The + * literal is substituted as "(%d)" so a negative never misparses. + */ +PG_PROP(int_to_bit_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%d)::int4::bit(32)::int4 = (%d)::int4", + n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Length is the declared width: length(n::bit(32)) = 32 for any int4 n. The + * cast must always yield exactly 32 bits regardless of the integer's value. + */ +PG_PROP(bit_length_32) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT length((%d)::int4::bit(32)) = 32", n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Idempotence of AND with self: (b & b) = b where b = n::bit(32). Bitwise + * AND of a value with itself is the value, so this must hold for the bits of + * any integer. + */ +PG_PROP(and_with_self) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%d)::int4::bit(32) & (%d)::int4::bit(32)) " + "= (%d)::int4::bit(32)", + n, n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * OR with all-zero identity: (b | 0::int::bit(32)) = b. The bit string of + * integer 0 is the 32-bit all-zero string, which is the identity element for + * bitwise OR, so OR-ing it with any value leaves the value unchanged. + */ +PG_PROP(or_with_zero) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%d)::int4::bit(32) | (0)::int4::bit(32)) " + "= (%d)::int4::bit(32)", + n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Shift/multiply correspondence: (n::bit(32) << k)::int = (n * 2^k)::int for + * small non-negative n and k, with the range guarded so n * 2^k stays below + * 2^31 and no bit shifts out of the top (which would break the equality) and + * no int4 overflow occurs. n is 0..1000 and k is 0..10, and the case is only + * asserted when n * 2^k < 2^31; otherwise the property trivially passes so a + * guarded-out draw is not a failure. + */ +PG_PROP(shift_eq_mul) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + int k; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 0, 1000); + k = (int) pg_prop_draw_int(tc, 0, 10); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN (%d)::int8 * (2^%d)::int8 < 2147483648 " + "THEN ((%d)::int4::bit(32) << %d)::int4 " + "= ((%d) * (2^%d)::int4)::int4 " + "ELSE true END", + n, k, n, k, n, k); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Double complement is the identity: ~(~b) = b where b = n::bit(32). Two + * applications of bitwise NOT flip every bit back, recovering the original + * bit string for any integer. + */ +PG_PROP(not_not_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ~(~((%d)::int4::bit(32))) = (%d)::int4::bit(32)", + n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Popcount cross-check: the number of 1 bits counted two independent ways + * must agree. The textual way is length(text) - length(text with 1s + * removed); the per-bit way is sum(get_bit(b, i)) over i in 0..31. Both count + * the set bits of n::bit(32), so they must be equal for every integer -- a + * disagreement means get_bit or the bit->text rendering is inconsistent with + * the stored bits. + */ +PG_PROP(popcount_consistency) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT sum(get_bit((%d)::int4::bit(32), i)) " + "FROM generate_series(0, 31) i) " + "= length((%d)::int4::bit(32)::text) " + "- length(replace((%d)::int4::bit(32)::text, '1', ''))", + n, n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Concatenation width: two 16-bit strings concatenate to exactly 32 bits. + * length(a::bit(16) || b::bit(16)) = 32 for any two int4 values a, b, since + * each half is a fixed 16-bit cast and || sums the widths. + */ +PG_PROP(bit_concat_int) +{ + pg_prop_conn *conn; + char *err = NULL; + int a; + int b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + b = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length((%d)::int4::bit(16) " + "|| (%d)::int4::bit(16)) = 32", + a, b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 103be3cbbb7aa..487eff9e66504 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From ed04f9239c2ac80b8becc30650db164510c59ce1 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:49 -0400 Subject: [PATCH 034/338] prop: add integer bitwise algebra identities property tests (bitlogic) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_bitlogic.c | 390 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 394 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_bitlogic.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 943e83c36170f..58b32a949e83e 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -49,7 +49,8 @@ OBJS = \ prop_floatcast.o \ prop_intops.o \ prop_intbit.o \ - prop_bitint.o + prop_bitint.o \ + prop_bitlogic.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 1a8250c9033a5..e5dc4e2241e3a 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -47,6 +47,7 @@ prop_area_sources = files( 'prop_intops.c', 'prop_intbit.c', 'prop_bitint.c', + 'prop_bitlogic.c', ) diff --git a/src/test/prop/prop_bitlogic.c b/src/test/prop/prop_bitlogic.c new file mode 100644 index 0000000000000..1efcf1dbc12ac --- /dev/null +++ b/src/test/prop/prop_bitlogic.c @@ -0,0 +1,390 @@ +/*------------------------------------------------------------------------- + * + * prop_bitlogic.c + * Property-based tests for int4 bitwise-logic identities. + * + * The oracle is Boolean algebra over two's-complement int4: idempotence, + * absorption, De Morgan's laws, XOR's self-inverse / commutativity / + * associativity, the shift round-trip, double complement, the identity + * elements of AND/OR/XOR, and the ~n = -n-1 two's-complement relation. Each + * is an exact algebraic law that must hold for every int4 input (shifts are + * range-guarded, and the complement relation is evaluated in bigint to dodge + * the one INT_MIN overflow), so any "f" result is a bug in the integer + * bitwise operators, not a numeric artefact. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_bitlogic.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "bitlogic" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw an int4 and render it as an SQL literal into buf, parenthesized so a + * leading minus never misparses in a larger expression and so INT_MIN is + * emitted as "(-2147483647 - 1)" rather than a bare -2147483648 (which would + * parse as -(2147483648) and overflow int4). + */ +static void +draw_int4(pg_prop_tc *tc, StringInfo buf) +{ + int64 n = pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + if (n == INT32_MIN) + appendStringInfoString(buf, "((-2147483647 - 1)::int4)"); + else + appendStringInfo(buf, "((%lld)::int4)", (long long) n); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operator errored or produced NULL on finite inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * n & n = n. AND is idempotent. + */ +PG_PROP(and_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + draw_int4(tc, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s & %s) = %s", n.data, n.data, n.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * n | n = n. OR is idempotent. + */ +PG_PROP(or_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + draw_int4(tc, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s | %s) = %s", n.data, n.data, n.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * n # n = 0. XOR is its own inverse, so a value XORed with itself clears. + */ +PG_PROP(xor_self_zero) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + draw_int4(tc, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s # %s) = 0", n.data, n.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Absorption: n & (n | m) = n AND n | (n & m) = n. + */ +PG_PROP(and_or_absorption) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + m, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + initStringInfo(&m); + draw_int4(tc, &n); + draw_int4(tc, &m); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s & (%s | %s)) = %s " + "AND (%s | (%s & %s)) = %s", + n.data, n.data, m.data, n.data, + n.data, n.data, m.data, n.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(m.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * De Morgan: ~(n & m) = (~n) | (~m) AND ~(n | m) = (~n) & (~m). + */ +PG_PROP(de_morgan) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + m, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + initStringInfo(&m); + draw_int4(tc, &n); + draw_int4(tc, &m); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (~(%s & %s)) = ((~%s) | (~%s)) " + "AND (~(%s | %s)) = ((~%s) & (~%s))", + n.data, m.data, n.data, m.data, + n.data, m.data, n.data, m.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(m.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * XOR commutes and associates: n # m = m # n AND (n # m) # k = n # (m # k). + */ +PG_PROP(xor_commutative_assoc) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + m, + k, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + initStringInfo(&m); + initStringInfo(&k); + draw_int4(tc, &n); + draw_int4(tc, &m); + draw_int4(tc, &k); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s # %s) = (%s # %s) " + "AND ((%s # %s) # %s) = (%s # (%s # %s))", + n.data, m.data, m.data, n.data, + n.data, m.data, k.data, n.data, m.data, k.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(m.data); + pfree(k.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Shift round-trip: for small non-negative n (0..1000) and k (0..10), when + * n << k still fits in int4, (n << k) >> k = n. n is drawn in 0..1000 and k + * in 0..10, so the maximum left-shifted value is 1000 << 10 = 1024000, well + * within int4; no overflow is possible and the right shift exactly undoes the + * left shift because no high bits are lost. + */ +PG_PROP(shift_inverse_small) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 0, 1000); + int k = (int) pg_prop_draw_int(tc, 0, 10); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%d::int4 << %d) >> %d) = %d::int4", + n, k, k, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Double complement: ~(~n) = n. NOT is an involution. + */ +PG_PROP(not_not) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + draw_int4(tc, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (~(~%s)) = %s", n.data, n.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Identity elements: n & 0 = 0 AND n | 0 = n AND n # 0 = n. 0 annihilates + * under AND and is the identity under OR and XOR. + */ +PG_PROP(and_zero) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + draw_int4(tc, &n); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s & 0) = 0 " + "AND (%s | 0) = %s " + "AND (%s # 0) = %s", + n.data, n.data, n.data, n.data, n.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Two's-complement relation: ~n = -n - 1. In int4 this overflows for + * n = INT_MIN (since -INT_MIN is not representable), so it is evaluated in + * bigint: (~n)::bigint = (-(n::bigint)) - 1. ~n is computed first in int4 + * (always well-defined) and then widened, so no int4 overflow can occur for + * any input, including INT_MIN. + */ +PG_PROP(complement_relation) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + draw_int4(tc, &n); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (~%s)::bigint = (-(%s::bigint)) - 1", + n.data, n.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 487eff9e66504..c495069a8f99d 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 0997333d47dbe9aa52cdcb4df14574c436249ecf Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:50 -0400 Subject: [PATCH 035/338] prop: add bit-string concat, shift, and bitwise laws property tests (bitstringops) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_bitstringops.c | 261 ++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 265 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_bitstringops.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 58b32a949e83e..a55a1b2c69ca8 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -50,7 +50,8 @@ OBJS = \ prop_intops.o \ prop_intbit.o \ prop_bitint.o \ - prop_bitlogic.o + prop_bitlogic.o \ + prop_bitstringops.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index e5dc4e2241e3a..51e0f60776a90 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -48,6 +48,7 @@ prop_area_sources = files( 'prop_intbit.c', 'prop_bitint.c', 'prop_bitlogic.c', + 'prop_bitstringops.c', ) diff --git a/src/test/prop/prop_bitstringops.c b/src/test/prop/prop_bitstringops.c new file mode 100644 index 0000000000000..dadff4a4d476b --- /dev/null +++ b/src/test/prop/prop_bitstringops.c @@ -0,0 +1,261 @@ +/*------------------------------------------------------------------------- + * + * prop_bitstringops.c + * Property-based tests for bit(n)/varbit string operations. + * + * Exercises the bit-string operators distinct from the "bit" area: string + * concatenation (||), left/right shift (<<, >>), bitwise AND/OR/XOR of + * equal-length operands, length(), position(), get_bit() and set_bit(). The + * oracle is the algebra these operations must satisfy -- De Morgan over an + * equal-length AND/OR/NOT, XOR-with-self is all zeros, set_bit then get_bit + * reads back the written bit, and both concatenation and shifting have exact + * length laws -- so any "f" result is a bug in varbit.c, not a tolerance. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_bitstringops.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "bitstringops" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a bit string literal B'<0/1 chars>' of length 1..16, drawing each + * character independently as 0 or 1. Returns the chosen length so callers + * that need a matching length (equal-length bitwise ops) can draw patterns of + * the same width. + */ +static int +draw_bitstring_len(pg_prop_tc *tc, StringInfo buf, int len) +{ + int i; + + appendStringInfoString(buf, "B'"); + for (i = 0; i < len; i++) + appendStringInfoChar(buf, pg_prop_draw_bool(tc) ? '1' : '0'); + appendStringInfoChar(buf, '\''); + return len; +} + +/* + * Draw a bit string of a length chosen here (1..16). Returns the length. + */ +static int +draw_bitstring(pg_prop_tc *tc, StringInfo buf) +{ + int len = (int) pg_prop_draw_int(tc, 1, 16); + + return draw_bitstring_len(tc, buf, len); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operation errored or produced NULL on valid inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Concatenation length law: length(a || b) = length(a) + length(b) for two + * varbit values of independently drawn lengths. + */ +PG_PROP(concat_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_bitstring(tc, &a); + draw_bitstring(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(%s::varbit || %s::varbit) " + "= length(%s::varbit) + length(%s::varbit)", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * De Morgan over equal-length bit(n): ~(a & b) = (~a) | (~b). One length n + * is drawn once, then two independent patterns of that length, so the AND/OR + * and NOT all operate on operands of identical width. + */ +PG_PROP(and_or_demorgan) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 1, 16); + initStringInfo(&a); + initStringInfo(&b); + draw_bitstring_len(tc, &a, n); + draw_bitstring_len(tc, &b, n); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ~(%s::bit(%d) & %s::bit(%d)) " + "= ((~ %s::bit(%d)) | (~ %s::bit(%d)))", + a.data, n, b.data, n, + a.data, n, b.data, n); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * XOR with self is all zeros: a # a has no 1 bits, so searching for a '1' in + * its text rendering yields position 0 (not found). This is sound for any + * drawn a of any length. + */ +PG_PROP(xor_self_zero) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_bitstring(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT position('1' in (%s::varbit # %s::varbit)::text) = 0", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * set_bit/get_bit round-trip: after set_bit(a, k, 1) the bit at position k + * reads back 1, for a drawn k in 0..length(a)-1. The length is drawn here so + * k can be bounded correctly. + */ +PG_PROP(setbit_getbit) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + int k; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 1, 16); + k = (int) pg_prop_draw_int(tc, 0, n - 1); + initStringInfo(&a); + draw_bitstring_len(tc, &a, n); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT get_bit(set_bit(%s::varbit, %d, 1), %d) = 1", + a.data, k, k); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Shifting preserves length: for a bit string a, length(a << 1) = length(a) + * and length(a >> 1) = length(a). Bit-string shift keeps the width and drops + * or zero-fills, so the length is invariant. + */ +PG_PROP(shift_length_preserved) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_bitstring(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(%s::varbit << 1) = length(%s::varbit) " + "AND length(%s::varbit >> 1) = length(%s::varbit)", + a.data, a.data, a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index c495069a8f99d..1961a5a13d483 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 734f2f008262cc9fe464fa9e4ec8709b5956821a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:50 -0400 Subject: [PATCH 036/338] prop: add integer/numeric overflow-safe widening property tests (safemath) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_safemath.c | 289 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 293 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_safemath.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index a55a1b2c69ca8..95dfce55ee3b3 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -51,7 +51,8 @@ OBJS = \ prop_intbit.o \ prop_bitint.o \ prop_bitlogic.o \ - prop_bitstringops.o + prop_bitstringops.o \ + prop_safemath.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 51e0f60776a90..7c0e025b6891e 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -49,6 +49,7 @@ prop_area_sources = files( 'prop_bitint.c', 'prop_bitlogic.c', 'prop_bitstringops.c', + 'prop_safemath.c', ) diff --git a/src/test/prop/prop_safemath.c b/src/test/prop/prop_safemath.c new file mode 100644 index 0000000000000..5a8a6a80ed663 --- /dev/null +++ b/src/test/prop/prop_safemath.c @@ -0,0 +1,289 @@ +/*------------------------------------------------------------------------- + * + * prop_safemath.c + * Property-based tests for integer/numeric overflow detection + * (src/backend/utils/adt/int.c, int8.c, numeric.c; src/include/common/int.h). + * + * PostgreSQL promises that integer arithmetic either returns the exact result + * or raises "integer out of range" -- it must never wrap silently. The + * properties here assert the POSITIVE side of that contract: when a value or + * operation is widened to a type with the headroom to hold it (int2->int8, + * int4->int8, int8->numeric), the widened result equals the exact + * mathematical answer. A silent wrap in the narrow type, or a wrong result + * after widening, breaks one of these identities; that is a correctness bug. + * + * These are deliberately positive invariants rather than "the server must + * error here" checks, so there is no skip/overflow gating to make the run + * flaky. Values are drawn near the type boundaries (sqrt(INT_MAX), INT*_MAX, + * INT*_MIN) because that is where wrap bugs live. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_safemath.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "safemath" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the widened arithmetic errored or produced NULL on inputs + * that are supposed to be in range, which is itself a bug -- so either is a + * failure, reported with the SQL. (Copied from prop_numeric.c's require_true.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * 1. int4_mul_overflow_widens + * + * For int4 a, b drawn near sqrt(INT_MAX) (~46340), the product a*b may exceed + * int4 but always fits int8. Computed in bigint it must equal the exact + * product -- cross-checked against the numeric product, which has no range at + * all. a and b are emitted parenthesized and cast so a negative draw never + * misparses, then cast to bigint before multiplying so the multiply itself + * happens in int8 (no int4 wrap, no int4 overflow error). + */ +PG_PROP(int4_mul_overflow_widens) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int a; + int b; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* near +-sqrt(INT_MAX): products straddle the int4 boundary */ + a = (int) pg_prop_draw_int(tc, -46341, 46341); + b = (int) pg_prop_draw_int(tc, -46341, 46341); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%d)::int4::bigint * (%d)::int4::bigint) " + "= ((%d)::numeric * (%d)::numeric)", + a, b, a, b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. int2_add_boundary + * + * 32767::int2 + 1 overflows int2, but the same addition widened to bigint + * yields exactly 32768. Asserting the bigint form confirms the correct value + * is reachable the moment the type has headroom; a silent int2 wrap would land + * at -32768 instead. + */ +PG_PROP(int2_add_boundary) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, "SELECT (32767::bigint + 1) = 32768"); + + pg_prop_disconnect(conn); +} + +/* + * 3. numeric_no_overflow + * + * numeric has no fixed range, so a large product can never overflow. Its + * only obligation is to be exact: a*b must equal b*a. Drawing wide integer + * values (up to ~18 decimal digits each, so products reach ~36 digits, far + * past int8) exercises the arbitrary-precision multiply with no range gating. + */ +PG_PROP(numeric_no_overflow) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a; + StringInfoData b; + StringInfoData q; + int i; + int na; + int nb; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* build two large decimal integer literals digit by digit */ + na = (int) pg_prop_draw_int(tc, 1, 18); + nb = (int) pg_prop_draw_int(tc, 1, 18); + + initStringInfo(&a); + initStringInfo(&b); + appendStringInfoString(&a, "("); + for (i = 0; i < na; i++) + appendStringInfoChar(&a, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + appendStringInfoString(&a, ")::numeric"); + appendStringInfoString(&b, "("); + for (i = 0; i < nb; i++) + appendStringInfoChar(&b, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + appendStringInfoString(&b, ")::numeric"); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s * %s) = (%s * %s)", + a.data, b.data, b.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. int8_near_max_add + * + * Staying strictly under INT64_MAX must never overflow: for k in 1..1000, + * (INT64_MAX - k)::bigint + k = INT64_MAX. This confirms addition that lands + * exactly on the maximum succeeds and is exact (no spurious overflow error, + * no wrap). + */ +PG_PROP(int8_near_max_add) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int k; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + k = (int) pg_prop_draw_int(tc, 1, 1000); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((9223372036854775807 - %d)::bigint + %d) " + "= 9223372036854775807", + k, k); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. abs_min_overflows_widens + * + * abs(INT32_MIN) overflows int4 (its magnitude is one past INT32_MAX), but + * widening to bigint first makes the magnitude representable: + * abs((-2147483648)::int4::bigint) = 2147483648. The negative literal is + * parenthesized so it is parsed as the value -2147483648, not -(2147483648). + */ +PG_PROP(abs_min_overflows_widens) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT abs((-2147483648)::int4::bigint) = 2147483648"); + + pg_prop_disconnect(conn); +} + +/* + * 6. div_by_type_min + * + * INT32_MIN / -1 overflows int4 (the quotient is one past INT32_MAX), but in + * bigint the quotient fits: (-2147483648)::bigint / (-1) = 2147483648. The + * negated operands are parenthesized so each parses as its intended value. + */ +PG_PROP(div_by_type_min) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT ((-2147483648)::bigint / (-1)) = 2147483648"); + + pg_prop_disconnect(conn); +} + +/* + * 7. mul_identity_safe + * + * The multiplicative identities a*1 = a and a*0 = 0 can never overflow for any + * int8 a, including INT64_MIN and INT64_MAX. INT64_MIN is written as a + * subtraction (-9223372036854775807 - 1) because 9223372036854775808 is not a + * valid int8 literal; the draw spans the full int8 range so both boundaries + * are exercised. + */ +PG_PROP(mul_identity_safe) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int64 a; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + + initStringInfo(&q); + if (a == INT64_MIN) + appendStringInfoString(&q, + "SELECT ((-9223372036854775807 - 1)::bigint * 1) " + "= (-9223372036854775807 - 1)::bigint " + "AND ((-9223372036854775807 - 1)::bigint * 0) = 0"); + else + appendStringInfo(&q, + "SELECT ((%lld)::bigint * 1) = (%lld)::bigint " + "AND ((%lld)::bigint * 0) = 0", + (long long) a, (long long) a, (long long) a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 1961a5a13d483..5e29121cafd53 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From c12be6a5e173bc6474f5c96f3f872081fd6b506a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:50 -0400 Subject: [PATCH 037/338] prop: add comparison-operator consistency across types property tests (arithcmp) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_arithcmp.c | 242 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 246 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_arithcmp.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 95dfce55ee3b3..49af0a7b3c056 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -52,7 +52,8 @@ OBJS = \ prop_bitint.o \ prop_bitlogic.o \ prop_bitstringops.o \ - prop_safemath.o + prop_safemath.o \ + prop_arithcmp.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 7c0e025b6891e..ed75806d7911a 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -50,6 +50,7 @@ prop_area_sources = files( 'prop_bitlogic.c', 'prop_bitstringops.c', 'prop_safemath.c', + 'prop_arithcmp.c', ) diff --git a/src/test/prop/prop_arithcmp.c b/src/test/prop/prop_arithcmp.c new file mode 100644 index 0000000000000..eb7655205fb86 --- /dev/null +++ b/src/test/prop/prop_arithcmp.c @@ -0,0 +1,242 @@ +/*------------------------------------------------------------------------- + * + * prop_arithcmp.c + * Property-based tests for comparison-operator consistency. + * + * The oracle is the algebra the comparison operators must obey: trichotomy + * (<, =, > partition any ordered pair), the derived operators (<=, >=, <>) + * expressed from < and =, cross-type agreement (the same integer compares + * equal across int4/int8/numeric/float8), and transitivity of <. Each + * property checks one of these identities in SQL and requires it to hold for + * every drawn input; a failure is a bug in a comparison operator or in a + * cross-type cast/coercion. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_arithcmp.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "arithcmp" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw an int literal across the full int4 range (favouring boundaries), as + * text, parenthesized if negative so a leading '-' does not misparse when + * substituted into a cast like (%s)::int8. + */ +static void +draw_int(pg_prop_tc *tc, StringInfo buf) +{ + int v = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + if (v < 0) + appendStringInfo(buf, "(%d)", v); + else + appendStringInfo(buf, "%d", v); +} + +/* Run a boolean SELECT and require "t"; NULL or error is a failure. */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* Trichotomy: exactly one of ab holds. */ +PG_PROP(cmp_trichotomy) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_int(tc, &a); + draw_int(tc, &b); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s < %s)::int + (%s = %s)::int + (%s > %s)::int) = 1", + a.data, b.data, a.data, b.data, a.data, b.data); + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* <= is < OR =. */ +PG_PROP(le_eq_lt_or_eq) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_int(tc, &a); + draw_int(tc, &b); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s <= %s) = (%s < %s OR %s = %s)", + a.data, b.data, a.data, b.data, a.data, b.data); + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* >= is the negation of <. */ +PG_PROP(ge_is_not_lt) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_int(tc, &a); + draw_int(tc, &b); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s >= %s) = (NOT (%s < %s))", + a.data, b.data, a.data, b.data); + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* <> is the negation of =. */ +PG_PROP(ne_is_not_eq) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_int(tc, &a); + draw_int(tc, &b); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s <> %s) = (NOT (%s = %s))", + a.data, b.data, a.data, b.data); + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* The same integer value compares equal across int4/int8/numeric/float8. */ +PG_PROP(cross_type_cmp) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_int(tc, &a); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s)::int4 = (%s)::int8 " + "AND (%s)::int4 = (%s)::numeric " + "AND (%s)::int4::float8 = (%s)::float8", + a.data, a.data, a.data, a.data, a.data, a.data); + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* Transitivity of <: a Date: Mon, 12 Oct 2026 21:42:51 -0400 Subject: [PATCH 038/338] prop: add BETWEEN/SYMMETRIC and cross-type comparison property tests (cmpops) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_cmpops.c | 357 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 361 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_cmpops.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 49af0a7b3c056..1e775644136d3 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -53,7 +53,8 @@ OBJS = \ prop_bitlogic.o \ prop_bitstringops.o \ prop_safemath.o \ - prop_arithcmp.o + prop_arithcmp.o \ + prop_cmpops.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index ed75806d7911a..b0c8bb505f179 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -51,6 +51,7 @@ prop_area_sources = files( 'prop_bitstringops.c', 'prop_safemath.c', 'prop_arithcmp.c', + 'prop_cmpops.c', ) diff --git a/src/test/prop/prop_cmpops.c b/src/test/prop/prop_cmpops.c new file mode 100644 index 0000000000000..df84302bacc4c --- /dev/null +++ b/src/test/prop/prop_cmpops.c @@ -0,0 +1,357 @@ +/*------------------------------------------------------------------------- + * + * prop_cmpops.c + * Property-based tests for BETWEEN / IS DISTINCT comparison constructs. + * + * The oracle is the standard's definition of each construct in terms of the + * primitive comparison operators: BETWEEN desugars to a pair of >= / <=, + * BETWEEN SYMMETRIC to a BETWEEN over the ordered endpoints, NOT BETWEEN to + * the negation, and IS [NOT] DISTINCT FROM to <> extended with a defined + * answer when operands are NULL. Each property checks one such identity in + * SQL and requires it to hold for every drawn integer; a failure is a bug in + * the construct's transformation or in a cross-type cast/coercion. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_cmpops.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "cmpops" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw an int4 literal (favouring boundaries), parenthesized if negative so a + * leading '-' does not misparse when substituted into a larger expression or + * a cast like (%s)::int8. + */ +static void +draw_int(pg_prop_tc *tc, StringInfo buf) +{ + int v = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + if (v < 0) + appendStringInfo(buf, "(%d)", v); + else + appendStringInfo(buf, "%d", v); +} + +/* Run a boolean SELECT and require "t"; NULL or error is a failure. */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * BETWEEN desugars to a conjunction of range bounds: + * (x BETWEEN a AND b) = (x >= a AND x <= b). + */ +PG_PROP(between_eq_and) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&a); + initStringInfo(&b); + draw_int(tc, &x); + draw_int(tc, &a); + draw_int(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s BETWEEN %s AND %s) = (%s >= %s AND %s <= %s)", + x.data, a.data, b.data, + x.data, a.data, x.data, b.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * BETWEEN SYMMETRIC orders its endpoints first: + * (x BETWEEN SYMMETRIC a AND b) = (x BETWEEN least(a,b) AND greatest(a,b)). + */ +PG_PROP(between_symmetric) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&a); + initStringInfo(&b); + draw_int(tc, &x); + draw_int(tc, &a); + draw_int(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s BETWEEN SYMMETRIC %s AND %s) = " + "(%s BETWEEN least(%s, %s) AND greatest(%s, %s))", + x.data, a.data, b.data, + x.data, a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * NOT BETWEEN is the negation of BETWEEN: + * (x NOT BETWEEN a AND b) = NOT (x BETWEEN a AND b). + */ +PG_PROP(not_between) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&a); + initStringInfo(&b); + draw_int(tc, &x); + draw_int(tc, &a); + draw_int(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s NOT BETWEEN %s AND %s) = (NOT (%s BETWEEN %s AND %s))", + x.data, a.data, b.data, + x.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The same integer compares equal across types: int4 = numeric, int4 = int8, + * and int4 widened to float8 = float8. An int4 value is representable exactly + * in each of these types, so every equality must hold. + */ +PG_PROP(cross_type_eq) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_int(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s)::int4 = (%s)::numeric " + "AND (%s)::int4 = (%s)::int8 " + "AND ((%s)::int4)::float8 = (%s)::float8", + x.data, x.data, x.data, x.data, x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * For non-NULL operands, IS DISTINCT FROM is exactly <>. + */ +PG_PROP(is_distinct_vs_eq) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + y, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&y); + draw_int(tc, &x); + draw_int(tc, &y); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s IS DISTINCT FROM %s) = (%s <> %s)", + x.data, y.data, x.data, y.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(y.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Two NULLs are not distinct: IS NOT DISTINCT FROM is true and IS DISTINCT + * FROM is false. This is the whole point of the construct -- plain = / <> + * would return NULL here. + */ +PG_PROP(is_not_distinct_null) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT (NULL::int IS NOT DISTINCT FROM NULL::int) " + "AND (NOT (NULL::int IS DISTINCT FROM NULL::int))"); + + pg_prop_disconnect(conn); +} + +/* + * An empty range (lower bound strictly above upper bound) contains nothing: + * when a > b, (x BETWEEN a AND b) is false for every x. We draw a strictly + * greater than b by drawing two ints and taking greatest/least, forcing them + * apart by +1 so the range is genuinely empty regardless of ties. + */ +PG_PROP(between_empty_range) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + p, + r, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&p); + initStringInfo(&r); + draw_int(tc, &x); + draw_int(tc, &p); + draw_int(tc, &r); + + /* + * a := greatest(p,r) + 1, b := least(p,r), computed in int8 so adding 1 to + * INT32_MAX cannot overflow. Then a > b holds unconditionally, so the + * range [a,b] is empty. + */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s BETWEEN " + "(greatest((%s)::int8, (%s)::int8) + 1) " + "AND least((%s)::int8, (%s)::int8)) = false", + x.data, p.data, r.data, p.data, r.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(p.data); + pfree(r.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Transitivity of <: NOT (x < y AND y < z) OR (x < z). Equivalently, if + * x < y and y < z then x < z. This is the material-implication form so the + * single boolean is true for every drawn triple. + */ +PG_PROP(ternary_order) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + y, + z, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&y); + initStringInfo(&z); + draw_int(tc, &x); + draw_int(tc, &y); + draw_int(tc, &z); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (NOT (%s < %s AND %s < %s)) OR (%s < %s)", + x.data, y.data, y.data, z.data, x.data, z.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(y.data); + pfree(z.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 9e706c0e45d56..d490d7b8ecb42 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 326ddac32b0c0332b1f811eaaa9232f6e0fb35f9 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:51 -0400 Subject: [PATCH 039/338] prop: add text property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_text.c | 435 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 439 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_text.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 1e775644136d3..e806fb5eb9858 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -54,7 +54,8 @@ OBJS = \ prop_bitstringops.o \ prop_safemath.o \ prop_arithcmp.o \ - prop_cmpops.o + prop_cmpops.o \ + prop_text.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index b0c8bb505f179..70934b15ce577 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -52,6 +52,7 @@ prop_area_sources = files( 'prop_safemath.c', 'prop_arithcmp.c', 'prop_cmpops.c', + 'prop_text.c', ) diff --git a/src/test/prop/prop_text.c b/src/test/prop/prop_text.c new file mode 100644 index 0000000000000..09fe2190c7fd8 --- /dev/null +++ b/src/test/prop/prop_text.c @@ -0,0 +1,435 @@ +/*------------------------------------------------------------------------- + * + * prop_text.c + * Property-based tests for text/string functions. + * + * The text/string functions in src/backend/utils/adt/varlena.c have had ~19 + * fixes over the years; these check algebraic identities of substring, + * length, concat, overlay, replace and split against each other, so the + * functions cross-check one another rather than re-deriving their semantics. + * + * The oracle is independent builtins -- length(), substr(), ||, reverse(), + * upper()/lower(), replace(), position(), overlay(), split_part() and + * string_to_array() -- and the identities they must satisfy together. + * Alphabets are ASCII, single-byte, and quote-free, so length() counts + * characters, reverse() is exact, and SQL string-literal quoting is trivial + * (multibyte case folding is unicode_case's job, not this file's). + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_text.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "text" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabets never contain a quote, but do it defensively + * so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..16 characters of plain text from a 7-char alphabet "abcABC " into + * buf as a bare string (no quoting). Letters (mixed case) and a space only -- + * all single-byte ASCII and never a single quote -- so length() counts + * characters exactly and quoting is trivial. + */ +static void +draw_text(pg_prop_tc *tc, StringInfo buf) +{ + static const char alphabet[] = "abcABC "; + int n = (int) pg_prop_draw_int(tc, 0, 16); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, alphabet[pg_prop_draw_int(tc, 0, 6)]); +} + +/* + * Draw 0..16 ASCII letters from "abcABC" (no space) into buf as a bare + * string, for the case-folding length property where only letters are + * interesting. + */ +static void +draw_letters(pg_prop_tc *tc, StringInfo buf) +{ + static const char alphabet[] = "abcABC"; + int n = (int) pg_prop_draw_int(tc, 0, 16); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, alphabet[pg_prop_draw_int(tc, 0, 5)]); +} + +/* + * Draw 0..16 characters from the 2-letter alphabet "ab" into buf as a bare + * string, so 'a' occurs often -- for the replace-removes-all property. + */ +static void +draw_ab(pg_prop_tc *tc, StringInfo buf) +{ + static const char alphabet[] = "ab"; + int n = (int) pg_prop_draw_int(tc, 0, 16); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, alphabet[pg_prop_draw_int(tc, 0, 1)]); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the expression errored or produced NULL on well-formed + * letter strings (usually an index slipped out of range), which is itself a + * bug -- so either is a failure, reported with the SQL. Modelled on + * prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Concatenation length is additive: length(A || B) = length(A) + length(B). + * With a single-byte alphabet, length() counts characters, so the character + * counts of two strings add exactly across concatenation. + */ +PG_PROP(length_of_concat_additive) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_text(tc, &a); + draw_text(tc, &b); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT length("); + append_sql_literal(&q, a.data); + appendStringInfoString(&q, " || "); + append_sql_literal(&q, b.data); + appendStringInfoString(&q, ") = length("); + append_sql_literal(&q, a.data); + appendStringInfoString(&q, ") + length("); + append_sql_literal(&q, b.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Splitting and rejoining reconstructs the string: + * substr(S, 1, k) || substr(S, k+1) = S + * for a split point k in [0, length(S)]. k is drawn in [0, 16] and clamped + * server-side with least(k, length(S)) so the split point is always in range + * and both halves cover S exactly. + */ +PG_PROP(substr_concat_reconstructs) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int k; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + k = (int) pg_prop_draw_int(tc, 0, 16); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT substr("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", 1, least(%d, length(", k); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "))) || substr("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", least(%d, length(", k); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) + 1) = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * reverse() is its own inverse: reverse(reverse(S)) = S. With a single-byte + * ASCII-ish alphabet, reverse() is an exact character reversal, so applying + * it twice yields the original. + */ +PG_PROP(reverse_involution) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT reverse(reverse("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ASCII case change preserves length: + * length(upper(S)) = length(S) AND length(lower(S)) = length(S) + * for an ASCII-letter string S. upper()/lower() on single-byte ASCII map + * each character to exactly one character, so neither changes the length. + */ +PG_PROP(upper_lower_length_preserved_ascii) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT length(upper("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) = length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") AND length(lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) = length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * replace() removes all occurrences: after replacing every 'a' with the empty + * string, no 'a' remains, so position('a' in replace(S, 'a', '')) = 0. + * position() returns 0 when the substring is not found. S is drawn from "ab" + * so 'a' occurs often and the property has teeth. + */ +PG_PROP(replace_removes_all_occurrences) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_ab(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT position('a' in replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a', '')) = 0"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * overlay() matches a manual substring splice: + * overlay(S placing R from p for L) + * = substr(S, 1, p-1) || R || substr(S, p+L) + * for a 1-based start p and length L. p is drawn 0..24 then clamped + * server-side to [1, length(S)+1] and L to >= 0, consistently on both sides, + * so overlay() and the manual splice always see the same in-range p/L. This + * cross-checks overlay() against substr() and ||. + */ +PG_PROP(overlay_matches_manual) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + r, + q; + int p; + int l; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&r); + draw_text(tc, &s); + draw_text(tc, &r); + + p = (int) pg_prop_draw_int(tc, 0, 24); + l = (int) pg_prop_draw_int(tc, 0, 24); + + initStringInfo(&q); + + /* overlay side: clamp p to [1, length(S)+1], L to >= 0 */ + appendStringInfoString(&q, "SELECT overlay("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " placing "); + append_sql_literal(&q, r.data); + appendStringInfo(&q, " from least(greatest(%d, 1), length(", p); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ") + 1) for greatest(%d, 0)) = ", l); + + /* manual splice side: identical clamps so both agree */ + appendStringInfoString(&q, "substr("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", 1, least(greatest(%d, 1), length(", p); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") + 1) - 1) || "); + append_sql_literal(&q, r.data); + appendStringInfoString(&q, " || substr("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", least(greatest(%d, 1), length(", p); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ") + 1) + greatest(%d, 0))", l); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(r.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * split_part() round-trips against string_to_array(): + * split_part(S, ',', i) = (string_to_array(S, ','))[i] + * for i in [1, ntokens]. S is built by joining 1..4 letter-tokens with ','. + * string_to_array() is an independent oracle, so this cross-checks + * split_part()'s i-th-field semantics rather than re-deriving them. + */ +PG_PROP(split_part_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int ntokens; + int i; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* + * build S = tok1,tok2,... with ntokens in [1, 4]. Each token must be + * NON-EMPTY: split_part() and string_to_array() agree only on non-empty + * input -- string_to_array('') is an empty array (so [1] is NULL) while + * split_part('',...) is the empty string, a documented difference. + */ + ntokens = (int) pg_prop_draw_int(tc, 1, 4); + + initStringInfo(&s); + for (int t = 0; t < ntokens; t++) + { + static const char letters[] = "abcABC"; + int len = (int) pg_prop_draw_int(tc, 1, 6); + + if (t > 0) + appendStringInfoChar(&s, ','); + for (int c = 0; c < len; c++) + appendStringInfoChar(&s, letters[pg_prop_draw_int(tc, 0, 5)]); + } + + /* pick which 1-based token to compare */ + i = (int) pg_prop_draw_int(tc, 1, ntokens); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT split_part("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", ',', %d) = (string_to_array(", i); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", ','))[%d]", i); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index d490d7b8ecb42..93f8475109723 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From e2c72a9ac4894fb0718462284e8d0021c53a7d94 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:52 -0400 Subject: [PATCH 040/338] prop: add concat_ws, format, reverse, and encode round-trips property tests (textfns) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_textfns.c | 372 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 376 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_textfns.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index e806fb5eb9858..7e1028fd134f2 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -55,7 +55,8 @@ OBJS = \ prop_safemath.o \ prop_arithcmp.o \ prop_cmpops.o \ - prop_text.o + prop_text.o \ + prop_textfns.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 70934b15ce577..292abc9e676ec 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -53,6 +53,7 @@ prop_area_sources = files( 'prop_arithcmp.c', 'prop_cmpops.c', 'prop_text.c', + 'prop_textfns.c', ) diff --git a/src/test/prop/prop_textfns.c b/src/test/prop/prop_textfns.c new file mode 100644 index 0000000000000..153b4f48df51c --- /dev/null +++ b/src/test/prop/prop_textfns.c @@ -0,0 +1,372 @@ +/*------------------------------------------------------------------------- + * + * prop_textfns.c + * Property-based tests for text functions not covered by text/strfunc/ + * substring/stringagg: concat, concat_ws, format, reverse, quote_literal, + * md5, and the encode/decode hex round-trip. + * + * The oracle is the SQL algebra these functions must satisfy: reverse is an + * involution and length-preserving; concat treats NULL as empty while + * concat_ws skips NULL arguments; format('%s', S) is just S; quote_literal + * over a quote-free string is S wrapped in single quotes; encode/decode hex + * composed with convert_to/convert_from round-trips exactly; and a hex md5 is + * always 32 characters. Each identity is checked with a quote-free [abc] + * alphabet so SQL literal quoting is trivial and the only variable is the + * function under test; a false result is a bug in that function, not a + * quoting or encoding artefact. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_textfns.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "textfns" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. The [abc] alphabet never contains a quote, but do it + * defensively so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..12 characters from the quote-free single-byte alphabet "abc" into + * buf as a bare string (no quoting). All ASCII, never a single quote, so + * length() counts characters exactly, reverse() is exact, and quoting is + * trivial. + */ +static void +draw_text(pg_prop_tc *tc, StringInfo buf) +{ + static const char alphabet[] = "abc"; + int n = (int) pg_prop_draw_int(tc, 0, 12); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, alphabet[pg_prop_draw_int(tc, 0, 2)]); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on valid inputs, + * which is itself a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * reverse is an involution: reverse(reverse(S)) = S. + */ +PG_PROP(reverse_involution) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT reverse(reverse("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * concat_ws skips NULL arguments: concat_ws(',', S, NULL, T) equals + * concat_ws(',', S, T). A NULL in the middle must not emit a stray separator. + */ +PG_PROP(concat_ws_skips_null) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + t, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&t); + draw_text(tc, &s); + draw_text(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT concat_ws(',', "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", NULL, "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ") = concat_ws(',', "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * concat treats NULL as empty: concat(S, NULL, T) = S || T. The dropped NULL + * contributes nothing, so the result is just the two literals concatenated. + */ +PG_PROP(concat_null_is_empty) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + t, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&t); + draw_text(tc, &s); + draw_text(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT concat("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", NULL, "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ") = ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " || "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * format('%s', S) is the identity on text: the %s conversion inserts the + * argument verbatim. + */ +PG_PROP(format_s_equals_value) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT format('%s', "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * quote_literal wraps a value in single quotes and doubles any embedded + * quote. For a quote-free [abc] string there is nothing to double, so the + * result is exactly '''' || S || '''' (SQL '''' is one literal quote char; + * in this C source each is written as a four-single-quote run inside the + * literal below). Comparing against the SQL-built expected keeps the oracle + * independent of any C-side rendering. + */ +PG_PROP(quote_literal_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT quote_literal("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = ('''' || "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " || '''')"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * encode/decode hex round-trips through the byte representation: + * convert_from(decode(encode(convert_to(S,'UTF8'),'hex'),'hex'),'UTF8') = S. + * convert_to turns S into its UTF-8 bytes, encode renders those as hex text, + * decode parses them back to the same bytes, and convert_from reads UTF-8, + * so the whole composition is the identity. + */ +PG_PROP(encode_decode_hex_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT convert_from(decode(encode(convert_to("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, + ", 'UTF8'), 'hex'), 'hex'), 'UTF8') = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A hex md5 digest is always 32 characters, independent of the input length. + */ +PG_PROP(md5_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT length(md5("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) = 32"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * reverse is length-preserving: length(reverse(S)) = length(S). Reordering + * characters never adds or drops one. + */ +PG_PROP(reverse_length_preserved) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT length(reverse("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) = length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 93f8475109723..53e08d6deea7b 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 8328af583971ec8fffaeed4b1ba2a57b5065eb31 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:52 -0400 Subject: [PATCH 041/338] prop: add text position, split, replace, and translate property tests (textpos) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_textpos.c | 406 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 410 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_textpos.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 7e1028fd134f2..c2da2717442e3 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -56,7 +56,8 @@ OBJS = \ prop_arithcmp.o \ prop_cmpops.o \ prop_text.o \ - prop_textfns.o + prop_textfns.o \ + prop_textpos.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 292abc9e676ec..04e3eaeadc8fb 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -54,6 +54,7 @@ prop_area_sources = files( 'prop_cmpops.c', 'prop_text.c', 'prop_textfns.c', + 'prop_textpos.c', ) diff --git a/src/test/prop/prop_textpos.c b/src/test/prop/prop_textpos.c new file mode 100644 index 0000000000000..effd6a045b7ee --- /dev/null +++ b/src/test/prop/prop_textpos.c @@ -0,0 +1,406 @@ +/*------------------------------------------------------------------------- + * + * prop_textpos.c + * Property-based tests for text position/split/replace functions not + * covered by text/strfunc/substring/textfns: position/strpos, split_part, + * replace, translate, overlay, and repeat. + * + * The oracle is the SQL algebra these functions must satisfy: a found + * position is a valid 1-based index and a not-found one is 0; replace that + * deletes a character leaves none behind, and replace of an absent needle is + * the identity; split_part on an absent delimiter yields the whole string as + * part 1, and split on a known delimiter recovers the leading segment; + * overlay inserting the empty string is the identity; translate composed with + * its inverse map is the identity; and repeat multiplies length exactly. All + * strings are drawn from the quote-free [abc] alphabet (or [bc] where 'a' is + * the delimiter) so SQL literal quoting is trivial and the only variable is + * the function under test; a false result is a bug in that function, not a + * quoting artefact. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_textpos.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "textpos" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. The [abc]/[bc] alphabets never contain a quote, but do it + * defensively so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..12 characters from a quote-free single-byte alphabet into buf as a + * bare string (no quoting). All ASCII, never a single quote, so length() + * counts characters exactly and quoting is trivial. + */ +static void +draw_from(pg_prop_tc *tc, StringInfo buf, const char *alphabet) +{ + int span = (int) strlen(alphabet) - 1; + int n = (int) pg_prop_draw_int(tc, 0, 12); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, alphabet[pg_prop_draw_int(tc, 0, span)]); +} + +/* Draw over [abc]. */ +static void +draw_abc(pg_prop_tc *tc, StringInfo buf) +{ + draw_from(tc, buf, "abc"); +} + +/* Draw over [bc] (no 'a'), for properties where 'a' is a delimiter. */ +static void +draw_bc(pg_prop_tc *tc, StringInfo buf) +{ + draw_from(tc, buf, "bc"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on valid inputs, + * which is itself a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * A position is either not-found (0) or a valid 1-based index: it lies in + * [1, length(S) - length(P) + 1]. When P is longer than S the needle cannot + * occur, so position is 0 and the first disjunct holds; otherwise the found + * index can be at most the last place a P-length window fits in S. P is + * drawn 1..3 characters over [abc]. + */ +PG_PROP(position_found_or_zero) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + p, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc(tc, &s); + + initStringInfo(&p); + { + static const char alphabet[] = "abc"; + int n = (int) pg_prop_draw_int(tc, 1, 3); + + for (int i = 0; i < n; i++) + appendStringInfoChar(&p, alphabet[pg_prop_draw_int(tc, 0, 2)]); + } + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT position("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, " in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = 0 OR (position("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, " in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") >= 1 AND position("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, " in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") <= length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") - length("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ") + 1)"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Deleting a character removes every occurrence: after replace(S, 'a', ''), + * position('a' in ...) is 0 (no 'a' remains). + */ +PG_PROP(replace_removes_all) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT position('a' in replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a', '')) = 0"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Replacing an absent needle is the identity: 'z' never occurs in an [abc] + * string, so replace(S, 'z', 'Q') = S. + */ +PG_PROP(replace_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'z', 'Q') = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * split_part on an absent delimiter yields the whole string as the single + * part: 'x' never occurs in an [abc] string, so split_part(S, 'x', 1) = S. + */ +PG_PROP(split_part_rejoin) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT split_part("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'x', 1) = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * split_part recovers the leading segment: with S and T drawn over [bc] (no + * 'a'), the string S || 'a' || T splits on 'a' into exactly [S, T], so + * split_part(S || 'a' || T, 'a', 1) = S. Because S has no 'a', the first + * delimiter is the one we inserted. + */ +PG_PROP(split_part_count) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + t, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&t); + draw_bc(tc, &s); + draw_bc(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT split_part(("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " || 'a' || "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, "), 'a', 1) = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Overlay inserting the empty string is the identity: overlay(S placing '' + * from p for 0) inserts nothing at position p and deletes zero characters, so + * the result is S. p is drawn 1..length(S)+1 (every legal insertion point, + * including just past the end). + */ +PG_PROP(overlay_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int slen; + int p; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc(tc, &s); + slen = (int) strlen(s.data); + p = (int) pg_prop_draw_int(tc, 1, slen + 1); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT overlay("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, " placing '' from %d for 0) = ", p); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * translate is invertible under a bijective map: translate(S, 'abc', 'xyz') + * renames a->x, b->y, c->z, and translating the result back with the swapped + * argument order restores S. Over [abc] the map is a bijection, so the + * composition is the identity. + */ +PG_PROP(translate_maps) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT translate(translate("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'abc', 'xyz'), 'xyz', 'abc') = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * repeat multiplies length exactly: length(repeat(S, k)) = length(S) * k for + * k in 0..5 (k = 0 yields the empty string, length 0). + */ +PG_PROP(repeat_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int k; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc(tc, &s); + k = (int) pg_prop_draw_int(tc, 0, 5); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT length(repeat("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", %d)) = length(", k); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ") * %d", k); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 53e08d6deea7b..71b768b19a3ef 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From f9f6c7499e34d93b268b8f2bcc0ea0a3c501bad3 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:53 -0400 Subject: [PATCH 042/338] prop: add upper/lower/initcap case-folding laws property tests (textcase) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_textcase.c | 400 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 404 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_textcase.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index c2da2717442e3..baf5fdad2830c 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -57,7 +57,8 @@ OBJS = \ prop_cmpops.o \ prop_text.o \ prop_textfns.o \ - prop_textpos.o + prop_textpos.o \ + prop_textcase.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 04e3eaeadc8fb..1163d352e1000 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -55,6 +55,7 @@ prop_area_sources = files( 'prop_text.c', 'prop_textfns.c', 'prop_textpos.c', + 'prop_textcase.c', ) diff --git a/src/test/prop/prop_textcase.c b/src/test/prop/prop_textcase.c new file mode 100644 index 0000000000000..5e4fcbc83ec3f --- /dev/null +++ b/src/test/prop/prop_textcase.c @@ -0,0 +1,400 @@ +/*------------------------------------------------------------------------- + * + * prop_textcase.c + * Property-based tests for ASCII upper/lower/initcap case-folding laws. + * + * The oracle is the algebra of ASCII case folding: on the alphabet [a-zA-Z] + * every character folds 1:1 (one lowercase <-> one uppercase, no multibyte + * expansion), so upper/lower are idempotent, length-preserving, mutually + * inverse on single letters, and distribute over concatenation. A broken + * identity here is a bug in the case-fold path for plain ASCII text, not a + * locale or Unicode-expansion artefact -- these inputs never leave ASCII. + * + * Distinct from the unicode_case / strfunc / textfns areas: those cover + * Unicode folding, assorted string builtins, and text utilities. This area + * asserts only the pure ASCII case-fold laws. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_textcase.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "textcase" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabet is [a-zA-Z] and never contains a quote, but do + * it defensively so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..12 ASCII letters from [a-zA-Z] into buf as a bare string. Letters + * only -- all single-byte, 1:1 case folding, never a quote -- so length() + * counts characters exactly and quoting is trivial. + */ +static void +draw_letters(pg_prop_tc *tc, StringInfo buf) +{ + static const char alphabet[] = + "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"; + int n = (int) pg_prop_draw_int(tc, 0, 12); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, alphabet[pg_prop_draw_int(tc, 0, 51)]); +} + +/* + * Draw a single lowercase ASCII letter [a-z] into buf as a bare string. + */ +static void +draw_lower_letter(pg_prop_tc *tc, StringInfo buf) +{ + appendStringInfoChar(buf, (char) ('a' + pg_prop_draw_int(tc, 0, 25))); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the expression errored or produced NULL on well-formed + * ASCII letter strings, which is itself a bug -- so either is a failure, + * reported with the SQL. Modelled on prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * lower is idempotent: lower(lower(S)) = lower(S). A second fold of + * already-lowercased ASCII text must be a no-op. + */ +PG_PROP(lower_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT lower(lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) = lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * upper is idempotent: upper(upper(S)) = upper(S). + */ +PG_PROP(upper_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT upper(upper("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) = upper("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Uppercasing then lowercasing equals just lowercasing: lower(upper(S)) = + * lower(S). For ASCII the uppercase of any letter lowercases back to the + * same letter as the original did, so the intervening upper() is invisible. + */ +PG_PROP(upper_lower_roundtrip_lower) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT lower(upper("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) = lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Case folding preserves length: length(upper(S)) = length(S) and + * length(lower(S)) = length(S). ASCII case folding is 1:1, so neither fold + * adds or drops characters. + */ +PG_PROP(length_preserved) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT length(upper("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) = length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") AND length(lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) = length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * initcap of a single ASCII word (letters only, no spaces) uppercases the + * first character and lowercases the rest: initcap(W) = upper(left(W,1)) || + * lower(substr(W,2)), for length(W) >= 1. Drawing a word of 1..12 letters + * from [a-zA-Z] with no spaces means there is exactly one word boundary (the + * start), so initcap's "first letter of each word" reduces to this. + */ +PG_PROP(initcap_lower_rest) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* a single word of >= 1 letters, no spaces */ + initStringInfo(&s); + n = (int) pg_prop_draw_int(tc, 1, 12); + { + static const char alphabet[] = + "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"; + + for (int i = 0; i < n; i++) + appendStringInfoChar(&s, alphabet[pg_prop_draw_int(tc, 0, 51)]); + } + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT initcap("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = upper(left("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 1)) || lower(substr("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 2))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Case folding is case-insensitive at the fold: lower(S) = lower(upper(S)) + * and upper(S) = upper(lower(S)). Folding to one case first and then folding + * again cannot change the final-case result for ASCII. + */ +PG_PROP(case_insensitive_eq) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = lower(upper("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) AND upper("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = upper(lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Case folding distributes over concatenation: lower(S || T) = + * lower(S) || lower(T). Folding is per-character and context-free for ASCII, + * so splitting the fold across a join cannot change the result. + */ +PG_PROP(concat_distributes) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + t, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&t); + draw_letters(tc, &s); + draw_letters(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " || "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ") = lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") || lower("); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * For a lowercase ASCII letter c, upper actually changes it and the fold is + * invertible: upper(c) <> c and lower(upper(c)) = c. This pins down that the + * [a-z] range genuinely has distinct uppercase images that round-trip back. + */ +PG_PROP(upper_lower_disjoint_for_letters) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&c); + draw_lower_letter(tc, &c); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT upper("); + append_sql_literal(&q, c.data); + appendStringInfoString(&q, ") <> "); + append_sql_literal(&q, c.data); + appendStringInfoString(&q, " AND lower(upper("); + append_sql_literal(&q, c.data); + appendStringInfoString(&q, ")) = "); + append_sql_literal(&q, c.data); + + require_true(tc, conn, q.data); + + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 71b768b19a3ef..c613e9a4148b3 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 5749bc31ff3f7bc0a08536a6de289dfd594095ed Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:53 -0400 Subject: [PATCH 043/338] prop: add length/octet_length/bit_length relationships property tests (textlength) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_textlength.c | 316 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 320 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_textlength.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index baf5fdad2830c..5c8618f07ae3d 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -58,7 +58,8 @@ OBJS = \ prop_text.o \ prop_textfns.o \ prop_textpos.o \ - prop_textcase.o + prop_textcase.o \ + prop_textlength.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 1163d352e1000..bbd3f73f4ea21 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -56,6 +56,7 @@ prop_area_sources = files( 'prop_textfns.c', 'prop_textpos.c', 'prop_textcase.c', + 'prop_textlength.c', ) diff --git a/src/test/prop/prop_textlength.c b/src/test/prop/prop_textlength.c new file mode 100644 index 0000000000000..11587c319be26 --- /dev/null +++ b/src/test/prop/prop_textlength.c @@ -0,0 +1,316 @@ +/*------------------------------------------------------------------------- + * + * prop_textlength.c + * Property-based tests for the text length family of functions. + * + * The oracle here is the algebra relating length/char_length (character + * counts), octet_length (byte count), bit_length (= 8 * octet_length), and + * their behaviour under concatenation, substr, lpad and the bytea cast. For + * single-byte ASCII input these have exact, server-encoding-independent + * relationships, so any "f" result is a bug in a length routine, not an + * encoding artefact. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_textlength.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "textlength" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Alphabet is single-byte ASCII [a-c]; it contains no single quote, so a + * drawn string is safe to drop between single quotes with no escaping. + */ +#define TL_ALPHABET "abc" +#define TL_MIN 0 +#define TL_MAX 15 + +/* + * Draw a string from TL_ALPHABET and append it as a single-quoted SQL literal. + */ +static void +draw_str_literal(pg_prop_tc *tc, StringInfo buf) +{ + size_t len = 0; + char *s = pg_prop_draw_from(tc, TL_ALPHABET, TL_MIN, TL_MAX, &len); + + appendStringInfoChar(buf, '\''); + appendBinaryStringInfo(buf, s, (int) len); + appendStringInfoChar(buf, '\''); + pg_free(s); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a length function errored or produced NULL on finite + * input, which is itself a bug -- so either is a failure, reported with the + * SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * length() and char_length() are documented synonyms: both count characters. + */ +PG_PROP(length_eq_char_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_str_literal(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT length(%s) = char_length(%s)", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * For single-byte ASCII [a-c], every character is exactly one byte, so the + * byte count equals the character count. + */ +PG_PROP(octet_eq_char_ascii) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_str_literal(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT octet_length(%s) = char_length(%s)", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * bit_length() is defined as 8 times the byte length. + */ +PG_PROP(bit_length_8x) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_str_literal(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT bit_length(%s) = 8 * octet_length(%s)", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Concatenation is additive in character length: length(S || T) = + * length(S) + length(T). + */ +PG_PROP(concat_length_additive) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + t, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&t); + draw_str_literal(tc, &s); + draw_str_literal(tc, &t); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(%s || %s) = length(%s) + length(%s)", + s.data, t.data, s.data, t.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A prefix is never longer than the string: length(substr(S, 1, k)) = + * least(k, length(S)) for k in 0..20 (k spans past TL_MAX so the clamp is + * exercised). substr with a negative/zero count returns the empty string, + * and least() matches since length is non-negative. + */ +PG_PROP(length_substr) +{ + pg_prop_conn *conn; + char *err = NULL; + int k = (int) pg_prop_draw_int(tc, 0, 20); + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_str_literal(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(substr(%s, 1, %d)) = least(%d, length(%s))", + s.data, k, k, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The empty string has zero characters, zero bytes, and zero bits. No input + * is drawn: this is a fixed boundary check. + */ +PG_PROP(empty_lengths) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT length('') = 0 AND octet_length('') = 0 " + "AND bit_length('') = 0"); + + pg_prop_disconnect(conn); +} + +/* + * lpad(S, n, 'x') produces a string of exactly n characters for n >= 0, + * padding or truncating as needed. n spans 0..20, past TL_MAX, so both the + * pad and truncate cases are exercised. (Padding is covered elsewhere; this + * is a cross-check against the length family.) + */ +PG_PROP(length_after_pad) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 0, 20); + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_str_literal(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT length(lpad(%s, %d, 'x')) = %d", + s.data, n, n); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Casting ASCII text to bytea preserves the byte count: for single-byte + * ASCII [a-c] the text and its bytea form have the same number of bytes. + */ +PG_PROP(octet_length_bytea) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_str_literal(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT octet_length((%s)::bytea) = octet_length(%s)", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index c613e9a4148b3..12517d02dc2b6 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 9967f7acee70684130a49855b4408d5ec6bc7356 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:54 -0400 Subject: [PATCH 044/338] prop: add text/number parsing round-trips property tests (textnum) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_textnum.c | 311 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 315 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_textnum.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 5c8618f07ae3d..b0ce2d7ce51ae 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -59,7 +59,8 @@ OBJS = \ prop_textfns.o \ prop_textpos.o \ prop_textcase.o \ - prop_textlength.o + prop_textlength.o \ + prop_textnum.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index bbd3f73f4ea21..392a96be760ff 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -57,6 +57,7 @@ prop_area_sources = files( 'prop_textpos.c', 'prop_textcase.c', 'prop_textlength.c', + 'prop_textnum.c', ) diff --git a/src/test/prop/prop_textnum.c b/src/test/prop/prop_textnum.c new file mode 100644 index 0000000000000..be5f4eaab5a33 --- /dev/null +++ b/src/test/prop/prop_textnum.c @@ -0,0 +1,311 @@ +/*------------------------------------------------------------------------- + * + * prop_textnum.c + * Property-based tests for text <-> number parsing round-trips. + * + * The oracle is the identity that parsing a number's own text rendering must + * reproduce the number: n = n::text::int4 for integers, X = X::text::numeric + * for exact numerics, and v ~= (v::text)::float8 within IEEE tolerance for + * float8. These exercise the type input/output paths (int4in, numeric_in, + * float8in and their *_out counterparts) plus the leading/trailing whitespace + * and leading-sign handling of integer input. A broken round-trip, a trimmed + * digit, or a rejected valid literal is an input/output bug, not a rounding + * artefact: exact round-trips are asserted exactly, the lossy float one only + * within a stated relative-plus-absolute tolerance. + * + * Distinct from castchain/decimalfmt/floatcast: this file is purely about the + * textual input/output parsers and their whitespace and sign conventions, not + * cast composition, numeric formatting functions, or binary float<->numeric + * width relationships. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_textnum.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "textnum" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the parse errored or produced NULL on valid input, which + * is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Append an int4-valued literal, parenthesizing negatives so that, when + * substituted into a larger cast expression, the sign binds to the literal + * rather than to a cast of the positive magnitude (which would overflow for + * INT_MIN: bare -2147483648::int4 parses as -(2147483648::int4)). + */ +static void +append_int4(StringInfo buf, int n) +{ + if (n < 0) + appendStringInfo(buf, "(%d)", n); + else + appendStringInfo(buf, "%d", n); +} + +/* + * int_text_roundtrip: for a drawn int4 n, n::text::int4 = n. The core + * identity of integer I/O: int4out followed by int4in must recover the value. + */ +PG_PROP(int_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + append_int4(&q, n); + appendStringInfoString(&q, "::int4::text::int4 = "); + append_int4(&q, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * text_int_leading_space: (' ' || n::text)::int4 = n. Integer input trims + * leading whitespace. n is drawn >= 0 so there is no leading-sign / space + * interaction to confuse the parser. + */ +PG_PROP(text_int_leading_space) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 0, INT32_MAX); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT (' ' || "); + append_int4(&q, n); + appendStringInfoString(&q, "::int4::text)::int4 = "); + append_int4(&q, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * text_int_trailing_space: (n::text || ' ')::int4 = n. Integer input trims + * trailing whitespace. n drawn >= 0 to keep sign handling out of it. + */ +PG_PROP(text_int_trailing_space) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 0, INT32_MAX); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_int4(&q, n); + appendStringInfoString(&q, "::int4::text || ' ')::int4 = "); + append_int4(&q, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * numeric_text_roundtrip: for a numeric built as '.', + * (X::numeric)::text::numeric = X::numeric. numeric_out then numeric_in is + * exact: the text rendering carries full precision, so the recovered value + * must equal the original exactly. + */ +PG_PROP(numeric_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int ipart = (int) pg_prop_draw_int(tc, 0, 2000000000); + int fpart = (int) pg_prop_draw_int(tc, 0, 999999); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* + * Build X as a '.' text literal cast to numeric. Both parts + * are non-negative so the literal is a plain unsigned decimal; the point + * of the property is the text<->numeric round-trip, not sign handling. + */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('%d.%d'::numeric)::text::numeric = '%d.%d'::numeric", + ipart, fpart, ipart, fpart); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * float_text_roundtrip_tol: for a float8 v built as '.', + * abs((v::text)::float8 - v) < 1e-9 * (abs(v) + 1). float8out uses enough + * significant digits to recover the exact binary value, so the round-trip + * error must stay inside a tight relative-plus-absolute tolerance. + */ +PG_PROP(float_text_roundtrip_tol) +{ + pg_prop_conn *conn; + char *err = NULL; + int ipart = (int) pg_prop_draw_int(tc, 0, 1000000000); + int fpart = (int) pg_prop_draw_int(tc, 0, 999999); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* + * v is a non-negative '.' float8. The tolerance is relative + * to abs(v) with a +1 floor so very small v is still covered. + */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(('%d.%d'::float8)::text::float8 - '%d.%d'::float8) " + "< 1e-9::float8 * (abs('%d.%d'::float8) + 1)", + ipart, fpart, ipart, fpart, ipart, fpart); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * plus_sign_accepted: ('+' || n::text)::int4 = n for n >= 0. Integer input + * accepts an explicit leading '+' sign; prefixing one to a non-negative + * rendering must not change the parsed value. + */ +PG_PROP(plus_sign_accepted) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 0, INT32_MAX); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ('+' || "); + append_int4(&q, n); + appendStringInfoString(&q, "::int4::text)::int4 = "); + append_int4(&q, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * widen_narrow_roundtrip: (n::text)::int8::int4 = n for n in int4 range. + * Rendering an int4, parsing it as the wider int8, then narrowing back must + * recover n since every int4 value fits in int8. This exercises both int I/O + * parsers (int4out, int8in) and the int8->int4 narrowing cast on in-range + * values. + */ +PG_PROP(widen_narrow_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_int4(&q, n); + appendStringInfoString(&q, "::int4::text)::int8::int4 = "); + append_int4(&q, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * bool_text: true::text::bool = true AND false::text::bool = false. Boolean + * output ('true'/'false') must parse back to the same value through boolin. + * No input is drawn: the property is a fixed two-way round-trip assertion. + */ +PG_PROP(bool_text) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT true::text::bool = true " + "AND false::text::bool = false"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 12517d02dc2b6..3313c224acbf8 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From c8e8f192865263f9c833bcec1e82864243825631 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:54 -0400 Subject: [PATCH 045/338] prop: add split_part, string_to_array/table consistency property tests (textsplit) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_textsplit.c | 365 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 369 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_textsplit.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index b0ce2d7ce51ae..089ed270a9142 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -60,7 +60,8 @@ OBJS = \ prop_textpos.o \ prop_textcase.o \ prop_textlength.o \ - prop_textnum.o + prop_textnum.o \ + prop_textsplit.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 392a96be760ff..070f6159e5a88 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -58,6 +58,7 @@ prop_area_sources = files( 'prop_textcase.c', 'prop_textlength.c', 'prop_textnum.c', + 'prop_textsplit.c', ) diff --git a/src/test/prop/prop_textsplit.c b/src/test/prop/prop_textsplit.c new file mode 100644 index 0000000000000..1e9fcdcdfaa40 --- /dev/null +++ b/src/test/prop/prop_textsplit.c @@ -0,0 +1,365 @@ +/*------------------------------------------------------------------------- + * + * prop_textsplit.c + * Property-based tests for text splitting/joining functions. + * + * Exercises split_part(), string_to_array(), string_to_table() and + * array_to_string() against one another. These functions share the delimiter + * scan that chops a string into fields; a mismatch between the array, scalar + * and set-returning forms (or between split and its join inverse) is a bug in + * that shared field logic. The oracle is each function's agreement with the + * others, so no invented formula is asserted -- only identities. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_textsplit.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "textsplit" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error is itself a failure, reported with the SQL. (Copied from + * prop_numeric.c's require_true.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Differential result-set oracle: two queries must return identical rows in + * identical order. Both callers below carry a total ORDER BY. (Copied from + * the area spec skeleton.) + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Draw a comma-joined string S of 1..5 pieces, each 1..3 chars from the + * alphabet [a-c]. The alphabet excludes the delimiter ',' and the letter + * 'q' (used as the "absent delimiter" below), so a piece can never contain a + * delimiter and the piece count is exactly (number of commas) + 1. Each + * piece has length >= 1, so there are no empty leading/trailing fields and + * the split/join round-trip is unambiguous. Returns the number of pieces. + */ +static int +draw_pieces(pg_prop_tc *tc, StringInfo buf) +{ + int npieces = (int) pg_prop_draw_int(tc, 1, 5); + int i, + j; + + for (i = 0; i < npieces; i++) + { + int plen = (int) pg_prop_draw_int(tc, 1, 3); + + if (i > 0) + appendStringInfoChar(buf, ','); + for (j = 0; j < plen; j++) + appendStringInfoChar(buf, + (char) ('a' + pg_prop_draw_int(tc, 0, 2))); + } + return npieces; +} + +/* + * 1. split_part(S, ',', k) = (string_to_array(S, ','))[k] for every part + * number k in 1..5. The scalar and array forms must pick the same field; + * array indexing is 1-based and matches split_part's 1-based part number. + * For k past the last part both yield SQL NULL/'' -- but we only check k up + * to npieces where both are a real field, and for k beyond we rely on + * properties 5/6; here the CASE compares the two forms with IS NOT DISTINCT + * FROM so a NULL array element (k > npieces) matches split_part's '' only + * when both agree. To keep it sound we bound k to the drawn piece count. + */ +PG_PROP(split_part_matches_array) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int npieces; + int k; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + npieces = draw_pieces(tc, &s); + k = (int) pg_prop_draw_int(tc, 1, npieces); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT split_part('%s', ',', %d) " + "= (string_to_array('%s', ','))[%d]", + s.data, k, s.data, k); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. cardinality(string_to_array(S, ',')) = (count of ',' in S) + 1. Each + * drawn piece is non-empty and contains no comma, so the number of fields is + * exactly one more than the number of delimiters. The comma count is derived + * independently as length(S) - length(replace(S, ',', '')). + */ +PG_PROP(string_to_array_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + (void) draw_pieces(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cardinality(string_to_array('%s', ',')) " + "= (length('%s') - length(replace('%s', ',', ''))) + 1", + s.data, s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. string_to_table(S, ',') WITH ORDINALITY must yield the same ordered + * fields as unnest(string_to_array(S, ',')) WITH ORDINALITY. Both forms + * split on the same delimiter; the set-returning function and the array + * expansion must agree field-for-field. A total ORDER BY ord makes the + * row-major comparison a true ordered comparison. + */ +PG_PROP(split_table_eq_array) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q1, + q2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + (void) draw_pieces(tc, &s); + + initStringInfo(&q1); + appendStringInfo(&q1, + "SELECT t FROM string_to_table('%s', ',') " + "WITH ORDINALITY AS st(t, ord) ORDER BY ord", + s.data); + + initStringInfo(&q2); + appendStringInfo(&q2, + "SELECT t FROM unnest(string_to_array('%s', ',')) " + "WITH ORDINALITY AS u(t, ord) ORDER BY ord", + s.data); + + require_same_rows(tc, conn, q1.data, q2.data); + + pfree(s.data); + pfree(q1.data); + pfree(q2.data); + pg_prop_disconnect(conn); +} + +/* + * 4. array_to_string(string_to_array(S, ','), ',') = S. Splitting then + * re-joining on the same delimiter is the identity when no field is NULL and + * no field contains the delimiter -- guaranteed here because every drawn + * piece is a non-empty run of [a-c] with no comma, so there are no ambiguous + * empty edge fields. + */ +PG_PROP(array_to_string_inverse) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + (void) draw_pieces(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT array_to_string(string_to_array('%s', ','), ',') " + "= '%s'", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. split_part(S, ',', 1000) = ''. A part number beyond the last field is + * the empty string, never NULL and never an error. 1000 is far past the + * maximum of 5 fields. + */ +PG_PROP(split_part_out_of_range) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + (void) draw_pieces(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT split_part('%s', ',', 1000) = ''", + s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. split_part(S, ',', -1) counts from the end, so it must equal the last + * field, which is (string_to_array(S, ','))[cardinality(...)]. This ties the + * negative-index scalar form back to the array form, catching an off-by-one + * in the reverse scan. + */ +PG_PROP(split_part_negative) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + (void) draw_pieces(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT split_part('%s', ',', -1) " + "= (string_to_array('%s', ','))" + "[cardinality(string_to_array('%s', ','))]", + s.data, s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. split_part(S, 'q', 1) = S when 'q' does not occur in S. The alphabet is + * [a-c] so 'q' is always absent; with no delimiter present the whole string + * is field 1. + */ +PG_PROP(empty_delimiter_whole) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + (void) draw_pieces(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT split_part('%s', 'q', 1) = '%s'", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 3313c224acbf8..141c0ce4db94c 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 918053a7d56aff035dc507df2cb30bf4d870eb63 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:55 -0400 Subject: [PATCH 046/338] prop: add strfunc property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_strfunc.c | 459 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 463 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_strfunc.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 089ed270a9142..1d52f29c2ccc9 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -61,7 +61,8 @@ OBJS = \ prop_textcase.o \ prop_textlength.o \ prop_textnum.o \ - prop_textsplit.o + prop_textsplit.o \ + prop_strfunc.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 070f6159e5a88..ac5a17e8f74a5 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -59,6 +59,7 @@ prop_area_sources = files( 'prop_textlength.c', 'prop_textnum.c', 'prop_textsplit.c', + 'prop_strfunc.c', ) diff --git a/src/test/prop/prop_strfunc.c b/src/test/prop/prop_strfunc.c new file mode 100644 index 0000000000000..e15392e75c51d --- /dev/null +++ b/src/test/prop/prop_strfunc.c @@ -0,0 +1,459 @@ +/*------------------------------------------------------------------------- + * + * prop_strfunc.c + * Property-based tests for assorted text/string functions. + * + * Covers string builtins not exercised by the "text" area (prop_text.c): + * lpad, rpad, btrim/ltrim/rtrim, translate, chr/ascii, initcap, repeat, + * left/right and strpos. These check algebraic identities of each function + * against independent oracles -- left(), right(), substr(), position() and + * replace() -- so the functions cross-check one another rather than + * re-deriving their semantics. + * + * Alphabets are ASCII, single-byte and quote-free, so length() counts + * characters, the padding/trim/translate semantics are exact, and SQL + * string-literal quoting is trivial. Where a target length depends on the + * operand's own length, it is built as a SQL expression (length('') + e) + * so the call and its oracle always see the same length. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_strfunc.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "strfunc" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabets never contain a quote, but do it defensively + * so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..12 ASCII letters from "abc" into buf as a bare string. Letters + * only -- all single-byte and never a single quote -- so length() counts + * characters exactly and quoting is trivial. + */ +static void +draw_letters(pg_prop_tc *tc, StringInfo buf) +{ + static const char alphabet[] = "abc"; + int n = (int) pg_prop_draw_int(tc, 0, 12); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, alphabet[pg_prop_draw_int(tc, 0, 2)]); +} + +/* + * Draw 0..12 characters from "ab" into buf as a bare string, so 'a' occurs + * often at the edges -- for the btrim/ltrim/rtrim properties where trimming + * 'a' needs to have teeth. + */ +static void +draw_ab(pg_prop_tc *tc, StringInfo buf) +{ + static const char alphabet[] = "ab"; + int n = (int) pg_prop_draw_int(tc, 0, 12); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, alphabet[pg_prop_draw_int(tc, 0, 1)]); +} + +/* + * Draw 0..12 characters from "abc " (letters plus space) into buf as a bare + * string, so word boundaries occur -- for the initcap property. + */ +static void +draw_words(pg_prop_tc *tc, StringInfo buf) +{ + static const char alphabet[] = "abc "; + int n = (int) pg_prop_draw_int(tc, 0, 12); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, alphabet[pg_prop_draw_int(tc, 0, 3)]); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the expression errored or produced NULL on well-formed + * letter strings (usually an index slipped out of range), which is itself a + * bug -- so either is a failure, reported with the SQL. Modelled on + * prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * lpad()/rpad() produce a result of exactly the requested length: + * length(lpad(S, L, 'x')) = L AND length(rpad(S, L, 'x')) = L + * for a target length L in [0, 20]. Both pad on their respective side when S + * is shorter than L and truncate to L when S is longer, so the result length + * is L regardless of length(S). + */ +PG_PROP(lpad_rpad_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int l; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + l = (int) pg_prop_draw_int(tc, 0, 20); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT length(lpad("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", %d, 'x')) = %d AND length(rpad(", l, l); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", %d, 'x')) = %d", l, l); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * When padding on the right (L >= length(S)), rpad leaves S as a prefix: + * left(rpad(S, length(S) + e, 'x'), length(S)) = S + * for extra padding e in [0, 8]. The target length length(S) + e is built in + * SQL so the call and the oracle agree on length(S); left() is the + * independent oracle for "the first length(S) characters". + */ +PG_PROP(rpad_prefix_preserved_when_padding) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int e; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + e = (int) pg_prop_draw_int(tc, 0, 8); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT left(rpad("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", length("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ") + %d, 'x'), length(", e); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * btrim() removes 'a' from both edges: + * left(btrim(S, 'a'), 1) <> 'a' AND right(btrim(S, 'a'), 1) <> 'a' + * S is drawn from "ab" so leading/trailing 'a' runs occur often. An empty + * result makes left()/right() return '' which is <> 'a', so the all-'a' case + * (btrim collapses to '') is naturally satisfied. + */ +PG_PROP(btrim_removes_edge_chars) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_ab(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT left(btrim("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a'), 1) <> 'a' AND right(btrim("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a'), 1) <> 'a'"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * btrim is the composition of one-sided trims: + * btrim(S, 'a') = ltrim(rtrim(S, 'a'), 'a') + * S is drawn from "ab" so both edges exercise 'a' runs. This cross-checks + * btrim against ltrim and rtrim: trimming both ends at once must equal + * trimming the right then the left. + */ +PG_PROP(ltrim_rtrim_compose_btrim) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_ab(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT btrim("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a') = ltrim(rtrim("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a'), 'a')"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * translate() with single-character from/to is a character map equivalent to + * replace(): + * translate(S, 'a', 'b') = replace(S, 'a', 'b') + * replace() is the independent oracle: for a single-char from/to the two + * agree exactly (every 'a' becomes 'b', everything else is left alone). S is + * drawn from "abc" so 'a' occurs and the map does real work. + */ +PG_PROP(translate_is_char_map) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT translate("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a', 'b') = replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a', 'b')"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * chr() and ascii() round-trip on printable ASCII: + * ascii(chr(p)) = p + * for a drawn code point p in [32, 126]. chr() builds the one-character + * string for the code point and ascii() reads it back, so composing them is + * the identity. + */ +PG_PROP(chr_ascii_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int p; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + p = (int) pg_prop_draw_int(tc, 32, 126); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ascii(chr(%d)) = %d", p, p); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * repeat() multiplies length: + * length(repeat(S, n)) = length(S) * n + * for a count n in [0, 10]. With a single-byte alphabet length() counts + * characters, so repeating S n times gives exactly n copies' worth of + * characters (0 for n = 0). + */ +PG_PROP(repeat_length_multiplies) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + n = (int) pg_prop_draw_int(tc, 0, 10); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT length(repeat("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", %d)) = length(", n); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ") * %d", n); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * initcap() is idempotent on its own output: + * initcap(initcap(S)) = initcap(S) + * initcap uppercases the first letter of each word and lowercases the rest; + * applying it again finds the same word boundaries and the same already-cased + * letters, so the result is unchanged. S is drawn from "abc " (letters plus + * space) so word boundaries actually occur. + */ +PG_PROP(initcap_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_words(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT initcap(initcap("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) = initcap("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * strpos() is consistent with substr()/position(): + * when strpos(S, c) = 0 the char is absent (position(c in S) = 0); + * otherwise substr(S, strpos(S, c), 1) = c -- strpos points at an actual + * occurrence of c. + * strpos() returns the 1-based index of the first match or 0 if absent; this + * cross-checks it against substr() (the character it claims to point at) and + * position() (the absence case). c is fixed 'a' and S is drawn from "abc" so + * both branches are exercised. + */ +PG_PROP(strpos_consistent_with_substring) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT CASE WHEN strpos("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a') = 0 THEN position('a' in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = 0 ELSE substr("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", strpos("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a'), 1) = 'a' END"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 141c0ce4db94c..0015a7858aed2 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 8ab1bee66519e8cf4a05d5a0224a7fa3b42588b8 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:55 -0400 Subject: [PATCH 047/338] prop: add position/strpos/starts_with/overlay semantics property tests (strpos) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_strpos.c | 411 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 415 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_strpos.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 1d52f29c2ccc9..fe4291c06d9b7 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -62,7 +62,8 @@ OBJS = \ prop_textlength.o \ prop_textnum.o \ prop_textsplit.o \ - prop_strfunc.o + prop_strfunc.o \ + prop_strpos.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index ac5a17e8f74a5..75d0dbd13574c 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -60,6 +60,7 @@ prop_area_sources = files( 'prop_textnum.c', 'prop_textsplit.c', 'prop_strfunc.c', + 'prop_strpos.c', ) diff --git a/src/test/prop/prop_strpos.c b/src/test/prop/prop_strpos.c new file mode 100644 index 0000000000000..745b610e881a4 --- /dev/null +++ b/src/test/prop/prop_strpos.c @@ -0,0 +1,411 @@ +/*------------------------------------------------------------------------- + * + * prop_strpos.c + * Property-based tests for the precise semantics of position/strpos, + * starts_with, and overlay. + * + * These functions have exact, documented contracts: strpos(S, P) and + * position(P in S) are synonyms returning a 1-based index or 0 when absent; + * the empty needle is found at position 1; starts_with(S, P) is true iff P is + * a prefix of S; and overlay(S placing R from p for n) splices R over n + * characters starting at p, so an interior replace of equal width preserves + * length while a zero-width insert grows it by length(R). The oracle is this + * algebra expressed two ways (synonym equality, substring recovery, prefix + * via left()), so a false result is a bug in the function, not a quoting + * artefact: all strings are drawn from the quote-free [abc] alphabet, with + * 'z' and 'XY' as sentinels guaranteed absent from that alphabet. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_strpos.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "strpos" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. The [abc] alphabet never contains a quote, but do it + * defensively so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw n..m characters from the quote-free [abc] alphabet into buf as a bare + * string (no quoting). All ASCII, never a single quote, so length() counts + * characters exactly and quoting is trivial. + */ +static void +draw_abc(pg_prop_tc *tc, StringInfo buf, int min, int max) +{ + static const char alphabet[] = "abc"; + int n = (int) pg_prop_draw_int(tc, min, max); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, alphabet[pg_prop_draw_int(tc, 0, 2)]); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on valid inputs, + * which is itself a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * strpos and position are synonyms: strpos(S, P) = position(P in S) for every + * S and P. strpos takes its arguments (haystack, needle) while position uses + * the "needle in haystack" syntax, so this also pins down the argument order. + * P is drawn 0..3 characters over [abc] (including empty, which both must + * handle identically). + */ +PG_PROP(strpos_eq_position) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + p, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&p); + draw_abc(tc, &s, 0, 12); + draw_abc(tc, &p, 0, 3); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT strpos("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ") = position("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, " in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A needle absent from the alphabet is never found: 'z' never occurs in an + * [abc] string, so position('z' in S) = 0. + */ +PG_PROP(position_zero_when_absent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc(tc, &s, 0, 12); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT position('z' in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = 0"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The empty needle is found at position 1: position('' in S) = 1 for every S + * (including the empty string, where the empty needle still reports 1). + */ +PG_PROP(position_empty_is_one) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc(tc, &s, 0, 12); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT position('' in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = 1"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * starts_with(S, P) is true iff P is a prefix of S, which is exactly + * left(S, length(P)) = P: left() takes the leading length(P) characters, and + * they equal P precisely when P prefixes S. This holds for every P including + * the empty string (left(S, 0) = '' = P). P is drawn 0..3 over [abc]. + */ +PG_PROP(starts_with_eq_prefix) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + p, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&p); + draw_abc(tc, &s, 0, 12); + draw_abc(tc, &p, 0, 3); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT starts_with("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ") = (left("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", length("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ")) = "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * When position(P in S) = k > 0, the substring of S at k of width length(P) + * is exactly P: substr(S, k, length(P)) = P. This is the defining property + * of a found position. A CASE guards k = 0 (needle absent) so the property + * is vacuously true there; when k > 0 the recovered substring must match. + * P is drawn 1..3 over [abc]. + */ +PG_PROP(position_found_substring) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + p, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&p); + draw_abc(tc, &s, 0, 12); + draw_abc(tc, &p, 1, 3); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT CASE WHEN position("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, " in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = 0 THEN true ELSE substr("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", position("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, " in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "), length("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ")) = "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, " END"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * An interior equal-width overlay preserves length: overlay(S placing 'XY' + * from p for 2) replaces the 2 characters at p with the 2-character 'XY', so + * the length is unchanged. p is drawn 1..length(S)-1, which keeps the whole + * 2-wide window inside S (p + 2 - 1 <= length(S)), so no growth from running + * past the end. Guarded to length(S) >= 3 so such an interior p exists. + */ +PG_PROP(overlay_replaces) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int slen; + int p; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc(tc, &s, 3, 12); + slen = (int) strlen(s.data); + p = (int) pg_prop_draw_int(tc, 1, slen - 1); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT length(overlay("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, " placing 'XY' from %d for 2)) = length(", p); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A zero-width overlay inserts: overlay(S placing 'XY' from p for 0) deletes + * nothing and splices in the 2-character 'XY', so the length grows by 2. p + * is drawn 1..length(S)+1 (every legal insertion point, including just past + * the end). + */ +PG_PROP(overlay_insert_grows) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int slen; + int p; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc(tc, &s, 0, 12); + slen = (int) strlen(s.data); + p = (int) pg_prop_draw_int(tc, 1, slen + 1); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT length(overlay("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, " placing 'XY' from %d for 0)) = length(", p); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") + 2"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A non-empty needle is found at position 1 in a string it prefixes: + * strpos(P || S, P) = 1, because P is a prefix of P || S and the first match + * is at the very start. P is drawn 1..3 over [abc] (empty is excluded + * because strpos('' prepended, '') is 1 too but trivially; the non-empty case + * is the interesting prefix contract). + */ +PG_PROP(strpos_prefix) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + p, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&p); + draw_abc(tc, &s, 0, 12); + draw_abc(tc, &p, 1, 3); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT strpos(("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, " || "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "), "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ") = 1"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 0015a7858aed2..1062c6dd4a35a 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 1a36bed84928abe87718a017f89432d1eec55e26 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:56 -0400 Subject: [PATCH 048/338] prop: add substring, left/right, and starts_with property tests (substring) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_substring.c | 437 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 441 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_substring.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index fe4291c06d9b7..59e3e2e0da9ae 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -63,7 +63,8 @@ OBJS = \ prop_textnum.o \ prop_textsplit.o \ prop_strfunc.o \ - prop_strpos.o + prop_strpos.o \ + prop_substring.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 75d0dbd13574c..e6621d9b3ade7 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -61,6 +61,7 @@ prop_area_sources = files( 'prop_textsplit.c', 'prop_strfunc.c', 'prop_strpos.c', + 'prop_substring.c', ) diff --git a/src/test/prop/prop_substring.c b/src/test/prop/prop_substring.c new file mode 100644 index 0000000000000..d433325af3373 --- /dev/null +++ b/src/test/prop/prop_substring.c @@ -0,0 +1,437 @@ +/*------------------------------------------------------------------------- + * + * prop_substring.c + * Property-based tests for substring extraction and position functions. + * + * Covers the substring/position builtins not exercised by the "text" area + * (prop_text.c) or the "strfunc" area (prop_strfunc.c): the SQL-standard + * substring(S FROM m FOR L) and substring(S FROM 1) forms, the POSIX + * substring(S FROM pattern) extraction, left()/right() edge cases + * (including negative lengths) and starts_with(). The "text" area owns + * substr()-split/overlay/split_part and the "strfunc" area owns + * lpad/rpad/btrim/translate/chr/ascii/initcap/repeat/strpos, so none of + * those are repeated here. + * + * The oracle is independent builtins -- left(), right(), substring() with + * FROM/FOR, length(), position() and starts_with() -- and the identities + * they must satisfy together, so the functions cross-check one another + * rather than re-deriving their semantics. Alphabets are ASCII single-byte + * letters "abc", so length() counts characters exactly, SQL string-literal + * quoting is trivial, and a single-letter POSIX pattern is a plain literal + * regex (no metacharacters), making the regex-extraction oracle exact. + * Indices are clamped server-side with least()/greatest() so the manual + * oracles always see the same in-range index and no negative-length surprise + * can arise; left() with a negative length is its own property. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_substring.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "substring" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabets never contain a quote, but do it defensively + * so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..14 ASCII letters from "abc" into buf as a bare string. Letters + * only -- all single-byte and never a single quote -- so length() counts + * characters exactly, quoting is trivial, and a single letter used as a POSIX + * pattern is a plain literal (no regex metacharacters). + */ +static void +draw_letters(pg_prop_tc *tc, StringInfo buf) +{ + static const char alphabet[] = "abc"; + int n = (int) pg_prop_draw_int(tc, 0, 14); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, alphabet[pg_prop_draw_int(tc, 0, 2)]); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the expression errored or produced NULL on well-formed + * letter strings (usually an index slipped out of range), which is itself a + * bug -- so either is a failure, reported with the SQL. Modelled on + * prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * left() and right() reconstruct the whole string: + * left(S, k) || right(S, length(S) - k) = S + * for a split point k in [0, length(S)]. left(S, k) is the first k + * characters and right(S, m) the last m, so the two halves cover S exactly + * and concatenate back to it. k is drawn 0..16 and clamped server-side with + * least(k, length(S)) so the split point is always in range (and the second + * argument length(S) - k is never negative, which for right() would mean + * something else entirely). + */ +PG_PROP(left_right_concat_reconstructs) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int k; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + k = (int) pg_prop_draw_int(tc, 0, 16); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT left("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", least(%d, length(", k); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "))) || right("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", length("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ") - least(%d, length(", k); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "))) = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A negative length to left() drops characters from the end: + * left(S, -n) = substring(S from 1 for greatest(length(S) - n, 0)) + * for n in [0, length(S) + 2]. left(S, -n) returns all but the last n + * characters (and the empty string once n reaches length(S)); the oracle + * takes the first length(S) - n characters, clamped to >= 0 so both sides + * agree on the over-trimmed case. n is drawn 0..16 which comfortably covers + * [0, length(S) + 2] since length(S) <= 14. + */ +PG_PROP(left_negative_drops_from_end) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + /* + * n must be >= 1: left(s, -0) is left(s, 0) = '' (SQL -0 == 0), which does + * not match the "drop the last n characters" oracle. Only a genuinely + * negative length triggers the drop-from-end behaviour. + */ + n = (int) pg_prop_draw_int(tc, 1, 16); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT left("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", -%d) = substring(", n); + append_sql_literal(&q, s.data); + appendStringInfo(&q, " from 1 for greatest(length("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ") - %d, 0))", n); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The length of a SQL-standard substring never exceeds the requested length + * nor the available tail: + * length(substring(S from m for L)) <= L + * AND length(substring(S from m for L)) <= greatest(length(S) - m + 1, 0) + * for a 1-based start m in [1, length(S) + 1] and length L in [0, length(S)]. + * This is a sound two-sided upper bound: substring(S from m for L) yields at + * most L characters and at most the length(S) - m + 1 characters available + * from m to the end (0 when m is past the end). m is drawn 1..16 and clamped + * server-side to [1, length(S) + 1]; L is drawn 0..16. + */ +PG_PROP(substring_from_for_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int m; + int l; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + m = (int) pg_prop_draw_int(tc, 1, 16); + l = (int) pg_prop_draw_int(tc, 0, 16); + + initStringInfo(&q); + + /* extracted length <= requested length L */ + appendStringInfoString(&q, "SELECT length(substring("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, " from least(greatest(%d, 1), length(", m); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ") + 1) for %d)) <= %d", l, l); + + /* extracted length <= available tail from m */ + appendStringInfoString(&q, " AND length(substring("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, " from least(greatest(%d, 1), length(", m); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ") + 1) for %d)) <= greatest(length(", l); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ") - least(greatest(%d, 1), length(", m); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") + 1) + 1, 0)"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * substring from position 1 with no length is the whole string: + * substring(S from 1) = S + * The SQL-standard substring with only a start of 1 and no FOR clause returns + * everything from the first character onward, i.e. S itself. + */ +PG_PROP(substring_from_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT substring("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " from 1) = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * starts_with() agrees with a prefix comparison via left(): + * starts_with(S, P) = (left(S, length(P)) = P) + * left(S, length(P)) is the first length(P) characters of S, so S starts with + * P exactly when that prefix equals P. Both S and the short prefix P are + * drawn from "abc" so P is a genuine candidate prefix often enough to exercise + * both the true and false branches. left() and length() are independent + * oracles for starts_with(). + */ +PG_PROP(starts_with_matches_left) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + p, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&p); + draw_letters(tc, &s); + draw_letters(tc, &p); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT starts_with("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ") = (left("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", length("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ")) = "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The POSIX substring(S from pattern) form extracts the matched text: + * when position(c in S) > 0, substring(S from c) = c; + * otherwise substring(S from c) IS NULL. + * The pattern c is a single letter from "abc", which is a plain literal regex + * (no metacharacters), so it matches exactly where that character occurs. + * position() is the independent oracle for whether c occurs at all; when it + * does the extraction returns that one character, and when it does not the + * POSIX form returns NULL. + */ +PG_PROP(substring_regex_extracts_match) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + char c; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + c = "abc"[pg_prop_draw_int(tc, 0, 2)]; + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT CASE WHEN position('"); + appendStringInfoChar(&q, c); + appendStringInfoString(&q, "' in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") > 0 THEN substring("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " from '"); + appendStringInfoChar(&q, c); + appendStringInfoString(&q, "') = '"); + appendStringInfoChar(&q, c); + appendStringInfoString(&q, "' ELSE substring("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " from '"); + appendStringInfoChar(&q, c); + appendStringInfoString(&q, "') IS NULL END"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * SQL-standard substring FROM/FOR splits and rejoins to reconstruct S: + * substring(S from 1 for k) || substring(S from k + 1) = S + * for a split point k in [0, length(S)]. substring(S from 1 for k) is the + * first k characters and substring(S from k + 1) the rest, so they concatenate + * back to S. k is drawn 0..16 and clamped server-side with least(k, + * length(S)) so the split point is always in range and both halves cover S + * exactly. This exercises the SQL-standard substring FROM/FOR form, distinct + * from the substr() in the "text" area. + */ +PG_PROP(substring_concat_pieces) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int k; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + k = (int) pg_prop_draw_int(tc, 0, 16); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT substring("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, " from 1 for least(%d, length(", k); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "))) || substring("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, " from least(%d, length(", k); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) + 1) = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 1062c6dd4a35a..fd5b0ba6bb6bf 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 9a0a9fd6375022ab0853ae9e33fef0fef6cebcf8 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:56 -0400 Subject: [PATCH 049/338] prop: add lpad/rpad/btrim/trim laws property tests (padtrim) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_padtrim.c | 401 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 405 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_padtrim.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 59e3e2e0da9ae..4e25ca44997a3 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -64,7 +64,8 @@ OBJS = \ prop_textsplit.o \ prop_strfunc.o \ prop_strpos.o \ - prop_substring.o + prop_substring.o \ + prop_padtrim.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index e6621d9b3ade7..7d1cdff0bca0a 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -62,6 +62,7 @@ prop_area_sources = files( 'prop_strfunc.c', 'prop_strpos.c', 'prop_substring.c', + 'prop_padtrim.c', ) diff --git a/src/test/prop/prop_padtrim.c b/src/test/prop/prop_padtrim.c new file mode 100644 index 0000000000000..119eb311cd398 --- /dev/null +++ b/src/test/prop/prop_padtrim.c @@ -0,0 +1,401 @@ +/*------------------------------------------------------------------------- + * + * prop_padtrim.c + * Property-based tests for lpad/rpad and btrim/ltrim/rtrim/trim. + * + * These check the padding and trimming laws against independent oracles -- + * length(), left(), right() and construction of a known-trimmable string -- + * rather than re-deriving the functions' semantics. The alphabet is [abc] + * (single-byte, quote-free, and crucially excluding the pad char 'x' and the + * space), so length() counts characters, a padded or trimmed region is + * exactly the 'x'/space run we added, and a failure is a bug in the function. + * + * Distinct from the strfunc/textfns/textpos areas: here the laws are the + * exact-length guarantee of pad, the suffix/prefix a pad preserves, the exact + * recovery btrim gives on a known-trimmable string, the ltrim/rtrim + * composition, trim's default (space) stripping, and pad's right-truncation. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_padtrim.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "padtrim" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabet never contains a quote, but do it defensively + * so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..10 characters from "abc" into buf as a bare string. Letters only + * -- all single-byte, never a quote, never the pad char 'x', never a space -- + * so length() counts characters and the pad/trim semantics are exact. + */ +static void +draw_letters(pg_prop_tc *tc, StringInfo buf) +{ + static const char alphabet[] = "abc"; + int n = (int) pg_prop_draw_int(tc, 0, 10); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, alphabet[pg_prop_draw_int(tc, 0, 2)]); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the expression errored or produced NULL on well-formed + * letter strings, which is itself a bug -- so either is a failure, reported + * with the SQL. Modelled on prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * lpad() produces a result of exactly the requested length: + * length(lpad(S, n, 'x')) = n + * for a target n in [0, 20]. lpad pads on the left when S is shorter than n + * and truncates to n when S is longer, so the result length is n regardless + * of length(S). + */ +PG_PROP(lpad_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + n = (int) pg_prop_draw_int(tc, 0, 20); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT length(lpad("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", %d, 'x')) = %d", n, n); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * rpad() produces a result of exactly the requested length: + * length(rpad(S, n, 'x')) = n + * for a target n in [0, 20]. Symmetric to lpad_length: rpad pads on the + * right or truncates, so the result length is n regardless of length(S). + */ +PG_PROP(rpad_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + n = (int) pg_prop_draw_int(tc, 0, 20); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT length(rpad("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", %d, 'x')) = %d", n, n); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Left-padding preserves the original as the suffix: + * right(lpad(S, length(S) + e, 'x'), length(S)) = S + * for extra padding e in [0, 10], i.e. the target length is >= length(S). + * The target is built in SQL as length('') + e so the call and oracle + * agree on length(S); right() is the independent oracle for "the last + * length(S) characters", which after left-padding is exactly S. + */ +PG_PROP(lpad_preserves_suffix) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int e; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + e = (int) pg_prop_draw_int(tc, 0, 10); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT right(lpad("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", length("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ") + %d, 'x'), length(", e); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Right-padding preserves the original as the prefix: + * left(rpad(S, length(S) + e, 'x'), length(S)) = S + * for extra padding e in [0, 10]. Symmetric to lpad_preserves_suffix: after + * right-padding, the first length(S) characters are exactly S. left() is the + * independent oracle. + */ +PG_PROP(rpad_preserves_prefix) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int e; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + e = (int) pg_prop_draw_int(tc, 0, 10); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT left(rpad("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", length("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ") + %d, 'x'), length(", e); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * btrim removes a known 'x' run from both ends exactly: + * btrim('xx' || S || 'xx', 'x') = S + * S is drawn from [abc] so it contains no 'x'; the only 'x' characters are the + * two we prepend and append. btrim(_, 'x') must strip exactly those and leave + * S, so the recovered value equals the original. (An all-trim result on S + * cannot over-trim since S has no 'x'.) + */ +PG_PROP(btrim_removes_both_ends) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT btrim('xx' || "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " || 'xx', 'x') = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * btrim is the composition of the two one-sided trims: + * btrim(S2, 'x') = ltrim(rtrim(S2, 'x'), 'x') + * where S2 = 'xx' || S || 'xx' with S from [abc], so both edges carry 'x' + * runs for the trims to bite on. Trimming both ends at once must equal + * trimming the right then the left; this cross-checks btrim against ltrim and + * rtrim. + */ +PG_PROP(ltrim_rtrim_compose) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT btrim('xx' || "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " || 'xx', 'x') = ltrim(rtrim('xx' || "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " || 'xx', 'x'), 'x')"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * trim() with no explicit character set strips leading/trailing spaces: + * trim(' ' || S || ' ') = S + * S is drawn from [abc] so it contains no space; the only spaces are the two + * we prepend and append. Default trim must remove exactly those, recovering + * S. (S cannot be over-trimmed since it has no space.) + */ +PG_PROP(trim_default_spaces) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT trim(' ' || "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " || ' ') = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * When the target length is shorter than S, lpad truncates from the right, + * keeping the first n characters: + * lpad(S, n, 'x') = left(S, n) + * for n in [0, length(S)]. n is drawn in SQL as a fraction of length('') + * via a drawn offset so the call and oracle see the same length; left(S, n) + * is the independent oracle for "the first n characters". At n = length(S) + * both sides are S, so the boundary is sound; below it, this exercises pad's + * right-truncation. + */ +PG_PROP(lpad_truncates) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int slen; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + /* n in [0, length(S)]; strlen is the character length for [abc]. */ + slen = (int) strlen(s.data); + n = (int) pg_prop_draw_int(tc, 0, slen); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT lpad("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", %d, 'x') = left(", n); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", %d)", n); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index fd5b0ba6bb6bf..33f35b60e6f80 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 17c95d949bf81d6064787c6d294c979b152a4e9d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:56 -0400 Subject: [PATCH 050/338] prop: add trim LEADING/TRAILING/BOTH variants property tests (trimfns) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_trimfns.c | 347 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 351 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_trimfns.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 4e25ca44997a3..5781e1ed4d648 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -65,7 +65,8 @@ OBJS = \ prop_strfunc.o \ prop_strpos.o \ prop_substring.o \ - prop_padtrim.o + prop_padtrim.o \ + prop_trimfns.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 7d1cdff0bca0a..29d99a68b2323 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -63,6 +63,7 @@ prop_area_sources = files( 'prop_strpos.c', 'prop_substring.c', 'prop_padtrim.c', + 'prop_trimfns.c', ) diff --git a/src/test/prop/prop_trimfns.c b/src/test/prop/prop_trimfns.c new file mode 100644 index 0000000000000..ba8b8fd4a1307 --- /dev/null +++ b/src/test/prop/prop_trimfns.c @@ -0,0 +1,347 @@ +/*------------------------------------------------------------------------- + * + * prop_trimfns.c + * Property-based tests for the SQL TRIM(...) syntactic forms. + * + * The SQL standard TRIM(LEADING|TRAILING|BOTH FROM ) is parsed + * into the btrim/ltrim/rtrim built-ins. These properties pin down the three + * directional variants and the two-forms-must-agree equivalence between the + * TRIM syntax and the function-call built-ins: TRIM BOTH = btrim, TRIM + * LEADING = ltrim, TRIM TRAILING = rtrim. An oracle built from a known- + * trimmable string (an 'x' sentinel not in the body alphabet) plus the + * syntax-vs-function differential make every assertion sound. A mismatch is + * a parse-analysis or built-in bug, not a rounding artefact. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_trimfns.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "trimfns" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Body alphabet. 'x' is deliberately excluded so it can serve as a trim + * sentinel that never collides with the body content. + */ +#define TRIM_ALPHABET "abc" + +/* + * Draw a quoted SQL string literal over the body alphabet [abc] into buf. + * The alphabet has no single quotes, so no escaping is needed. Length 0..12. + */ +static void +draw_body(pg_prop_tc *tc, StringInfo buf) +{ + size_t len = 0; + char *s = pg_prop_draw_from(tc, TRIM_ALPHABET, 0, 12, &len); + + appendStringInfoChar(buf, '\''); + appendBinaryStringInfo(buf, s, (int) len); + appendStringInfoChar(buf, '\''); + free(s); +} + +/* + * Draw a quoted SQL string literal that interleaves 'x' sentinels with body + * characters from [abc], so trims have runs to bite on at both ends and in + * the middle. Each position is independently either an 'x' or a body char. + * This produces the general S2 used by the equivalence properties. + */ +static void +draw_mixed(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 12); + + appendStringInfoChar(buf, '\''); + for (int i = 0; i < n; i++) + { + if (pg_prop_draw_bool(tc)) + appendStringInfoChar(buf, 'x'); + else + appendStringInfoChar(buf, + TRIM_ALPHABET[pg_prop_draw_int(tc, 0, 2)]); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the trim errored or produced NULL on finite text inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * TRIM(BOTH ' ' FROM ' ' || S || ' ') = S. S is over [abc] (no spaces), so + * the two leading and two trailing spaces are exactly what default trim must + * strip, recovering S. (S cannot be over-trimmed: it holds no spaces.) + */ +PG_PROP(trim_both_default) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_body(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT trim(BOTH ' ' FROM ' ' || %s || ' ') = %s", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * TRIM(LEADING 'x' FROM 'xxx' || S) = S. S is over [abc] so it never starts + * with 'x'; the three prepended sentinels are exactly what leading trim must + * strip, leaving S untouched. + */ +PG_PROP(trim_leading) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_body(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT trim(LEADING 'x' FROM 'xxx' || %s) = %s", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * TRIM(TRAILING 'x' FROM S || 'xxx') = S. S is over [abc] so it never ends + * with 'x'; the three appended sentinels are exactly what trailing trim must + * strip, leaving S untouched. + */ +PG_PROP(trim_trailing) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_body(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT trim(TRAILING 'x' FROM %s || 'xxx') = %s", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * TRIM(BOTH 'x' FROM 'xx' || S || 'xx') = S. S is over [abc] so it has no + * 'x' at either edge; the two prepended and two appended sentinels are + * exactly what both-ends trim with the explicit 'x' set must strip, + * recovering S. + */ +PG_PROP(trim_both_chars) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_body(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT trim(BOTH 'x' FROM 'xx' || %s || 'xx') = %s", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * btrim(S2, 'x') = TRIM(BOTH 'x' FROM S2) for an arbitrary mixed S2. The two + * spellings of both-ends trimming must agree on every input; S2 interleaves + * 'x' sentinels with body chars so there are edge and interior runs to test. + */ +PG_PROP(btrim_eq_trim_both) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s2); + draw_mixed(tc, &s2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT btrim(%s, 'x') = trim(BOTH 'x' FROM %s)", + s2.data, s2.data); + + require_true(tc, conn, q.data); + + pfree(s2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ltrim(S2, 'x') = TRIM(LEADING 'x' FROM S2) for an arbitrary mixed S2. The + * function and syntax forms of leading trim must agree on every input. + */ +PG_PROP(ltrim_eq_trim_leading) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s2); + draw_mixed(tc, &s2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ltrim(%s, 'x') = trim(LEADING 'x' FROM %s)", + s2.data, s2.data); + + require_true(tc, conn, q.data); + + pfree(s2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * rtrim(S2, 'x') = TRIM(TRAILING 'x' FROM S2) for an arbitrary mixed S2. The + * function and syntax forms of trailing trim must agree on every input. + */ +PG_PROP(rtrim_eq_trim_trailing) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s2); + draw_mixed(tc, &s2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT rtrim(%s, 'x') = trim(TRAILING 'x' FROM %s)", + s2.data, s2.data); + + require_true(tc, conn, q.data); + + pfree(s2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * btrim is idempotent: btrim(btrim(S2, 'x'), 'x') = btrim(S2, 'x'). A single + * both-ends trim already removes every leading and trailing 'x', so a second + * pass must be a no-op. S2 is an arbitrary mixed string. + */ +PG_PROP(trim_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s2); + draw_mixed(tc, &s2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT btrim(btrim(%s, 'x'), 'x') = btrim(%s, 'x')", + s2.data, s2.data); + + require_true(tc, conn, q.data); + + pfree(s2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 33f35b60e6f80..3c1727090c88a 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From c33de82330dec9de7cba465a145af39da1d185c5 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:57 -0400 Subject: [PATCH 051/338] prop: add like property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_like.c | 397 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 401 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_like.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 5781e1ed4d648..e40106cff8aa5 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -66,7 +66,8 @@ OBJS = \ prop_strpos.o \ prop_substring.o \ prop_padtrim.o \ - prop_trimfns.o + prop_trimfns.o \ + prop_like.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 29d99a68b2323..339381ff163e7 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -64,6 +64,7 @@ prop_area_sources = files( 'prop_substring.c', 'prop_padtrim.c', 'prop_trimfns.c', + 'prop_like.c', ) diff --git a/src/test/prop/prop_like.c b/src/test/prop/prop_like.c new file mode 100644 index 0000000000000..3b4b8e35d1c62 --- /dev/null +++ b/src/test/prop/prop_like.c @@ -0,0 +1,397 @@ +/*------------------------------------------------------------------------- + * + * prop_like.c + * Property-based tests for LIKE/ILIKE pattern matching. + * + * LIKE/ILIKE and the % _ wildcards (src/backend/utils/adt/like_match.c) have + * had matching and escaping fixes; these check matching against an + * independent oracle built from the same strings, plus escaping and + * case-insensitivity invariants. + * + * The oracle is independent builtins -- starts_with(), repeat(), length(), + * lower() -- so the restricted pattern classes we generate (literal prefix + * plus a single trailing %, a run of _, a wildcard-free pattern) are + * cross-checked without re-deriving LIKE's own semantics. Alphabets for the + * text operand are letters only, so a wildcard never leaks into the text and + * every % or _ under test comes from a pattern. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_like.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "like" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabets never contain a quote, but do it defensively + * so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..10 characters of plain text from a 3-letter alphabet "abc" into buf + * as a bare string (no quoting). Letters only -- never % or _ -- so the text + * cannot accidentally contain a wildcard. + */ +static void +draw_text(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abc"; + int n = (int) pg_prop_draw_int(tc, 0, 10); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 2)]); +} + +/* + * Draw 0..8 tokens into buf as a bare LIKE pattern. Each token is a literal + * letter from "abc", a '%' (match any run), or a '_' (match any one char). + * The tiny alphabet keeps patterns matching often. + */ +static void +draw_pattern(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abc"; + int n = (int) pg_prop_draw_int(tc, 0, 8); + + for (int i = 0; i < n; i++) + { + int kind = (int) pg_prop_draw_int(tc, 0, 4); + + if (kind == 3) + appendStringInfoChar(buf, '%'); + else if (kind == 4) + appendStringInfoChar(buf, '_'); + else + appendStringInfoChar(buf, letters[kind]); + } +} + +/* + * Draw 0..10 mixed-case characters from "abAB" into buf as a bare string, for + * exercising case folding (ILIKE vs lower()+LIKE). Letters only. + */ +static void +draw_mixed_text(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abAB"; + int n = (int) pg_prop_draw_int(tc, 0, 10); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 3)]); +} + +/* + * Draw 0..8 mixed-case tokens into buf as a bare LIKE pattern: a literal + * letter from "abAB", a '%', or a '_'. For the case-insensitivity property. + */ +static void +draw_mixed_pattern(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abAB"; + int n = (int) pg_prop_draw_int(tc, 0, 8); + + for (int i = 0; i < n; i++) + { + int kind = (int) pg_prop_draw_int(tc, 0, 5); + + if (kind == 4) + appendStringInfoChar(buf, '%'); + else if (kind == 5) + appendStringInfoChar(buf, '_'); + else + appendStringInfoChar(buf, letters[kind]); + } +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the comparison errored or produced NULL on non-NULL + * letter strings, which is itself a bug -- so either is a failure, reported + * with the SQL. Modelled on prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * The single-% pattern matches every string. Trivial, but catches gross + * breakage of the "% matches any run including empty" rule. + */ +PG_PROP(percent_wildcard_matches_all) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " LIKE '%'"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A wildcard-free pattern matches exactly its own text. Since S has no % or + * _, "S LIKE S" matches iff S equals S, which it does. + */ +PG_PROP(literal_pattern_matches_itself) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " LIKE "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * S always matches its own prefix followed by '%'. The prefix length k is + * drawn in [0, length(S)] and the substring is taken server-side so no C-side + * substring logic can disagree with the backend's. + */ +PG_PROP(prefix_pattern_consistent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int k; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + /* prefix length in [0, len(S)] */ + k = (int) pg_prop_draw_int(tc, 0, (int) strlen(s.data)); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " LIKE (substring("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, " from 1 for %d) || '%%')", k); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * LIKE of a literal prefix plus a trailing '%' agrees with starts_with(). + * starts_with(text, prefix) is an independent builtin, so this genuinely + * cross-checks LIKE's prefix semantics rather than re-deriving them. + */ +PG_PROP(like_prefix_matches_oracle) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + l, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&l); + draw_text(tc, &s); /* text */ + draw_text(tc, &l); /* literal prefix (letters only) */ + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " LIKE ("); + append_sql_literal(&q, l.data); + appendStringInfoString(&q, " || '%')) = starts_with("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, l.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(l.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A pattern of exactly n underscores (and nothing else) matches S iff S has + * length n. length() is an independent builtin, so this cross-checks the '_' + * (match exactly one char) semantics. + */ +PG_PROP(like_underscores_match_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + n = (int) pg_prop_draw_int(tc, 0, 12); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, " LIKE repeat('_', %d)) = (length(", n); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ") = %d)", n); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ILIKE is case-insensitive LIKE: "S ILIKE P" equals "lower(S) LIKE lower(P)". + * Our pattern alphabet is letters + % + _, all case-fold safe (% and _ are + * unaffected by lower()), so there are no escape subtleties. Text and + * pattern are drawn mixed-case to actually exercise the folding. + */ +PG_PROP(ilike_is_case_insensitive_like) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + p, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&p); + draw_mixed_text(tc, &s); + draw_mixed_pattern(tc, &p); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " ILIKE "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ") = (lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") LIKE lower("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, "))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * An escaped wildcard is a literal. With the default backslash escape and + * modern standard_conforming_strings (so '100\%' in SQL is a literal + * backslash followed by a percent), LIKE treats '\%' as a literal '%': + * '100%' matches '100\%' but '100x' does not. Fixed literals, no draws. + */ +PG_PROP(escape_literal_percent) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT ('100%' LIKE '100\\%') AND NOT ('100x' LIKE '100\\%')"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 3c1727090c88a..f531983c7db36 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 5f76a7a7fcf41e04b04460ea028ce68f203ddd15 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:57 -0400 Subject: [PATCH 052/338] prop: add LIKE wildcard and escape semantics property tests (likepattern) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_likepattern.c | 385 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 389 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_likepattern.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index e40106cff8aa5..a649b017b95b3 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -67,7 +67,8 @@ OBJS = \ prop_substring.o \ prop_padtrim.o \ prop_trimfns.o \ - prop_like.o + prop_like.o \ + prop_likepattern.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 339381ff163e7..b522f6bab58c6 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -65,6 +65,7 @@ prop_area_sources = files( 'prop_padtrim.c', 'prop_trimfns.c', 'prop_like.c', + 'prop_likepattern.c', ) diff --git a/src/test/prop/prop_likepattern.c b/src/test/prop/prop_likepattern.c new file mode 100644 index 0000000000000..2c14dff365fad --- /dev/null +++ b/src/test/prop/prop_likepattern.c @@ -0,0 +1,385 @@ +/*------------------------------------------------------------------------- + * + * prop_likepattern.c + * Property-based tests for LIKE wildcard semantics (_ % ESCAPE). + * + * These isolate the structural meaning of LIKE's wildcards -- '_' matches + * exactly one character, '%' matches any run including empty, and ESCAPE + * turns a following metacharacter into a literal -- and cross-check each + * against an independent builtin (length(), position(), left()). This is a + * different cut than the "like"/"stringmatch"/"regexmatch" areas: it is about + * the wildcard algebra itself (how many chars a pattern consumes), not about + * agreement between operators. A mismatch means LIKE miscounts a wildcard -- + * a matching bug in like_match.c. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_likepattern.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "likepattern" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabet never contains a quote, but do it defensively + * so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + const char *p; + + appendStringInfoChar(buf, '\''); + for (p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..10 characters of plain text from the 3-letter alphabet "abc" into + * buf as a bare string (no quoting). Letters only -- never %, _, or \ -- so + * the text can never contain a LIKE metacharacter, and every wildcard under + * test comes from the pattern we build, not the operand. + */ +static void +draw_text(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abc"; + int n = (int) pg_prop_draw_int(tc, 0, 10); + int i; + + for (i = 0; i < n; i++) + appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 2)]); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the match errored or produced NULL on non-NULL letter + * strings, which is itself a bug -- so either is a failure, reported with the + * SQL. Modelled on prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * A single '_' matches S iff S has length exactly 1. length() is an + * independent builtin, so this pins the "_ consumes exactly one character" + * rule at its smallest instance. + */ +PG_PROP(underscore_matches_one) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " LIKE '_') = (length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = 1)"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The single-'%' pattern matches every string, the empty string included. + * Pins "% matches any run including empty" at its base case. + */ +PG_PROP(percent_matches_all) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " LIKE '%'"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * '%' || c || '%' matches S iff the single letter c occurs somewhere in S. + * position(c in S) > 0 is an independent builtin, so this cross-checks the + * "contains" idiom (a run before, a run after) against substring search. + */ +PG_PROP(percent_x_percent_contains) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + char c; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + c = "abc"[pg_prop_draw_int(tc, 0, 2)]; + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, " LIKE ('%%' || '%c' || '%%')) = (position('%c' in ", + c, c); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") > 0)"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A pattern of exactly n underscores matches S iff S has length n. Scales + * the single-'_' rule to a run via repeat('_', n); length() is the oracle. + */ +PG_PROP(underscores_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + n = (int) pg_prop_draw_int(tc, 0, 12); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, " LIKE repeat('_', %d)) = (length(", n); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ") = %d)", n); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A wildcard-free prefix P followed by '%' matches S iff S's first length(P) + * characters equal P. left(S, length(P)) = P is an independent builtin, so + * this cross-checks LIKE's prefix handling. P is drawn letters-only, so it + * contains no wildcard and the only metacharacter is the trailing '%'. + */ +PG_PROP(prefix_percent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + p, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&p); + draw_text(tc, &s); + draw_text(tc, &p); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " LIKE ("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, " || '%')) = (left("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", length("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ")) = "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * An escaped '%' is a literal '%'. With ESCAPE '\', the pattern '\%' matches + * only the one-character string "%". Our text S is drawn over [abc], so S is + * never "%": the left side (S LIKE '\%' ESCAPE '\') is always false, and so + * is the right side (S = '%'); they therefore agree. This pins that ESCAPE + * strips the wildcard meaning of the following '%' rather than matching an + * arbitrary run. + */ +PG_PROP(escape_literal_percent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " LIKE '\\%' ESCAPE '\\') = ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " = '%')"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The empty pattern matches only the empty string: '' LIKE '' is true, and + * for any S, (S LIKE '') equals (S = ''). Pins the base case where the + * pattern consumes nothing. + */ +PG_PROP(empty_pattern_empty_string) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ('' LIKE '') AND (("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " LIKE '') = ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " = ''))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * S always matches any of its own prefixes followed by '%'. The prefix + * length k is drawn in [0, length(S)] and the prefix itself is taken + * server-side with left(), so no C-side substring logic can disagree with the + * backend's. Since S starts with its own first k characters, the match must + * always succeed. + */ +PG_PROP(like_concat_split) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int k; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + k = (int) pg_prop_draw_int(tc, 0, (int) strlen(s.data)); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " LIKE (left("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", %d) || '%%')", k); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index f531983c7db36..9824cbe7552cd 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 3d93b02f9c6f194220015ed04e8cca9143e86ec0 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:58 -0400 Subject: [PATCH 053/338] prop: add LIKE/ILIKE/SIMILAR TO/regex consistency property tests (stringmatch) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_stringmatch.c | 411 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 415 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_stringmatch.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index a649b017b95b3..f2258275d29dc 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -68,7 +68,8 @@ OBJS = \ prop_padtrim.o \ prop_trimfns.o \ prop_like.o \ - prop_likepattern.o + prop_likepattern.o \ + prop_stringmatch.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index b522f6bab58c6..b9a5d79eadff9 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -66,6 +66,7 @@ prop_area_sources = files( 'prop_trimfns.c', 'prop_like.c', 'prop_likepattern.c', + 'prop_stringmatch.c', ) diff --git a/src/test/prop/prop_stringmatch.c b/src/test/prop/prop_stringmatch.c new file mode 100644 index 0000000000000..032634aba8e88 --- /dev/null +++ b/src/test/prop/prop_stringmatch.c @@ -0,0 +1,411 @@ +/*------------------------------------------------------------------------- + * + * prop_stringmatch.c + * Property-based tests for LIKE/ILIKE/SIMILAR TO/~ consistency. + * + * These cross-check the pattern-matching operators against *different* + * oracles than the "like" area does: LIKE prefix/contains are checked against + * position()/left(), ILIKE against lower()+LIKE, ~ (regex) against position() + * for literal letters, and SIMILAR TO against plain equality for metachar-free + * patterns. All text and all patterns are built from the letters-only + * alphabet [abc] (or [abcABC] for the case-folding property), so a % or _ + * never leaks into the operands and every operator sees only the literal + * patterns the property intends. A mismatch means one operator disagrees + * with an independent builtin -- a matching bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_stringmatch.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "stringmatch" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabets never contain a quote, but do it defensively + * so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw a bare string of length [min, max] from a letters-only alphabet. No % + * or _ or any LIKE/SIMILAR/regex metacharacter can appear, so the drawn value + * is always a plain literal both as text and as a pattern. + */ +static void +draw_letters(pg_prop_tc *tc, StringInfo buf, const char *alphabet, + int min, int max) +{ + int span = (int) strlen(alphabet) - 1; + int n = (int) pg_prop_draw_int(tc, min, max); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, alphabet[pg_prop_draw_int(tc, 0, span)]); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the comparison errored or produced NULL on non-NULL + * letter strings, which is itself a bug -- so either is a failure, reported + * with the SQL. Modelled on prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * 1. like_prefix_eq_leftmatch: a literal prefix P followed by '%' matches S + * exactly when the first length(P) characters of S equal P. P is a plain + * [abc] string (length 0..4) with no wildcards, so "P || '%'" is a pure + * prefix pattern; left(S, length(P)) is an independent builtin. + */ +PG_PROP(like_prefix_eq_leftmatch) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + p, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&p); + draw_letters(tc, &s, "abc", 0, 12); + draw_letters(tc, &p, "abc", 0, 4); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " LIKE ("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, " || '%')) = (left("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", length("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ")) = "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. like_contains_eq_position: "'%' || P || '%'" matches S exactly when P + * occurs somewhere in S. P is a non-empty [abc] string (length 1..3) with no + * wildcards, so the surrounding %s make a pure substring pattern; position(P + * in S) > 0 is the independent oracle (position returns 0 when not found). + */ +PG_PROP(like_contains_eq_position) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + p, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&p); + draw_letters(tc, &s, "abc", 0, 12); + draw_letters(tc, &p, "abc", 1, 3); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " LIKE ('%' || "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, " || '%')) = (position("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, " in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") > 0)"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. ilike_eq_lower_like: ILIKE is case-insensitive LIKE, so "S ILIKE P" + * equals "lower(S) LIKE lower(P)". S and P are drawn mixed-case over + * [abcABC] to actually exercise folding; P is a wildcard-free literal, so no + * escaping or metachar subtlety is in play. + */ +PG_PROP(ilike_eq_lower_like) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + p, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&p); + draw_letters(tc, &s, "abcABC", 0, 12); + draw_letters(tc, &p, "abcABC", 0, 12); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " ILIKE "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ") = (lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") LIKE lower("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, "))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. like_literal_eq_equals: with no wildcard characters, LIKE degrades to + * equality, so "S LIKE P" equals "S = P". P is a plain [abc] string + * (length 0..12) with no % or _, so there is nothing for LIKE to match + * loosely. + */ +PG_PROP(like_literal_eq_equals) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + p, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&p); + draw_letters(tc, &s, "abc", 0, 12); + draw_letters(tc, &p, "abc", 0, 12); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " LIKE "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ") = ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " = "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. regex_literal_eq_contains: for a single literal letter c from [abc] + * (none of which is a regex metacharacter), the POSIX regex match "S ~ c" is + * a plain containment test, so it equals "position(c in S) > 0". + */ +PG_PROP(regex_literal_eq_contains) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&c); + draw_letters(tc, &s, "abc", 0, 12); + draw_letters(tc, &c, "abc", 1, 1); /* exactly one literal letter */ + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " ~ "); + append_sql_literal(&q, c.data); + appendStringInfoString(&q, ") = (position("); + append_sql_literal(&q, c.data); + appendStringInfoString(&q, " in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") > 0)"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. similar_to_literal: SIMILAR TO with a pattern that has no SIMILAR + * metacharacters degrades to equality. P is a plain [abc] string -- none of + * a, b, c are SIMILAR metachars (% _ | * + ? { } ( ) [ ] \) -- so + * "S SIMILAR TO P" equals "S = P". + */ +PG_PROP(similar_to_literal) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + p, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&p); + draw_letters(tc, &s, "abc", 0, 12); + draw_letters(tc, &p, "abc", 0, 12); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " SIMILAR TO "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ") = ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " = "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. not_like_complement: NOT LIKE is the boolean complement of LIKE, so + * "S NOT LIKE P" equals "NOT (S LIKE P)" for any pattern. Here P may contain + * wildcards (literal letters, %, or _), since the identity holds whatever P + * is as long as both operands are non-NULL (which letters-only strings + * always are). + */ +PG_PROP(not_like_complement) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + p, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s, "abc", 0, 12); + + /* + * Build a pattern that may contain the wildcards %/_ as well as literal + * letters; only build a % or _ where deliberately chosen so no accidental + * wildcard leaks in. + */ + initStringInfo(&p); + n = (int) pg_prop_draw_int(tc, 0, 8); + for (int i = 0; i < n; i++) + { + int kind = (int) pg_prop_draw_int(tc, 0, 4); + + if (kind == 3) + appendStringInfoChar(&p, '%'); + else if (kind == 4) + appendStringInfoChar(&p, '_'); + else + appendStringInfoChar(&p, "abc"[kind]); + } + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " NOT LIKE "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ") = (NOT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " LIKE "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, "))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 9824cbe7552cd..078e077b7f794 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 097de35f191533e1e4e24b0370cd22534b3d3757 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:58 -0400 Subject: [PATCH 054/338] prop: add regexp property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_regexp.c | 353 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 357 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_regexp.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index f2258275d29dc..511c602561b4e 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -69,7 +69,8 @@ OBJS = \ prop_trimfns.o \ prop_like.o \ prop_likepattern.o \ - prop_stringmatch.o + prop_stringmatch.o \ + prop_regexp.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index b9a5d79eadff9..d7508031d5db3 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -67,6 +67,7 @@ prop_area_sources = files( 'prop_like.c', 'prop_likepattern.c', 'prop_stringmatch.c', + 'prop_regexp.c', ) diff --git a/src/test/prop/prop_regexp.c b/src/test/prop/prop_regexp.c new file mode 100644 index 0000000000000..59a40f94f332d --- /dev/null +++ b/src/test/prop/prop_regexp.c @@ -0,0 +1,353 @@ +/*------------------------------------------------------------------------- + * + * prop_regexp.c + * Property-based tests for regular-expression functions. + * + * The regexp functions (src/backend/utils/adt/regexp.c) -- + * regexp_replace, regexp_count, regexp_split_to_array, + * regexp_split_to_table, and the ~ match operator -- are checked here + * against independent string builtins so the oracle is never the function + * under test. For a single-literal-char or single-delimiter pattern the + * regexp semantics coincide exactly with replace()/position()/length()/ + * string_to_array(), which gives sound, tolerance-free equalities. + * + * Every generated string and pattern is letters-only ("abc"/"ab") plus the + * fixed literal delimiters 'x' and 'z', so no regex metacharacter is ever + * produced: the patterns are plain literals and the string-builtin oracles + * are exact. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_regexp.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "regexp" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabets never contain a quote, but do it defensively + * so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..12 characters of plain text from a 3-letter alphabet "abc" into buf + * as a bare string (no quoting). Letters only -- never a regex metacharacter + * -- so the text cannot accidentally contain a special regex token and the + * literal-pattern oracles stay exact. + */ +static void +draw_text(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abc"; + int n = (int) pg_prop_draw_int(tc, 0, 12); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 2)]); +} + +/* + * Draw a single literal char from "abc" into *c as a one-character bare + * string, usable both as a one-char text and as a one-char literal pattern. + */ +static void +draw_char(pg_prop_tc *tc, char *c) +{ + static const char letters[] = "abc"; + + c[0] = letters[pg_prop_draw_int(tc, 0, 2)]; + c[1] = '\0'; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on non-NULL letter + * strings, which is itself a bug -- so either is a failure, reported with the + * SQL. Modelled on prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Global replace of a single literal char removes every occurrence and agrees + * with replace(). For a literal single-char pattern c, regexp_replace(S, c, + * '', 'g') must equal replace(S, c, '') -- replace() is the independent + * oracle for a literal pattern -- which in turn contains no c. + */ +PG_PROP(replace_g_removes_all_matches) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + char c[2]; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + draw_char(tc, c); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT regexp_replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, c); + appendStringInfoString(&q, ", '', 'g') = replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, c); + appendStringInfoString(&q, ", '')"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The ~ match operator with a single-literal-char pattern matches iff the + * char occurs. (S ~ c) must equal (position(c in S) > 0), with position() + * the independent occurrence oracle for a literal char. + */ +PG_PROP(match_consistent_with_position) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + char c[2]; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + draw_char(tc, c); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " ~ "); + append_sql_literal(&q, c); + appendStringInfoString(&q, ") = (position("); + append_sql_literal(&q, c); + appendStringInfoString(&q, " in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") > 0)"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Splitting on the empty pattern yields individual characters, and rejoining + * them reconstructs the original. array_to_string(regexp_split_to_array(S, + * ''), '') must equal S. + */ +PG_PROP(split_to_array_rejoin) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT array_to_string(regexp_split_to_array("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", ''), '') = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Splitting on a literal delimiter produces as many pieces as the delimiter + * induces. The number of rows from regexp_split_to_table(S, 'x') must equal + * cardinality(string_to_array(S, 'x')) -- string_to_array is the independent + * oracle for literal-delimiter splitting. S is built as "ab" tokens joined + * by 'x', so 'x' occurs only as a delimiter. + */ +PG_PROP(split_count_matches_delimiter) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + + /* + * Build a guaranteed non-empty 'x'-joined string. On the empty string + * regexp_split_to_table yields one empty row while string_to_array yields + * an empty array, a documented difference that is not under test here; a + * non-empty string makes the two agree. + */ + { + static const char letters[] = "ab"; + int ntok = (int) pg_prop_draw_int(tc, 1, 6); + + for (int i = 0; i < ntok; i++) + { + int tlen = (int) pg_prop_draw_int(tc, 1, 3); + + if (i > 0) + appendStringInfoChar(&s, 'x'); + for (int j = 0; j < tlen; j++) + appendStringInfoChar(&s, letters[pg_prop_draw_int(tc, 0, 1)]); + } + } + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT (SELECT count(*) FROM regexp_split_to_table("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'x')) = cardinality(string_to_array("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'x'))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Replacing a pattern that cannot occur leaves S unchanged. 'z' is not in the + * "abc" alphabet S is drawn from, so regexp_replace(S, 'z', 'Q', 'g') must + * equal S. + */ +PG_PROP(replace_identity_on_nonmatch) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT regexp_replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'z', 'Q', 'g') = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * regexp_count of a single literal char equals the number of occurrences, + * oracled by the length difference after removing that char: each removed + * single-char match shortens S by exactly 1, so length(S) - length(replace(S, + * c, '')) is the exact occurrence count. + */ +PG_PROP(count_matches_occurrences) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + char c[2]; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + draw_char(tc, c); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT regexp_count("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, c); + appendStringInfoString(&q, ") = (length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") - length(replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, c); + appendStringInfoString(&q, ", '')))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 078e077b7f794..87ab51b89bacc 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From dbaca97c3d6fd1621eb89520ac46fdc86c704800 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:59 -0400 Subject: [PATCH 055/338] prop: add regexp replace, count, and split round-trips property tests (regexfns) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_regexfns.c | 345 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 349 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_regexfns.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 511c602561b4e..07b2469cfb180 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -70,7 +70,8 @@ OBJS = \ prop_like.o \ prop_likepattern.o \ prop_stringmatch.o \ - prop_regexp.o + prop_regexp.o \ + prop_regexfns.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index d7508031d5db3..962001044f620 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -68,6 +68,7 @@ prop_area_sources = files( 'prop_likepattern.c', 'prop_stringmatch.c', 'prop_regexp.c', + 'prop_regexfns.c', ) diff --git a/src/test/prop/prop_regexfns.c b/src/test/prop/prop_regexfns.c new file mode 100644 index 0000000000000..214a9d4498a38 --- /dev/null +++ b/src/test/prop/prop_regexfns.c @@ -0,0 +1,345 @@ +/*------------------------------------------------------------------------- + * + * prop_regexfns.c + * Property-based tests for the regexp text-processing functions. + * + * Exercises regexp_replace, regexp_match, regexp_split_to_array and + * regexp_count (src/backend/utils/adt/regexp.c) against independent string + * builtins (replace/length/position/array_to_string). Strings are letters- + * only over "abc" and every pattern/delimiter is a single literal letter + * ('a' or 'z'), so no regex metacharacter is ever produced and the literal- + * pattern semantics coincide exactly with the string-builtin oracles. A "f" + * means a regexp function disagrees with its string-builtin oracle, which is + * a bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_regexfns.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "regexfns" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabets never contain a quote, but do it defensively + * so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..12 characters of plain text from the 3-letter alphabet "abc" into + * buf as a bare string (no quoting). Letters only -- never a regex + * metacharacter -- so the literal-pattern oracles stay exact. + */ +static void +draw_text(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abc"; + int n = (int) pg_prop_draw_int(tc, 0, 12); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 2)]); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on non-NULL letter + * strings, which is itself a bug -- so either is a failure, reported with the + * SQL. Modelled on prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Property 1 -- split_to_array_rejoin. + * + * The delimiter 'x' is not in the "abc" alphabet, so it never occurs in S: + * regexp_split_to_array(S, 'x') cannot split and must yield the single- + * element array ARRAY[S]. Rejoining with any separator therefore reproduces + * S exactly, so array_to_string(regexp_split_to_array(S, 'x'), 'x') = S. + */ +PG_PROP(split_to_array_rejoin) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT array_to_string(regexp_split_to_array("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'x'), 'x') = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 2 -- split_count_plus_one. + * + * Splitting S on the literal delimiter 'a' produces (occurrences of 'a') + 1 + * pieces. The occurrence count is oracled by the length difference after + * removing 'a' (each single-char removal shortens S by one), so + * cardinality(regexp_split_to_array(S, 'a')) + * = (length(S) - length(replace(S, 'a', ''))) + 1. + */ +PG_PROP(split_count_plus_one) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT cardinality(regexp_split_to_array("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a')) = ((length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") - length(replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a', ''))) + 1)"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 3 -- replace_global_removes. + * + * Global replace of the literal 'a' with the empty string removes every 'a', + * so the result contains no 'a': position('a' in regexp_replace(S, 'a', '', + * 'g')) = 0. + */ +PG_PROP(replace_global_removes) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT position('a' in regexp_replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a', '', 'g')) = 0"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 4 -- replace_first_only. + * + * Without the 'g' flag, regexp_replace(S, 'a', 'X') replaces at most the + * first 'a'. Using the empty replacement so the length oracle is clean: + * removing exactly the first 'a' shortens S by one when S had an 'a', and + * leaves it unchanged otherwise. So + * length(regexp_replace(S, 'a', '')) + * = length(S) - (CASE WHEN position('a' in S) > 0 THEN 1 ELSE 0 END). + */ +PG_PROP(replace_first_only) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT length(regexp_replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a', '')) = (length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") - (CASE WHEN position('a' in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") > 0 THEN 1 ELSE 0 END))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 5 -- regexp_count_matches. + * + * regexp_count of the literal 'a' equals its number of occurrences, oracled + * by the length difference after removing 'a' (each single-char removal + * shortens S by one): + * regexp_count(S, 'a') = length(S) - length(replace(S, 'a', '')). + */ +PG_PROP(regexp_count_matches) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT regexp_count("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a') = (length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") - length(replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a', '')))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 6 -- match_returns_null_when_absent. + * + * 'z' is not in the "abc" alphabet S is drawn from, so there is no match and + * regexp_match(S, 'z') must be SQL NULL: regexp_match(S, 'z') IS NULL. + */ +PG_PROP(match_returns_null_when_absent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT regexp_match("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'z') IS NULL"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 7 -- replace_identity_when_absent. + * + * Replacing a pattern that cannot occur leaves S unchanged. 'z' is not in the + * "abc" alphabet S is drawn from, so regexp_replace(S, 'z', 'Q', 'g') = S. + */ +PG_PROP(replace_identity_when_absent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT regexp_replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'z', 'Q', 'g') = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 87ab51b89bacc..d84c95965a1e1 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 82fece44b3e7bd97691ef2439728b831e0151fe0 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:42:59 -0400 Subject: [PATCH 056/338] prop: add POSIX regex match operators and anchors property tests (regexmatch) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_regexmatch.c | 409 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 413 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_regexmatch.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 07b2469cfb180..543a7d0e1f976 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -71,7 +71,8 @@ OBJS = \ prop_likepattern.o \ prop_stringmatch.o \ prop_regexp.o \ - prop_regexfns.o + prop_regexfns.o \ + prop_regexmatch.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 962001044f620..8c5362dda6e5c 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -69,6 +69,7 @@ prop_area_sources = files( 'prop_stringmatch.c', 'prop_regexp.c', 'prop_regexfns.c', + 'prop_regexmatch.c', ) diff --git a/src/test/prop/prop_regexmatch.c b/src/test/prop/prop_regexmatch.c new file mode 100644 index 0000000000000..e6f863c9955b1 --- /dev/null +++ b/src/test/prop/prop_regexmatch.c @@ -0,0 +1,409 @@ +/*------------------------------------------------------------------------- + * + * prop_regexmatch.c + * Property-based tests for POSIX regex match operators and regexp_matches. + * + * The POSIX regex match operators (~, ~*, !~, !~*) and regexp_matches() are + * checked against independent builtins (lower/left/right/position) and + * against each other, using single-letter patterns and the simple anchors ^, + * $, . only. All text and all literal patterns come from a letters-only + * alphabet, so no regex metacharacter leaks in except the ones a property + * deliberately builds; a mismatch means a match operator or regexp_matches + * disagrees with an independent oracle -- a regex-matching bug. Distinct from + * the regexp/regexfns/stringmatch areas, which check different oracles. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_regexmatch.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "regexmatch" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabets never contain a quote, but do it defensively + * so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw a bare string of length [min, max] from a letters-only alphabet. No + * regex metacharacter can appear, so the drawn value is always plain literal + * text (and, when used as a pattern, a literal pattern). + */ +static void +draw_letters(pg_prop_tc *tc, StringInfo buf, const char *alphabet, + int min, int max) +{ + int span = (int) strlen(alphabet) - 1; + int n = (int) pg_prop_draw_int(tc, min, max); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, alphabet[pg_prop_draw_int(tc, 0, span)]); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the match errored or produced NULL on non-NULL letter + * strings, which is itself a bug -- so either is a failure, reported with the + * SQL. Modelled on prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * 1. match_case_insensitive: the case-insensitive match "S ~* P" equals the + * case-sensitive match of the folded operands, "lower(S) ~ lower(P)". S is + * drawn mixed-case over [a-cA-C] to exercise folding; P is a single literal + * letter (also mixed-case), which is not a regex metacharacter, so both sides + * perform a plain case-folded containment. + */ +PG_PROP(match_case_insensitive) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + p, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&p); + draw_letters(tc, &s, "abcABC", 0, 12); + draw_letters(tc, &p, "abcABC", 1, 1); /* one literal letter */ + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " ~* "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ") = (lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") ~ lower("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, "))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. not_match_complement: "S !~ P" is the boolean complement of "S ~ P", so + * it equals "NOT (S ~ P)" for any S and P (both non-NULL letter strings). P + * is a single letter here, but the identity holds for any pattern; a mismatch + * means the negated operator disagrees with its own base operator. + */ +PG_PROP(not_match_complement) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + p, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&p); + draw_letters(tc, &s, "abc", 0, 12); + draw_letters(tc, &p, "abc", 1, 1); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " !~ "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, ") = (NOT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " ~ "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, "))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. anchored_start: a start anchor followed by one literal letter c, pattern + * '^c', matches S exactly when S begins with c, i.e. left(S, 1) = c. c is a + * single [abc] letter (no metachar), so '^' || c is a pure "starts-with-c" + * pattern; left(S, 1) is the independent oracle (and equals '' for empty S, + * never matching c). + */ +PG_PROP(anchored_start) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&c); + draw_letters(tc, &s, "abc", 0, 12); + draw_letters(tc, &c, "abc", 1, 1); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " ~ ('^' || "); + append_sql_literal(&q, c.data); + appendStringInfoString(&q, ")) = (left("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 1) = "); + append_sql_literal(&q, c.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. anchored_end: a single literal letter c followed by an end anchor, + * pattern 'c$', matches S exactly when S ends with c, i.e. right(S, 1) = c. + * Guarded to non-empty S: for empty S, right(S,1) = '' can never equal c, and + * 'c$' never matches, so the identity still holds -- but we keep the explicit + * guard so the oracle is obviously sound even on the empty case. + */ +PG_PROP(anchored_end) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&c); + draw_letters(tc, &s, "abc", 1, 12); /* non-empty S */ + draw_letters(tc, &c, "abc", 1, 1); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT CASE WHEN length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = 0 THEN true ELSE ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " ~ ("); + append_sql_literal(&q, c.data); + appendStringInfoString(&q, " || '$')) = (right("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 1) = "); + append_sql_literal(&q, c.data); + appendStringInfoString(&q, ") END"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. dot_matches_any_single: for a non-empty S, the pattern '.' (dot matches + * any single character) always finds a match, so "S ~ '.'" is true. S is + * drawn non-empty over [abc]; '.' is the only metachar here and it is built + * directly, so there is nothing else to interpret. + */ +PG_PROP(dot_matches_any_single) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s, "abc", 1, 12); /* non-empty S */ + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " ~ '.')"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. empty_pattern_matches: the empty regex matches every string, so + * "S ~ ''" is true for any S (including empty S). S is drawn over [abc], + * length 0..12. + */ +PG_PROP(empty_pattern_matches) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s, "abc", 0, 12); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " ~ '')"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. literal_contains: for a single literal letter c (not a regex metachar), + * the match "S ~ c" is a plain containment test, so it equals + * "position(c in S) > 0" (position returns 0 when c is absent). + */ +PG_PROP(literal_contains) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&c); + draw_letters(tc, &s, "abc", 0, 12); + draw_letters(tc, &c, "abc", 1, 1); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " ~ "); + append_sql_literal(&q, c.data); + appendStringInfoString(&q, ") = (position("); + append_sql_literal(&q, c.data); + appendStringInfoString(&q, " in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") > 0)"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 8. matches_returns_group: regexp_matches(S, '(a)') returns the matched + * group text when 'a' occurs in S. Fixing c = 'a' keeps the pattern a plain + * single-group literal. When position('a' in S) > 0 the captured group must + * be exactly 'a'; when 'a' is absent, regexp_matches yields no row (the + * scalar subquery is NULL) and the CASE short-circuits to true, so the oracle + * is sound on both branches. + */ +PG_PROP(matches_returns_group) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s, "abc", 0, 12); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT CASE WHEN position('a' in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") > 0 THEN (SELECT (regexp_matches("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", '(a)'))[1]) = 'a' ELSE true END"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index d84c95965a1e1..188e688d4a229 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 77c7a2438d3fbe0ca2cd13ff81e5d957652b3c2f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:00 -0400 Subject: [PATCH 057/338] prop: add regexp_replace backreferences and flags property tests (regexpreplace) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_regexpreplace.c | 413 +++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 417 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_regexpreplace.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 543a7d0e1f976..da585cdc73d13 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -72,7 +72,8 @@ OBJS = \ prop_stringmatch.o \ prop_regexp.o \ prop_regexfns.o \ - prop_regexmatch.o + prop_regexmatch.o \ + prop_regexpreplace.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 8c5362dda6e5c..164440dbeeb66 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -70,6 +70,7 @@ prop_area_sources = files( 'prop_regexp.c', 'prop_regexfns.c', 'prop_regexmatch.c', + 'prop_regexpreplace.c', ) diff --git a/src/test/prop/prop_regexpreplace.c b/src/test/prop/prop_regexpreplace.c new file mode 100644 index 0000000000000..4f5c09ab7ec0d --- /dev/null +++ b/src/test/prop/prop_regexpreplace.c @@ -0,0 +1,413 @@ +/*------------------------------------------------------------------------- + * + * prop_regexpreplace.c + * Property-based tests for regexp_replace() with backreferences, flags + * and capture groups. + * + * Exercises the substitution machinery of regexp_replace + * (src/backend/utils/adt/regexp.c): the 'g' global flag, the 'i' + * case-insensitive flag, anchors (^ and $), the whole-string ".*" match and + * the backreference substitution path (\1, \2) over capture groups. Each + * property is a scalar boolean oracle built from independent string builtins + * (replace/length/left/substr) or a hand-computed constant, so a "f" means + * the replacement engine disagrees with an oracle that does not use regexes, + * which is a bug. Strings are letters-only over "abc" (length 0..12) so no + * drawn character is ever a regex metacharacter. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_regexpreplace.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "regexpreplace" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabet never contains a quote, but do it defensively + * so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..12 characters of plain text from the 3-letter alphabet "abc" into + * buf as a bare string (no quoting). Letters only -- never a regex + * metacharacter -- so the literal/anchored-pattern oracles stay exact. + */ +static void +draw_text(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abc"; + int n = (int) pg_prop_draw_int(tc, 0, 12); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 2)]); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on non-NULL letter + * strings, which is itself a bug -- so either is a failure, reported with the + * SQL. Modelled on prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Property 1 -- replace_all_g. + * + * The 'g' flag replaces every occurrence, so regexp_replace(S, 'a', 'X', 'g') + * turns each 'a' into one 'X'. The number of 'X' in the result therefore + * equals the number of 'a' in S, oracled without regexes by the length + * difference after removing each character with replace(): + * length(R) - length(replace(R, 'X', '')) = length(S) - length(replace(S, 'a', '')) + * where R = regexp_replace(S, 'a', 'X', 'g'). (S has no 'X', so counting 'X' + * in R counts exactly the substitutions.) + */ +PG_PROP(replace_all_g) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT (length(regexp_replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a', 'X', 'g')) - length(replace(regexp_replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a', 'X', 'g'), 'X', ''))) = (length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") - length(replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a', '')))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 2 -- replace_first_only. + * + * Without 'g', regexp_replace(S, 'a', 'X') replaces at most one 'a'. Using + * the empty replacement so the length oracle is clean: removing exactly the + * first 'a' shortens S by one when S had an 'a', and leaves it unchanged + * otherwise. So the 'a' count decreases by 1 iff S contained an 'a': + * count_a(regexp_replace(S, 'a', '')) = count_a(S) - (1 if S has 'a' else 0) + * where count_a(T) = length(T) - length(replace(T, 'a', '')). + */ +PG_PROP(replace_first_only) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT (length(regexp_replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a', '')) - length(replace(regexp_replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a', ''), 'a', ''))) = ((length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") - length(replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'a', ''))) - (CASE WHEN position('a' in "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") > 0 THEN 1 ELSE 0 END))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 3 -- backreference. + * + * Two capture groups (a)(b) with the replacement E'\2\1' swap the matched + * fragments. The constant case regexp_replace('ab', '(a)(b)', E'\2\1') = 'ba' + * is checked alongside a drawn 2-character string c1||c2 from "abc": + * regexp_replace(c1||c2, '(.)(.)', E'\2\1') = c2||c1. '(.)(.)' matches any + * two characters and the backreferences reverse them, oracled by the literal + * concatenation c2||c1. (In the C source each backslash is doubled so the + * SQL text sees E'\2\1'.) + */ +PG_PROP(backreference) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData c1, + c2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* draw exactly two characters from "abc" */ + initStringInfo(&c1); + initStringInfo(&c2); + { + static const char letters[] = "abc"; + + appendStringInfoChar(&c1, letters[pg_prop_draw_int(tc, 0, 2)]); + appendStringInfoChar(&c2, letters[pg_prop_draw_int(tc, 0, 2)]); + } + + initStringInfo(&q); + /* constant case: literal (a)(b) swap */ + appendStringInfoString(&q, "SELECT regexp_replace('ab', '(a)(b)', E'\\\\2\\\\1') = 'ba'"); + /* drawn case: (.)(.) swaps the two drawn characters */ + appendStringInfoString(&q, " AND regexp_replace("); + append_sql_literal(&q, c1.data); + appendStringInfoString(&q, " || "); + append_sql_literal(&q, c2.data); + appendStringInfoString(&q, ", '(.)(.)', E'\\\\2\\\\1') = ("); + append_sql_literal(&q, c2.data); + appendStringInfoString(&q, " || "); + append_sql_literal(&q, c1.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(c1.data); + pfree(c2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 4 -- case_insensitive_flag. + * + * The 'i' flag makes the match case-insensitive, so the lowercase pattern + * 'abc' matches the uppercase subject 'ABC': + * regexp_replace('ABC', 'abc', 'X', 'i') = 'X'. + * A constant check -- no draw needed -- that the 'i' flag reaches the matcher. + */ +PG_PROP(case_insensitive_flag) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT regexp_replace('ABC', 'abc', 'X', 'i') = 'X'"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 5 -- replace_empty_match. + * + * 'z' is not in the "abc" alphabet S is drawn from, so the pattern never + * matches and a global replace leaves S unchanged: + * regexp_replace(S, 'z', 'X', 'g') = S. + */ +PG_PROP(replace_empty_match) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT regexp_replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'z', 'X', 'g') = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 6 -- replace_whole. + * + * The anchored pattern '^.*$' matches the entire string in one go, so without + * 'g' a non-empty S is replaced wholesale by the replacement: + * regexp_replace(S, '^.*$', 'Y') = 'Y' for non-empty S. + * For an empty S the single empty match still yields 'Y', so the identity + * holds for every drawn S; the CASE is left in only to document the + * non-empty reasoning while remaining true at length 0. + */ +PG_PROP(replace_whole) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT regexp_replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", '^.*$', 'Y') = 'Y'"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 7 -- anchored_replace. + * + * The '^' anchor restricts the match to the start of the string, so + * regexp_replace(S, '^a', 'X') rewrites 'a' only when it is the first + * character. Oracled by the string builtins left()/substr(): + * = CASE WHEN left(S,1) = 'a' THEN 'X' || substr(S,2) ELSE S END. + * (substr(S,2) is the empty string when S is 'a' alone, so the single-char + * and length-0 cases both come out right.) + */ +PG_PROP(anchored_replace) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT regexp_replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", '^a', 'X') = (CASE WHEN left("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 1) = 'a' THEN 'X' || substr("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 2) ELSE "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " END)"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 8 -- identity_no_match. + * + * The pattern 'zzz' uses only characters outside the "abc" alphabet, so it + * never matches and a global replace is the identity: + * regexp_replace(S, 'zzz', 'Q', 'g') = S. + */ +PG_PROP(identity_no_match) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT regexp_replace("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 'zzz', 'Q', 'g') = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 188e688d4a229..de59e96fd4557 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From cc17969c6a31e1ae0042852f3ff42e1f0d68c52f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:00 -0400 Subject: [PATCH 058/338] prop: add char property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_char.c | 375 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 379 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_char.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index da585cdc73d13..7d5ec2716ca16 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -73,7 +73,8 @@ OBJS = \ prop_regexp.o \ prop_regexfns.o \ prop_regexmatch.o \ - prop_regexpreplace.o + prop_regexpreplace.o \ + prop_char.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 164440dbeeb66..f894a596e63ba 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -71,6 +71,7 @@ prop_area_sources = files( 'prop_regexfns.c', 'prop_regexmatch.c', 'prop_regexpreplace.c', + 'prop_char.c', ) diff --git a/src/test/prop/prop_char.c b/src/test/prop/prop_char.c new file mode 100644 index 0000000000000..4494ff0d0589a --- /dev/null +++ b/src/test/prop/prop_char.c @@ -0,0 +1,375 @@ +/*------------------------------------------------------------------------- + * + * prop_char.c + * Property-based tests for char(n)/varchar(n) length semantics. + * + * bpchar (char(n)) blank-pads and blank-trims on comparison; varchar(n) + * rejects over-length input; these check the documented padding/trimming and + * length-limit semantics (src/backend/utils/adt/varchar.c). + * + * The oracle is independent builtins -- length(), char_length(), rtrim() -- + * together with the type's own cast and comparison rules. Two subtleties + * drive the queries here: + * + * - length()/char_length() on a bpchar IGNORE trailing blanks and return + * the trimmed length, so padding is observed by casting the bpchar to + * text first ((x::char(n))::text keeps the n-width padding) and taking + * the length of that. + * + * - an EXPLICIT cast '...'::varchar(n) or '...'::char(n) TRUNCATES silently + * (no error); the length-violation error path fires only on assignment / + * INSERT. So over-length explicit casts are checked for truncation to n, + * not for an error. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_char.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "char" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabets never contain a quote, but do it defensively + * so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..maxlen characters from the alphabet "ab " (two letters plus a + * space) into buf as a bare string (no quoting). Including the space means + * generated strings routinely carry trailing blanks, which is exactly what + * the bpchar blank-pad / blank-trim semantics must handle. + */ +static void +draw_str(pg_prop_tc *tc, StringInfo buf, int maxlen) +{ + static const char alphabet[] = "ab "; + int n = (int) pg_prop_draw_int(tc, 0, maxlen); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, alphabet[pg_prop_draw_int(tc, 0, 2)]); +} + +/* + * Draw a run of 0..maxlen non-space characters from "ab" into buf. Used for + * over-length strings whose truncation must be observable: a trailing space + * that got dropped by truncation would be indistinguishable from blank + * trimming, so these strings never contain a space. + */ +static void +draw_str_nonspace(pg_prop_tc *tc, StringInfo buf, int maxlen) +{ + static const char letters[] = "ab"; + int n = (int) pg_prop_draw_int(tc, 0, maxlen); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 1)]); +} + +/* + * Draw a declared length n in [1, 20]. + */ +static int +draw_len(pg_prop_tc *tc) +{ + return (int) pg_prop_draw_int(tc, 1, 20); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the query errored or produced NULL on the generated + * inputs, which is itself a bug -- so either is a failure, reported with the + * SQL. Modelled on prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * char(n) blank-pads its value to exactly n characters. length() on the + * bpchar itself would ignore the trailing blanks, so cast to text first -- + * ((S::char(n))::text) preserves the n-width padding -- and require its + * length to be n. S is drawn no longer than n so no truncation is involved. + */ +PG_PROP(bpchar_blank_pads_to_n) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = draw_len(tc); + + initStringInfo(&s); + draw_str(tc, &s, n); /* length(S) <= n */ + + initStringInfo(&q); + + /* + * char(n) pads its value to n characters of STORAGE. That padding is only + * observable via octet_length (the raw stored bytes): length(), + * char_length(), a ::text cast, and even || all trim trailing blanks by + * bpchar's SQL semantics. S is drawn from a single-byte ASCII alphabet, + * so n padded characters are exactly n bytes. + */ + appendStringInfoString(&q, "SELECT octet_length("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, "::char(%d)) = %d", n, n); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * char(n) comparison ignores trailing blanks: S and S with three extra + * trailing spaces compare equal as char(n). S is kept short enough that both + * operands fit in n (length(S) + 3 <= n), so the extra spaces are genuine + * trailing blanks rather than truncated padding. This is the documented + * bpchar blank-insensitive comparison. + */ +PG_PROP(bpchar_trailing_blanks_ignored_in_compare) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* need room for S plus three trailing spaces within n */ + n = (int) pg_prop_draw_int(tc, 3, 20); + + initStringInfo(&s); + draw_str(tc, &s, n - 3); /* length(S) <= n - 3 */ + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, "::char(%d)) = (", n); + /* same string with three extra trailing spaces */ + appendStringInfoChar(&q, '\''); + for (const char *p = s.data; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(&q, '\''); + appendStringInfoChar(&q, *p); + } + appendStringInfoString(&q, " '"); + appendStringInfo(&q, "::char(%d))", n); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * varchar within its limit stores the string unchanged -- no padding, no + * trimming. S is drawn no longer than n, so (S::varchar(n))::text must equal + * S exactly. + */ +PG_PROP(varchar_roundtrip_within_limit) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = draw_len(tc); + + initStringInfo(&s); + draw_str(tc, &s, n); /* length(S) <= n */ + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, "::varchar(%d))::text = ", n); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * An explicit cast of an over-length string to varchar(n) TRUNCATES silently + * to n characters (it does NOT error -- the error path is only on + * assignment/INSERT). S is drawn strictly longer than n and from non-space + * letters so the truncation is observable (a dropped trailing blank would be + * ambiguous with trimming). length() on the varchar is its real length, so + * require it to equal n. + */ +PG_PROP(varchar_explicit_cast_truncates) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int n; + int extra; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = draw_len(tc); + + /* S strictly longer than n: length in [n+1, n+5] */ + extra = (int) pg_prop_draw_int(tc, 1, 5); + + initStringInfo(&s); + draw_str_nonspace(tc, &s, n + extra); + + /* + * draw_str_nonspace draws 0..n+extra; force the length strictly above n + * by padding with 'a' if the draw came up short. + */ + while ((int) strlen(s.data) <= n) + appendStringInfoChar(&s, 'a'); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT length("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, "::varchar(%d)) = %d", n, n); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Casting S to varchar(n) and then to char(n) equals casting S directly to + * char(n), for S fitting in n. bpchar's padding is reached by the same final + * value regardless of the intermediate varchar hop. + */ +PG_PROP(bpchar_cast_from_varchar_consistent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = draw_len(tc); + + initStringInfo(&s); + draw_str(tc, &s, n); /* length(S) <= n */ + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT (("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, "::varchar(%d))::char(%d)) = (", n, n); + append_sql_literal(&q, s.data); + appendStringInfo(&q, "::char(%d))", n); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Trailing blanks do not affect a char(n) value: S::char(n) equals + * rtrim(S)::char(n), for S fitting in n. rtrim() is an independent builtin, + * so this cross-checks that bpchar's stored value is blank-canonical. + */ +PG_PROP(bpchar_rtrim_equivalence) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = draw_len(tc); + + initStringInfo(&s); + draw_str(tc, &s, n); /* length(S) <= n */ + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, "::char(%d)) = (rtrim(", n); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ")::char(%d))", n); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index de59e96fd4557..ef2d15266cb94 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 54a719732d8817bfdfe39c24f045631005c2687f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:00 -0400 Subject: [PATCH 059/338] prop: add char/varchar/text/name interconversion property tests (charconv) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_charconv.c | 319 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 323 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_charconv.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 7d5ec2716ca16..6eb43b479ae4e 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -74,7 +74,8 @@ OBJS = \ prop_regexfns.o \ prop_regexmatch.o \ prop_regexpreplace.o \ - prop_char.o + prop_char.o \ + prop_charconv.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index f894a596e63ba..86bc4caf3a9b6 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -72,6 +72,7 @@ prop_area_sources = files( 'prop_regexmatch.c', 'prop_regexpreplace.c', 'prop_char.c', + 'prop_charconv.c', ) diff --git a/src/test/prop/prop_charconv.c b/src/test/prop/prop_charconv.c new file mode 100644 index 0000000000000..f80e2847f6af5 --- /dev/null +++ b/src/test/prop/prop_charconv.c @@ -0,0 +1,319 @@ +/*------------------------------------------------------------------------- + * + * prop_charconv.c + * Property-based tests for character-type interconversion. + * + * Casts among text, varchar, bpchar (char(n)) and name must preserve string + * content within each type's capacity: name holds up to NAMEDATALEN-1 (63) + * bytes, varchar with no length limit behaves as text, char(n) blank-pads to + * n and strips that padding on the way back to text, and varchar(n) truncates + * to n characters. The oracle is the identity/round-trip itself: a cast + * chain that should return the original string but does not is a bug in the + * type I/O or the cast implementation, not a rounding artefact. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_charconv.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "charconv" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a string over the alphabet [abc] of length in [min_len, max_len] and + * append it as a quoted SQL literal to buf. The alphabet is letters only, so + * no quote-doubling is needed; the trailing-space-free guarantee comes for + * free since the alphabet contains no spaces, which keeps the char(n) padding + * properties sound (strip of blank padding recovers the original exactly). + */ +static void +draw_abc_literal(pg_prop_tc *tc, StringInfo buf, int min_len, int max_len) +{ + static const char alpha[] = "abc"; + size_t len = 0; + char *s; + + s = pg_prop_draw_from(tc, alpha, (size_t) min_len, (size_t) max_len, &len); + + appendStringInfoChar(buf, '\''); + appendBinaryStringInfo(buf, s, (int) len); + appendStringInfoChar(buf, '\''); + + pg_free(s); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the cast chain errored or returned NULL on finite inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * text -> name -> text round-trips. A [abc] string of length <= 10 is well + * within NAMEDATALEN-1 (63), so name stores it whole and casting back to text + * recovers the original: S::name::text = S. + */ +PG_PROP(text_name_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc_literal(tc, &s, 0, 10); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s::name::text) = %s", s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * varchar with no length limit is content-identical to text, so a round-trip + * through it changes nothing: S::varchar::text = S. + */ +PG_PROP(varchar_unlimited_eq_text) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc_literal(tc, &s, 0, 10); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s::varchar::text) = %s", s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * char(n) blank-pads to n characters; casting back to text strips that + * trailing blank padding. With a [abc] string (no trailing spaces of its + * own) of length <= 10 stored in char(20), the strip recovers S exactly: + * (S::char(20))::text = S. + */ +PG_PROP(char_strips_on_text) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc_literal(tc, &s, 0, 10); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s::char(20))::text) = %s", s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * name truncates to NAMEDATALEN-1 (63) bytes. A string of 70 'a's cast to + * name must therefore have length exactly 63. This property takes no drawn + * input; it is a fixed boundary check that fails if NAMEDATALEN or the name + * truncation rule changes. + */ +PG_PROP(name_truncates_at_63) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, "SELECT length(repeat('a', 70)::name) = 63"); + + pg_prop_disconnect(conn); +} + +/* + * Casting text to text is the identity: S::text = S. A trivial but sound + * anchor -- it fails only if text input/output stops being faithful. + */ +PG_PROP(text_text_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc_literal(tc, &s, 0, 10); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s::text) = %s", s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * varchar(n) truncates to n characters. For S of length >= 3, + * (S::varchar(3))::text keeps exactly the first 3 characters, which is + * left(S, 3). S is drawn length 3..10 so the truncation is always exercised. + */ +PG_PROP(varchar_n_text_prefix) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc_literal(tc, &s, 3, 10); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s::varchar(3))::text) = left(%s, 3)", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * name equality agrees with text equality. name uses C-collation byte + * ordering, so for two short [abc] strings (both well within 63 bytes and + * free of trailing spaces) equality of their name casts matches equality of + * the strings themselves: (S::name = T::name) = (S = T). Stated as an + * equality rather than ordering so it is sound regardless of collation. + */ +PG_PROP(name_comparison) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + t, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&t); + draw_abc_literal(tc, &s, 0, 10); + draw_abc_literal(tc, &t, 0, 10); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s::name = %s::name) = (%s = %s)", + s.data, t.data, s.data, t.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A longer cast chain exercising several conversions at once: char(10) pads S + * to 10, ::text strips the padding back to S, ::name stores it whole (<= 63 + * bytes), and a final ::text recovers S. With no trailing spaces in S the + * whole chain is the identity: ((S::char(10))::text)::name::text = S. S is + * drawn length 0..10 so char(10) never truncates. + */ +PG_PROP(char_name_text_chain) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc_literal(tc, &s, 0, 10); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((((%s::char(10))::text)::name)::text) = %s", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index ef2d15266cb94..854c6be525d40 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 71361b5f2a0b9c5920d0596d872dff91f6f8c8f6 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:01 -0400 Subject: [PATCH 060/338] prop: add chr/ascii and single-character functions property tests (charfuncs) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_charfuncs.c | 306 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 310 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_charfuncs.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 6eb43b479ae4e..df06cdccc23f1 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -75,7 +75,8 @@ OBJS = \ prop_regexmatch.o \ prop_regexpreplace.o \ prop_char.o \ - prop_charconv.o + prop_charconv.o \ + prop_charfuncs.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 86bc4caf3a9b6..1cb4237734dfb 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -73,6 +73,7 @@ prop_area_sources = files( 'prop_regexpreplace.c', 'prop_char.c', 'prop_charconv.c', + 'prop_charfuncs.c', ) diff --git a/src/test/prop/prop_charfuncs.c b/src/test/prop/prop_charfuncs.c new file mode 100644 index 0000000000000..175ff6e0cc0ab --- /dev/null +++ b/src/test/prop/prop_charfuncs.c @@ -0,0 +1,306 @@ +/*------------------------------------------------------------------------- + * + * prop_charfuncs.c + * Property-based tests for single-character SQL functions: chr(), ascii(), + * and the one-character behaviour of length/octet_length/upper/lower/initcap. + * + * The oracle is the ASCII algebra of these functions: over the printable + * ASCII range 33..126, chr() and ascii() are mutual inverses, a single ASCII + * codepoint is exactly one character and one byte, upper/lower shift the ASCII + * letter case by 32, and ASCII codepoint order is byte/collation order. Each + * property asserts an identity that must hold for every drawn codepoint; any + * "f" result is a bug in these functions, not a rounding or locale artefact + * (all inputs are single-byte ASCII, below any multibyte boundary). + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_charfuncs.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "charfuncs" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on a valid ASCII + * codepoint, which is itself a bug -- so either is a failure, reported with + * the SQL. Copied from prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * ascii(chr(n)) = n for every printable ASCII codepoint n in 33..126. + * chr()/ascii() must be mutual inverses on the single-byte ASCII range. + */ +PG_PROP(chr_ascii_inverse) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 33, 126); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ascii(chr(%d)) = %d", n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * length(chr(n)) = 1 for n in 33..126. A single ASCII codepoint is exactly + * one character. + */ +PG_PROP(chr_length_one) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 33, 126); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT length(chr(%d)) = 1", n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ascii(S) = ascii(left(S,1)) for a non-empty string S: ascii() reads only + * the first character. S is built from drawn printable ASCII codepoints so it + * carries no quoting hazard, and is always at least one character long. + */ +PG_PROP(ascii_first_char) +{ + pg_prop_conn *conn; + char *err = NULL; + int len; + int i; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* length 1..8; every char a printable ASCII codepoint 33..126 */ + len = (int) pg_prop_draw_int(tc, 1, 8); + initStringInfo(&s); + for (i = 0; i < len; i++) + { + int n = (int) pg_prop_draw_int(tc, 33, 126); + + /* double single quotes to keep the SQL literal safe */ + if (n == '\'') + appendStringInfoChar(&s, '\''); + appendStringInfoChar(&s, (char) n); + } + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ascii('%s') = ascii(left('%s', 1))", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * For a lowercase letter codepoint n in 97..122, ascii(upper(chr(n))) = n - 32. + * upper() shifts an ASCII lowercase letter to its uppercase counterpart, which + * is exactly 32 codepoints lower. + */ +PG_PROP(upper_lower_ascii) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 97, 122); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ascii(upper(chr(%d))) = %d", n, n - 32); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * chr(a) || chr(b) is a two-character string whose first character's ascii() + * is a. Concatenating two single ASCII codepoints yields length 2 and + * preserves the leading codepoint. + */ +PG_PROP(chr_concat) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (int) pg_prop_draw_int(tc, 33, 126); + b = (int) pg_prop_draw_int(tc, 33, 126); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(chr(%d) || chr(%d)) = 2 " + "AND ascii(left(chr(%d) || chr(%d), 1)) = %d", + a, b, a, b, a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * initcap() uppercases the first letter of a word: for a lowercase letter + * codepoint n in 97..122, left(initcap(chr(n) || 'bc'), 1) = upper(chr(n)). + * The literal suffix 'bc' is letters-only so no quoting hazard arises. + */ +PG_PROP(initcap_first_upper) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 97, 122); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT left(initcap(chr(%d) || 'bc'), 1) = upper(chr(%d))", + n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ASCII codepoint order is string order: for n1 < n2 in 33..126, + * chr(n1) < chr(n2). Draw n1 and n2, then compare the smaller codepoint's + * character against the larger's; assume n1 != n2 is not needed because we + * order the two draws and skip the equal case via a strict comparison. + */ +PG_PROP(ascii_order) +{ + pg_prop_conn *conn; + char *err = NULL; + int x, + y, + n1, + n2; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x = (int) pg_prop_draw_int(tc, 33, 126); + y = (int) pg_prop_draw_int(tc, 33, 126); + /* equal codepoints make no strict ordering claim; discard that case */ + pg_prop_assume(tc, x != y); + n1 = (x < y) ? x : y; + n2 = (x < y) ? y : x; + + initStringInfo(&q); + appendStringInfo(&q, "SELECT chr(%d) < chr(%d)", n1, n2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * octet_length(chr(n)) = 1 for n in 33..126: every printable ASCII codepoint + * encodes to a single byte in a UTF-8 (or any ASCII-superset) database. + */ +PG_PROP(octet_length_ascii) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 33, 126); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT octet_length(chr(%d)) = 1", n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 854c6be525d40..7260a42f74d61 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From d01d17f89f3ea8fc17609d09fe8e01c51e1452bb Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:01 -0400 Subject: [PATCH 061/338] prop: add varchar/char length enforcement and padding property tests (varchartypmod) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_varchartypmod.c | 331 +++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 335 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_varchartypmod.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index df06cdccc23f1..14872d7363818 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -76,7 +76,8 @@ OBJS = \ prop_regexpreplace.o \ prop_char.o \ prop_charconv.o \ - prop_charfuncs.o + prop_charfuncs.o \ + prop_varchartypmod.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 1cb4237734dfb..abd4903e3c715 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -74,6 +74,7 @@ prop_area_sources = files( 'prop_char.c', 'prop_charconv.c', 'prop_charfuncs.c', + 'prop_varchartypmod.c', ) diff --git a/src/test/prop/prop_varchartypmod.c b/src/test/prop/prop_varchartypmod.c new file mode 100644 index 0000000000000..70912964a6af3 --- /dev/null +++ b/src/test/prop/prop_varchartypmod.c @@ -0,0 +1,331 @@ +/*------------------------------------------------------------------------- + * + * prop_varchartypmod.c + * Property-based tests for varchar(n)/char(n) typmod length semantics. + * + * An explicit cast to varchar(n) truncates to n characters when the input is + * longer and leaves it unchanged when it fits; char(n) blank-pads to exactly + * n characters and compares/trims blank-insensitively. The oracle is the + * independent builtins left(), length(), char_length(), rtrim() plus the + * type's own cast and comparison rules, so any disagreement is a bug in the + * typmod coercion path (src/backend/utils/adt/varchar.c). + * + * This file keeps S over [abc] with NO trailing spaces, so every truncation + * or padding effect is unambiguous (a dropped or added blank can never be + * confused with a character of S). That is deliberately distinct from + * prop_char.c, whose alphabet includes a space to exercise blank trimming. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_varchartypmod.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "varchartypmod" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. The [abc] alphabet never contains a quote, but do it + * defensively so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw a string of length 0..12 from the alphabet [abc] into buf as a bare + * string (no quoting). No space is ever produced, so S has no trailing + * blanks and truncation/padding effects are always observable. Returns the + * drawn length. + */ +static int +draw_str(pg_prop_tc *tc, StringInfo buf) +{ + static const char alphabet[] = "abc"; + int n = (int) pg_prop_draw_int(tc, 0, 12); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, alphabet[pg_prop_draw_int(tc, 0, 2)]); + return n; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the query errored or produced NULL on the generated + * inputs, which is itself a bug -- so either is a failure, reported with the + * SQL. Modelled on prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * varchar truncation: for n <= length(S), the explicit cast S::varchar(n) + * truncates to the first n characters, i.e. equals left(S, n). n is drawn in + * [0, length(S)] so the cast is always in truncation (or exact-fit) range. + */ +PG_PROP(varchar_truncates_on_cast) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int len; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + len = draw_str(tc, &s); + + /* varchar(n) requires n >= 1; skip length-0 draws (nothing to truncate) */ + if (len == 0) + { + pfree(s.data); + pg_prop_disconnect(conn); + return; + } + n = (int) pg_prop_draw_int(tc, 1, len); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, "::varchar(%d))::text = left(", n); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", %d)", n); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * varchar no-truncate when it fits: for n >= length(S), S::varchar(n) equals + * S unchanged (no padding for varchar). n is drawn in + * [length(S), length(S)+5], clamped to a minimum of 1 since varchar(n) + * requires n >= 1. + */ +PG_PROP(varchar_no_truncate_when_fits) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int len; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + len = draw_str(tc, &s); + + n = len + (int) pg_prop_draw_int(tc, 0, 5); + if (n < 1) + n = 1; /* varchar(n) requires n >= 1 */ + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, "::varchar(%d))::text = ", n); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * char(n) blank-pads in storage, but the cast back to text strips the trailing + * padding (unlike a raw bpchar, which keeps it internally). So for S with no + * trailing spaces and length <= n, the round-trip (S::char(n))::text recovers + * S exactly. S is single-byte ASCII with no trailing space; n is drawn in + * [length(S), 20] (clamped to >= 1 for the empty-S case). + */ +PG_PROP(char_pads_to_n) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int len; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + len = draw_str(tc, &s); + + n = (int) pg_prop_draw_int(tc, len < 1 ? 1 : len, 20); + + /* + * char(n) blank-pads in storage, but casting back to text strips the + * trailing padding, so for S (no trailing spaces, length <= n) the + * round-trip S::char(n)::text recovers S exactly. + */ + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, "::char(%d))::text = ", n); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * char trim recovers the original: since S over [abc] has no trailing space, + * rtrim(S::char(20)) equals S. char(20) pads S with trailing blanks to width + * 20; rtrim strips exactly those blanks, and because S itself ends in no + * blank the recovered string is S exactly. (length(S) <= 12 <= 20, so no + * truncation.) + */ +PG_PROP(char_trim_recovers) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + (void) draw_str(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT rtrim(("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "::char(20))) = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * char(n) comparison ignores trailing blanks: S::char(5) equals + * (S || ' ')::char(5) whenever length(S) < 5, so both operands pad to width 5 + * and differ only in trailing blanks, which bpchar equality ignores. S is + * redrawn until it is short enough (length < 5) rather than rejecting cases. + */ +PG_PROP(char_compares_ignore_trailing) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int len; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + len = draw_str(tc, &s); + + /* keep S short enough that both operands fit within char(5) */ + if (len >= 5) + s.data[s.len = 4] = '\0'; + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "::char(5)) = (("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " || ' ')::char(5))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * varchar does not pad: S::varchar(20)::text equals S for any S of length + * <= 12 (well within 20), confirming varchar stores the value verbatim with + * no trailing-blank padding, unlike char(n). + */ +PG_PROP(varchar_text_equal) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + (void) draw_str(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "::varchar(20))::text = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 7260a42f74d61..4574846a57ade 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 007356a85e8533c76f7e1b5ca75a1bb11a60d50b Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:01 -0400 Subject: [PATCH 062/338] prop: add cash property tests (cash) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_cash.c | 375 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 379 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_cash.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 14872d7363818..668557344d315 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -77,7 +77,8 @@ OBJS = \ prop_char.o \ prop_charconv.o \ prop_charfuncs.o \ - prop_varchartypmod.o + prop_varchartypmod.o \ + prop_cash.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index abd4903e3c715..07a17a7cd8090 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -75,6 +75,7 @@ prop_area_sources = files( 'prop_charconv.c', 'prop_charfuncs.c', 'prop_varchartypmod.c', + 'prop_cash.c', ) diff --git a/src/test/prop/prop_cash.c b/src/test/prop/prop_cash.c new file mode 100644 index 0000000000000..2304521c551e4 --- /dev/null +++ b/src/test/prop/prop_cash.c @@ -0,0 +1,375 @@ +/*------------------------------------------------------------------------- + * + * prop_cash.c + * Property-based tests for the money type (src/backend/utils/adt/cash.c). + * + * The money type is a fixed-point int64 of minor units (cents, in the C + * locale's two-fractional-digit format): a value like $1.23 is stored as the + * integer 123. Because the storage is a bare int64, its arithmetic has had a + * run of overflow fixes -- in addition and subtraction (where two large + * amounts can wrap the int64), in multiplication (cash * numeric and the + * integer multiply paths), and in the rounding of division back to the + * nearest cent. A wrap instead of an error is a silent wrong-results bug. + * + * These properties check arithmetic identities that must hold on finite, + * non-overflowing inputs -- additive inverse, commutativity, a text + * round-trip, multiply-as-numeric, a division-rounding bound, and total + * order -- and require that whenever an operation does run past the int64 + * range it raises an "out of range" error rather than wrapping. Any case + * that legitimately overflows is discarded (an assume), not reported, so the + * identities are only asserted where they are supposed to hold. + * + * Locale note: money's text input and output depend on lc_monetary (the + * currency symbol, the sign placement, the grouping). To make the text + * round-trip deterministic every property forces lc_monetary = 'C' on its + * connection; in the C locale money prints like "$1.23" or "-$1.23". The + * VALUE of a money literal is kept locale-independent regardless, by building + * it as (::numeric / 100)::money rather than parsing a formatted + * string, so the arithmetic properties do not depend on the locale at all. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_cash.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "cash" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * True when a SQL error is a legitimate int64 out-of-range/overflow condition + * rather than a wrong-results bug. money arithmetic near the ends of the + * int64 minor-unit range can legally overflow; those cases are discarded, not + * reported. + */ +static bool +is_overflow(const char *errmsg) +{ + if (errmsg == NULL) + return false; + return strstr(errmsg, "out of range") != NULL || + strstr(errmsg, "overflow") != NULL; +} + +/* + * Open a connection to the test cluster and force lc_monetary = 'C' so money + * text output/parse is deterministic. Fails the case if either step fails. + */ +static pg_prop_conn * +connect_C(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, "SET lc_monetary = 'C'", &err)) + pg_prop_fail(tc, "SET lc_monetary = 'C' failed: %s", err ? err : ""); + + return conn; +} + +/* + * Draw a money amount as a locale-independent SQL expression into buf: + * + * ((::numeric / 100)::money) + * + * money is an int64 count of minor units (cents, two fractional digits in the + * C locale), so an amount is fully described by its integer cent count. + * Casting a numeric to money is defined by value and does not depend on + * lc_monetary, which is why we build the literal this way instead of parsing a + * formatted "$1.23" string. + * + * Most draws use a moderate cent range so ordinary arithmetic (sums, + * small-factor products) does not overflow and the identities get exercised; + * about one draw in eight instead uses nearly the full int64 range so the + * overflow paths are hit and checked to error rather than wrap. The wide + * bound stops short of INT64_MAX (9223372036854775807 cents) so that a single + * numeric->money cast of the drawn value itself does not overflow -- only the + * subsequent arithmetic can. + */ +static void +draw_money(pg_prop_tc *tc, StringInfo buf) +{ + int64 cents; + + if (pg_prop_draw_int(tc, 0, 7) == 0) + cents = pg_prop_draw_int(tc, INT64_C(-92233720368547758), + INT64_C(92233720368547758)); + else + cents = pg_prop_draw_int(tc, -1000000, 1000000); + + appendStringInfo(buf, "((" INT64_FORMAT "::numeric / 100)::money)", cents); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operation errored or produced NULL on finite, + * non-overflowing inputs, which is itself a bug -- so either is a failure, + * reported with the SQL. Use this for properties that cannot overflow. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Like require_true, but a SQL error whose message names an int64 overflow is + * a legitimate out-of-range condition, not a bug: the case is discarded with + * an assume(false) (which longjmps and does not return). Any other error, a + * NULL result, or an "f" result is a failure. Use this for properties whose + * arithmetic can legally run past the int64 minor-unit range. + */ +static void +require_true_or_skip(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + if (is_overflow(err)) + { + /* ran off the int64 range: a legal error, discard the case */ + pg_prop_note(tc, "overflow (skipped): %s", err); + if (err) + pg_free(err); + pg_prop_assume(tc, false); + return; /* unreachable: assume(false) longjmps */ + } + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Additive inverse: (M1 + M2) - M2 = M1. money add/subtract are exact on + * int64 minor units, so this holds whenever the intermediate sum does not + * overflow; if the addition does overflow the server errors and the case is + * skipped. + */ +PG_PROP(add_sub_inverse) +{ + pg_prop_conn *conn; + StringInfoData m1, + m2, + q; + + conn = connect_C(tc); + + initStringInfo(&m1); + initStringInfo(&m2); + draw_money(tc, &m1); + draw_money(tc, &m2); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s + %s) - %s) = %s", + m1.data, m2.data, m2.data, m1.data); + + require_true_or_skip(tc, conn, q.data); + + pfree(m1.data); + pfree(m2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Addition commutes: M1 + M2 = M2 + M1. A carry/overflow bug that depends on + * operand order would surface as asymmetry here. Both sides overflow or + * neither does, so an overflow error discards the case cleanly. + */ +PG_PROP(add_commutes) +{ + pg_prop_conn *conn; + StringInfoData m1, + m2, + q; + + conn = connect_C(tc); + + initStringInfo(&m1); + initStringInfo(&m2); + draw_money(tc, &m1); + draw_money(tc, &m2); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s + %s) = (%s + %s)", + m1.data, m2.data, m2.data, m1.data); + + require_true_or_skip(tc, conn, q.data); + + pfree(m1.data); + pfree(m2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Text round-trip in the C locale: M::text::money = M. With lc_monetary='C' + * money prints like "$1.23"/"-$1.23"; parsing that text back must recover the + * exact same value. This is the property that needs the forced C locale -- + * under a different lc_monetary the symbol and sign placement change but the + * round-trip must still hold. No arithmetic, so no overflow: require "t". + */ +PG_PROP(text_roundtrip_C_locale) +{ + pg_prop_conn *conn; + StringInfoData m, + q; + + conn = connect_C(tc); + + initStringInfo(&m); + draw_money(tc, &m); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s::text::money) = %s", m.data, m.data); + + require_true(tc, conn, q.data); + + pfree(m.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Multiplying money by an integer equals doing the multiply in numeric and + * casting back: M * k = (M::numeric * k)::money. The oracle is independent of + * the money * int code path (it routes through numeric arithmetic and a single + * numeric->money cast), so a bug in the integer multiply surfaces as a + * mismatch. k is small (0..6) so moderate amounts stay in range; a product + * that does overflow errors and the case is skipped. + */ +PG_PROP(mul_by_int_is_repeated_add) +{ + pg_prop_conn *conn; + int k; + StringInfoData m, + q; + + conn = connect_C(tc); + + k = (int) pg_prop_draw_int(tc, 0, 6); + + initStringInfo(&m); + draw_money(tc, &m); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s * %d) = ((%s::numeric * %d)::money)", + m.data, k, m.data, k); + + require_true_or_skip(tc, conn, q.data); + + pfree(m.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Dividing money by a nonzero integer TRUNCATES toward zero (it is plain + * integer division of the int64 minor units, not round-to-nearest), so the + * money quotient differs from the exact numeric quotient by strictly less + * than one cent: abs((M / k)::numeric - M::numeric / k) < 0.01. This bounds + * the truncation without re-implementing it. k is drawn nonzero by + * construction (in {-6..-1, 1..6}), so no division-by-zero is expected; an + * overflow in the division skips the case. + */ +PG_PROP(div_by_nonzero_times_back_close) +{ + pg_prop_conn *conn; + int k; + StringInfoData m, + q; + + conn = connect_C(tc); + + /* draw k in {-6..-1, 1..6}: nonzero by construction */ + k = (int) pg_prop_draw_int(tc, 1, 6); + if (pg_prop_draw_bool(tc)) + k = -k; + + initStringInfo(&m); + draw_money(tc, &m); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(((%s / %d)::numeric) - (%s::numeric / %d)) " + "< 0.01::numeric", + m.data, k, m.data, k); + + require_true_or_skip(tc, conn, q.data); + + pfree(m.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * money is a total order: for any two values exactly one of <, =, > holds, so + * the three comparisons sum to exactly 1. A broken comparator (e.g. a signed/ + * unsigned mix-up at the int64 boundary) would make two of them true or none + * true. Pure comparison, no arithmetic: no overflow, require "t". + */ +PG_PROP(comparison_total_order) +{ + pg_prop_conn *conn; + StringInfoData m1, + m2, + q; + + conn = connect_C(tc); + + initStringInfo(&m1); + initStringInfo(&m2); + draw_money(tc, &m1); + draw_money(tc, &m2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (((%s < %s)::int + (%s = %s)::int " + "+ (%s > %s)::int) = 1)", + m1.data, m2.data, m1.data, m2.data, m1.data, m2.data); + + require_true(tc, conn, q.data); + + pfree(m1.data); + pfree(m2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 4574846a57ade..2ccfdc8a8b1d9 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 77040da3bf6a30c84a0e447bacc619a72d1392d1 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:01 -0400 Subject: [PATCH 063/338] prop: add bool property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_bool.c | 322 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 326 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_bool.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 668557344d315..ede970948e2a2 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -78,7 +78,8 @@ OBJS = \ prop_charconv.o \ prop_charfuncs.o \ prop_varchartypmod.o \ - prop_cash.o + prop_cash.o \ + prop_bool.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 07a17a7cd8090..156064dd53031 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -76,6 +76,7 @@ prop_area_sources = files( 'prop_charfuncs.c', 'prop_varchartypmod.c', 'prop_cash.c', + 'prop_bool.c', ) diff --git a/src/test/prop/prop_bool.c b/src/test/prop/prop_bool.c new file mode 100644 index 0000000000000..c40a62a86f240 --- /dev/null +++ b/src/test/prop/prop_bool.c @@ -0,0 +1,322 @@ +/*------------------------------------------------------------------------- + * + * prop_bool.c + * Property-based tests for boolean type input/output. + * + * boolean input (boolin, src/backend/utils/adt/bool.c) accepts many spellings + * (t/true/yes/on/1 and f/false/no/off/0, case-insensitive, with surrounding + * whitespace) and must map them consistently; these check that tolerance and + * boolean algebra. The oracle is the type's own semantics: every accepted + * true spelling must equal true, every false spelling false, output is + * canonical 'true'/'false', and the usual laws (double negation, De Morgan, + * the false < true total order) hold. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_bool.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "bool" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * The documented true-ish spellings, case-insensitive: true, t, yes, y, on, 1. + * Mixed case and a couple of whitespace-padded forms are included to exercise + * boolin's case folding and leading/trailing whitespace trimming. None + * contain a single quote, so they embed into a '...' literal unescaped. + */ +static const char *const true_spellings[] = { + "t", "T", "true", "TRUE", "True", + "yes", "YES", "y", "on", "ON", "1", + " true ", " 1 " +}; + +/* + * The documented false-ish spellings, case-insensitive: false, f, no, n, off, + * 0. Same mixed-case and whitespace coverage as the true set. + */ +static const char *const false_spellings[] = { + "f", "F", "false", "FALSE", "False", + "no", "NO", "n", "off", "OFF", "0", + " false ", " 0 " +}; + +#define N_TRUE_SPELLINGS ((int) (sizeof(true_spellings) / sizeof(true_spellings[0]))) +#define N_FALSE_SPELLINGS ((int) (sizeof(false_spellings) / sizeof(false_spellings[0]))) + +/* + * Draw one accepted spelling of "true". + */ +static const char * +draw_true_spelling(pg_prop_tc *tc) +{ + int i = (int) pg_prop_draw_int(tc, 0, N_TRUE_SPELLINGS - 1); + + return true_spellings[i]; +} + +/* + * Draw one accepted spelling of "false". + */ +static const char * +draw_false_spelling(pg_prop_tc *tc) +{ + int i = (int) pg_prop_draw_int(tc, 0, N_FALSE_SPELLINGS - 1); + + return false_spellings[i]; +} + +/* + * Draw a boolean literal: coin-flip the truth value, pick a spelling of it, + * and emit it into buf as a single-quoted SQL literal ''. The + * spellings contain only letters, digits and spaces -- no quotes -- so no + * escaping is needed. *expected receives the C bool the literal denotes. + */ +static void +draw_bool_literal(pg_prop_tc *tc, StringInfo buf, bool *expected) +{ + bool want = pg_prop_draw_bool(tc); + const char *spelling = want ? draw_true_spelling(tc) : draw_false_spelling(tc); + + appendStringInfoChar(buf, '\''); + appendStringInfoString(buf, spelling); + appendStringInfoChar(buf, '\''); + + *expected = want; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means boolin rejected an accepted spelling or the algebra + * produced NULL on non-NULL inputs, which is itself a bug -- so either is a + * failure, reported with the SQL. Modelled on prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Every accepted true spelling parses to true. + */ +PG_PROP(true_spellings_parse_true) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *spelling; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + spelling = draw_true_spelling(tc); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT '%s'::boolean = true", spelling); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Every accepted false spelling parses to false. + */ +PG_PROP(false_spellings_parse_false) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *spelling; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + spelling = draw_false_spelling(tc); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT '%s'::boolean = false", spelling); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Text round-trip: boolean output is canonical 'true'/'false' and matches the + * parsed value. Draw a literal with a known expected bool and compare its + * ::text rendering to the canonical word for that bool. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData b, + q; + bool expected = false; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&b); + draw_bool_literal(tc, &b, &expected); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s::boolean)::text = '%s'", + b.data, expected ? "true" : "false"); + + require_true(tc, conn, q.data); + + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Double negation is identity: NOT (NOT B) = B. + */ +PG_PROP(not_involution) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData b, + q; + bool expected = false; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&b); + draw_bool_literal(tc, &b, &expected); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (NOT (NOT (%s::boolean))) = (%s::boolean)", + b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * De Morgan over the boolean type: NOT (P AND Q) = (NOT P) OR (NOT Q). + */ +PG_PROP(and_or_demorgan) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData p, + qq, + q; + bool ep = false, + eq = false; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&p); + initStringInfo(&qq); + draw_bool_literal(tc, &p, &ep); + draw_bool_literal(tc, &qq, &eq); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (NOT ((%s::boolean) AND (%s::boolean))) " + "= ((NOT (%s::boolean)) OR (NOT (%s::boolean)))", + p.data, qq.data, p.data, qq.data); + + require_true(tc, conn, q.data); + + pfree(p.data); + pfree(qq.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * boolin is case-insensitive: a true spelling parses the same whether upper- + * or lower-cased. (The spellings are already true-ish; folding case must not + * change the parse.) + */ +PG_PROP(case_insensitive) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *spelling; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + spelling = draw_true_spelling(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (upper('%s')::boolean) = (lower('%s')::boolean)", + spelling, spelling); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The boolean total order: false < true, not true < false, and false <> true. + * A fixed check with no draws -- it anchors the ordering operators. + */ +PG_PROP(comparison_consistent) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT (false < true) AND NOT (true < false) " + "AND (false <> true)"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 2ccfdc8a8b1d9..a5453dbf4d1e4 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From b0acd0768f139352a7ce4278e1931f1b26851ac6 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:02 -0400 Subject: [PATCH 064/338] prop: add boolean text I/O and cast variants property tests (boolcast) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_boolcast.c | 223 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 227 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_boolcast.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index ede970948e2a2..71a4810476dc5 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -79,7 +79,8 @@ OBJS = \ prop_charfuncs.o \ prop_varchartypmod.o \ prop_cash.o \ - prop_bool.o + prop_bool.o \ + prop_boolcast.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 156064dd53031..64a4aaebe67b7 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -77,6 +77,7 @@ prop_area_sources = files( 'prop_varchartypmod.c', 'prop_cash.c', 'prop_bool.c', + 'prop_boolcast.c', ) diff --git a/src/test/prop/prop_boolcast.c b/src/test/prop/prop_boolcast.c new file mode 100644 index 0000000000000..a54da8ad7637e --- /dev/null +++ b/src/test/prop/prop_boolcast.c @@ -0,0 +1,223 @@ +/*------------------------------------------------------------------------- + * + * prop_boolcast.c + * Property-based tests for boolean text I/O and cast variants. + * + * The boolean type accepts many spellings on input ('true'/'yes'/'on'/'1' + * and 'false'/'no'/'off'/'0', case-insensitively) and casts to and from int4 + * (true<->1, false<->0). boolin/boolout and the bool<->int4 casts implement + * this; a bug there shows up as a spelling that no longer parses to the right + * value, a broken bool<->int round-trip, or a text round-trip that does not + * recover the input. These oracles are exact fixed-input identities (plus a + * drawn 0/1 for the int->bool direction), so any "f" is a bug, not a tolerance + * artefact. This area is distinct from booltest (IS TRUE/FALSE/UNKNOWN) and + * boolexpr (AND/OR/NOT short-circuit algebra): it targets only I/O and casts. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_boolcast.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "boolcast" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a cast or I/O conversion errored or produced NULL on an + * input that must succeed, which is itself a bug -- so either is a failure, + * reported with the SQL. (Copied from prop_numeric.c's require_true.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * All the accepted "true" and "false" spellings parse to the same boolean + * value. 'true'/'yes'/'on'/'1' are true; 'false'/'no'/'off'/'0' are false. + * Exercises boolin's spelling table directly. + */ +PG_PROP(bool_text_variants) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT 'true'::bool = 't'::bool " + "AND 'yes'::bool = 'true'::bool " + "AND 'on'::bool = 'true'::bool " + "AND '1'::bool = 'true'::bool " + "AND 'false'::bool = 'f'::bool " + "AND 'no'::bool = 'false'::bool " + "AND 'off'::bool = 'false'::bool " + "AND '0'::bool = 'false'::bool"); + + pg_prop_disconnect(conn); +} + +/* + * bool -> int4 cast: true is 1, false is 0. Exercises the bool->int4 cast. + */ +PG_PROP(bool_to_int) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT 'true'::bool::int = 1 " + "AND 'false'::bool::int = 0"); + + pg_prop_disconnect(conn); +} + +/* + * int4 -> bool cast: 0 is false, nonzero is true. Drawn n in {0,1} keeps the + * oracle on the two values PostgreSQL round-trips cleanly; the generic rule + * (0 false, nonzero true) is checked against the n=0 vs n<>0 split. + */ +PG_PROP(int_to_bool) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_bool(tc) ? 1 : 0; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (CASE WHEN %d = 0 THEN false ELSE true END) " + "= ((%d)::int::bool)", + n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * bool text round-trip: boolout renders 'true'/'false', and reparsing the + * output recovers the value. b::text::bool = b for both booleans. + */ +PG_PROP(bool_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT 'true'::bool::text = 'true' " + "AND 'false'::bool::text = 'false' " + "AND 'true'::bool::text::bool = 'true'::bool " + "AND 'false'::bool::text::bool = 'false'::bool"); + + pg_prop_disconnect(conn); +} + +/* + * NOT toggles: NOT true is false, NOT false is true. + */ +PG_PROP(not_toggles) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT (NOT 'true'::bool) = 'false'::bool " + "AND (NOT 'false'::bool) = 'true'::bool"); + + pg_prop_disconnect(conn); +} + +/* + * AND/OR identity elements: (b AND true) = b and (b OR false) = b for both + * booleans. + */ +PG_PROP(bool_and_or_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT ('true'::bool AND 'true'::bool) = 'true'::bool " + "AND ('false'::bool AND 'true'::bool) = 'false'::bool " + "AND ('true'::bool OR 'false'::bool) = 'true'::bool " + "AND ('false'::bool OR 'false'::bool) = 'false'::bool"); + + pg_prop_disconnect(conn); +} + +/* + * Input is case-insensitive: 'TRUE'/'False'/'YES' parse to the expected + * values. Exercises boolin's case folding. + */ +PG_PROP(case_insensitive_input) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT 'TRUE'::bool = 'true'::bool " + "AND 'False'::bool = 'false'::bool " + "AND 'YES'::bool = 'true'::bool"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index a5453dbf4d1e4..6adf2fcec9fe6 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From ea6152996c054075b6abddea8943033406983085 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:02 -0400 Subject: [PATCH 065/338] prop: add IS TRUE/FALSE/UNKNOWN boolean tests property tests (booltest) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_booltest.c | 193 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 197 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_booltest.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 71a4810476dc5..07b9ddf221d89 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -80,7 +80,8 @@ OBJS = \ prop_varchartypmod.o \ prop_cash.o \ prop_bool.o \ - prop_boolcast.o + prop_boolcast.o \ + prop_booltest.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 64a4aaebe67b7..181b950a49771 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -78,6 +78,7 @@ prop_area_sources = files( 'prop_cash.c', 'prop_bool.c', 'prop_boolcast.c', + 'prop_booltest.c', ) diff --git a/src/test/prop/prop_booltest.c b/src/test/prop/prop_booltest.c new file mode 100644 index 0000000000000..c244de5881322 --- /dev/null +++ b/src/test/prop/prop_booltest.c @@ -0,0 +1,193 @@ +/*------------------------------------------------------------------------- + * + * prop_booltest.c + * Property-based tests for IS TRUE / IS FALSE / IS UNKNOWN boolean tests. + * + * The oracle is the defining expansion of each boolean test: IS TRUE is + * coalesce(x = true, false), IS UNKNOWN is x IS NULL, and the three states + * partition the input (exactly one holds for any bool-or-NULL x). Each + * property reconstructs a test from its definition in SQL and requires the + * two sides to agree; a disagreement is a bug in the BooleanTest code. These + * tests never produce NULL -- that is the whole point of IS TRUE/FALSE/UNKNOWN + * -- so a NULL oracle result is itself a failure. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_booltest.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "booltest" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* Draw a bool-or-NULL literal: 'true', 'false', or NULL (~1/3 each). */ +static void +draw_bool_or_null(pg_prop_tc *tc, StringInfo buf) +{ + int k = (int) pg_prop_draw_int(tc, 0, 2); + + if (k == 0) + appendStringInfoString(buf, "NULL::bool"); + else if (k == 1) + appendStringInfoString(buf, "true"); + else + appendStringInfoString(buf, "false"); +} + +/* Run a boolean SELECT and require "t"; NULL or error is a failure. */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* IS TRUE equals coalesce(x = true, false). */ +PG_PROP(is_true_definition) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_bool_or_null(tc, &x); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s) IS TRUE) = coalesce((%s) = true, false)", + x.data, x.data); + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* IS NOT TRUE is the complement of IS TRUE. */ +PG_PROP(is_not_true_complement) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_bool_or_null(tc, &x); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s) IS NOT TRUE) = (NOT ((%s) IS TRUE))", + x.data, x.data); + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* IS UNKNOWN equals IS NULL for a boolean. */ +PG_PROP(is_unknown_eq_is_null) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_bool_or_null(tc, &x); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s) IS UNKNOWN) = ((%s) IS NULL)", + x.data, x.data); + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The three states partition the input: exactly one of IS TRUE, IS FALSE, + * IS UNKNOWN holds for any bool-or-NULL x. + */ +PG_PROP(exactly_one_state) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_bool_or_null(tc, &x); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (((%s) IS TRUE)::int + ((%s) IS FALSE)::int + " + "((%s) IS UNKNOWN)::int) = 1", + x.data, x.data, x.data); + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* Double negation is identity, including for NULL: NOT NOT x = x (3VL). */ +PG_PROP(double_negation) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_bool_or_null(tc, &x); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (NOT (NOT (%s))) IS NOT DISTINCT FROM (%s)", + x.data, x.data); + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 6adf2fcec9fe6..3640c4d619979 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 9b6f008feb56a3b6a9eee49daecb4b7177df6371 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:02 -0400 Subject: [PATCH 066/338] prop: add boolean algebra laws in three-valued logic property tests (boolexpr) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_boolexpr.c | 314 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 318 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_boolexpr.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 07b9ddf221d89..c2387dd0ee53d 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -81,7 +81,8 @@ OBJS = \ prop_cash.o \ prop_bool.o \ prop_boolcast.o \ - prop_booltest.o + prop_booltest.o \ + prop_boolexpr.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 181b950a49771..adff3d81a0e61 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -79,6 +79,7 @@ prop_area_sources = files( 'prop_bool.c', 'prop_boolcast.c', 'prop_booltest.c', + 'prop_boolexpr.c', ) diff --git a/src/test/prop/prop_boolexpr.c b/src/test/prop/prop_boolexpr.c new file mode 100644 index 0000000000000..0afec66c45019 --- /dev/null +++ b/src/test/prop/prop_boolexpr.c @@ -0,0 +1,314 @@ +/*------------------------------------------------------------------------- + * + * prop_boolexpr.c + * Property-based tests for boolean-expression algebra under 3VL. + * + * PostgreSQL booleans are three-valued (true, false, NULL/unknown). The + * laws of boolean algebra -- associativity, distributivity, absorption, + * commutativity, identity/annihilator -- all survive the extension to 3VL + * when AND/OR are given their Kleene truth tables, and this file checks each + * law directly against the server's AND/OR evaluation. Because a 3VL result + * can itself be NULL, both sides of every law are compared with IS NOT + * DISTINCT FROM (NULL-safe equality) rather than =; a disagreement is a bug + * in boolean expression evaluation, not a NULL artefact. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_boolexpr.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "boolexpr" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw one three-valued boolean literal: 'true', 'false', or NULL (~1/3 + * each). NULL is cast to bool so AND/OR resolve to the boolean operators + * even when every operand is NULL. + */ +static void +draw_bool3(pg_prop_tc *tc, StringInfo buf) +{ + int k = (int) pg_prop_draw_int(tc, 0, 2); + + if (k == 0) + appendStringInfoString(buf, "true"); + else if (k == 1) + appendStringInfoString(buf, "false"); + else + appendStringInfoString(buf, "NULL::bool"); +} + +/* Run a boolean SELECT and require "t"; NULL or error is a failure. */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* (a AND b) AND c is not distinct from a AND (b AND c). */ +PG_PROP(and_associative) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_bool3(tc, &a); + initStringInfo(&b); + draw_bool3(tc, &b); + initStringInfo(&c); + draw_bool3(tc, &c); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (((%s) AND (%s)) AND (%s)) " + "IS NOT DISTINCT FROM ((%s) AND ((%s) AND (%s)))", + a.data, b.data, c.data, a.data, b.data, c.data); + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* (a OR b) OR c is not distinct from a OR (b OR c). */ +PG_PROP(or_associative) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_bool3(tc, &a); + initStringInfo(&b); + draw_bool3(tc, &b); + initStringInfo(&c); + draw_bool3(tc, &c); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (((%s) OR (%s)) OR (%s)) " + "IS NOT DISTINCT FROM ((%s) OR ((%s) OR (%s)))", + a.data, b.data, c.data, a.data, b.data, c.data); + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* a AND (b OR c) is not distinct from (a AND b) OR (a AND c). */ +PG_PROP(and_distributes_over_or) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_bool3(tc, &a); + initStringInfo(&b); + draw_bool3(tc, &b); + initStringInfo(&c); + draw_bool3(tc, &c); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s) AND ((%s) OR (%s))) " + "IS NOT DISTINCT FROM " + "(((%s) AND (%s)) OR ((%s) AND (%s)))", + a.data, b.data, c.data, + a.data, b.data, a.data, c.data); + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* a OR (b AND c) is not distinct from (a OR b) AND (a OR c). */ +PG_PROP(or_distributes_over_and) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_bool3(tc, &a); + initStringInfo(&b); + draw_bool3(tc, &b); + initStringInfo(&c); + draw_bool3(tc, &c); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s) OR ((%s) AND (%s))) " + "IS NOT DISTINCT FROM " + "(((%s) OR (%s)) AND ((%s) OR (%s)))", + a.data, b.data, c.data, + a.data, b.data, a.data, c.data); + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Absorption, both forms, which hold in 3VL: + * a AND (a OR b) is not distinct from a + * a OR (a AND b) is not distinct from a + */ +PG_PROP(absorption) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_bool3(tc, &a); + initStringInfo(&b); + draw_bool3(tc, &b); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (((%s) AND ((%s) OR (%s))) IS NOT DISTINCT FROM (%s)) " + "AND (((%s) OR ((%s) AND (%s))) IS NOT DISTINCT FROM (%s))", + a.data, a.data, b.data, a.data, + a.data, a.data, b.data, a.data); + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Commutativity of both connectives: + * (a AND b) is not distinct from (b AND a) + * (a OR b) is not distinct from (b OR a) + */ +PG_PROP(and_commutative_or_commutative) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_bool3(tc, &a); + initStringInfo(&b); + draw_bool3(tc, &b); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (((%s) AND (%s)) IS NOT DISTINCT FROM ((%s) AND (%s))) " + "AND (((%s) OR (%s)) IS NOT DISTINCT FROM ((%s) OR (%s)))", + a.data, b.data, b.data, a.data, + a.data, b.data, b.data, a.data); + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Identity and annihilator elements: + * a AND true is not distinct from a (true is AND-identity) + * a OR false is not distinct from a (false is OR-identity) + * a AND false = false (false annihilates AND) + * a OR true = true (true annihilates OR) + * The annihilator results are never NULL, so plain = is sound there. + */ +PG_PROP(identity_annihilator) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_bool3(tc, &a); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (((%s) AND true) IS NOT DISTINCT FROM (%s)) " + "AND (((%s) OR false) IS NOT DISTINCT FROM (%s)) " + "AND (((%s) AND false) = false) " + "AND (((%s) OR true) = true)", + a.data, a.data, a.data, a.data, a.data, a.data); + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 3640c4d619979..22f6a4f8a9b01 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 990fd15fb7b5500002026337daf3a5e0dbeae8a9 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:02 -0400 Subject: [PATCH 067/338] prop: add CASE, COALESCE, NULLIF, GREATEST/LEAST property tests (casewhen) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_casewhen.c | 335 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 339 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_casewhen.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index c2387dd0ee53d..94de1d10658fd 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -82,7 +82,8 @@ OBJS = \ prop_bool.o \ prop_boolcast.o \ prop_booltest.o \ - prop_boolexpr.o + prop_boolexpr.o \ + prop_casewhen.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index adff3d81a0e61..852f8144fcf5a 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -80,6 +80,7 @@ prop_area_sources = files( 'prop_boolcast.c', 'prop_booltest.c', 'prop_boolexpr.c', + 'prop_casewhen.c', ) diff --git a/src/test/prop/prop_casewhen.c b/src/test/prop/prop_casewhen.c new file mode 100644 index 0000000000000..b8d7bd8fbd9a3 --- /dev/null +++ b/src/test/prop/prop_casewhen.c @@ -0,0 +1,335 @@ +/*------------------------------------------------------------------------- + * + * prop_casewhen.c + * Property-based tests for CASE / COALESCE / NULLIF / GREATEST / LEAST. + * + * These SQL constructs all compile to the same conditional-expression + * machinery (CaseExpr, CoalesceExpr, MinMaxExpr, NullIfExpr in the executor), + * and each has a textbook definition in terms of the others. The oracle is + * that equivalence: COALESCE must equal the first non-NULL via a nested CASE, + * NULLIF must equal its CASE expansion, a searched and a simple CASE must + * agree, GREATEST/LEAST must bound their arguments and ignore NULLs. Any "f" + * result is a wrong-results bug in that machinery, not a modelling artefact, + * because every comparison that can see NULL uses IS NOT DISTINCT FROM. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_casewhen.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "casewhen" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* Integer domain for drawn arguments. */ +#define VAL_MIN (-20) +#define VAL_MAX 20 + +/* + * Emit an int literal in [VAL_MIN, VAL_MAX], or SQL NULL (~25% of the time). + * A negative is wrapped as "(%d)" so a leading minus never misparses when the + * literal is substituted into a larger expression. + */ +static void +draw_int_or_null(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 3) == 0) + { + appendStringInfoString(buf, "NULL::int"); + return; + } + appendStringInfo(buf, "(%d)", (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); +} + +/* Emit a non-NULL int literal in [VAL_MIN, VAL_MAX], as "(%d)". */ +static void +draw_int(pg_prop_tc *tc, StringInfo buf) +{ + appendStringInfo(buf, "(%d)", (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the expression errored or produced NULL where the oracle + * guarantees a definite true, which is itself a bug -- so either is a failure, + * reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * A searched CASE and a simple CASE over the same scrutinee must agree. Draw + * x in 0..3 so some cases hit a WHEN arm and some fall to ELSE. + */ +PG_PROP(case_searched_equals_simple) +{ + pg_prop_conn *conn; + char *err = NULL; + int x; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x = (int) pg_prop_draw_int(tc, 0, 3); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (CASE (%d) WHEN 0 THEN 'a' WHEN 1 THEN 'b' " + "ELSE 'z' END) " + "= (CASE WHEN (%d)=0 THEN 'a' WHEN (%d)=1 THEN 'b' " + "ELSE 'z' END)", + x, x, x); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * COALESCE(a,b,c) returns the first non-NULL argument (or NULL if all are + * NULL), which is exactly a nested CASE on IS NOT NULL. IS NOT DISTINCT FROM + * makes the all-NULL case pass soundly. + */ +PG_PROP(coalesce_first_nonnull) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + initStringInfo(&c); + draw_int_or_null(tc, &a); + draw_int_or_null(tc, &b); + draw_int_or_null(tc, &c); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT COALESCE(%s, %s, %s) IS NOT DISTINCT FROM " + "(CASE WHEN %s IS NOT NULL THEN %s " + "WHEN %s IS NOT NULL THEN %s ELSE %s END)", + a.data, b.data, c.data, + a.data, a.data, b.data, b.data, c.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * NULLIF(a,b) is defined as CASE WHEN a=b THEN NULL ELSE a END. IS NOT + * DISTINCT FROM covers the NULL-on-both-sides cases. + */ +PG_PROP(nullif_definition) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_int_or_null(tc, &a); + draw_int_or_null(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT NULLIF(%s, %s) IS NOT DISTINCT FROM " + "(CASE WHEN %s = %s THEN NULL ELSE %s END)", + a.data, b.data, a.data, b.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * For non-NULL a,b,c: LEAST(a,b,c) <= each argument <= GREATEST(a,b,c). A + * sound total bound -- the min is below every element and the max above every + * element with no ties or NULLs to worry about. + */ +PG_PROP(greatest_least_bounds) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + initStringInfo(&c); + draw_int(tc, &a); + draw_int(tc, &b); + draw_int(tc, &c); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT LEAST(%s, %s, %s) <= %s AND %s <= GREATEST(%s, %s, %s) " + "AND LEAST(%s, %s, %s) <= %s AND %s <= GREATEST(%s, %s, %s) " + "AND LEAST(%s, %s, %s) <= %s AND %s <= GREATEST(%s, %s, %s)", + a.data, b.data, c.data, a.data, a.data, a.data, b.data, c.data, + a.data, b.data, c.data, b.data, b.data, a.data, b.data, c.data, + a.data, b.data, c.data, c.data, c.data, a.data, b.data, c.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * GREATEST and LEAST skip NULL arguments, so inserting a NULL among non-NULL + * operands must not change the result. + */ +PG_PROP(greatest_ignores_null) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_int(tc, &a); + draw_int(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT GREATEST(%s, NULL::int, %s) = GREATEST(%s, %s) " + "AND LEAST(%s, NULL::int, %s) = LEAST(%s, %s)", + a.data, b.data, a.data, b.data, + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A CASE with no ELSE and no matching WHEN yields NULL. One constant form + * (WHEN false) and one drawn form (WHEN x > 1000, never true for x in the + * domain) both must be NULL. + */ +PG_PROP(case_no_match_is_null) +{ + pg_prop_conn *conn; + char *err = NULL; + int x; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x = (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (CASE WHEN false THEN 1 END) IS NULL " + "AND (CASE WHEN (%d) > 1000 THEN 1 END) IS NULL", + x); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * COALESCE(a,a) is a (idempotent), including when a is NULL. IS NOT DISTINCT + * FROM makes the NULL case sound. + */ +PG_PROP(coalesce_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_int_or_null(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT COALESCE(%s, %s) IS NOT DISTINCT FROM %s", + a.data, a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 22f6a4f8a9b01..9f731ecb257e0 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From c289ca65d0f1b08d512529dc407964305a53ec8f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:03 -0400 Subject: [PATCH 068/338] prop: add NULLIF chains and conditional expressions property tests (condexpr) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_condexpr.c | 333 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 337 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_condexpr.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 94de1d10658fd..33f37cb9c3cc9 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -83,7 +83,8 @@ OBJS = \ prop_boolcast.o \ prop_booltest.o \ prop_boolexpr.o \ - prop_casewhen.o + prop_casewhen.o \ + prop_condexpr.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 852f8144fcf5a..836bea96aa2b0 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -81,6 +81,7 @@ prop_area_sources = files( 'prop_booltest.c', 'prop_boolexpr.c', 'prop_casewhen.c', + 'prop_condexpr.c', ) diff --git a/src/test/prop/prop_condexpr.c b/src/test/prop/prop_condexpr.c new file mode 100644 index 0000000000000..79c7ef8243fba --- /dev/null +++ b/src/test/prop/prop_condexpr.c @@ -0,0 +1,333 @@ +/*------------------------------------------------------------------------- + * + * prop_condexpr.c + * Property-based tests for conditional-expression semantics. + * + * NULLIF, multi-branch simple CASE, GREATEST/LEAST NULL handling, ELSE-less + * searched CASE, and nested COALESCE all compile to the same executor nodes + * (NullIfExpr, CaseExpr, MinMaxExpr, CoalesceExpr). Each property pins one + * specified behaviour against an independent reconstruction of its definition, + * so any "f" is a wrong-results bug in that machinery rather than a modelling + * artefact: NULLIF reflexively nulls, NULLIF equals its = based CASE + * expansion, a four-branch simple CASE selects the first match, GREATEST/LEAST + * skip NULLs (and all-NULL yields NULL), an ELSE-less CASE with no true WHEN + * is NULL, and COALESCE flattens. Comparisons that can see NULL use IS NOT + * DISTINCT FROM; the NULLIF oracle uses plain = to match NULLIF's own equality. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_condexpr.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "condexpr" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw one int-or-NULL argument into buf: with probability ~1/4 the literal + * SQL NULL (cast to ::int so an all-NULL expression still type-resolves), + * otherwise a small integer in [-50, 50] emitted as "(%d)::int" so a leading + * minus never misparses when substituted into a larger expression. + */ +static void +draw_arg(pg_prop_tc *tc, StringInfo buf) +{ + bool is_null = (pg_prop_draw_int(tc, 0, 3) == 0); + + if (is_null) + appendStringInfoString(buf, "NULL::int"); + else + appendStringInfo(buf, "(%d)::int", + (int) pg_prop_draw_int(tc, -50, 50)); +} + +/* + * Draw a guaranteed non-NULL int argument into buf, for properties that + * compare GREATEST/LEAST with interleaved NULLs against the NULL-free form. + */ +static void +draw_arg_nonnull(pg_prop_tc *tc, StringInfo buf) +{ + appendStringInfo(buf, "(%d)::int", + (int) pg_prop_draw_int(tc, -50, 50)); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the expression did not behave as the oracle demands, + * which is itself a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * NULLIF(a, a) IS NULL for any int-or-NULL a. When a is non-NULL, a = a is + * true so NULLIF returns NULL; when a is NULL, NULLIF(NULL, NULL) is NULL by + * definition. Either way the result is NULL, checked with IS NULL. + */ +PG_PROP(nullif_self_null) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arg(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT nullif(%s, %s) IS NULL", a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * NULLIF(a, b) equals its = based CASE expansion: + * CASE WHEN a = b THEN NULL ELSE a END. + * NULLIF decides on SQL = (not IS NOT DISTINCT FROM), so when a or b is NULL + * the WHEN is unknown and NULLIF returns a -- the oracle reproduces that. The + * two sides are compared with IS NOT DISTINCT FROM so a NULL result matches. + */ +PG_PROP(coalesce_nullif_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_arg(tc, &a); + draw_arg(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT nullif(%s, %s) IS NOT DISTINCT FROM " + "CASE WHEN %s = %s THEN NULL ELSE %s END", + a.data, b.data, + a.data, b.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A four-branch simple CASE selects the first matching branch. For a drawn + * x in 0..3, + * CASE x WHEN 0 THEN 10 WHEN 1 THEN 11 WHEN 2 THEN 12 ELSE 13 END + * must equal 10 + least(x, 3) (10,11,12 for x=0,1,2; 13 for x=3 via ELSE). + */ +PG_PROP(case_multibranch) +{ + pg_prop_conn *conn; + char *err = NULL; + int x = (int) pg_prop_draw_int(tc, 0, 3); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (CASE %d WHEN 0 THEN 10 WHEN 1 THEN 11 " + "WHEN 2 THEN 12 ELSE 13 END) = (10 + least(%d, 3))", + x, x); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * GREATEST and LEAST skip NULL arguments: with non-NULL a, b, c, + * GREATEST(a, NULL, b, NULL, c) = GREATEST(a, b, c) + * and likewise for LEAST. The interleaved NULLs must have no effect. Both + * operands are non-NULL so plain = is sound here. + */ +PG_PROP(greatest_least_null_skip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + initStringInfo(&c); + draw_arg_nonnull(tc, &a); + draw_arg_nonnull(tc, &b); + draw_arg_nonnull(tc, &c); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT greatest(%s, NULL::int, %s, NULL::int, %s) = " + "greatest(%s, %s, %s) " + "AND least(%s, NULL::int, %s, NULL::int, %s) = " + "least(%s, %s, %s)", + a.data, b.data, c.data, + a.data, b.data, c.data, + a.data, b.data, c.data, + a.data, b.data, c.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * GREATEST (and LEAST) over all-NULL arguments is NULL: every argument is + * skipped, leaving nothing, and the specified result is NULL. + */ +PG_PROP(greatest_all_null) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT greatest(NULL::int, NULL::int) IS NULL " + "AND least(NULL::int, NULL::int) IS NULL"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A searched CASE with no ELSE and no true WHEN yields NULL. A statically + * false WHEN is NULL, and a WHEN whose predicate is false for every drawn a in + * the small range (a > 1000000) is also never taken, so both CASEs are NULL. + */ +PG_PROP(case_else_omitted_null) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arg(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (CASE WHEN false THEN 1 END) IS NULL " + "AND (CASE WHEN (%s > 1000000) THEN 1 END) IS NULL", + a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * COALESCE flattens: COALESCE(a, COALESCE(b, c)) returns the first non-NULL of + * a, b, c, exactly as the flat COALESCE(a, b, c) does. IS NOT DISTINCT FROM + * makes the all-NULL case (both sides NULL) pass. + */ +PG_PROP(nested_coalesce) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + initStringInfo(&c); + draw_arg(tc, &a); + draw_arg(tc, &b); + draw_arg(tc, &c); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(%s, coalesce(%s, %s)) " + "IS NOT DISTINCT FROM coalesce(%s, %s, %s)", + a.data, b.data, c.data, + a.data, b.data, c.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 9f731ecb257e0..81914b04c80af 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From e764659035f8f622bfdd975a5f3c51bfb653335d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:03 -0400 Subject: [PATCH 069/338] prop: add coalesce property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_coalesce.c | 322 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 326 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_coalesce.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 33f37cb9c3cc9..7e504a4703c27 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -84,7 +84,8 @@ OBJS = \ prop_booltest.o \ prop_boolexpr.o \ prop_casewhen.o \ - prop_condexpr.o + prop_condexpr.o \ + prop_coalesce.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 836bea96aa2b0..82d334fdd5bbf 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -82,6 +82,7 @@ prop_area_sources = files( 'prop_boolexpr.c', 'prop_casewhen.c', 'prop_condexpr.c', + 'prop_coalesce.c', ) diff --git a/src/test/prop/prop_coalesce.c b/src/test/prop/prop_coalesce.c new file mode 100644 index 0000000000000..3deef9a28a949 --- /dev/null +++ b/src/test/prop/prop_coalesce.c @@ -0,0 +1,322 @@ +/*------------------------------------------------------------------------- + * + * prop_coalesce.c + * Property-based tests for PostgreSQL's NULL-handling expressions. + * + * COALESCE, NULLIF, GREATEST, LEAST and CASE each have precise, specified + * semantics: COALESCE returns the first non-NULL argument; NULLIF returns NULL + * when its two arguments are equal (by SQL = semantics) and the first argument + * otherwise; GREATEST and LEAST skip NULL arguments and return NULL only when + * every argument is NULL; a searched CASE yields the result of its first true + * WHEN, or the ELSE (NULL absent) when none matches. These properties pin + * those semantics down against explicit CASE / aggregate oracles. + * + * The oracle is an independent reconstruction of each operator's definition: + * a hand-written searched CASE for COALESCE, the = operator for NULLIF, and + * the max()/min() aggregates (which skip NULLs just like GREATEST/LEAST) over + * a VALUES list. Every result comparison uses IS NOT DISTINCT FROM so a NULL + * result on both sides compares equal; the NULLIF oracle uses plain = to match + * NULLIF's own equality semantics. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_coalesce.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "coalesce" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw one argument into buf: with probability ~1/4 the literal SQL NULL, + * otherwise a small integer in [-50, 50]. Every argument is cast to ::int so + * the expression's type resolves even when all arguments are NULL (otherwise + * COALESCE/GREATEST/LEAST over bare NULLs raise "could not determine type of + * NULL"). + */ +static void +draw_arg(pg_prop_tc *tc, StringInfo buf) +{ + bool is_null = (pg_prop_draw_int(tc, 0, 3) == 0); + + if (is_null) + appendStringInfoString(buf, "NULL::int"); + else + appendStringInfo(buf, "(%d)::int", + (int) pg_prop_draw_int(tc, -50, 50)); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the expression did not behave as the oracle demands, + * which is itself a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * COALESCE returns the first non-NULL argument. An explicit searched CASE + * with one WHEN ... IS NOT NULL per argument computes exactly that; the two + * must agree. IS NOT DISTINCT FROM makes the all-NULL case (both sides NULL) + * pass. + */ +PG_PROP(coalesce_first_nonnull) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a1, + a2, + a3, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a1); + initStringInfo(&a2); + initStringInfo(&a3); + draw_arg(tc, &a1); + draw_arg(tc, &a2); + draw_arg(tc, &a3); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(%s, %s, %s) IS NOT DISTINCT FROM " + "CASE WHEN %s IS NOT NULL THEN %s " + "WHEN %s IS NOT NULL THEN %s " + "WHEN %s IS NOT NULL THEN %s END", + a1.data, a2.data, a3.data, + a1.data, a1.data, + a2.data, a2.data, + a3.data, a3.data); + + require_true(tc, conn, q.data); + + pfree(a1.data); + pfree(a2.data); + pfree(a3.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * COALESCE is idempotent and absorbs a leading NULL: coalesce(a, a) = a and + * coalesce(NULL, a) = a, for any a (including a = NULL, handled by IS NOT + * DISTINCT FROM). + */ +PG_PROP(coalesce_idempotent_absorb) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arg(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (coalesce(%s, %s) IS NOT DISTINCT FROM %s) " + "AND (coalesce(NULL::int, %s) IS NOT DISTINCT FROM %s)", + a.data, a.data, a.data, + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * NULLIF(a, b) is NULL when a = b and a otherwise. The oracle uses plain = + * (not IS NOT DISTINCT FROM) because NULLIF decides on SQL equality: if a or b + * is NULL then a = b is NULL, the CASE WHEN is not true, and NULLIF returns a + * -- which this oracle reproduces exactly. + */ +PG_PROP(nullif_semantics) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_arg(tc, &a); + draw_arg(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT nullif(%s, %s) IS NOT DISTINCT FROM " + "CASE WHEN %s = %s THEN NULL ELSE %s END", + a.data, b.data, + a.data, b.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * GREATEST and LEAST skip NULL arguments and return NULL only when every + * argument is NULL. The max()/min() aggregates over a VALUES list of the same + * arguments skip NULLs identically, so they are an exact independent oracle. + */ +PG_PROP(greatest_least_skip_nulls) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a1, + a2, + a3, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a1); + initStringInfo(&a2); + initStringInfo(&a3); + draw_arg(tc, &a1); + draw_arg(tc, &a2); + draw_arg(tc, &a3); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT greatest(%s, %s, %s) IS NOT DISTINCT FROM " + "(SELECT max(v) FROM (VALUES (%s), (%s), (%s)) t(v)) " + "AND least(%s, %s, %s) IS NOT DISTINCT FROM " + "(SELECT min(v) FROM (VALUES (%s), (%s), (%s)) t(v))", + a1.data, a2.data, a3.data, + a1.data, a2.data, a3.data, + a1.data, a2.data, a3.data, + a1.data, a2.data, a3.data); + + require_true(tc, conn, q.data); + + pfree(a1.data); + pfree(a2.data); + pfree(a3.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * When at least one argument is non-NULL, GREATEST >= LEAST. All-NULL yields + * NULL on both sides, the comparison is NULL, and coalesce(..., true) makes + * that case pass -- so the property holds unconditionally. + */ +PG_PROP(greatest_ge_least) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a1, + a2, + a3, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a1); + initStringInfo(&a2); + initStringInfo(&a3); + draw_arg(tc, &a1); + draw_arg(tc, &a2); + draw_arg(tc, &a3); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(greatest(%s, %s, %s) >= " + "least(%s, %s, %s), true)", + a1.data, a2.data, a3.data, + a1.data, a2.data, a3.data); + + require_true(tc, conn, q.data); + + pfree(a1.data); + pfree(a2.data); + pfree(a3.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A searched CASE whose WHENs are all false yields its ELSE expression, or + * NULL when no ELSE is present. CASE WHEN false THEN 1 ELSE a END = a, and + * CASE WHEN false THEN 1 END IS NULL. + */ +PG_PROP(case_exhaustive_matches_coalesce) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arg(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (CASE WHEN false THEN 1 ELSE %s END) " + "IS NOT DISTINCT FROM %s " + "AND (CASE WHEN false THEN 1 END) IS NULL", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 81914b04c80af..9840e70fc6baa 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From cefe2ccadd00d313f592596c94c88b4ba06ea7d0 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:03 -0400 Subject: [PATCH 070/338] prop: add COALESCE/NULLIF/GREATEST/LEAST deeper laws property tests (coalescefn) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_coalescefn.c | 377 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 381 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_coalescefn.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 7e504a4703c27..efa88344612c0 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -85,7 +85,8 @@ OBJS = \ prop_boolexpr.o \ prop_casewhen.o \ prop_condexpr.o \ - prop_coalesce.o + prop_coalesce.o \ + prop_coalescefn.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 82d334fdd5bbf..ab5249c18b844 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -83,6 +83,7 @@ prop_area_sources = files( 'prop_casewhen.c', 'prop_condexpr.c', 'prop_coalesce.c', + 'prop_coalescefn.c', ) diff --git a/src/test/prop/prop_coalescefn.c b/src/test/prop/prop_coalescefn.c new file mode 100644 index 0000000000000..184c4bfc3594c --- /dev/null +++ b/src/test/prop/prop_coalescefn.c @@ -0,0 +1,377 @@ +/*------------------------------------------------------------------------- + * + * prop_coalescefn.c + * Property-based tests for deeper COALESCE/NULLIF/GREATEST/LEAST laws. + * + * The first-match and NULL-skip semantics of these operators imply a layer of + * algebraic laws beyond the one-step definitions checked in prop_coalesce: + * COALESCE is associative and absorbs NULL on either side; NULLIF composes + * with COALESCE so that COALESCE(NULLIF(a,b),b) collapses back to + * COALESCE(a,b); GREATEST is idempotent and both GREATEST and LEAST ignore + * NULL arguments entirely, so dropping a NULL argument changes nothing and + * LEAST never exceeds GREATEST. A failure of any of these is a bug in the + * operator, not an artefact of the test. + * + * The oracle is each law stated as a SQL boolean that must be "t" for every + * input: identities compared with IS NOT DISTINCT FROM (so NULL = NULL passes) + * and ordering stated with <= over non-NULL draws. No invented formulas -- + * every right-hand side is a rearrangement of the same operators or a direct + * consequence of their published semantics. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_coalescefn.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "coalescefn" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw one int-or-NULL argument into buf: with probability ~30% the literal + * SQL NULL, otherwise a small integer in [-50, 50]. Each argument is cast to + * ::int so the expression type resolves even when every argument is NULL, and + * a negative value is wrapped in parens as "(%d)::int" so a leading sign never + * misparses when substituted into a larger expression. + */ +static void +draw_arg(pg_prop_tc *tc, StringInfo buf) +{ + bool is_null = (pg_prop_draw_int(tc, 0, 9) < 3); + + if (is_null) + appendStringInfoString(buf, "NULL::int"); + else + appendStringInfo(buf, "(%d)::int", + (int) pg_prop_draw_int(tc, -50, 50)); +} + +/* + * Draw a guaranteed-non-NULL int argument into buf, for the ordering and + * idempotence laws that are only stated over non-NULL values. + */ +static void +draw_arg_nonnull(pg_prop_tc *tc, StringInfo buf) +{ + appendStringInfo(buf, "(%d)::int", + (int) pg_prop_draw_int(tc, -50, 50)); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the law did not hold as the oracle demands, which is + * itself a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * COALESCE is associative: nesting the fold left or right over three + * arguments yields the same value. Both arrangements return the first + * non-NULL of a, b, c (or NULL if all three are NULL), so they must agree; + * IS NOT DISTINCT FROM makes the all-NULL case pass. + */ +PG_PROP(coalesce_associative) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + initStringInfo(&c); + draw_arg(tc, &a); + draw_arg(tc, &b); + draw_arg(tc, &c); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(%s, coalesce(%s, %s)) " + "IS NOT DISTINCT FROM " + "coalesce(coalesce(%s, %s), %s)", + a.data, b.data, c.data, + a.data, b.data, c.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * COALESCE absorbs a NULL on either side: coalesce(NULL, a) = a and + * coalesce(a, NULL) = a for any a (including a = NULL, where both sides are + * NULL and IS NOT DISTINCT FROM holds). + */ +PG_PROP(coalesce_absorbs_null) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arg(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (coalesce(NULL::int, %s) IS NOT DISTINCT FROM %s) " + "AND (coalesce(%s, NULL::int) IS NOT DISTINCT FROM %s)", + a.data, a.data, + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * coalesce(a, b) returns a when a is non-NULL and b otherwise -- exactly the + * value of CASE WHEN a IS NOT NULL THEN a ELSE b END. The two must agree for + * all a, b; IS NOT DISTINCT FROM handles the both-NULL case. + */ +PG_PROP(coalesce_first_nonnull_wins) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_arg(tc, &a); + draw_arg(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(%s, %s) IS NOT DISTINCT FROM " + "(CASE WHEN %s IS NOT NULL THEN %s ELSE %s END)", + a.data, b.data, + a.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * NULLIF composes with COALESCE to cancel back to COALESCE: + * coalesce(nullif(a, b), b) IS NOT DISTINCT FROM coalesce(a, b). + * + * Case a = b (both non-NULL): nullif(a,b)=NULL so the left is b, and the + * right is a = b -- equal. Case a <> b: nullif(a,b)=a so both sides are a. + * Case a NULL: nullif(NULL,b)=NULL so the left is b, and coalesce(NULL,b)=b. + * In every case the two sides match, so the law holds unconditionally. + */ +PG_PROP(nullif_coalesce_cancel) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_arg(tc, &a); + draw_arg(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(nullif(%s, %s), %s) " + "IS NOT DISTINCT FROM coalesce(%s, %s)", + a.data, b.data, b.data, + a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * GREATEST is idempotent: greatest(a, a) = a for non-NULL a. Drawn non-NULL + * so the result is well defined; IS NOT DISTINCT FROM is used for uniformity. + */ +PG_PROP(greatest_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arg_nonnull(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT greatest(%s, %s) IS NOT DISTINCT FROM %s", + a.data, a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * For non-NULL a, b, c the minimum never exceeds the maximum: + * least(a, b, c) <= greatest(a, b, c). All arguments are drawn non-NULL so + * both sides are non-NULL and the comparison is a plain boolean. + */ +PG_PROP(least_le_greatest) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + initStringInfo(&c); + draw_arg_nonnull(tc, &a); + draw_arg_nonnull(tc, &b); + draw_arg_nonnull(tc, &c); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT least(%s, %s, %s) <= greatest(%s, %s, %s)", + a.data, b.data, c.data, + a.data, b.data, c.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * GREATEST skips NULL arguments, so a NULL in the middle changes nothing: + * greatest(a, NULL, b) = greatest(a, b) for non-NULL a, b. Both are non-NULL + * so the result is well defined; IS NOT DISTINCT FROM is used for uniformity. + */ +PG_PROP(greatest_null_skip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_arg_nonnull(tc, &a); + draw_arg_nonnull(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT greatest(%s, NULL::int, %s) " + "IS NOT DISTINCT FROM greatest(%s, %s)", + a.data, b.data, + a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * COALESCE over nothing but NULLs is NULL: coalesce(NULL, NULL, NULL) IS NULL. + * A fixed law with no drawn inputs -- it must hold on every run. + */ +PG_PROP(coalesce_all_null) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT coalesce(NULL::int, NULL::int, NULL::int) IS NULL"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 9840e70fc6baa..493f622681a26 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From b59daf43b6591ed0e6f977bc182d027460244903 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:03 -0400 Subject: [PATCH 071/338] prop: add three-valued logic and NULL propagation property tests (nullsemantics) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_nullsemantics.c | 326 +++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 330 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_nullsemantics.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index efa88344612c0..be00ac70b2dd9 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -86,7 +86,8 @@ OBJS = \ prop_casewhen.o \ prop_condexpr.o \ prop_coalesce.o \ - prop_coalescefn.o + prop_coalescefn.o \ + prop_nullsemantics.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index ab5249c18b844..456768452c2ac 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -84,6 +84,7 @@ prop_area_sources = files( 'prop_condexpr.c', 'prop_coalesce.c', 'prop_coalescefn.c', + 'prop_nullsemantics.c', ) diff --git a/src/test/prop/prop_nullsemantics.c b/src/test/prop/prop_nullsemantics.c new file mode 100644 index 0000000000000..2379c99dabdea --- /dev/null +++ b/src/test/prop/prop_nullsemantics.c @@ -0,0 +1,326 @@ +/*------------------------------------------------------------------------- + * + * prop_nullsemantics.c + * Property-based tests for SQL three-valued logic and NULL propagation. + * + * SQL boolean expressions evaluate in three-valued logic (true/false/unknown), + * and NULL propagates through comparisons, IN, NOT IN, and boolean aggregates + * in ways that are easy to get subtly wrong in the executor. Each property + * builds a scalar boolean oracle: a SELECT that must return "t" for every + * drawn combination of bool-or-NULL and int-or-NULL operands. Where both + * sides of a comparison can themselves be NULL we compare with IS NOT DISTINCT + * FROM (NULL-safe equality) so the oracle never collapses to NULL on a + * boundary; a "t" is required, so an "f", NULL, or SQL error is a failure. + * + * A failure here is a bug in 3VL evaluation or NULL propagation, not a + * tolerance artefact: these identities are exact for every input. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_nullsemantics.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "nullsemantics" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a boolean-or-NULL literal: one of "true", "false", "NULL::boolean". + * The cast on NULL keeps the operand well-typed in any context. + */ +static void +draw_bool_null(pg_prop_tc *tc, StringInfo buf) +{ + int k = (int) pg_prop_draw_int(tc, 0, 2); + + if (k == 0) + appendStringInfoString(buf, "NULL::boolean"); + else if (k == 1) + appendStringInfoString(buf, "true"); + else + appendStringInfoString(buf, "false"); +} + +/* + * Draw an integer-or-NULL literal from a small domain, ~25% NULL. Negative + * integers are emitted parenthesized and cast as "(%d)" so a leading sign + * never misparses when substituted into a larger expression. + */ +static void +draw_int_null(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 3) == 0) + appendStringInfoString(buf, "NULL::int"); + else + appendStringInfo(buf, "(%d)", (int) pg_prop_draw_int(tc, -5, 5)); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the oracle produced unknown or errored on inputs that must + * yield true, which is itself a failure -- reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * De Morgan in 3VL: NOT (a AND b) is equivalent to (NOT a) OR (NOT b) for a,b + * ranging over true/false/unknown. Both sides can be NULL (when a or b is + * NULL), so compare with IS NOT DISTINCT FROM. + */ +PG_PROP(and_or_demorgan_3vl) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_bool_null(tc, &a); + draw_bool_null(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (NOT (%s AND %s)) " + "IS NOT DISTINCT FROM ((NOT %s) OR (NOT %s))", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * IS NULL is the exact complement of IS NOT NULL: (x IS NULL) = NOT (x IS NOT + * NULL). Both IS NULL and IS NOT NULL always return a non-NULL boolean, so a + * plain = is sound here. + */ +PG_PROP(is_null_complement) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_int_null(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s IS NULL) = (NOT (%s IS NOT NULL))", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Reflexive equality is NULL exactly when x is NULL, and true otherwise: + * (x = x) IS NOT DISTINCT FROM (CASE WHEN x IS NULL THEN NULL ELSE true END). + * The left side is NULL when x is NULL, so IS NOT DISTINCT FROM is required. + */ +PG_PROP(null_eq_is_null) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_int_null(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s = %s) IS NOT DISTINCT FROM " + "(CASE WHEN %s IS NULL THEN NULL ELSE true END)", + x.data, x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * IS NOT DISTINCT FROM equals coalesce(a = b, a IS NULL AND b IS NULL): a = + * b is NULL when either operand is NULL, and in that case the second argument + * supplies the NULL-safe answer (true only when both are NULL). coalesce + * never returns NULL here because its second argument is a non-NULL boolean, + * so the two always-non-NULL sides compare with a plain =. + */ +PG_PROP(coalesce_vs_is_not_distinct) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_int_null(tc, &a); + draw_int_null(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s IS NOT DISTINCT FROM %s) = " + "coalesce(%s = %s, %s IS NULL AND %s IS NULL)", + a.data, b.data, a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * IN is the 3VL OR-of-equals: (x IN (l1, l2)) is equivalent to (x = l1 OR x = + * l2), including when any operand is NULL (then both can be NULL/unknown). + * Compare with IS NOT DISTINCT FROM because both sides can be NULL. + */ +PG_PROP(in_with_null) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + l1, + l2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&l1); + initStringInfo(&l2); + draw_int_null(tc, &x); + draw_int_null(tc, &l1); + draw_int_null(tc, &l2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s IN (%s, %s)) IS NOT DISTINCT FROM " + "(%s = %s OR %s = %s)", + x.data, l1.data, l2.data, + x.data, l1.data, x.data, l2.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(l1.data); + pfree(l2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The classic NOT IN NULL trap: when the list contains NULL and x does not + * match any non-NULL item, the result is NULL (unknown), not true. We draw + * two distinct non-NULL integers x and k (redrawing k until x <> k) so x is + * guaranteed not to equal the only non-NULL list element, then require + * (x NOT IN (k, NULL)) IS NULL -- which is itself always a non-NULL boolean, + * so a plain comparison to "t" is sound. + */ +PG_PROP(not_in_null_is_unknown) +{ + pg_prop_conn *conn; + char *err = NULL; + int x, + k; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x = (int) pg_prop_draw_int(tc, -5, 5); + /* k != x without a reject loop: add a non-zero offset */ + k = x + (int) pg_prop_draw_int(tc, 1, 10); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%d) NOT IN ((%d), NULL)) IS NULL", + x, k); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Constant 3VL truth-table check for AND/OR with unknown: true AND NULL is + * unknown; false AND NULL is false (determined); true OR NULL is true + * (determined); false OR NULL is unknown. No drawn inputs -- a fixed sanity + * oracle guarding the determining-vs-propagating cases of boolean 3VL. + */ +PG_PROP(boolean_agg_3vl) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT (true AND NULL) IS NULL " + "AND (false AND NULL) = false " + "AND (true OR NULL) = true " + "AND (false OR NULL) IS NULL"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 493f622681a26..2574418024d84 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From fea83e78ebfeaefbe4da4909d9be63331857451e Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:03 -0400 Subject: [PATCH 072/338] prop: add UNIQUE NULLS DISTINCT and NULL grouping property tests (nulldistinct) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_nulldistinct.c | 292 ++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 296 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_nulldistinct.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index be00ac70b2dd9..213b22755f5a6 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -87,7 +87,8 @@ OBJS = \ prop_condexpr.o \ prop_coalesce.o \ prop_coalescefn.o \ - prop_nullsemantics.o + prop_nullsemantics.o \ + prop_nulldistinct.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 456768452c2ac..85d98ea414dc7 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -85,6 +85,7 @@ prop_area_sources = files( 'prop_coalesce.c', 'prop_coalescefn.c', 'prop_nullsemantics.c', + 'prop_nulldistinct.c', ) diff --git a/src/test/prop/prop_nulldistinct.c b/src/test/prop/prop_nulldistinct.c new file mode 100644 index 0000000000000..fc5deb19129cd --- /dev/null +++ b/src/test/prop/prop_nulldistinct.c @@ -0,0 +1,292 @@ +/*------------------------------------------------------------------------- + * + * prop_nulldistinct.c + * Property-based tests for NULL distinctness in DISTINCT / GROUP BY / + * UNIQUE NULLS [NOT] DISTINCT. + * + * SQL treats NULL as a single value for duplicate elimination (DISTINCT, + * GROUP BY) but as distinct for aggregation (count(col) skips NULL) and, by + * default, for UNIQUE constraints (NULLS DISTINCT, so many NULLs coexist). + * PostgreSQL 15 added UNIQUE NULLS NOT DISTINCT, which collapses NULLs so at + * most one NULL row is allowed. These properties pin that algebra: all NULLs + * form exactly one DISTINCT/GROUP group, count(col) excludes them, a default + * UNIQUE admits many NULLs, and NULLS NOT DISTINCT rejects the second. A + * mismatch is a bug in grouping, aggregation, or index null handling. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_nulldistinct.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "nulldistinct" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small domain for a: 0..3 so collisions are frequent, with ~30% NULL to + * exercise the single-null-group semantics hard. + */ +#define VAL_MIN 0 +#define VAL_MAX 3 + +/* Draw a value for column a: SQL NULL ~30% of the time, else 0..3. */ +static void +draw_val(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 9) < 3) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); +} + +/* + * (Re)build nd_t(id int PRIMARY KEY, a int) with 0..25 random rows. id is + * 0..nrows-1 so it is a total key; a is 0..3 or NULL (~30% NULL). Fresh in + * every property so cases never see each other's rows. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS nd_t; " + "CREATE TABLE nd_t (id int PRIMARY KEY, a int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 25); + for (int r = 0; r < nrows; r++) + { + StringInfoData a; + + initStringInfo(&a); + draw_val(tc, &a); + appendStringInfo(&ddl, "INSERT INTO nd_t VALUES (%d, %s); ", r, a.data); + pfree(a.data); + } + appendStringInfoString(&ddl, "ANALYZE nd_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build nd_t failed: %s\n ddl: %s", + err ? err : "(no message)", ddl.data); + pfree(ddl.data); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error is itself a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Property 1 (shape A): SELECT DISTINCT a treats all NULLs as one value. The + * cardinality of SELECT DISTINCT a equals count(DISTINCT a) (which excludes + * NULL) plus one if any NULL row exists. + */ +PG_PROP(distinct_nulls_one_group) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *sql = + "SELECT (SELECT count(*) FROM (SELECT DISTINCT a FROM nd_t) s) " + "= (SELECT count(DISTINCT a) FROM nd_t) " + "+ (CASE WHEN EXISTS(SELECT 1 FROM nd_t WHERE a IS NULL) " + "THEN 1 ELSE 0 END)"; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + require_true(tc, conn, sql); + pg_prop_disconnect(conn); +} + +/* + * Property 2 (shape A): GROUP BY a puts all NULLs in one group, so the group + * count equals the SELECT DISTINCT a cardinality (property 1's LHS). + */ +PG_PROP(groupby_null_one_group) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *sql = + "SELECT (SELECT count(*) FROM (SELECT a FROM nd_t GROUP BY a) s) " + "= (SELECT count(*) FROM (SELECT DISTINCT a FROM nd_t) t)"; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + require_true(tc, conn, sql); + pg_prop_disconnect(conn); +} + +/* + * Property 3 (shape A): count(a) ignores NULLs, so it equals count(*) minus + * the number of NULL a rows. + */ +PG_PROP(count_col_excludes_null) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *sql = + "SELECT (SELECT count(a) FROM nd_t) " + "= (SELECT count(*) FROM nd_t) " + "- (SELECT count(*) FROM nd_t WHERE a IS NULL)"; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + require_true(tc, conn, sql); + pg_prop_disconnect(conn); +} + +/* + * Property 4: a default UNIQUE(a) constraint uses NULLS DISTINCT, so multiple + * NULL rows are allowed. Insert three NULL rows and verify all three persist. + * Scratch table nd_u is built and dropped within the property. + */ +PG_PROP(unique_nulls_distinct_allows_dups) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *build = + "DROP TABLE IF EXISTS nd_u; " + "CREATE TABLE nd_u (a int UNIQUE); " + "INSERT INTO nd_u VALUES (NULL); " + "INSERT INTO nd_u VALUES (NULL); " + "INSERT INTO nd_u VALUES (NULL); "; + const char *sql = "SELECT (SELECT count(*) FROM nd_u) = 3"; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, build, &err)) + pg_prop_fail(tc, "build nd_u failed: %s", err ? err : "(no message)"); + + require_true(tc, conn, sql); + + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS nd_u", &err); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} + +/* + * Property 5: UNIQUE NULLS NOT DISTINCT collapses NULLs, so at most one NULL + * row is allowed. Insert one NULL (must succeed), then a second NULL (must + * fail with a constraint violation), then verify exactly one NULL row remains. + * This asserts the second insert returns false (failed), which is the whole + * point of NULLS NOT DISTINCT. + */ +PG_PROP(unique_nulls_not_distinct_rejects) +{ + pg_prop_conn *conn; + char *err = NULL; + bool ok; + const char *build = + "DROP TABLE IF EXISTS nd_u2; " + "CREATE TABLE nd_u2 (a int UNIQUE NULLS NOT DISTINCT); "; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, build, &err)) + pg_prop_fail(tc, "build nd_u2 failed: %s", err ? err : "(no message)"); + if (err) + { + pg_free(err); + err = NULL; + } + + /* First NULL must succeed. */ + if (!pg_prop_sql_exec(conn, "INSERT INTO nd_u2 VALUES (NULL)", &err)) + pg_prop_fail(tc, "first NULL insert failed under NULLS NOT DISTINCT: %s", + err ? err : "(no message)"); + if (err) + { + pg_free(err); + err = NULL; + } + + /* Second NULL must be rejected by the unique constraint. */ + ok = pg_prop_sql_exec(conn, "INSERT INTO nd_u2 VALUES (NULL)", &err); + if (ok) + pg_prop_fail(tc, "second NULL insert succeeded under NULLS NOT DISTINCT"); + if (err) + { + pg_free(err); + err = NULL; + } + + /* + * The failed insert aborts its own implicit transaction, so the table + * still holds exactly one NULL row. pg_prop_sql_exec runs each statement + * in its own transaction, so the prior successful insert is durable. + */ + require_true(tc, conn, + "SELECT (SELECT count(*) FROM nd_u2 WHERE a IS NULL) = 1"); + + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS nd_u2", &err); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} + +/* + * Property 6 (shape B flavour via scalar count): SELECT DISTINCT ON (a) picks + * one row per distinct a with NULL as a single group, so its cardinality + * equals that of SELECT DISTINCT a. The ORDER BY a NULLS FIRST, id is a total + * order making the DISTINCT ON choice deterministic. + */ +PG_PROP(distinct_on_null) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *sql = + "SELECT (SELECT count(*) FROM " + "(SELECT DISTINCT ON (a) a FROM nd_t ORDER BY a NULLS FIRST, id) s) " + "= (SELECT count(*) FROM (SELECT DISTINCT a FROM nd_t) t)"; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + require_true(tc, conn, sql); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 2574418024d84..7a2e687f0f83c 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 228403de23ac2df565e456f9248455904b98219d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:04 -0400 Subject: [PATCH 073/338] prop: add oid property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_oid.c | 277 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 281 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_oid.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 213b22755f5a6..00fca5725572f 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -88,7 +88,8 @@ OBJS = \ prop_coalesce.o \ prop_coalescefn.o \ prop_nullsemantics.o \ - prop_nulldistinct.o + prop_nulldistinct.o \ + prop_oid.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 85d98ea414dc7..6efdbe838c241 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -86,6 +86,7 @@ prop_area_sources = files( 'prop_coalescefn.c', 'prop_nullsemantics.c', 'prop_nulldistinct.c', + 'prop_oid.c', ) diff --git a/src/test/prop/prop_oid.c b/src/test/prop/prop_oid.c new file mode 100644 index 0000000000000..a42354fd57ce7 --- /dev/null +++ b/src/test/prop/prop_oid.c @@ -0,0 +1,277 @@ +/*------------------------------------------------------------------------- + * + * prop_oid.c + * Property-based tests for the oid type and oidvector + * (src/backend/utils/adt/oid.c). + * + * oid is an unsigned 32-bit identifier. Its text I/O prints and parses the + * full unsigned range (values above 2^31 must appear as unsigned decimals, + * not negatives, and the range wraps/overflows at 2^32), and oidvector input + * parses a space-separated list of oids -- both have had fixes. These + * properties check unsigned text round-trip, that output is the unsigned form, + * that comparison is a total order under *unsigned* ordering, and that an + * oidvector round-trips through text and preserves its length. + * + * The oracle is the type's own text I/O and comparison: an oid formatted to + * text and parsed back must equal the original across the whole range + * (including 0 and 4294967295), the printed form of a value above 2^31 must be + * its unsigned decimal, comparison must be a total order with unsigned + * semantics, and an oidvector cast to text and back must be unchanged and keep + * its element count. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_oid.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "oid" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw one oid value. oid is unsigned 32-bit, so the full range + * 0..4294967295 is drawn -- the region above 2^31 is the whole point, since a + * signed misinterpretation would print or compare those values as negatives. + */ +static uint32 +draw_oid(pg_prop_tc *tc) +{ + return (uint32) pg_prop_draw_int(tc, 0, UINT32_MAX); +} + +/* + * Draw an oidvector literal into buf as a single-quoted, space-separated list + * of k oids in 0..6 elements, cast to oidvector. The literal contains only + * digits and spaces, so single-quoting is safe. Returns k via *k_out. + */ +static void +draw_oidvector(pg_prop_tc *tc, StringInfo buf, int *k_out) +{ + int k = (int) pg_prop_draw_int(tc, 0, 6); + + *k_out = k; + + appendStringInfoChar(buf, '\''); + for (int i = 0; i < k; i++) + { + if (i > 0) + appendStringInfoChar(buf, ' '); + appendStringInfo(buf, "%u", draw_oid(tc)); + } + appendStringInfoString(buf, "'::oidvector"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the query errored or produced NULL on valid inputs, which + * is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Text round-trip: an oid formatted to text and parsed back equals the + * original, across the whole range including 0 and 4294967295. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + uint32 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = draw_oid(tc); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%u::oid)::text::oid = %u::oid", n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Unsigned text: oid prints as an unsigned decimal. For n > 2147483647 the + * output must be the unsigned form (e.g. 4000000000, not -294967296). The + * expected text is built in C with %u, which is the unsigned decimal, and + * compared against the oid's own ::text output. + */ +PG_PROP(unsigned_text) +{ + pg_prop_conn *conn; + char *err = NULL; + uint32 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = draw_oid(tc); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%u::oid)::text = '%u'::text", n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Comparison is a total order: for any two oids exactly one of <, =, > holds, + * so the three predicates sum to 1. + */ +PG_PROP(comparison_total_order) +{ + pg_prop_conn *conn; + char *err = NULL; + uint32 o1, + o2; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + o1 = draw_oid(tc); + o2 = draw_oid(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%u::oid < %u::oid)::int " + "+ (%u::oid = %u::oid)::int " + "+ (%u::oid > %u::oid)::int) = 1", + o1, o2, o1, o2, o1, o2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Ordering is unsigned: a value above 2^31 compares greater than a small + * value. A signed misinterpretation would make the large value negative and + * thus smaller, so this directly targets unsigned-comparison correctness. + */ +PG_PROP(order_is_unsigned) +{ + pg_prop_conn *conn; + char *err = NULL; + uint32 s, + l; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + s = (uint32) pg_prop_draw_int(tc, 0, 1000); + l = (uint32) pg_prop_draw_int(tc, INT64_C(2147483648), UINT32_MAX); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%u::oid > %u::oid)", l, s); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * oidvector text round-trip: an oidvector cast to text and parsed back equals + * the original. Exercises oidvector output and input (space-separated oids). + */ +PG_PROP(oidvector_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int k; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_oidvector(tc, &v, &k); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)::text::oidvector = (%s)", + v.data, v.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * oidvector length: an oidvector built from k oids casts to oid[] with + * cardinality k. The empty oidvector (k = 0) has cardinality 0, so coalesce + * guards the NULL that cardinality() returns for an empty array. + */ +PG_PROP(oidvector_length_matches) +{ + pg_prop_conn *conn; + char *err = NULL; + int k; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_oidvector(tc, &v, &k); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT coalesce(cardinality((%s)::oid[]), 0) = %d", + v.data, k); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 7a2e687f0f83c..92d26991a25aa 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From bf5852f0b134453039eaf40136140f36a5399bf1 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:04 -0400 Subject: [PATCH 074/338] prop: add uuid property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_uuid.c | 304 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 308 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_uuid.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 00fca5725572f..c067d8774fa15 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -89,7 +89,8 @@ OBJS = \ prop_coalescefn.o \ prop_nullsemantics.o \ prop_nulldistinct.o \ - prop_oid.o + prop_oid.o \ + prop_uuid.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 6efdbe838c241..26e409ee832f6 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -87,6 +87,7 @@ prop_area_sources = files( 'prop_nullsemantics.c', 'prop_nulldistinct.c', 'prop_oid.c', + 'prop_uuid.c', ) diff --git a/src/test/prop/prop_uuid.c b/src/test/prop/prop_uuid.c new file mode 100644 index 0000000000000..943bd7be1753f --- /dev/null +++ b/src/test/prop/prop_uuid.c @@ -0,0 +1,304 @@ +/*------------------------------------------------------------------------- + * + * prop_uuid.c + * Property-based tests for the uuid type. + * + * uuid in/out parsing (dashes, braces, case) and ordering have had fixes; + * these check that parsing is tolerant and canonicalizing, and that + * comparison is a total order consistent with the byte string. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_uuid.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "uuid" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw 16 random bytes and render them as a 32-character lowercase hex string + * (no dashes) into "hex", a caller-provided buffer of at least 33 bytes. Each + * byte is drawn with pg_prop_draw_int(0, 255) and printed "%02x", so the + * result is always fixed-width and lowercase. + */ +static void +draw_uuid_hex(pg_prop_tc *tc, char *hex) +{ + int i; + + for (i = 0; i < 16; i++) + { + int b = (int) pg_prop_draw_int(tc, 0, 255); + + snprintf(hex + i * 2, 3, "%02x", b); + } + hex[32] = '\0'; +} + +/* + * Insert dashes into a 32-char hex string to form the canonical 8-4-4-4-12 + * dashed UUID text, appended to buf. Dashes go after hex positions 8, 12, 16 + * and 20. + */ +static void +append_dashed(StringInfo buf, const char *hex) +{ + appendBinaryStringInfo(buf, hex, 8); + appendStringInfoChar(buf, '-'); + appendBinaryStringInfo(buf, hex + 8, 4); + appendStringInfoChar(buf, '-'); + appendBinaryStringInfo(buf, hex + 12, 4); + appendStringInfoChar(buf, '-'); + appendBinaryStringInfo(buf, hex + 16, 4); + appendStringInfoChar(buf, '-'); + appendBinaryStringInfo(buf, hex + 20, 12); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a validly-generated value parsed to NULL or errored, which + * is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Text round-trip: parsing a canonical dashed uuid and printing it yields the + * lowercase canonical form. Since the drawn literal U is already lowercase + * canonical, (U::uuid)::text must equal lower(U). This exercises uuid_in then + * uuid_out. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex[33]; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_uuid_hex(tc, hex); + initStringInfo(&d); + append_dashed(&d, hex); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ('%s'::uuid)::text = lower('%s')", + d.data, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Dash-insensitive parse: uuid_in accepts both the 32-hex form without dashes + * and the standard dashed form, and they denote the same value. So H::uuid = + * D::uuid where H and D come from the same bytes. + */ +PG_PROP(dash_insensitive_parse) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex[33]; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_uuid_hex(tc, hex); + initStringInfo(&d); + append_dashed(&d, hex); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ('%s'::uuid) = ('%s'::uuid)", + hex, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Case-insensitive parse: uuid_in treats the hex digits case-insensitively, so + * the lowercase dashed form and its uppercase version denote the same value. + * upper() produces the uppercase form in SQL so the C side keeps one encoding. + */ +PG_PROP(case_insensitive_parse) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex[33]; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_uuid_hex(tc, hex); + initStringInfo(&d); + append_dashed(&d, hex); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ('%s'::uuid) = (upper('%s')::uuid)", + d.data, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Brace-form parse: uuid_in accepts the {dashed} braced form and it denotes + * the same value as the bare dashed form. + */ +PG_PROP(brace_form_parse) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex[33]; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_uuid_hex(tc, hex); + initStringInfo(&d); + append_dashed(&d, hex); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ('%s'::uuid) = ('{%s}'::uuid)", + d.data, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Total order: for two independently drawn uuids exactly one of <, =, > holds, + * so the sum of those three booleans cast to int is 1. This checks uuid + * comparison is a total order (trichotomy). + */ +PG_PROP(comparison_total_order) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex1[33], + hex2[33]; + StringInfoData d1, + d2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_uuid_hex(tc, hex1); + draw_uuid_hex(tc, hex2); + initStringInfo(&d1); + initStringInfo(&d2); + append_dashed(&d1, hex1); + append_dashed(&d2, hex2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (('%s'::uuid < '%s'::uuid)::int " + "+ ('%s'::uuid = '%s'::uuid)::int " + "+ ('%s'::uuid > '%s'::uuid)::int) = 1", + d1.data, d2.data, d1.data, d2.data, d1.data, d2.data); + + require_true(tc, conn, q.data); + + pfree(d1.data); + pfree(d2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Order matches the hex string: uuid comparison must agree with the + * lexicographic comparison of the canonical lowercase 32-char hex (no dashes). + * Both hex strings are lowercase fixed-width, so SQL text "<" is the right + * oracle, and the dashed and raw forms are built from the same bytes. + */ +PG_PROP(order_matches_hex_string) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex1[33], + hex2[33]; + StringInfoData d1, + d2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_uuid_hex(tc, hex1); + draw_uuid_hex(tc, hex2); + initStringInfo(&d1); + initStringInfo(&d2); + append_dashed(&d1, hex1); + append_dashed(&d2, hex2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('%s'::uuid < '%s'::uuid) = ('%s' < '%s')", + d1.data, d2.data, hex1, hex2); + + require_true(tc, conn, q.data); + + pfree(d1.data); + pfree(d2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 92d26991a25aa..35d7489b89784 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From d07b4d9d776e3fc5bfb3f5482384b613784b64d8 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:04 -0400 Subject: [PATCH 075/338] prop: add UUID ordering, round-trip, and comparison property tests (uuidops) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_uuidops.c | 270 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 274 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_uuidops.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index c067d8774fa15..aeebd45615eb8 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -90,7 +90,8 @@ OBJS = \ prop_nullsemantics.o \ prop_nulldistinct.o \ prop_oid.o \ - prop_uuid.o + prop_uuid.o \ + prop_uuidops.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 26e409ee832f6..aefdc16972e96 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -88,6 +88,7 @@ prop_area_sources = files( 'prop_nulldistinct.c', 'prop_oid.c', 'prop_uuid.c', + 'prop_uuidops.c', ) diff --git a/src/test/prop/prop_uuidops.c b/src/test/prop/prop_uuidops.c new file mode 100644 index 0000000000000..ab636d5ff06d7 --- /dev/null +++ b/src/test/prop/prop_uuidops.c @@ -0,0 +1,270 @@ +/*------------------------------------------------------------------------- + * + * prop_uuidops.c + * Property-based tests for uuid ordering, text round-trip and comparison. + * + * The oracle is the uuid type's own algebra: output is the canonical + * lowercase 8-4-4-4-12 text, comparison is a total order equal to the + * lexicographic order of that canonical text, and text <-> uuid round-trips + * exactly. A failure means uuid in/out or comparison disagrees with the + * documented canonical form -- a bug in the type, not a tolerance artefact. + * This complements the "uuid" area (byte-oriented) by driving uuids built + * from drawn hex nibbles in the canonical dashed form, and by asserting the + * order-matches-text and nil-is-minimum laws. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_uuidops.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "uuidops" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw 32 hex nibbles (0-9a-f) and format them into the canonical dashed + * 8-4-4-4-12 uuid form into "buf" (at least 37 bytes). Lowercase hex. + */ +static void +draw_uuid_canon(pg_prop_tc *tc, char *buf) +{ + static const char hexdig[16] = "0123456789abcdef"; + int i; + int p = 0; + + for (i = 0; i < 32; i++) + { + if (i == 8 || i == 12 || i == 16 || i == 20) + buf[p++] = '-'; + buf[p++] = hexdig[(int) pg_prop_draw_int(tc, 0, 15)]; + } + buf[p] = '\0'; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error is itself a bug (uuid I/O or comparison should never error or + * yield NULL on these finite, valid inputs), so either is a failure. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * text_roundtrip: (U)::text::uuid = U. Casting a uuid out to text and back + * must reproduce the same uuid. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + char u[37]; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_uuid_canon(tc, u); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('%s'::uuid)::text::uuid = '%s'::uuid", + u, u); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * case_insensitive: the lower-case and upper-case hex forms of one uuid parse + * to the same uuid. Build the upper-case form in C so the two literals + * differ only in letter case. + */ +PG_PROP(case_insensitive) +{ + pg_prop_conn *conn; + char *err = NULL; + char lo[37]; + char up[37]; + int i; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_uuid_canon(tc, lo); + for (i = 0; lo[i] != '\0'; i++) + { + if (lo[i] >= 'a' && lo[i] <= 'f') + up[i] = (char) (lo[i] - 'a' + 'A'); + else + up[i] = lo[i]; + } + up[i] = '\0'; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT '%s'::uuid = '%s'::uuid", + lo, up); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * total_order: for two drawn uuids A, B exactly one of A < B, A = B, A > B + * holds. (num of true among the three predicates) = 1. + */ +PG_PROP(total_order) +{ + pg_prop_conn *conn; + char *err = NULL; + char a[37]; + char b[37]; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_uuid_canon(tc, a); + draw_uuid_canon(tc, b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (('%s'::uuid < '%s'::uuid)::int " + "+ ('%s'::uuid = '%s'::uuid)::int " + "+ ('%s'::uuid > '%s'::uuid)::int) = 1", + a, b, a, b, a, b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * self_equal: U = U AND NOT (U <> U). Reflexivity of equality and its + * agreement with the <> operator. + */ +PG_PROP(self_equal) +{ + pg_prop_conn *conn; + char *err = NULL; + char u[37]; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_uuid_canon(tc, u); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('%s'::uuid = '%s'::uuid) " + "AND NOT ('%s'::uuid <> '%s'::uuid)", + u, u, u, u); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * order_matches_text: (A < B) = (A::text < B::text). uuid comparison is the + * lexicographic order of the canonical lowercase hex text; since output is + * always canonical lowercase, the uuid order must agree with text order. + */ +PG_PROP(order_matches_text) +{ + pg_prop_conn *conn; + char *err = NULL; + char a[37]; + char b[37]; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_uuid_canon(tc, a); + draw_uuid_canon(tc, b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('%s'::uuid < '%s'::uuid) " + "= (('%s'::uuid)::text < ('%s'::uuid)::text)", + a, b, a, b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * nil_is_min: the nil uuid (all zero) is <= any drawn uuid. The all-zero + * value is the minimum of the total order. + */ +PG_PROP(nil_is_min) +{ + pg_prop_conn *conn; + char *err = NULL; + char u[37]; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_uuid_canon(tc, u); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT '00000000-0000-0000-0000-000000000000'::uuid " + "<= '%s'::uuid", + u); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 35d7489b89784..63cd644c6e1e1 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From d501b4ce71245b6d9297e174a451f7062975d1ea Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:04 -0400 Subject: [PATCH 076/338] prop: add bytea property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_bytea.c | 320 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 324 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_bytea.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index aeebd45615eb8..d5b80a7112dcd 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -91,7 +91,8 @@ OBJS = \ prop_nulldistinct.o \ prop_oid.o \ prop_uuid.o \ - prop_uuidops.o + prop_uuidops.o \ + prop_bytea.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index aefdc16972e96..f11133411e074 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -89,6 +89,7 @@ prop_area_sources = files( 'prop_oid.c', 'prop_uuid.c', 'prop_uuidops.c', + 'prop_bytea.c', ) diff --git a/src/test/prop/prop_bytea.c b/src/test/prop/prop_bytea.c new file mode 100644 index 0000000000000..d2ff02f62ca54 --- /dev/null +++ b/src/test/prop/prop_bytea.c @@ -0,0 +1,320 @@ +/*------------------------------------------------------------------------- + * + * prop_bytea.c + * Property-based tests for bytea and encode()/decode(). + * + * encode()/decode() (hex, base64, escape) and bytea operations are codecs + * with exact round-trip properties plus length/concatenation laws; this + * exercises all three encodings over arbitrary bytes. The targets are + * src/backend/utils/adt/encode.c (the hex/base64/escape codecs) and + * varlena.c (bytea length, concatenation and substring). + * + * The oracle is the type's own algebra: for every encoding, decode . encode + * is the identity on bytea; length() of a bytea is its byte count; + * concatenation adds lengths; and splitting a bytea at any point and + * rejoining the pieces reconstructs it. None of these admit any tolerance, + * so any "f" result is a bug in the codec or in the bytea operators. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_bytea.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "bytea" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw 0..128 arbitrary bytes and append a bytea literal for them into buf, + * in the hex input format: '\x'::bytea with two hex chars per + * byte. The leading sequence is a literal backslash followed by 'x', which + * is how bytea's hex input is written under the default + * standard_conforming_strings=on (the backslash is a literal character in the + * single-quoted string, not an escape). The empty bytea '\x'::bytea is a + * valid and interesting case and is included (nbytes may be 0). + * + * *nbytes receives the number of bytes drawn, so a property can compare SQL + * length() against the exact count it generated. + */ +static void +draw_bytea_literal(pg_prop_tc *tc, StringInfo buf, size_t *nbytes) +{ + size_t len; + unsigned char *bytes = pg_prop_draw_bytes(tc, 0, 128, &len); + + appendStringInfoString(buf, "'\\x"); + for (size_t i = 0; i < len; i++) + appendStringInfo(buf, "%02x", bytes[i]); + appendStringInfoString(buf, "'::bytea"); + + *nbytes = len; + pg_free(bytes); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the codec or operator errored or produced NULL on valid + * inputs, which is itself a bug -- so either is a failure, reported with the + * SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Hex round-trip: decode(encode(B, 'hex'), 'hex') = B for every byte string. + * encode() renders each byte as two lowercase hex digits and decode() parses + * them back, so this is the identity on bytea with no exceptions. + */ +PG_PROP(hex_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + size_t nbytes; + StringInfoData b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&b); + draw_bytea_literal(tc, &b, &nbytes); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT decode(encode(%s, 'hex'), 'hex') = %s", + b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Base64 round-trip: decode(encode(B, 'base64'), 'base64') = B. base64 is a + * pure codec, so encode-then-decode must reproduce every input exactly. + */ +PG_PROP(base64_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + size_t nbytes; + StringInfoData b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&b); + draw_bytea_literal(tc, &b, &nbytes); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT decode(encode(%s, 'base64'), 'base64') = %s", + b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Escape round-trip: decode(encode(B, 'escape'), 'escape') = B. The 'escape' + * format leaves printable non-backslash bytes as-is and octal-escapes the + * rest (and doubles backslashes), which is a known fiddly path -- a boundary + * bug there would show up as a failed round-trip here. + */ +PG_PROP(escape_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + size_t nbytes; + StringInfoData b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&b); + draw_bytea_literal(tc, &b, &nbytes); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT decode(encode(%s, 'escape'), 'escape') = %s", + b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Length matches the drawn byte count: length(B) = . For bytea, + * length() returns the number of bytes, which must equal the count this + * property generated into the literal. + */ +PG_PROP(length_matches_byte_count) +{ + pg_prop_conn *conn; + char *err = NULL; + size_t nbytes; + StringInfoData b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&b); + draw_bytea_literal(tc, &b, &nbytes); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT length(%s) = %zu", b.data, nbytes); + + require_true(tc, conn, q.data); + + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Concatenation is length-additive: length(B1 || B2) = length(B1) + + * length(B2). Appending two bytea values must produce exactly the sum of + * their lengths, with no truncation or padding. + */ +PG_PROP(concat_length_additive) +{ + pg_prop_conn *conn; + char *err = NULL; + size_t n1, + n2; + StringInfoData b1, + b2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&b1); + initStringInfo(&b2); + draw_bytea_literal(tc, &b1, &n1); + draw_bytea_literal(tc, &b2, &n2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(%s || %s) = length(%s) + length(%s)", + b1.data, b2.data, b1.data, b2.data); + + require_true(tc, conn, q.data); + + pfree(b1.data); + pfree(b2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Split-and-rejoin reconstructs: substr(B, 1, k) || substr(B, k+1) = B for + * any split point k in 0..length(B). The split point is clamped in SQL with + * least(, length(B)) so the drawn index can never run past the end + * and the oracle matches regardless of the generated byte count. + */ +PG_PROP(substr_concat_reconstructs) +{ + pg_prop_conn *conn; + char *err = NULL; + size_t nbytes; + int64 k; + StringInfoData b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&b); + draw_bytea_literal(tc, &b, &nbytes); + k = pg_prop_draw_int(tc, 0, 128); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT substr(%s, 1, least(%lld, length(%s))) " + "|| substr(%s, least(%lld, length(%s)) + 1) = %s", + b.data, (long long) k, b.data, + b.data, (long long) k, b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Hex encoding length: length(encode(B, 'hex')) = 2 * . The hex + * encoding emits exactly two characters per input byte, so the encoded text + * length is twice the byte count with no exceptions. + */ +PG_PROP(hex_encode_length) +{ + pg_prop_conn *conn; + char *err = NULL; + size_t nbytes; + StringInfoData b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&b); + draw_bytea_literal(tc, &b, &nbytes); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT length(encode(%s, 'hex')) = %zu", + b.data, (size_t) 2 * nbytes); + + require_true(tc, conn, q.data); + + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 63cd644c6e1e1..4412a75a86384 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 55c9abec1438178c4ce4a3e4b57561058c827e5a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:04 -0400 Subject: [PATCH 077/338] prop: add bytea concat, substring, and encode round-trips property tests (byteaops) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_byteaops.c | 339 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 343 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_byteaops.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index d5b80a7112dcd..3f3c660612e07 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -92,7 +92,8 @@ OBJS = \ prop_oid.o \ prop_uuid.o \ prop_uuidops.o \ - prop_bytea.o + prop_bytea.o \ + prop_byteaops.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index f11133411e074..f9dcf00e2d1a5 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -90,6 +90,7 @@ prop_area_sources = files( 'prop_uuid.c', 'prop_uuidops.c', 'prop_bytea.c', + 'prop_byteaops.c', ) diff --git a/src/test/prop/prop_byteaops.c b/src/test/prop/prop_byteaops.c new file mode 100644 index 0000000000000..6450ff9aed1e0 --- /dev/null +++ b/src/test/prop/prop_byteaops.c @@ -0,0 +1,339 @@ +/*------------------------------------------------------------------------- + * + * prop_byteaops.c + * Property-based tests for bytea operations and codecs. + * + * bytea's operators and codecs form a small exact algebra: concatenation is + * length-additive, substring()/substr() partition a value cleanly, get_byte + * reads back what set_byte wrote, and encode/decode is the identity for every + * encoding. None of these admit any tolerance, so a "t" is required for all + * inputs; the targets are varlena.c (bytea length, concat, substring, + * get_byte/set_byte) and encode.c (hex/base64/escape codecs). + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_byteaops.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "byteaops" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a byte string of 0..20 bytes and append it as decode('', + * 'hex') into buf, where is an even-length run of hex digits (two + * per byte). Drawing the byte count directly keeps the hex length predictable + * (always 2 * nbytes) so length-based oracles can compute the exact count. + * + * *nbytes receives the number of bytes, so a property can bound indices and + * split points against the real length without a round-trip to SQL. + */ +static void +draw_bytea(pg_prop_tc *tc, StringInfo buf, int *nbytes) +{ + int n = (int) pg_prop_draw_int(tc, 0, 20); + int i; + + appendStringInfoString(buf, "decode('"); + for (i = 0; i < n; i++) + { + int b = (int) pg_prop_draw_int(tc, 0, 255); + + appendStringInfo(buf, "%02x", b); + } + appendStringInfoString(buf, "', 'hex')"); + + *nbytes = n; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means an operator or codec errored or produced NULL on valid + * inputs, which is itself a bug -- so either is a failure, reported with the + * SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Concatenation is length-additive: octet_length(A || B) = octet_length(A) + + * octet_length(B). Appending two bytea values yields exactly the sum of their + * byte counts, with no truncation or padding. + */ +PG_PROP(concat_length) +{ + pg_prop_conn *conn; + char *err = NULL; + int na, + nb; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_bytea(tc, &a, &na); + draw_bytea(tc, &b, &nb); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT octet_length(%s || %s) " + "= octet_length(%s) + octet_length(%s)", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Hex round-trip: decode(encode(A, 'hex'), 'hex') = A. encode renders each + * byte as two hex digits and decode parses them back, so this is the identity + * on bytea with no exceptions. + */ +PG_PROP(hex_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int na; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_bytea(tc, &a, &na); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT decode(encode(%s, 'hex'), 'hex') = %s", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Base64 round-trip: decode(encode(A, 'base64'), 'base64') = A. base64 is a + * pure codec, so encode-then-decode must reproduce every input exactly. + */ +PG_PROP(base64_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int na; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_bytea(tc, &a, &na); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT decode(encode(%s, 'base64'), 'base64') = %s", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * set_byte then get_byte reads back the written value: for non-empty A and a + * valid index k in 0..octet_length(A)-1, get_byte(set_byte(A, k, 65), k) = 65. + * k is drawn in the parent and clamped in SQL to the last valid index so it + * can never run past the end regardless of the generated byte count. + */ +PG_PROP(getbyte_setbyte) +{ + pg_prop_conn *conn; + char *err = NULL; + int na; + int64 k; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_bytea(tc, &a, &na); + + /* Empty bytea has no valid index; nothing to assert. */ + if (na == 0) + { + pfree(a.data); + pg_prop_disconnect(conn); + return; + } + + k = pg_prop_draw_int(tc, 0, na - 1); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT get_byte(set_byte(%s, %lld, 65), %lld) = 65", + a.data, (long long) k, (long long) k); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * substring length is doubly bounded: octet_length(substring(A from s for l)) + * <= l AND <= greatest(octet_length(A) - s + 1, 0). A substring can never be + * longer than the requested length, nor longer than the bytes available from + * the start position; s is drawn in 1..nbytes+1 and l in 0..nbytes. + */ +PG_PROP(substring_length) +{ + pg_prop_conn *conn; + char *err = NULL; + int na; + int64 s, + l; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_bytea(tc, &a, &na); + s = pg_prop_draw_int(tc, 1, na + 1); + l = pg_prop_draw_int(tc, 0, na); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT octet_length(substring(%s from %lld for %lld)) <= %lld " + "AND octet_length(substring(%s from %lld for %lld)) " + "<= greatest(octet_length(%s) - %lld + 1, 0)", + a.data, (long long) s, (long long) l, (long long) l, + a.data, (long long) s, (long long) l, + a.data, (long long) s); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Split-and-rejoin reconstructs: substring(A from 1 for k) || + * substring(A from k+1) = A for any split point k in 0..octet_length(A). k is + * drawn in the parent in 0..nbytes, so it already lies in range. + */ +PG_PROP(substr_concat_whole) +{ + pg_prop_conn *conn; + char *err = NULL; + int na; + int64 k; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_bytea(tc, &a, &na); + k = pg_prop_draw_int(tc, 0, na); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT substring(%s from 1 for %lld) " + "|| substring(%s from %lld) = %s", + a.data, (long long) k, + a.data, (long long) (k + 1), a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Escape round-trip: decode(encode(A, 'escape'), 'escape') = A. The 'escape' + * format leaves printable non-backslash bytes as-is and octal-escapes the rest + * (doubling backslashes), a fiddly path whose boundary bugs show up here as a + * failed round-trip. + */ +PG_PROP(escape_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int na; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_bytea(tc, &a, &na); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT decode(encode(%s, 'escape'), 'escape') = %s", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 4412a75a86384..85cbc4dd133d1 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 204a2dac0ed95169546491f0382baec9b39454a3 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:04 -0400 Subject: [PATCH 078/338] prop: add bit property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_bit.c | 345 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 349 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_bit.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 3f3c660612e07..40de4dece36c3 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -93,7 +93,8 @@ OBJS = \ prop_uuid.o \ prop_uuidops.o \ prop_bytea.o \ - prop_byteaops.o + prop_byteaops.o \ + prop_bit.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index f9dcf00e2d1a5..8cf4fefb82f51 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -91,6 +91,7 @@ prop_area_sources = files( 'prop_uuidops.c', 'prop_bytea.c', 'prop_byteaops.c', + 'prop_bit.c', ) diff --git a/src/test/prop/prop_bit.c b/src/test/prop/prop_bit.c new file mode 100644 index 0000000000000..a02b3d5c1cd82 --- /dev/null +++ b/src/test/prop/prop_bit.c @@ -0,0 +1,345 @@ +/*------------------------------------------------------------------------- + * + * prop_bit.c + * Property-based tests for the bit and bit varying types. + * + * These exercise bit-string operations (concatenation, shifts, bitwise ops, + * length) implemented in src/backend/utils/adt/varbit.c, checking + * boolean-algebra and length identities over generated bit strings. + * + * The oracle is the bit-string algebra itself: concatenation length is + * additive, De Morgan's laws hold for the bitwise operators, XOR of a value + * with itself is the all-zeros string, shifts preserve length, the all-ones + * mask is the AND identity and the all-zeros mask its annihilator, text I/O + * round-trips, and get_bit observes what set_bit wrote. Any "f" result is a + * bug in varbit.c, not a representation artefact. + * + * The bitwise operators (~, &, |, #) require equal operand lengths, so the + * properties that use them draw a single length L and then draw bodies of + * exactly that length. All-ones and all-zeros masks are derived from the + * drawn value X itself (via ~(X & ~X) and X & ~X) so they match X's length + * automatically rather than hardcoding a literal of a particular width. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_bit.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "bit" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a bit-varying literal of a drawn length n in 1..40 into buf, emitted as + * B'' where each bit is '0' or '1'. Zero-length bit strings are legal + * but awkward to use in the identities below, so lengths start at 1; the + * engine favours the boundary, so length-1 strings get plenty of coverage. + */ +static void +draw_bits(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 1, 40); + + appendStringInfoString(buf, "B'"); + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 1))); + appendStringInfoChar(buf, '\''); +} + +/* + * Draw a bit-varying literal of a SPECIFIC length len into buf. Used by the + * properties whose operators (~, &, |, #) require equal-length operands: the + * caller draws L once with draw_bits_len_choice() and then draws each body of + * exactly that length. + */ +static void +draw_bits_len(pg_prop_tc *tc, StringInfo buf, int len) +{ + appendStringInfoString(buf, "B'"); + for (int i = 0; i < len; i++) + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 1))); + appendStringInfoChar(buf, '\''); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operation errored or produced NULL on well-formed bit + * strings, which is itself a bug -- so either is a failure, reported with the + * SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Concatenation length is additive: length(X || Y) = length(X) + length(Y). + * length() of a bit string is its bit count, and || appends the bits of Y + * after those of X, so no bits are gained or lost. + */ +PG_PROP(length_of_concat_additive) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + y, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&y); + draw_bits(tc, &x); + draw_bits(tc, &y); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(%s || %s) = length(%s) + length(%s)", + x.data, y.data, x.data, y.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(y.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * De Morgan's law for bit strings: NOT (X & Y) = (NOT X) | (NOT Y). The + * bitwise operators require equal-length operands, so draw a single length L + * and then draw X and Y of exactly that length. + */ +PG_PROP(and_or_not_demorgan) +{ + pg_prop_conn *conn; + char *err = NULL; + int len; + StringInfoData x, + y, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + len = (int) pg_prop_draw_int(tc, 1, 40); + + initStringInfo(&x); + initStringInfo(&y); + draw_bits_len(tc, &x, len); + draw_bits_len(tc, &y, len); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (~ (%s & %s)) = ((~ %s) | (~ %s))", + x.data, y.data, x.data, y.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(y.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * XOR of a value with itself is the all-zeros string of the same length: + * X # X = all zeros. The all-zeros mask is derived from X itself as X & ~X + * so it matches X's length without building a separate literal. + */ +PG_PROP(xor_self_is_zero) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_bits(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s # %s) = (%s & ~ %s)", + x.data, x.data, x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Shifts preserve length: for a shift s in 0..L, length(X << s) = length(X) + * and length(X >> s) = length(X). Bit-string shifts keep the width, filling + * vacated positions with zeros rather than growing or shrinking the string. + */ +PG_PROP(shift_left_right_length_preserved) +{ + pg_prop_conn *conn; + char *err = NULL; + int len; + int s; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + len = (int) pg_prop_draw_int(tc, 1, 40); + s = (int) pg_prop_draw_int(tc, 0, len); + + initStringInfo(&x); + draw_bits_len(tc, &x, len); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(%s << %d) = length(%s) " + "AND length(%s >> %d) = length(%s)", + x.data, s, x.data, x.data, s, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * AND identity and annihilator: X & (all ones) = X, and X & (all zeros) = + * all zeros. The masks are derived from X itself -- all zeros is X & ~X and + * all ones is ~(X & ~X) -- so they automatically match X's length instead of + * being hardcoded literals of a particular width. + */ +PG_PROP(and_identity_and_annihilator) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_bits(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s & (~ (%s & ~ %s))) = %s " + "AND (%s & (%s & ~ %s)) = (%s & ~ %s)", + x.data, x.data, x.data, x.data, + x.data, x.data, x.data, x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Text I/O round-trips: X::text::bit varying = X. The bit-string output + * function renders the value as '0'/'1' characters, and the input function + * parses that back to the same value, so casting out to text and back in is + * the identity. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_bits(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s::text::bit varying = %s", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * get_bit observes what set_bit wrote: for a position p in 0..L-1, + * get_bit(set_bit(X, p, 1), p) = 1 and get_bit(set_bit(X, p, 0), p) = 0. + * get_bit/set_bit are 0-indexed; this checks that a written bit reads back as + * written regardless of X's prior content at p. + */ +PG_PROP(get_bit_set_bit_consistent) +{ + pg_prop_conn *conn; + char *err = NULL; + int len; + int p; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + len = (int) pg_prop_draw_int(tc, 1, 40); + p = (int) pg_prop_draw_int(tc, 0, len - 1); + + initStringInfo(&x); + draw_bits_len(tc, &x, len); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT get_bit(set_bit(%s, %d, 1), %d) = 1 " + "AND get_bit(set_bit(%s, %d, 0), %d) = 0", + x.data, p, p, x.data, p, p); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 85cbc4dd133d1..ce1116b4ba7fb 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From b4135a8b654560d7acd20681f85765283c9d0d94 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:05 -0400 Subject: [PATCH 079/338] prop: add record property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_record.c | 365 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 369 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_record.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 40de4dece36c3..6a092de63dd6a 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -94,7 +94,8 @@ OBJS = \ prop_uuidops.o \ prop_bytea.o \ prop_byteaops.o \ - prop_bit.o + prop_bit.o \ + prop_record.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 8cf4fefb82f51..2f7bfd3c38075 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -92,6 +92,7 @@ prop_area_sources = files( 'prop_bytea.c', 'prop_byteaops.c', 'prop_bit.c', + 'prop_record.c', ) diff --git a/src/test/prop/prop_record.c b/src/test/prop/prop_record.c new file mode 100644 index 0000000000000..12993a7ca1492 --- /dev/null +++ b/src/test/prop/prop_record.c @@ -0,0 +1,365 @@ +/*------------------------------------------------------------------------- + * + * prop_record.c + * Property-based tests for composite/record type I/O and comparison. + * + * record_in/record_out and record comparison (src/backend/utils/adt/rowtypes.c) + * have had quoting and NULL-handling fixes; these check anonymous-record text + * round-trip and that record comparison is lexicographic over the fields. + * + * record_out must quote and escape field values that contain the record + * delimiters -- commas, parentheses, double quotes, backslashes, spaces -- and + * record_in must parse that text back to the identical record; a quoting bug + * breaks the text round-trip. Record comparison (record_cmp and the + * record_lt/eq/gt operators) walks the fields left to right, so it must be a + * lexicographic total order; a short-circuit or sign bug breaks that. + * + * The oracle is independent of rowtypes.c: round-trip compares a value to + * itself through text::composite, and the ordering properties compare the + * record operators to an explicit lexicographic expression built from the + * scalar int comparisons. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_record.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "record" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Alphabet for text fields. It deliberately includes exactly the characters + * record_out must quote/escape -- comma, double quote, backslash, open and + * close parens, and space -- alongside a few letters. These are the + * round-trip-critical characters; if they are drawn into a field value, + * record_out has to quote the field and record_in has to parse it back. + */ +static const char text_alphabet[] = ",\"\\() ab"; + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the record I/O or comparison errored or produced NULL on + * valid inputs, which is itself a bug -- so either is a failure, reported with + * the SQL. Mirrors require_true in prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Connect to the test cluster and create the composite types this area needs. + * Each property opens its own connection (the Tier-B pattern), so the types + * are created per connection; CASCADE + IF EXISTS makes the DDL idempotent + * against a connection that reused a backend. On any failure the property is + * failed with the message. + */ +static pg_prop_conn * +connect_and_setup(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, + "DROP TYPE IF EXISTS prop_rec2 CASCADE; " + "CREATE TYPE prop_rec2 AS (a int, b int)", + &err)) + pg_prop_fail(tc, "setup failed (prop_rec2): %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, + "DROP TYPE IF EXISTS prop_rec3 CASCADE; " + "CREATE TYPE prop_rec3 AS (a int, b int, c int)", + &err)) + pg_prop_fail(tc, "setup failed (prop_rec3): %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, + "DROP TYPE IF EXISTS prop_rect CASCADE; " + "CREATE TYPE prop_rect AS (a text, b text)", + &err)) + pg_prop_fail(tc, "setup failed (prop_rect): %s", err ? err : ""); + + return conn; +} + +/* + * Draw one integer field value into buf as a SQL expression: either a small + * integer (possibly negative) cast to int, or NULL. NULL fields exercise the + * NULL-handling path in record_out/record_in. + */ +static void +draw_int_field(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_bool(tc)) + appendStringInfoString(buf, "NULL::int"); + else + appendStringInfo(buf, "(%d)::int", + (int) pg_prop_draw_int(tc, -5, 5)); +} + +/* + * Draw one non-NULL integer field value into buf, for the ordering + * properties (comparison of NULL fields is three-valued and would not match + * the two-valued lexicographic oracle). + */ +static void +draw_int_field_nonnull(pg_prop_tc *tc, StringInfo buf) +{ + appendStringInfo(buf, "(%d)::int", (int) pg_prop_draw_int(tc, -3, 3)); +} + +/* + * Draw one text field value into buf as a single-quoted SQL literal, or NULL. + * The literal is built from text_alphabet (the quoting-critical characters), + * and single quotes are doubled defensively even though the alphabet has + * none, so the literal is always well-formed. + */ +static void +draw_text_field(pg_prop_tc *tc, StringInfo buf) +{ + char *s; + size_t len; + + if (pg_prop_draw_bool(tc)) + { + appendStringInfoString(buf, "NULL::text"); + return; + } + + s = pg_prop_draw_from(tc, text_alphabet, 0, 6, &len); + + appendStringInfoChar(buf, '\''); + for (size_t i = 0; i < len; i++) + { + if (s[i] == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, s[i]); + } + appendStringInfoString(buf, "'::text"); + + pg_free(s); +} + +/* + * Text round-trip through record_out/record_in for an all-int composite: + * + * (ROW(v1,...)::prop_rec3)::text::prop_rec3 + * IS NOT DISTINCT FROM ROW(v1,...)::prop_rec3 + * + * The ncols-wide prop_rec3 fixes the shape so the cast is well-typed; building + * the three literals once and substituting them on both sides makes it a + * genuine out-then-in round-trip rather than a tautology. IS NOT DISTINCT + * FROM so an all-NULL row still compares equal. (ncols is 2..3 fields of + * prop_rec3's three; extra fields are filled with NULL so the shape matches.) + */ +PG_PROP(text_roundtrip_int_record) +{ + pg_prop_conn *conn; + StringInfoData f[3]; + StringInfoData row, + q; + int ncols = (int) pg_prop_draw_int(tc, 2, 3); + int i; + + conn = connect_and_setup(tc); + + initStringInfo(&row); + appendStringInfoString(&row, "ROW("); + for (i = 0; i < 3; i++) + { + initStringInfo(&f[i]); + if (i < ncols) + draw_int_field(tc, &f[i]); + else + appendStringInfoString(&f[i], "NULL::int"); + + if (i > 0) + appendStringInfoString(&row, ", "); + appendStringInfoString(&row, f[i].data); + } + appendStringInfoString(&row, ")::prop_rec3"); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s)::text::prop_rec3 IS NOT DISTINCT FROM (%s)", + row.data, row.data); + + require_true(tc, conn, q.data); + + for (i = 0; i < 3; i++) + pfree(f[i].data); + pfree(row.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Text round-trip for a two-field text composite whose field values are drawn + * from the quoting-critical alphabet (comma, quote, backslash, parens, + * space). This is the high-value property: it forces record_out to quote and + * escape those characters and record_in to parse the quoted text back: + * + * (ROW(v1,v2)::prop_rect)::text::prop_rect + * IS NOT DISTINCT FROM ROW(v1,v2)::prop_rect + * + * IS NOT DISTINCT FROM so an all-NULL row still compares equal. + */ +PG_PROP(text_roundtrip_text_record) +{ + pg_prop_conn *conn; + StringInfoData f1, + f2, + row, + q; + + conn = connect_and_setup(tc); + + initStringInfo(&f1); + initStringInfo(&f2); + draw_text_field(tc, &f1); + draw_text_field(tc, &f2); + + initStringInfo(&row); + appendStringInfo(&row, "ROW(%s, %s)::prop_rect", f1.data, f2.data); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s)::text::prop_rect IS NOT DISTINCT FROM (%s)", + row.data, row.data); + + require_true(tc, conn, q.data); + + pfree(f1.data); + pfree(f2.data); + pfree(row.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Record comparison is lexicographic over the fields. For two two-int + * records R1 = ROW(a1,b1), R2 = ROW(a2,b2): + * + * (R1 < R2) = ((a1 < a2) OR (a1 = a2 AND b1 < b2)) + * + * The right side is an independent lexicographic oracle built from the scalar + * int comparisons, so a short-circuit or sign bug in record_cmp fails here. + * Fields are non-NULL to keep both sides two-valued. + */ +PG_PROP(comparison_lexicographic) +{ + pg_prop_conn *conn; + StringInfoData a1, + b1, + a2, + b2, + q; + + conn = connect_and_setup(tc); + + initStringInfo(&a1); + initStringInfo(&b1); + initStringInfo(&a2); + initStringInfo(&b2); + draw_int_field_nonnull(tc, &a1); + draw_int_field_nonnull(tc, &b1); + draw_int_field_nonnull(tc, &a2); + draw_int_field_nonnull(tc, &b2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (ROW(%s, %s)::prop_rec2 < ROW(%s, %s)::prop_rec2) " + "= ((%s < %s) OR (%s = %s AND %s < %s))", + a1.data, b1.data, a2.data, b2.data, + a1.data, a2.data, a1.data, a2.data, b1.data, b2.data); + + require_true(tc, conn, q.data); + + pfree(a1.data); + pfree(b1.data); + pfree(a2.data); + pfree(b2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Record comparison is a total order: for any two non-NULL two-int records + * exactly one of <, =, > holds, so the three boolean-to-int results sum to 1: + * + * (R1R2)::int = 1 + * + * A missing or overlapping case in record_cmp (e.g. a value that is neither + * less, equal, nor greater, or two of them firing at once) breaks this. + */ +PG_PROP(comparison_total_order) +{ + pg_prop_conn *conn; + StringInfoData a1, + b1, + a2, + b2, + q; + + conn = connect_and_setup(tc); + + initStringInfo(&a1); + initStringInfo(&b1); + initStringInfo(&a2); + initStringInfo(&b2); + draw_int_field_nonnull(tc, &a1); + draw_int_field_nonnull(tc, &b1); + draw_int_field_nonnull(tc, &a2); + draw_int_field_nonnull(tc, &b2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((ROW(%s, %s)::prop_rec2 < ROW(%s, %s)::prop_rec2)::int " + "+ (ROW(%s, %s)::prop_rec2 = ROW(%s, %s)::prop_rec2)::int " + "+ (ROW(%s, %s)::prop_rec2 > ROW(%s, %s)::prop_rec2)::int) = 1", + a1.data, b1.data, a2.data, b2.data, + a1.data, b1.data, a2.data, b2.data, + a1.data, b1.data, a2.data, b2.data); + + require_true(tc, conn, q.data); + + pfree(a1.data); + pfree(b1.data); + pfree(a2.data); + pfree(b2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index ce1116b4ba7fb..ce6185f9b039e 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From a3a1000609bea0f16e35dba88c32378eb2bc07d9 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:05 -0400 Subject: [PATCH 080/338] prop: add enum property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_enum.c | 298 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 302 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_enum.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 6a092de63dd6a..e09bd5b45aead 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -95,7 +95,8 @@ OBJS = \ prop_bytea.o \ prop_byteaops.o \ prop_bit.o \ - prop_record.o + prop_record.o \ + prop_enum.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 2f7bfd3c38075..a75c8f18f5753 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -93,6 +93,7 @@ prop_area_sources = files( 'prop_byteaops.c', 'prop_bit.c', 'prop_record.c', + 'prop_enum.c', ) diff --git a/src/test/prop/prop_enum.c b/src/test/prop/prop_enum.c new file mode 100644 index 0000000000000..2dfb251de8671 --- /dev/null +++ b/src/test/prop/prop_enum.c @@ -0,0 +1,298 @@ +/*------------------------------------------------------------------------- + * + * prop_enum.c + * Property-based tests for enum types (src/backend/utils/adt/enum.c). + * + * Enum ordering follows declaration order, and enum_range/enum_first/enum_last + * reflect it. Each property builds a random enum type per case and checks the + * ordering and range invariants against that known declaration order. + * + * The oracle is the declaration order itself: labels are named "l0".."l(n-1)" + * so a label's index is exactly its sort position. Enum comparison, enum_first + * and enum_last, and the two enum_range forms must agree with those indices -- + * '<' tracks index order, enum_first/last are the first/last labels, and + * enum_range returns the inclusive run of labels. Any disagreement is a bug in + * enum.c, not a modelling slack, so every property requires an exact "t". + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_enum.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "enum" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means an enum operation errored or produced NULL on valid + * inputs, which is itself a bug -- so either is a failure, reported with the + * SQL. Mirrors require_true in prop_numeric.c / prop_record.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Connect to the test cluster and build a FRESH enum type named prop_enum. + * + * A label count n is drawn in 2..6 (at least 2 so a strict i 0) + appendStringInfoString(&ddl, ", "); + appendStringInfo(&ddl, "'l%d'", i); + } + appendStringInfoChar(&ddl, ')'); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "setup failed (prop_enum): %s", err ? err : ""); + + pfree(ddl.data); + + *n = nlabels; + return conn; +} + +/* + * Enum comparison follows declaration order: for two distinct indices i'::prop_enum < 'l'::prop_enum. i j) + { + int t = i; + + i = j; + j = t; + } + + initStringInfo(&q); + appendStringInfo(&q, "SELECT 'l%d'::prop_enum < 'l%d'::prop_enum", i, j); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Enum comparison is a total order: for any two labels a, b exactly one of + * <, =, > holds, so the three boolean-to-int results sum to 1. A missing or + * overlapping case in the enum comparison breaks this. + */ +PG_PROP(comparison_total_order) +{ + pg_prop_conn *conn; + StringInfoData q; + int n; + int a, + b; + + conn = connect_and_setup(tc, &n); + + a = (int) pg_prop_draw_int(tc, 0, n - 1); + b = (int) pg_prop_draw_int(tc, 0, n - 1); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (('l%d'::prop_enum < 'l%d'::prop_enum)::int " + "+ ('l%d'::prop_enum = 'l%d'::prop_enum)::int " + "+ ('l%d'::prop_enum > 'l%d'::prop_enum)::int) = 1", + a, b, a, b, a, b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * enum_first and enum_last return the first and last declared labels: 'l0' and + * 'l(n-1)'. NULL::prop_enum selects the type for the type-only functions. + */ +PG_PROP(enum_first_last) +{ + pg_prop_conn *conn; + StringInfoData q; + int n; + + conn = connect_and_setup(tc, &n); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT enum_first(NULL::prop_enum) = 'l0'::prop_enum " + "AND enum_last(NULL::prop_enum) = 'l%d'::prop_enum", + n - 1); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * enum_range(NULL::prop_enum) returns all n labels in declaration order: the + * array has cardinality n, its first element is 'l0' and its last is 'l(n-1)'. + * A missing label or wrong order in the range function breaks this. + */ +PG_PROP(enum_range_cardinality) +{ + pg_prop_conn *conn; + StringInfoData q; + int n; + + conn = connect_and_setup(tc, &n); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cardinality(enum_range(NULL::prop_enum)) = %d " + "AND (enum_range(NULL::prop_enum))[1] = 'l0'::prop_enum " + "AND (enum_range(NULL::prop_enum))[%d] = 'l%d'::prop_enum", + n, n, n - 1); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * enum_range(a, b) for labels at indices i<=j returns exactly the inclusive + * run from i to j, which has j-i+1 labels. i<=j is enforced in C by swapping. + * An off-by-one or wrong-bound in the two-argument range function breaks this. + */ +PG_PROP(enum_range_bounded) +{ + pg_prop_conn *conn; + StringInfoData q; + int n; + int i, + j; + + conn = connect_and_setup(tc, &n); + + i = (int) pg_prop_draw_int(tc, 0, n - 1); + j = (int) pg_prop_draw_int(tc, 0, n - 1); + if (i > j) + { + int t = i; + + i = j; + j = t; + } + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cardinality(enum_range('l%d'::prop_enum, " + "'l%d'::prop_enum)) = (%d - %d + 1)", + i, j, j, i); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * An enum label cast to text and back to the enum equals the original label: + * ('l'::prop_enum)::text::prop_enum = 'l'::prop_enum. A bug in enum + * output (enum_out) or input (enum_in) breaks this round-trip. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + StringInfoData q; + int n; + int i; + + conn = connect_and_setup(tc, &n); + + i = (int) pg_prop_draw_int(tc, 0, n - 1); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('l%d'::prop_enum)::text::prop_enum " + "= 'l%d'::prop_enum", + i, i); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index ce6185f9b039e..76d0bd4b69608 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 281e9520635abf9341beb6901f1d4b672bfc9fdf Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:05 -0400 Subject: [PATCH 081/338] prop: add jsonb property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonb.c | 324 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 328 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonb.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index e09bd5b45aead..85abe0cb40429 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -96,7 +96,8 @@ OBJS = \ prop_byteaops.o \ prop_bit.o \ prop_record.o \ - prop_enum.o + prop_enum.o \ + prop_jsonb.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index a75c8f18f5753..10d0bb0675e94 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -94,6 +94,7 @@ prop_area_sources = files( 'prop_bit.c', 'prop_record.c', 'prop_enum.c', + 'prop_jsonb.c', ) diff --git a/src/test/prop/prop_jsonb.c b/src/test/prop/prop_jsonb.c new file mode 100644 index 0000000000000..673948ffa52bc --- /dev/null +++ b/src/test/prop/prop_jsonb.c @@ -0,0 +1,324 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonb.c + * Property-based tests for the jsonb type (src/backend/utils/adt/jsonb*). + * + * jsonb normalizes on input: it deduplicates object keys keeping the last + * value, drops insignificant whitespace, and canonicalizes numbers. These + * properties check that round-trips through text are stable, that the + * normalization is idempotent (applying it again changes nothing), and that + * structure is preserved -- the builder agrees with the parser, duplicate + * keys resolve "last wins", and every value contains itself. + * + * The oracle is jsonb's own normalized algebra: because input is already + * normalized, jsonb -> text -> jsonb is the identity, so these equalities + * must hold exactly. Any "f" (or a NULL/error on finite valid JSON) is a + * normalization bug, not a representational artefact. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonb.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonb" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * JSON string/key contents are drawn only from this alphabet. Restricting to + * [a-zA-Z0-9] means a generated value never contains a single quote (or a + * backslash, or a double quote), so the whole JSON text can be dropped into a + * single-quoted SQL literal with no escaping hazard. + */ +#define JSON_ALPHABET \ + "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789" + +/* Draw a quoted JSON string of 0..4 safe characters into buf. */ +static void +draw_json_string(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 4); + + appendStringInfoChar(buf, '"'); + for (int i = 0; i < n; i++) + { + int idx = (int) pg_prop_draw_int(tc, 0, (int) strlen(JSON_ALPHABET) - 1); + + appendStringInfoChar(buf, JSON_ALPHABET[idx]); + } + appendStringInfoChar(buf, '"'); +} + +/* Draw one JSON scalar (integer, string, true, false, or null) into buf. */ +static void +draw_json_scalar(pg_prop_tc *tc, StringInfo buf) +{ + switch ((int) pg_prop_draw_int(tc, 0, 4)) + { + case 0: + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, -1000, 1000)); + break; + case 1: + draw_json_string(tc, buf); + break; + case 2: + appendStringInfoString(buf, "true"); + break; + case 3: + appendStringInfoString(buf, "false"); + break; + default: + appendStringInfoString(buf, "null"); + break; + } +} + +/* + * Draw a random, always-valid JSON text value into buf. + * + * The depth budget bounds nesting so a counterexample stays legible and the + * recursion always terminates: at depth <= 0, or ~1/3 of the time otherwise, + * emit a scalar; else emit an array or object with 0..3 children one level + * shallower. Objects sometimes repeat a key name so the "last value wins" + * dedup path is exercised by the general round-trip properties too. + */ +static void +draw_json(pg_prop_tc *tc, StringInfo buf, int depth) +{ + int nchildren; + + if (depth <= 0 || pg_prop_draw_int(tc, 0, 2) == 0) + { + draw_json_scalar(tc, buf); + return; + } + + if (pg_prop_draw_bool(tc)) + { + /* array of 0..3 elements */ + nchildren = (int) pg_prop_draw_int(tc, 0, 3); + appendStringInfoChar(buf, '['); + for (int i = 0; i < nchildren; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + draw_json(tc, buf, depth - 1); + } + appendStringInfoChar(buf, ']'); + } + else + { + /* object of 0..3 members, sometimes reusing a key to force dedup */ + bool dup = pg_prop_draw_bool(tc); + + nchildren = (int) pg_prop_draw_int(tc, 0, 3); + appendStringInfoChar(buf, '{'); + for (int i = 0; i < nchildren; i++) + { + int key = (dup && i > 0) ? i - 1 : i; + + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "\"k%d\": ", key); + draw_json(tc, buf, depth - 1); + } + appendStringInfoChar(buf, '}'); + } +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means jsonb errored or produced NULL on finite valid JSON, + * which is itself a bug -- so either is a failure, reported with the context + * (the JSON text or expression under test) and the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql, + const char *context) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for %s\n sql: %s\n error: %s", + context, sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for %s\n sql: %s", + context, sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for %s\n sql: %s", + res, context, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * jsonb -> text -> jsonb is stable. Input is already normalized, so casting + * out to text and back must reproduce the identical jsonb value; a difference + * means the text output and the parser disagree about the normal form. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_json(tc, &j, 3); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('%s'::jsonb)::text::jsonb = '%s'::jsonb", + j.data, j.data); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Normalization is idempotent: two extra text round-trips do not change the + * value. jsonb normalizes on parse, so re-parsing already-normalized text + * must be a fixed point. + */ +PG_PROP(normalization_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_json(tc, &j, 3); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT '%s'::jsonb = '%s'::jsonb::text::jsonb::text::jsonb", + j.data, j.data); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Duplicate object keys resolve "last value wins": {"k": v1, "k": v2} must + * equal {"k": v2}. This is the documented dedup behaviour and a classic + * edge; the two values are drawn independently so v1 != v2 is common. + */ +PG_PROP(object_key_dedup_last_wins) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v1, + v2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v1); + initStringInfo(&v2); + draw_json_scalar(tc, &v1); + draw_json_scalar(tc, &v2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT '{\"k\": %s, \"k\": %s}'::jsonb = '{\"k\": %s}'::jsonb", + v1.data, v2.data, v2.data); + + require_true(tc, conn, q.data, q.data); + + pfree(v1.data); + pfree(v2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The builder agrees with the parser: jsonb_build_object('a', n, 'b', m) + * equals the parse of the equivalent literal {"a": n, "b": m}. Two + * independent construction paths must land on the same normalized value. + */ +PG_PROP(build_matches_parse) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, -1000000, 1000000); + int m = (int) pg_prop_draw_int(tc, -1000000, 1000000); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_build_object('a', %d, 'b', %d) = " + "'{\"a\": %d, \"b\": %d}'::jsonb", + n, m, n, m); + + require_true(tc, conn, q.data, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Containment is reflexive: every jsonb value contains itself under @>. + */ +PG_PROP(containment_reflexive) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_json(tc, &j, 3); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT '%s'::jsonb @> '%s'::jsonb", + j.data, j.data); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 76d0bd4b69608..bdeb3ee26ba5f 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From fde4f64a54508c55ea9e5f05f45fdad03f959249 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:05 -0400 Subject: [PATCH 082/338] prop: add jsonbops property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonbops.c | 402 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 406 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonbops.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 85abe0cb40429..a479f27cf477b 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -97,7 +97,8 @@ OBJS = \ prop_bit.o \ prop_record.o \ prop_enum.o \ - prop_jsonb.o + prop_jsonb.o \ + prop_jsonbops.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 10d0bb0675e94..95b3f47b0bed2 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -95,6 +95,7 @@ prop_area_sources = files( 'prop_record.c', 'prop_enum.c', 'prop_jsonb.c', + 'prop_jsonbops.c', ) diff --git a/src/test/prop/prop_jsonbops.c b/src/test/prop/prop_jsonbops.c new file mode 100644 index 0000000000000..6b9c0995e0cff --- /dev/null +++ b/src/test/prop/prop_jsonbops.c @@ -0,0 +1,402 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonbops.c + * Property-based tests for jsonb mutation operators (src/backend/utils/ + * adt/jsonfuncs.c, jsonb_util.c). + * + * jsonb's concatenation and delete operators have mutation semantics + * (right-wins merge, key removal) that must compose predictably; these check + * them against accessors. + * + * The operators under test are || (concat: top-level merge where the right + * operand wins on key conflict -- it is not a deep merge), - text (delete an + * object key), - int (delete an array element at an index), and #- text[] + * (delete the value at a path). Each property mutates a value and reads the + * result back with an accessor (->, #>, jsonb_array_length), so the oracle is + * jsonb's own algebra of mutation and extraction: a concat must expose the + * right operand's value at a shared key and the new keys it adds, a delete + * must make the removed key/element/path unobservable, and deleting something + * absent (or the empty object under ||) must be the identity. Any "f" (or a + * NULL/error on finite valid input) is a bug in a mutation operator, not a + * representational artefact. + * + * Distinct from prop_jsonb.c (text round-trips, normalization, key dedup, + * build-vs-parse, containment) and prop_jsonbuild.c / prop_jsonpath.c + * (builders, accessor inverses, path queries): this area targets the ||, - + * and #- mutation operators specifically. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonbops.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonbops" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Keys and string values are drawn only from [a-z]. Restricting the alphabet + * means a generated key or value never contains a single quote (or a + * backslash, or a double quote), so the whole JSON text can be dropped into a + * single-quoted SQL literal with no escaping hazard. + */ +#define KEY_ALPHABET "abcdefghijklmnopqrstuvwxyz" + +/* + * Draw one JSON scalar value into buf, written inline into a JSON literal: a + * small integer, or a quoted [a-z] string ("abc"). Both forms are legal + * inside a single-quoted jsonb literal with no escaping. + */ +static void +draw_value(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_bool(tc)) + { + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, -1000, 1000)); + } + else + { + int n = (int) pg_prop_draw_int(tc, 1, 4); + + appendStringInfoChar(buf, '"'); + for (int i = 0; i < n; i++) + { + int idx = (int) pg_prop_draw_int(tc, 0, (int) strlen(KEY_ALPHABET) - 1); + + appendStringInfoChar(buf, KEY_ALPHABET[idx]); + } + appendStringInfoChar(buf, '"'); + } +} + +/* + * Build a jsonb object literal '{"k0":V0,"k1":V1,...}' with nkeys members into + * buf. Keys are generated by index (k0, k1, ...) so they are distinct and + * none are dropped by jsonb's last-wins dedup; each value is a drawn scalar. + * The result is a complete single-quoted token with the ::jsonb cast. + */ +static void +draw_obj(pg_prop_tc *tc, StringInfo buf, int nkeys) +{ + appendStringInfoString(buf, "'{"); + for (int i = 0; i < nkeys; i++) + { + StringInfoData v; + + if (i > 0) + appendStringInfoChar(buf, ','); + initStringInfo(&v); + draw_value(tc, &v); + appendStringInfo(buf, "\"k%d\":%s", i, v.data); + pfree(v.data); + } + appendStringInfoString(buf, "}'::jsonb"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a mutation operator or accessor errored or produced NULL + * on finite valid input, which is itself a bug -- so either is a failure, + * reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Concat is right-wins on a shared key: for objects A and B that both contain + * k0, (A || B) -> 'k0' equals B -> 'k0'. || is a top-level merge, so the + * right operand's value for a conflicting key must survive. Both operands are + * built to contain k0 (nkeys >= 1). + */ +PG_PROP(concat_right_wins_on_key) +{ + pg_prop_conn *conn; + char *err = NULL; + int na = (int) pg_prop_draw_int(tc, 1, 4); + int nb = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_obj(tc, &a, na); + draw_obj(tc, &b, nb); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s || %s) -> 'k0') " + "IS NOT DISTINCT FROM (%s -> 'k0')", + a.data, b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Concat adds new keys: a key present only in B appears in A || B with B's + * value. A is built with the index keys k0..; B is built with those plus a + * key 'bonly' that A never has, so (A || B) -> 'bonly' must equal B -> 'bonly'. + */ +PG_PROP(concat_adds_new_keys) +{ + pg_prop_conn *conn; + char *err = NULL; + int na = (int) pg_prop_draw_int(tc, 0, 4); + int nb = (int) pg_prop_draw_int(tc, 0, 4); + StringInfoData a, + b, + v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_obj(tc, &a, na); + + /* Build B as an object with the index keys plus a dedicated 'bonly'. */ + initStringInfo(&v); + draw_value(tc, &v); + initStringInfo(&b); + appendStringInfoString(&b, "'{"); + for (int i = 0; i < nb; i++) + { + StringInfoData bv; + + initStringInfo(&bv); + draw_value(tc, &bv); + appendStringInfo(&b, "\"k%d\":%s,", i, bv.data); + pfree(bv.data); + } + appendStringInfo(&b, "\"bonly\":%s}'::jsonb", v.data); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s || %s) -> 'bonly') " + "IS NOT DISTINCT FROM (%s -> 'bonly')", + a.data, b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Delete removes a key: with k0 present in A, (A - 'k0') no longer contains + * k0, so (A - 'k0') -> 'k0' is SQL NULL. nkeys >= 1 guarantees k0 is present. + */ +PG_PROP(delete_key_removes_it) +{ + pg_prop_conn *conn; + char *err = NULL; + int na = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_obj(tc, &a, na); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s - 'k0') -> 'k0' IS NULL", a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Deleting an absent key is a no-op: 'zzz' is never an index key (those are + * k0, k1, ...), so (A - 'zzz') must equal A unchanged. + */ +PG_PROP(delete_absent_key_noop) +{ + pg_prop_conn *conn; + char *err = NULL; + int na = (int) pg_prop_draw_int(tc, 0, 4); + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_obj(tc, &a, na); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s - 'zzz') IS NOT DISTINCT FROM %s", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The empty object is the concat identity on both sides: A || '{}' and + * '{}' || A must each equal A. '{}' adds no keys and conflicts on none, so it + * cannot change the merged value from either direction. + */ +PG_PROP(concat_empty_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + int na = (int) pg_prop_draw_int(tc, 0, 4); + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_obj(tc, &a, na); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s || '{}'::jsonb) IS NOT DISTINCT FROM %s " + "AND ('{}'::jsonb || %s) IS NOT DISTINCT FROM %s", + a.data, a.data, a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Deleting an array element by index shrinks the array by one: for an array of + * n elements (n in 1..6) and an index i in 0..n-1, (arr - i) has length n - 1. + * (arr - int deletes the element at that index.) Element identity after a + * delete is fiddly in the presence of duplicate values, so this checks length + * only. + */ +PG_PROP(array_delete_index) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 1, 6); + int i = (int) pg_prop_draw_int(tc, 0, n - 1); + StringInfoData arr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&arr); + appendStringInfoString(&arr, "'["); + for (int e = 0; e < n; e++) + { + StringInfoData v; + + if (e > 0) + appendStringInfoChar(&arr, ','); + initStringInfo(&v); + draw_value(tc, &v); + appendStringInfoString(&arr, v.data); + pfree(v.data); + } + appendStringInfoString(&arr, "]'::jsonb"); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT jsonb_array_length(%s - %d) = %d", + arr.data, i, n - 1); + + require_true(tc, conn, q.data); + + pfree(arr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Path delete removes a nested key: for J = {"k0": {"k1": V}}, J #- '{k0,k1}' + * removes the inner key, so (J #- '{k0,k1}') #> '{k0}' is the empty object. + * J is built so k0 maps to a single-key object {"k1": V}. + */ +PG_PROP(path_delete_removes_nested) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_value(tc, &v); + + initStringInfo(&j); + appendStringInfo(&j, "'{\"k0\":{\"k1\":%s}}'::jsonb", v.data); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s #- '{k0,k1}') #> '{k0}') " + "IS NOT DISTINCT FROM '{}'::jsonb", + j.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index bdeb3ee26ba5f..b836688c99b19 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 4443378561de84a64827134e503b878bbb0fc6a4 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:05 -0400 Subject: [PATCH 083/338] prop: add jsonb_set, delete-path, and strip-nulls property tests (jsonbpath) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonbpath.c | 326 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 330 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonbpath.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index a479f27cf477b..5c57218ea9e32 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -98,7 +98,8 @@ OBJS = \ prop_record.o \ prop_enum.o \ prop_jsonb.o \ - prop_jsonbops.o + prop_jsonbops.o \ + prop_jsonbpath.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 95b3f47b0bed2..e58529e810509 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -96,6 +96,7 @@ prop_area_sources = files( 'prop_enum.c', 'prop_jsonb.c', 'prop_jsonbops.c', + 'prop_jsonbpath.c', ) diff --git a/src/test/prop/prop_jsonbpath.c b/src/test/prop/prop_jsonbpath.c new file mode 100644 index 0000000000000..0d4ca57b64678 --- /dev/null +++ b/src/test/prop/prop_jsonbpath.c @@ -0,0 +1,326 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonbpath.c + * Property-based tests for jsonb path-manipulation functions. + * + * These exercise the jsonb mutation functions and operators that address a + * value by a text[] path or by key: jsonb_set, jsonb_set with create_missing, + * the #- (delete-at-path) operator, jsonb_strip_nulls, and || (concat). The + * oracle is each function's documented contract -- set-then-get returns the + * value set, mutating one key leaves its siblings untouched, delete removes a + * key, delete-then-reinsert round-trips to the original, concat lets the right + * operand win on a conflicting key, strip_nulls drops null-valued members, and + * containment is reflexive -- so any "f" is a bug in the function, not a + * modelling slop on our side. (Distinct from areas "jsonb"/"jsonbops"/ + * "jsonbuild"/"jsonpath": those cover normalization/containment, the raw + * ||/-/#- operators on generated values, the build_* constructors, and the + * jsonpath TYPE respectively; this area is the jsonb path-manipulation + * FUNCTIONS over small built-in object literals.) + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonbpath.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonbpath" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a jsonb object literal with 1..4 distinct keys k0..k3 and small int + * values, as text '{"k0": 1, "k1": 2}'::jsonb. k0 is always present so the + * path '{k0}' always exists; nkeys-1 further keys are k1..k{nkeys-1}, all + * distinct by construction. Values are small ints 0..9 so the literal is + * trivially valid jsonb with no quoting hazard. + */ +static void +draw_object(pg_prop_tc *tc, StringInfo buf) +{ + int nkeys = (int) pg_prop_draw_int(tc, 1, 4); + int i; + + appendStringInfoString(buf, "'{"); + for (i = 0; i < nkeys; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "\"k%d\": %d", + i, (int) pg_prop_draw_int(tc, 0, 9)); + } + appendStringInfoString(buf, "}'::jsonb"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on a valid input, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * set_then_get: jsonb_set(J, '{k0}', '99') replaces k0 with 99, so reading + * k0 back yields exactly 99. k0 is always present in J. + */ +PG_PROP(set_then_get) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (jsonb_set(%s, '{k0}', '99'::jsonb) -> 'k0') " + "= '99'::jsonb", + j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * set_preserves_other_keys: replacing k0 does not change k1. Uses IS NOT + * DISTINCT FROM so the case where J has no k1 (both sides NULL) still passes. + */ +PG_PROP(set_preserves_other_keys) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (jsonb_set(%s, '{k0}', '99'::jsonb) -> 'k1') " + "IS NOT DISTINCT FROM (%s -> 'k1')", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * delete_path_removes: after J #- '{k0}' the key k0 no longer exists, so the + * key-exists test ? 'k0' is false. + */ +PG_PROP(delete_path_removes) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT NOT ((%s #- '{k0}') ? 'k0')", j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * delete_then_set_roundtrip: removing k0 and re-adding its original value + * (with create_missing = true) reconstructs J exactly. + */ +PG_PROP(delete_then_set_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_set(%s #- '{k0}', '{k0}', " + "%s -> 'k0', true) = %s", + j.data, j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * insert_adds_new_key: jsonb_set with create_missing = true adds a key that + * was absent. 'knew' is never a drawn key (keys are k0..k3), so it is always + * new and must exist after the set. + */ +PG_PROP(insert_adds_new_key) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (jsonb_set(%s, '{knew}', '7'::jsonb, true)) " + "? 'knew'", + j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * strip_nulls_removes_null_values: jsonb_strip_nulls drops object members + * whose value is JSON null, and keeps the rest. Fixed literal, so this is a + * constant-input regression check for the function's core contract. + */ +PG_PROP(strip_nulls_removes_null_values) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT NOT (jsonb_strip_nulls('{\"a\":1,\"b\":null}'::jsonb) " + "? 'b') AND (jsonb_strip_nulls('{\"a\":1,\"b\":null}'::jsonb) " + "? 'a')"); + + pg_prop_disconnect(conn); +} + +/* + * concat_right_wins: on a key present in both operands, || takes the right + * operand's value, so (J || '{"k0": 1234}') -> 'k0' = 1234. k0 is always in + * J, exercising the conflict case. + */ +PG_PROP(concat_right_wins) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s || '{\"k0\": 1234}'::jsonb) -> 'k0') " + "= '1234'::jsonb", + j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * containment_self: every jsonb value contains itself and is contained by + * itself, so J @> J AND J <@ J always holds. + */ +PG_PROP(containment_self) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s @> %s AND %s <@ %s", + j.data, j.data, j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index b836688c99b19..311275c3cd66a 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From ce9e3384232310834e331628cabc4e015d460051 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:05 -0400 Subject: [PATCH 084/338] prop: add jsonb aggregate and builder round-trips property tests (jsonbagg2) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonbagg2.c | 266 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 270 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonbagg2.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 5c57218ea9e32..d52db3767672a 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -99,7 +99,8 @@ OBJS = \ prop_enum.o \ prop_jsonb.o \ prop_jsonbops.o \ - prop_jsonbpath.o + prop_jsonbpath.o \ + prop_jsonbagg2.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index e58529e810509..b602d718f9bee 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -97,6 +97,7 @@ prop_area_sources = files( 'prop_jsonb.c', 'prop_jsonbops.c', 'prop_jsonbpath.c', + 'prop_jsonbagg2.c', ) diff --git a/src/test/prop/prop_jsonbagg2.c b/src/test/prop/prop_jsonbagg2.c new file mode 100644 index 0000000000000..0249ef0a4e27c --- /dev/null +++ b/src/test/prop/prop_jsonbagg2.c @@ -0,0 +1,266 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonbagg2.c + * Property-based tests for jsonb aggregate/builder round-trips. + * + * Exercises jsonb construction distinct from the jsonbuild/jsonagg areas: + * to_jsonb() of scalars and arrays, jsonb_object_agg() over a built table, + * jsonb_build_object(), jsonb_array_length() and jsonb_each(). The oracle is + * a round-trip or a count identity -- aggregating key/value pairs then looking + * one up returns the value that went in, extracting a scalar back from its + * to_jsonb equals the original, the array's reported length equals the number + * of elements, and the object has exactly as many keys as the (distinct-key) + * table has rows -- so any "f" result is a bug in the jsonb builders, not a + * tolerance. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonbagg2.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonbagg2" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operation errored or produced NULL on valid inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build jb_t(k text, v int) with a drawn number of rows 0..12. Keys are + * DISTINCT (k0, k1, ...) and values are small ints. Returns the row count so + * properties can guard the empty case. + */ +static int +build_jb_t(pg_prop_tc *tc, pg_prop_conn *conn) +{ + int nrows = (int) pg_prop_draw_int(tc, 0, 12); + int i; + char *err = NULL; + StringInfoData sql; + + initStringInfo(&sql); + appendStringInfoString(&sql, + "DROP TABLE IF EXISTS jb_t; " + "CREATE TABLE jb_t(k text, v int); "); + for (i = 0; i < nrows; i++) + { + int v = (int) pg_prop_draw_int(tc, -5, 5); + + appendStringInfo(&sql, + "INSERT INTO jb_t VALUES ('k%d', (%d)); ", + i, v); + } + appendStringInfoString(&sql, "ANALYZE jb_t;"); + + if (!pg_prop_sql_exec(conn, sql.data, &err)) + pg_prop_fail(tc, "build jb_t failed: %s\n SQL: %s", + err ? err : "", sql.data); + + pfree(sql.data); + return nrows; +} + +/* + * object_agg lookup: when k0 exists, jsonb_object_agg(k,v) -> 'k0' equals + * to_jsonb of the v stored for 'k0'. Guarded on nrows >= 1 so k0 is present; + * with no rows the aggregate is NULL and there is nothing to look up, so the + * empty case is skipped by building at least one row's worth of data only + * when nrows >= 1 (otherwise the property trivially passes a true constant). + */ +PG_PROP(object_agg_lookup) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + nrows = build_jb_t(tc, conn); + + initStringInfo(&q); + if (nrows >= 1) + appendStringInfoString(&q, + "SELECT (jsonb_object_agg(k, v) -> 'k0') " + "= to_jsonb((SELECT v FROM jb_t WHERE k = 'k0')) " + "FROM jb_t"); + else + appendStringInfoString(&q, "SELECT true"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Scalar round-trip through to_jsonb: for a drawn int n, + * (to_jsonb(n) #>> '{}')::int = n. #>> '{}' extracts the scalar as text. + */ +PG_PROP(to_jsonb_int_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, -2147483647, 2147483647); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (to_jsonb((%d)::int) #>> '{}')::int = (%d)::int", + n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Array length law: jsonb_array_length(to_jsonb(ARRAY[1,2,3])) = 3 (constant) + * and for a drawn array of m ints it equals m. m is 0..10. + */ +PG_PROP(array_to_jsonb_length) +{ + pg_prop_conn *conn; + char *err = NULL; + int m; + int i; + StringInfoData arr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + m = (int) pg_prop_draw_int(tc, 0, 10); + + initStringInfo(&arr); + appendStringInfoString(&arr, "ARRAY["); + for (i = 0; i < m; i++) + { + int v = (int) pg_prop_draw_int(tc, -5, 5); + + if (i > 0) + appendStringInfoString(&arr, ", "); + appendStringInfo(&arr, "(%d)", v); + } + appendStringInfoString(&arr, "]::int[]"); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_array_length(to_jsonb(ARRAY[1,2,3])) = 3 " + "AND jsonb_array_length(to_jsonb(%s)) = %d", + arr.data, m); + + require_true(tc, conn, q.data); + + pfree(arr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Key count: the number of entries in jsonb_object_agg(k,v) equals the number + * of (distinct-key) rows in jb_t. coalesce handles the empty table, where the + * aggregate is NULL and jsonb_each over NULL yields no rows (count 0 = 0). + */ +PG_PROP(object_agg_key_count) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + (void) build_jb_t(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT (SELECT count(*) FROM " + "jsonb_each(coalesce((SELECT jsonb_object_agg(k, v) " + "FROM jb_t), '{}'::jsonb))) " + "= (SELECT count(*) FROM jb_t)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * build_object round-trip: jsonb_build_object('a', n, 'b', m) -> 'a' equals + * to_jsonb(n), for drawn ints n and m. The 'a' slot must round-trip to its + * input value. + */ +PG_PROP(jsonb_build_object_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int n, + m; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, -2147483647, 2147483647); + m = (int) pg_prop_draw_int(tc, -2147483647, 2147483647); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (jsonb_build_object('a', (%d)::int, 'b', (%d)::int) " + "-> 'a') = to_jsonb((%d)::int)", + n, m, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 311275c3cd66a..258c1b2de47f7 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 6827287ffb46a28d5d1331a218fec947e7ec28a7 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:05 -0400 Subject: [PATCH 085/338] prop: add jsonb numeric value preservation and comparison property tests (jsonbnum) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonbnum.c | 301 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 305 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonbnum.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index d52db3767672a..aa6410e5d40bc 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -100,7 +100,8 @@ OBJS = \ prop_jsonb.o \ prop_jsonbops.o \ prop_jsonbpath.o \ - prop_jsonbagg2.o + prop_jsonbagg2.o \ + prop_jsonbnum.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index b602d718f9bee..d717fe71fe0b0 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -98,6 +98,7 @@ prop_area_sources = files( 'prop_jsonbops.c', 'prop_jsonbpath.c', 'prop_jsonbagg2.c', + 'prop_jsonbnum.c', ) diff --git a/src/test/prop/prop_jsonbnum.c b/src/test/prop/prop_jsonbnum.c new file mode 100644 index 0000000000000..3d594f90c036c --- /dev/null +++ b/src/test/prop/prop_jsonbnum.c @@ -0,0 +1,301 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonbnum.c + * Property-based tests for jsonb numeric value preservation. + * + * jsonb stores numbers as the arbitrary-precision NUMERIC type, so a number + * put into a jsonb value and extracted again (#>> '{}', ->>, ->) must come + * back with its full value and precision -- unlike IEEE float, 2^53+1 and a + * fractional numeric survive intact, and jsonb number comparison orders by + * numeric value. A lost digit, a float round-trip, or a wrong comparison + * here is a bug in jsonb's numeric handling, distinct from jsonbtype + * (type dispatch) and jsonconstruct (document building). + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonbnum.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonbnum" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a decimal NUMERIC literal as text, formatted "." and + * parenthesized so a leading sign never misparses when substituted into a + * larger expression. Always emits an integer digit and a fractional part so + * the literal exercises both integer and scaled numeric paths through jsonb. + */ +static void +draw_numeric(pg_prop_tc *tc, StringInfo buf) +{ + int nint = (int) pg_prop_draw_int(tc, 1, 12); + int nfrac = (int) pg_prop_draw_int(tc, 1, 8); + bool neg = pg_prop_draw_bool(tc); + + appendStringInfoString(buf, "("); + if (neg) + appendStringInfoChar(buf, '-'); + for (int i = 0; i < nint; i++) + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + appendStringInfoChar(buf, '.'); + for (int i = 0; i < nfrac; i++) + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + appendStringInfoString(buf, ")"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means jsonb produced the wrong answer or errored on a finite + * input, which is itself a bug -- so either is a failure, reported with SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * An integer put into jsonb and extracted as text equals its own ::text: + * to_jsonb() #>> '{}' = ::text. Draws the full int4 range so the + * boundary values (INT_MIN/MAX, 0, -1) are tried. + */ +PG_PROP(int_preserved) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n = pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (to_jsonb((%lld)::int) #>> '{}') = (%lld)::int::text", + (long long) n, (long long) n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * jsonb preserves numeric value: a scaled numeric cast to jsonb and back is + * equal (as numeric) to the original. + * (('x'::numeric)::jsonb #>> '{}')::numeric = 'x'::numeric. + */ +PG_PROP(numeric_preserved) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((to_jsonb(%s::numeric)) #>> '{}')::numeric = %s::numeric", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A negated integer extracts as its negation: + * (to_jsonb((-n)::int) #>> '{}')::int = -n for drawn n >= 0. The negative is + * emitted parenthesized as (-n) so it is cast cleanly rather than as -(..). + */ +PG_PROP(negative_preserved) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n = pg_prop_draw_int(tc, 0, INT32_MAX); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (to_jsonb((-%lld)::int) #>> '{}')::int = (-%lld)", + (long long) n, (long long) n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * jsonb numbers are arbitrary precision, so an integer beyond the exact range + * of IEEE double (2^53 + 1 = 9007199254740993) round-trips without loss -- + * unlike a float, which would collapse it to 2^53. + */ +PG_PROP(large_int_preserved) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT ((to_jsonb(9007199254740993::numeric)) #>> '{}')::numeric " + "= 9007199254740993::numeric"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * jsonb comparison orders numbers by value: for drawn a < b, + * to_jsonb(a) < to_jsonb(b), plus the fixed anchor that 1 < 2 and 2 > 1. + * a and b are drawn ordered to keep the property sound. + */ +PG_PROP(jsonb_number_compare) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a = pg_prop_draw_int(tc, INT32_MIN, INT32_MAX - 1); + int64 b = pg_prop_draw_int(tc, a + 1, INT32_MAX); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (to_jsonb(1) < to_jsonb(2)) " + "AND (to_jsonb(2) > to_jsonb(1)) " + "AND (to_jsonb((%lld)::int) < to_jsonb((%lld)::int))", + (long long) a, (long long) b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Indexing a jsonb scalar array returns the element as a jsonb number equal to + * to_jsonb of that value: ('[1,2,3]'::jsonb -> 1) = to_jsonb(2). + */ +PG_PROP(scalar_array_number) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT ('[1,2,3]'::jsonb -> 1) = to_jsonb(2)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A numeric placed into a jsonb object and extracted as text (->> 'n') casts + * back to the same numeric value: (jsonb_build_object('n', x) ->> 'n')::numeric + * = x::numeric. + */ +PG_PROP(numeric_in_object) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_numeric(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (jsonb_build_object('n', %s::numeric) ->> 'n')::numeric " + "= %s::numeric", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Zero forms: to_jsonb(0) equals '0'::jsonb, and to_jsonb(0.0::numeric) + * extracts to either '0' or '0.0' -- jsonb may keep the trailing scale of a + * scaled zero, so both renderings are accepted. + */ +PG_PROP(zero_forms) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT (to_jsonb(0) = '0'::jsonb) " + "AND ((to_jsonb(0.0::numeric) #>> '{}') IN ('0', '0.0'))"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 258c1b2de47f7..8dc1206fa98f7 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From d0c507713d0ba4ddf04c33c1157ca01fc4bebe7b Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:05 -0400 Subject: [PATCH 086/338] prop: add jsonb_typeof and scalar extraction property tests (jsonbtype) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonbtype.c | 285 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 289 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonbtype.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index aa6410e5d40bc..53a50bd98ba04 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -101,7 +101,8 @@ OBJS = \ prop_jsonbops.o \ prop_jsonbpath.o \ prop_jsonbagg2.o \ - prop_jsonbnum.o + prop_jsonbnum.o \ + prop_jsonbtype.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index d717fe71fe0b0..87d81b2968c1a 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -99,6 +99,7 @@ prop_area_sources = files( 'prop_jsonbpath.c', 'prop_jsonbagg2.c', 'prop_jsonbnum.c', + 'prop_jsonbtype.c', ) diff --git a/src/test/prop/prop_jsonbtype.c b/src/test/prop/prop_jsonbtype.c new file mode 100644 index 0000000000000..11bbeef9f5e06 --- /dev/null +++ b/src/test/prop/prop_jsonbtype.c @@ -0,0 +1,285 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonbtype.c + * Property-based tests for jsonb type classification and scalar extraction. + * + * The oracle is jsonb's own type system: jsonb_typeof() must report the + * JSON type of a value, and the scalar extraction operators (#>>, ->, ->>) + * must round-trip a value back to the SQL value it was built from. These + * are exact identities with no tolerance -- a wrong typeof result, a lost + * value, or a missing-key classification error is a bug in the jsonb type + * code, not a representational artefact. Properties cover scalar typeof, + * object/array typeof, int/bool/text extraction round-trips, numeric value + * preservation, nested typeof via key navigation, and element typeof via + * array index navigation. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonbtype.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonbtype" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means jsonb produced the wrong classification/value (or errored + * on valid input), which is itself a bug -- so either is a failure, reported + * with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Draw a string over the alphabet [abc] into buf as a single-quoted SQL text + * literal body (no surrounding quotes added). letters-only, so no quote + * doubling is needed. + */ +static void +draw_abc(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 8); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, (char) ('a' + pg_prop_draw_int(tc, 0, 2))); +} + +/* + * jsonb_typeof reports the JSON type of each scalar kind: a number, a string, + * a boolean, and the json null literal. + */ +PG_PROP(typeof_scalars) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_typeof(to_jsonb((%d))) = 'number'" + " AND jsonb_typeof(to_jsonb('%s'::text)) = 'string'" + " AND jsonb_typeof(to_jsonb(true)) = 'boolean'" + " AND jsonb_typeof('null'::jsonb) = 'null'", + n, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * jsonb_typeof of an empty object is 'object' and of an empty array is + * 'array'. + */ +PG_PROP(typeof_object_array) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT jsonb_typeof('{}'::jsonb) = 'object'" + " AND jsonb_typeof('[]'::jsonb) = 'array'"); + + pg_prop_disconnect(conn); +} + +/* + * An int survives a round-trip through a jsonb scalar: extracting the whole + * document as text (#>> '{}') and casting back to int yields the original. + * The negative literal is parenthesized before to_jsonb so a leading sign + * never misparses, and INT32_MIN cannot overflow because the engine's widest + * draw here is int4-ranged. + */ +PG_PROP(int_extract_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (to_jsonb((%d)) #>> '{}')::int = (%d)", + n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A boolean jsonb scalar extracts to the SQL text 'true' / 'false'. + */ +PG_PROP(bool_extract) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT (to_jsonb(true) #>> '{}') = 'true'" + " AND (to_jsonb(false) #>> '{}') = 'false'"); + + pg_prop_disconnect(conn); +} + +/* + * A text value survives a round-trip through a jsonb string scalar: building + * it with to_jsonb and extracting the whole document as text (#>> '{}') + * yields the original string. + */ +PG_PROP(text_extract_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (to_jsonb('%s'::text) #>> '{}') = '%s'", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * to_jsonb of an int equals the jsonb produced by parsing that int's text: + * the numeric value and its canonical representation are preserved. + */ +PG_PROP(number_preserves_value) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_jsonb((%d)) = ((%d))::text::jsonb", + n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Navigating into a nested document by key classifies each child correctly, + * and jsonb_typeof of a missing key is SQL NULL (not an error, not a type). + */ +PG_PROP(nested_typeof) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT jsonb_typeof('{\"a\": [1,2], \"b\": {}}'::jsonb -> 'a')" + " = 'array'" + " AND jsonb_typeof('{\"a\": [1,2], \"b\": {}}'::jsonb -> 'b')" + " = 'object'" + " AND jsonb_typeof('{\"a\": [1,2], \"b\": {}}'::jsonb -> 'nope')" + " IS NULL"); + + pg_prop_disconnect(conn); +} + +/* + * Navigating into an array by index classifies each element by its JSON type. + */ +PG_PROP(array_elem_typeof) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT jsonb_typeof('[1,\"x\",true,null]'::jsonb -> 0)" + " = 'number'" + " AND jsonb_typeof('[1,\"x\",true,null]'::jsonb -> 1)" + " = 'string'" + " AND jsonb_typeof('[1,\"x\",true,null]'::jsonb -> 2)" + " = 'boolean'" + " AND jsonb_typeof('[1,\"x\",true,null]'::jsonb -> 3)" + " = 'null'"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 8dc1206fa98f7..087993a180dcb 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 81cfac1721783caa54f4ba6b7d6a3e7e9f003863 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:06 -0400 Subject: [PATCH 087/338] prop: add jsonb subscripting and assignment property tests (jsonbsub) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonbsub.c | 314 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 318 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonbsub.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 53a50bd98ba04..283fa785e0714 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -102,7 +102,8 @@ OBJS = \ prop_jsonbpath.o \ prop_jsonbagg2.o \ prop_jsonbnum.o \ - prop_jsonbtype.o + prop_jsonbtype.o \ + prop_jsonbsub.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 87d81b2968c1a..3b3c624e2b1af 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -100,6 +100,7 @@ prop_area_sources = files( 'prop_jsonbagg2.c', 'prop_jsonbnum.c', 'prop_jsonbtype.c', + 'prop_jsonbsub.c', ) diff --git a/src/test/prop/prop_jsonbsub.c b/src/test/prop/prop_jsonbsub.c new file mode 100644 index 0000000000000..8d5c7ae95314a --- /dev/null +++ b/src/test/prop/prop_jsonbsub.c @@ -0,0 +1,314 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonbsub.c + * Property-based tests for jsonb subscripting (J['key'], J[idx]). + * + * SQL-level subscripting of jsonb (container[expr]) is a separate access path + * from the ->/->> operators and jsonb_set, implemented by jsonb's subscript + * handler (jsonb_subscript_handler and the SubscriptingRef executor). Each + * property pins subscripting against the operator it is defined to mirror: + * J['k'] must equal J -> 'k', an integer subscript must equal the numeric -> + * element, a missing key or out-of-bounds index must be NULL, and an + * assignment-style update via a built container must land the new value. A + * mismatch is a bug in the subscript handler, not in the operators. + * + * The oracle is the equivalence between the subscript syntax and the arrow + * operators (and IS NULL for absent elements), checked with no tolerance. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonbsub.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonbsub" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the subscript access errored or disagreed with its oracle + * on a well-formed input, which is itself a bug -- so either is a failure, + * reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build a jsonb object literal {"k0":,"k1":} into buf as a cast + * expression, with both keys present and small integer values. + */ +static void +draw_object(pg_prop_tc *tc, StringInfo buf) +{ + int v0 = (int) pg_prop_draw_int(tc, -5, 5); + int v1 = (int) pg_prop_draw_int(tc, -5, 5); + + appendStringInfo(buf, + "('{\"k0\": %d, \"k1\": %d}'::jsonb)", + v0, v1); +} + +/* + * Build a jsonb array literal [,...] with 1..6 small integer elements + * into buf as a cast expression. Always non-empty so element-0 access is + * well-defined; *n receives the element count. + */ +static void +draw_array(pg_prop_tc *tc, StringInfo buf, int *n) +{ + int len = (int) pg_prop_draw_int(tc, 1, 6); + int i; + + appendStringInfoString(buf, "('["); + for (i = 0; i < len; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, -5, 5)); + } + appendStringInfoString(buf, "]'::jsonb)"); + *n = len; +} + +/* + * subscript_eq_arrow: for an object J, J['k0'] = J -> 'k0'. The text subscript + * must select the same member value as the -> operator. + */ +PG_PROP(subscript_eq_arrow) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT %s['k0'] IS NOT DISTINCT FROM %s -> 'k0'", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_subscript_eq_arrow: for an array A, A[0] = A -> 0. jsonb integer + * subscripts are 0-based, matching the -> integer operator. The array is + * always non-empty, so element 0 exists. + */ +PG_PROP(array_subscript_eq_arrow) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_array(tc, &a, &n); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT %s[0] IS NOT DISTINCT FROM %s -> 0", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * subscript_assign: subscripting an object built by appending a new member + * reads back the appended value. (J || jsonb_build_object('k0', to_jsonb(99))) + * sets k0 to 99, and subscripting that container must return 99. This + * simulates assignment through the subscript read path. + */ +PG_PROP(subscript_assign) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s || jsonb_build_object('k0', to_jsonb(99)))['k0'] " + "= '99'::jsonb", + j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * missing_key_null: for an object J without key 'nope', J['nope'] IS NULL. + * A subscript on an absent member yields SQL NULL, not an error. + */ +PG_PROP(missing_key_null) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s['nope'] IS NULL", j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * nested_subscript: for J = {"o": {"i": }}, chained subscripting + * J['o']['i'] returns to_jsonb(n). This exercises the recursive descent of + * the subscript executor through nested containers. + */ +PG_PROP(nested_subscript) +{ + pg_prop_conn *conn; + char *err = NULL; + int nval = (int) pg_prop_draw_int(tc, -5, 5); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('{\"o\": {\"i\": %d}}'::jsonb)['o']['i'] " + "= to_jsonb(%d)", + nval, nval); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_oob_null: for an array A, an out-of-bounds integer subscript A[1000] + * IS NULL. An index past the end yields SQL NULL, not an error. + */ +PG_PROP(array_oob_null) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_array(tc, &a, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s[1000] IS NULL", a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * subscript_text: for an object J whose k0 value is a scalar int, the subscript + * value equals the ->> text parsed back to int and re-boxed: + * J['k0'] = to_jsonb((J ->> 'k0')::int). This pins the subscript result + * against the text extraction path round-tripped through to_jsonb. + */ +PG_PROP(subscript_text) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT %s['k0'] = to_jsonb((%s ->> 'k0')::int)", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 087993a180dcb..609fd3b4a25b8 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 7b00b1b723ab86af4a0c3bb3a405d7fd4e53d81b Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:06 -0400 Subject: [PATCH 088/338] prop: add jsonb key/value iteration and extraction property tests (jsonbkeys) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonbkeys.c | 348 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 352 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonbkeys.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 283fa785e0714..a3a56cb0a46e5 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -103,7 +103,8 @@ OBJS = \ prop_jsonbagg2.o \ prop_jsonbnum.o \ prop_jsonbtype.o \ - prop_jsonbsub.o + prop_jsonbsub.o \ + prop_jsonbkeys.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 3b3c624e2b1af..19f5582e19368 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -101,6 +101,7 @@ prop_area_sources = files( 'prop_jsonbnum.c', 'prop_jsonbtype.c', 'prop_jsonbsub.c', + 'prop_jsonbkeys.c', ) diff --git a/src/test/prop/prop_jsonbkeys.c b/src/test/prop/prop_jsonbkeys.c new file mode 100644 index 0000000000000..fcae881d65c40 --- /dev/null +++ b/src/test/prop/prop_jsonbkeys.c @@ -0,0 +1,348 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonbkeys.c + * Property-based tests for jsonb key/value iteration and extraction. + * + * Cross-checks the set-returning iterators (jsonb_object_keys, jsonb_each, + * jsonb_each_text) against the scalar extraction and membership operators + * (-> , ->> , ? , -) and the re-aggregation inverse (jsonb_object_agg). The + * oracle in every case is an agreement identity: the keys an iterator reports + * must match the keys the object actually has, the value it reports for a key + * must equal the value the arrow operator extracts, and expanding then + * re-aggregating must reproduce the object. Any disagreement -- a dropped, + * duplicated, or mis-valued key -- breaks one of these, so any "f" is a bug. + * Distinct from prop_jsoneach.c (iterator round-trips/counts over n in 0..5 + * and arrays) and prop_jsonbcontain.c (containment): here n is 1..5 so k0 is + * always present and the key-level cross-checks below are never vacuous. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonbkeys.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonbkeys" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the iteration/extraction errored or produced NULL on a + * well-formed jsonb object, which is itself a bug -- so either is a failure, + * reported with the SQL. (Same contract as prop_numeric.c's require_true.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build a jsonb object literal with n distinct keys k0..k(n-1), each mapped to + * a random small integer, into buf. n is drawn in [1, 5], so k0 is always + * present and the key-level properties below (which probe 'k0') are never + * vacuous. Keys are ASCII "k" so they are distinct by construction and + * need no quote-escaping. Returns n. + */ +static int +draw_object(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 1, 5); + + appendStringInfoString(buf, "'{"); + for (int i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(buf, ","); + appendStringInfo(buf, "\"k%d\": %d", + i, (int) pg_prop_draw_int(tc, -5, 5)); + } + appendStringInfoString(buf, "}'::jsonb"); + return n; +} + +/* + * Property 1 -- keys_match_each: jsonb_object_keys and jsonb_each enumerate the + * same key set, so they emit the same number of rows for any object. A + * difference means one iterator drops or duplicates a key relative to the + * other. + */ +PG_PROP(keys_match_each) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + (void) draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM jsonb_object_keys(%s)) " + "= (SELECT count(*) FROM jsonb_each(%s))", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 2 -- each_value_matches_arrow: the value jsonb_each reports for key + * k0 equals what the -> operator extracts for 'k0'. k0 is always present + * (n >= 1), so the subquery yields exactly one row; a mismatch means the + * iterator and the arrow operator disagree on a key's value. + */ +PG_PROP(each_value_matches_arrow) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + (void) draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT value FROM jsonb_each(%s) WHERE key = 'k0') " + "= (%s -> 'k0')", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 3 -- each_text_matches: the text value jsonb_each_text reports for + * key k0 equals what the ->> operator extracts for 'k0'. Both render the + * value as text, so they must agree exactly. + */ +PG_PROP(each_text_matches) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + (void) draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT value FROM jsonb_each_text(%s) WHERE key = 'k0') " + "= (%s ->> 'k0')", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 4 -- keys_exist: every key jsonb_object_keys reports is actually + * present in the object, i.e. J ? key holds for all of them. bool_and over + * all keys must be true; n >= 1 guarantees at least one key so the aggregate + * is never NULL. + */ +PG_PROP(keys_exist) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + (void) draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT bool_and(%s ? k) FROM jsonb_object_keys(%s) k", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 5 -- values_reconstruct: expanding the object with jsonb_each and + * re-aggregating with jsonb_object_agg(key, value) reproduces the original + * object. n >= 1 means there is always at least one row, so the aggregate is + * never NULL and no coalesce is needed. + */ +PG_PROP(values_reconstruct) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + (void) draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT jsonb_object_agg(key, value) " + "FROM jsonb_each(%s)) = %s", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 6 -- key_count_eq_n: the keys are distinct by construction, so + * jsonb_object_keys emits exactly n rows for an object built with n distinct + * keys. + */ +PG_PROP(key_count_eq_n) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + n = draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM jsonb_object_keys(%s)) = %d", + j.data, n); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 7 -- object_keys_distinct: a jsonb object has no duplicate keys, so + * jsonb_object_keys emits each key once -- count(*) equals count(DISTINCT key). + */ +PG_PROP(object_keys_distinct) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + (void) draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT count(*) = count(DISTINCT key) " + "FROM jsonb_object_keys(%s) AS key", + j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 8 -- delete_reduces_keys: deleting the always-present key k0 with + * the "- text" operator removes exactly one key, so the key count of (J - 'k0') + * is n - 1. n >= 1 guarantees k0 is present, so the delete always removes a + * real key. + */ +PG_PROP(delete_reduces_keys) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + n = draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM jsonb_object_keys(%s - 'k0')) = %d", + j.data, n - 1); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 609fd3b4a25b8..1fae479274579 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 5257cec70e123dedb8f1b023602ed07c8e68565b Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:06 -0400 Subject: [PATCH 089/338] prop: add jsonb containment and key-existence laws property tests (jsonbcontain) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonbcontain.c | 349 ++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 353 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonbcontain.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index a3a56cb0a46e5..10faf32052218 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -104,7 +104,8 @@ OBJS = \ prop_jsonbnum.o \ prop_jsonbtype.o \ prop_jsonbsub.o \ - prop_jsonbkeys.o + prop_jsonbkeys.o \ + prop_jsonbcontain.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 19f5582e19368..b1791849f8131 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -102,6 +102,7 @@ prop_area_sources = files( 'prop_jsonbtype.c', 'prop_jsonbsub.c', 'prop_jsonbkeys.c', + 'prop_jsonbcontain.c', ) diff --git a/src/test/prop/prop_jsonbcontain.c b/src/test/prop/prop_jsonbcontain.c new file mode 100644 index 0000000000000..ced6ccdc6bb0a --- /dev/null +++ b/src/test/prop/prop_jsonbcontain.c @@ -0,0 +1,349 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonbcontain.c + * Property-based tests for jsonb containment and key-existence operators + * (src/backend/utils/adt/jsonb_op.c, jsonb_util.c). + * + * jsonb's containment (@>, <@) and key-existence (?, ?|, ?&) operators form a + * small algebra over objects: a value contains itself, contains any projection + * of its own keys, and contains the empty object; a key test is true exactly + * when the key is present; and containment is transitive and survives concat. + * The oracle here is that algebra -- every property is a SELECT of a boolean + * that must be "t" for every generated object, so a "f" (or a NULL/error on + * finite valid input) is a bug in a containment or existence operator. + * + * Distinct from prop_jsonbops.c (the ||, - and #- mutation operators) and + * prop_jsonb.c / prop_jsonbuild.c / prop_jsonpath.c (text round-trips, + * builders, path queries): this area targets @>, <@, ?, ?| and ?&. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonbcontain.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonbcontain" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Build a jsonb object literal '{"k0":v0,"k1":v1,...}' with nkeys members into + * buf. Keys are generated by index (k0, k1, ...) so they are distinct and + * none are dropped by jsonb's last-wins dedup; each value is a drawn small + * integer. Integer values carry no quote/backslash hazard, so the whole token + * drops into a single-quoted SQL literal with no escaping. The result is a + * complete single-quoted token with the ::jsonb cast. + */ +static void +draw_obj(pg_prop_tc *tc, StringInfo buf, int nkeys) +{ + appendStringInfoString(buf, "'{"); + for (int i = 0; i < nkeys; i++) + { + if (i > 0) + appendStringInfoChar(buf, ','); + appendStringInfo(buf, "\"k%d\":%d", i, + (int) pg_prop_draw_int(tc, -1000, 1000)); + } + appendStringInfoString(buf, "}'::jsonb"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a containment/existence operator errored or produced NULL + * on finite valid input, which is itself a bug -- so either is a failure, + * reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * A value contains itself in both directions: J @> J AND J <@ J. Containment + * is reflexive, so this must hold for every object regardless of its keys. + */ +PG_PROP(contains_self) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_obj(tc, &j, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s @> %s) AND (%s <@ %s)", + j.data, j.data, j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A value contains a single-key projection of itself: with k0 present (nkeys + * >= 1), J @> jsonb_build_object('k0', J -> 'k0'). The projected object has + * one key with exactly J's value for it, so J must contain it. + */ +PG_PROP(contains_subset) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_obj(tc, &j, n); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT %s @> jsonb_build_object('k0', %s -> 'k0')", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The empty object is contained in every object: '{}'::jsonb <@ J. <@ is + * "is contained by", and the empty object demands nothing, so it is contained + * in any J (and the trivial J @> J is checked by contains_self). + */ +PG_PROP(empty_contained) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_obj(tc, &j, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT '{}'::jsonb <@ %s", j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Key existence tracks presence: with k0 present (nkeys >= 1), J ? 'k0' is + * true, and 'nope' is never an index key (those are k0, k1, ...), so + * NOT (J ? 'nope'). The ? operator must be true exactly for present keys. + */ +PG_PROP(key_exists) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_obj(tc, &j, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s ? 'k0') AND NOT (%s ? 'nope')", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ?| is true when at least one of the listed keys is present: with k0 present + * (nkeys >= 1) and 'nope' absent, J ?| ARRAY['k0','nope'] must be true because + * k0 alone satisfies the "any" test. + */ +PG_PROP(any_keys) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_obj(tc, &j, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s ?| ARRAY['k0','nope']", j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ?& is true when every listed key is present: J is built with at least k0 and + * k1 (nkeys drawn 2..4), so J ?& ARRAY['k0','k1'] must be true because both + * keys are present. + */ +PG_PROP(all_keys) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 2, 4); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_obj(tc, &j, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s ?& ARRAY['k0','k1']", j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Concat preserves the right operand under containment: (A || B) @> B always. + * || is a right-wins top-level merge, so every key of B survives with B's + * value in A || B; therefore the result must contain B. + */ +PG_PROP(concat_contains_operands) +{ + pg_prop_conn *conn; + char *err = NULL; + int na = (int) pg_prop_draw_int(tc, 1, 4); + int nb = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_obj(tc, &a, na); + draw_obj(tc, &b, nb); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s || %s) @> %s", a.data, b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Containment is transitive: A @> B and B @> C imply A @> C. Built from + * J so the hypotheses hold by construction: A = J (full, nkeys >= 2), B = a + * two-key projection {k0,k1} of J, C = a one-key projection {k0} of J. Each + * projection takes J's own values, so A @> B, B @> C and the implied A @> C + * must all be true. + */ +PG_PROP(contains_transitive) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 2, 4); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_obj(tc, &j, n); + + /* + * A = J; B = {k0,k1} projection; C = {k0} projection. Assert all three + * containments including the transitively implied A @> C in one SELECT. + */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s @> jsonb_build_object('k0', %s -> 'k0', 'k1', %s -> 'k1')) " + "AND (jsonb_build_object('k0', %s -> 'k0', 'k1', %s -> 'k1') " + "@> jsonb_build_object('k0', %s -> 'k0')) " + "AND (%s @> jsonb_build_object('k0', %s -> 'k0'))", + j.data, j.data, j.data, + j.data, j.data, j.data, + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 1fae479274579..766c0e66d1da9 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 6f1175bf4a6a034d6c4ac4bb65bc2e73b7a1d7c5 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:06 -0400 Subject: [PATCH 090/338] prop: add jsonb delete operators and relationships property tests (jsonbdelete) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonbdelete.c | 345 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 349 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonbdelete.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 10faf32052218..bca76e8e37e33 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -105,7 +105,8 @@ OBJS = \ prop_jsonbtype.o \ prop_jsonbsub.o \ prop_jsonbkeys.o \ - prop_jsonbcontain.o + prop_jsonbcontain.o \ + prop_jsonbdelete.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index b1791849f8131..71794fd089b46 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -103,6 +103,7 @@ prop_area_sources = files( 'prop_jsonbsub.c', 'prop_jsonbkeys.c', 'prop_jsonbcontain.c', + 'prop_jsonbdelete.c', ) diff --git a/src/test/prop/prop_jsonbdelete.c b/src/test/prop/prop_jsonbdelete.c new file mode 100644 index 0000000000000..680dfaea064d5 --- /dev/null +++ b/src/test/prop/prop_jsonbdelete.c @@ -0,0 +1,345 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonbdelete.c + * Property-based tests for the jsonb delete operators. + * + * These exercise the jsonb value-removal operators and their relationships: + * "- text" (delete object key), "- int" (delete array element by index), + * "- text[]" (delete several keys at once), and "#- text[]" (delete at path). + * The oracle is each operator's documented contract and the identities that + * tie them together -- deleting a present key makes the key-exists test false + * while leaving siblings intact, deleting an array index shrinks the array by + * one, a single-level path delete equals the key delete, delete-then-reinsert + * round-trips to the original, and delete is idempotent -- so any "f" is a bug + * in the operator, not a modelling slop on our side. (Distinct from areas + * "jsonbpath"/"jsonnest": those cover the jsonb path-manipulation FUNCTIONS + * (jsonb_set/strip_nulls/concat) and nesting/structure respectively; this area + * is specifically the delete OPERATORS and the algebra among them.) + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonbdelete.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonbdelete" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a jsonb object literal with n distinct keys k0..k{n-1} (n drawn 2..4) + * and small int values, as text '{"k0": 1, "k1": 2}'::jsonb. k0 and k1 are + * always present (n >= 2), so the properties that reference them always hit a + * present key. Values are small ints 0..9 so the literal is trivially valid + * jsonb with no quoting hazard, and the key "nope" is never drawn. + */ +static void +draw_object(pg_prop_tc *tc, StringInfo buf) +{ + int nkeys = (int) pg_prop_draw_int(tc, 2, 4); + int i; + + appendStringInfoString(buf, "'{"); + for (i = 0; i < nkeys; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "\"k%d\": %d", + i, (int) pg_prop_draw_int(tc, 0, 9)); + } + appendStringInfoString(buf, "}'::jsonb"); +} + +/* + * Draw a jsonb array literal of m elements (m drawn 1..5) with small int + * values, as text '[1, 2, 3]'::jsonb, and report m via *mout so the oracle + * can assert on the post-delete length. + */ +static void +draw_array(pg_prop_tc *tc, StringInfo buf, int *mout) +{ + int m = (int) pg_prop_draw_int(tc, 1, 5); + int i; + + appendStringInfoString(buf, "'["); + for (i = 0; i < m; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, 0, 9)); + } + appendStringInfoString(buf, "]'::jsonb"); + *mout = m; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operator errored or produced NULL on a valid input, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * delete_key_removes: after J - 'k0' the key k0 no longer exists, so the + * key-exists test ? 'k0' is false. k0 is always present in J. + */ +PG_PROP(delete_key_removes) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT NOT ((%s - 'k0') ? 'k0')", j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * delete_key_preserves_others: deleting k0 leaves k1 in place, so (J - 'k0') + * still answers true to ? 'k1'. k1 is always present (n >= 2). + */ +PG_PROP(delete_key_preserves_others) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s - 'k0') ? 'k1'", j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * delete_array_of_keys: J - ARRAY['k0','k1'] removes both keys at once, so + * neither key exists afterward. Both keys are always present (n >= 2). + */ +PG_PROP(delete_array_of_keys) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT NOT ((%s - ARRAY['k0','k1']) ? 'k0') " + "AND NOT ((%s - ARRAY['k0','k1']) ? 'k1')", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * delete_absent_key_noop: deleting a key that is not present changes nothing, + * so J - 'nope' = J. "nope" is never a drawn key (keys are k0..k3). + */ +PG_PROP(delete_absent_key_noop) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s - 'nope') = %s", j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_delete_index: for a jsonb array of m >= 1 elements, A - 0 deletes the + * element at index 0, so jsonb_array_length(A - 0) = m - 1. + */ +PG_PROP(array_delete_index) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int m = 0; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_array(tc, &a, &m); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT jsonb_array_length(%s - 0) = %d", + a.data, m - 1); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * path_delete_eq_key_delete: a single-level path delete and a key delete are + * the same operation, so J #- '{k0}' = J - 'k0'. k0 is always present. + */ +PG_PROP(path_delete_eq_key_delete) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s #- '{k0}') = (%s - 'k0')", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * delete_then_set_roundtrip: removing k0 and re-adding its original value + * (with create_missing = true) reconstructs J exactly. k0 is always present. + */ +PG_PROP(delete_then_set_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_set(%s - 'k0', '{k0}', " + "%s -> 'k0', true) = %s", + j.data, j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * double_delete_idempotent: deleting the same key twice is the same as + * deleting it once, so (J - 'k0') - 'k0' = J - 'k0'. + */ +PG_PROP(double_delete_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s - 'k0') - 'k0') = (%s - 'k0')", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 766c0e66d1da9..2f9f31602ed3a 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From c9e8c3f7ebed77c75dce19ee86d622b317a0d0a9 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:06 -0400 Subject: [PATCH 091/338] prop: add jsonb existence operators ?/?|/?& property tests (jsonbexists) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonbexists.c | 350 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 354 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonbexists.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index bca76e8e37e33..974caf08cf29a 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -106,7 +106,8 @@ OBJS = \ prop_jsonbsub.o \ prop_jsonbkeys.o \ prop_jsonbcontain.o \ - prop_jsonbdelete.o + prop_jsonbdelete.o \ + prop_jsonbexists.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 71794fd089b46..d6a70050b1c0a 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -104,6 +104,7 @@ prop_area_sources = files( 'prop_jsonbkeys.c', 'prop_jsonbcontain.c', 'prop_jsonbdelete.c', + 'prop_jsonbexists.c', ) diff --git a/src/test/prop/prop_jsonbexists.c b/src/test/prop/prop_jsonbexists.c new file mode 100644 index 0000000000000..2178f625e5376 --- /dev/null +++ b/src/test/prop/prop_jsonbexists.c @@ -0,0 +1,350 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonbexists.c + * Property-based tests for the jsonb existence operators ?, ?| and ?&. + * + * The existence operators probe presence, not value: J ? k is true exactly + * when k is a top-level object key (or, for an array, a string element); ?| + * is "any of these present" and ?& is "all of these present". The oracle is + * that presence algebra over objects built with distinct keys k0..k(n-1) and + * over jsonb string arrays: a present key tests true, an absent key tests + * false, "any" needs one hit, "all" needs every hit, and both quantifiers over + * an empty key list are false. Every property is a SELECT of a boolean that + * must be "t"; a "f" (or a NULL/error on finite valid input) is a bug in ?, + * ?| or ?&. + * + * Distinct from prop_jsonbcontain.c (the @>/<@ containment algebra) and + * prop_jsonbkeys.c (iterator/extraction cross-checks): this area targets only + * the three existence operators, including their behaviour on string arrays + * and on an empty key list. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonbexists.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonbexists" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means an existence operator errored or produced NULL on finite + * valid input, which is itself a bug -- so either is a failure, reported with + * the SQL. (Same contract as prop_numeric.c's require_true.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build a jsonb object literal '{"k0":v0,...,"k(n-1):v(n-1)"}' with n distinct + * keys into buf. Keys are generated by index (k0, k1, ...) so they are + * distinct and never equal the literal 'absent' names used below; each value + * is a drawn small integer, which carries no quote/backslash hazard, so the + * whole token drops into a single-quoted SQL literal with no escaping. The + * result is a complete single-quoted token with the ::jsonb cast. + */ +static void +draw_obj(pg_prop_tc *tc, StringInfo buf, int nkeys) +{ + appendStringInfoString(buf, "'{"); + for (int i = 0; i < nkeys; i++) + { + if (i > 0) + appendStringInfoChar(buf, ','); + appendStringInfo(buf, "\"k%d\":%d", i, + (int) pg_prop_draw_int(tc, -1000, 1000)); + } + appendStringInfoString(buf, "}'::jsonb"); +} + +/* + * Build a jsonb array literal of n distinct strings "s0".."s(n-1)" into buf. + * The element names are ASCII "s" so they need no quote-escaping and never + * collide with the object key names; the first element is always 's0' so the + * array-element existence probe below is never vacuous. Returns a complete + * single-quoted token with the ::jsonb cast. + */ +static void +draw_str_array(StringInfo buf, int nelem) +{ + appendStringInfoString(buf, "'["); + for (int i = 0; i < nelem; i++) + { + if (i > 0) + appendStringInfoChar(buf, ','); + appendStringInfo(buf, "\"s%d\"", i); + } + appendStringInfoString(buf, "]'::jsonb"); +} + +/* + * Property 1 -- key_exists_present: with k0 present (nkeys >= 1), J ? 'k0' is + * true. The ? operator must report a present top-level key as existing. + */ +PG_PROP(key_exists_present) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_obj(tc, &j, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s ? 'k0'", j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 2 -- key_not_exists_absent: 'absent' is never an index key (those + * are k0, k1, ...), so NOT (J ? 'absent') must hold. The ? operator must + * report a missing key as not existing. + */ +PG_PROP(key_not_exists_absent) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_obj(tc, &j, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT NOT (%s ? 'absent')", j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 3 -- any_keys_present: ?| is true when at least one listed key is + * present. k0 is present and 'absent' is not, so J ?| ARRAY['k0','absent'] + * must be true because k0 alone satisfies the "any" test. + */ +PG_PROP(any_keys_present) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_obj(tc, &j, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s ?| ARRAY['k0','absent']", j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 4 -- all_keys_present: ?& is true when every listed key is present. + * J is built with at least k0 and k1 (nkeys drawn 2..4), so J ?& + * ARRAY['k0','k1'] must be true because both keys are present. + */ +PG_PROP(all_keys_present) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 2, 4); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_obj(tc, &j, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s ?& ARRAY['k0','k1']", j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 5 -- all_keys_fails_one_absent: ?& demands every key, so a single + * absent key makes it false. k0 is present and 'absent' is not, so + * NOT (J ?& ARRAY['k0','absent']) must hold. + */ +PG_PROP(all_keys_fails_one_absent) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_obj(tc, &j, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT NOT (%s ?& ARRAY['k0','absent'])", j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 6 -- any_keys_all_absent: ?| is false when none of the listed keys + * is present. Neither 'absent1' nor 'absent2' is an index key, so + * NOT (J ?| ARRAY['absent1','absent2']) must hold. + */ +PG_PROP(any_keys_all_absent) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_obj(tc, &j, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT NOT (%s ?| ARRAY['absent1','absent2'])", + j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 7 -- array_element_exists: on a jsonb array of strings, ? tests + * string-element existence. The array always contains 's0' (nelem >= 1), so + * (arr ? 's0') must be true. + */ +PG_PROP(array_element_exists) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_str_array(&a, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s ? 's0'", a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 8 -- empty_any_false: "any" over an empty key list has no witness, + * so J ?| ARRAY[]::text[] is false; NOT (J ?| ARRAY[]::text[]) must hold for + * every object. + */ +PG_PROP(empty_any_false) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_obj(tc, &j, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT NOT (%s ?| ARRAY[]::text[])", j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 2f9f31602ed3a..66e1ccbf98d5f 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From db3f805b380629da82351c7603a8b5cbe46434ed Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:06 -0400 Subject: [PATCH 092/338] prop: add jsonb concatenation and shallow-merge semantics property tests (jsonbmerge) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonbmerge.c | 364 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 368 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonbmerge.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 974caf08cf29a..d125219203f45 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -107,7 +107,8 @@ OBJS = \ prop_jsonbkeys.o \ prop_jsonbcontain.o \ prop_jsonbdelete.o \ - prop_jsonbexists.o + prop_jsonbexists.o \ + prop_jsonbmerge.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index d6a70050b1c0a..f4667d2d1f755 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -105,6 +105,7 @@ prop_area_sources = files( 'prop_jsonbcontain.c', 'prop_jsonbdelete.c', 'prop_jsonbexists.c', + 'prop_jsonbmerge.c', ) diff --git a/src/test/prop/prop_jsonbmerge.c b/src/test/prop/prop_jsonbmerge.c new file mode 100644 index 0000000000000..2b50c65491eae --- /dev/null +++ b/src/test/prop/prop_jsonbmerge.c @@ -0,0 +1,364 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonbmerge.c + * Property-based tests for the jsonb concatenation/merge operator (||). + * + * These exercise the deep semantics of jsonb "||" that are distinct from the + * containment, existence, and delete operators: object-key union with + * right-hand precedence on shared keys, '{}' as a two-sided identity, + * associativity over pairwise-disjoint objects, and the crucial SHALLOW rule + * -- "||" replaces a shared nested value wholesale rather than deep-merging it. + * It also covers array concatenation (append) and the object||array wrapping + * rule. The oracle is each behaviour's documented contract plus the key-count + * identity for disjoint operands, so any "f" is a bug in the operator, not a + * modelling slop on our side. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonbmerge.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonbmerge" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a jsonb object literal with n distinct keys named "k%d" starting at + * index "base", with small int values, as text '{"k3": 7, "k4": 2}'::jsonb. + * n is drawn 1..nmax. By choosing non-overlapping [base, base+n) ranges for + * two or three objects, the caller gets pairwise-disjoint key sets. Values + * are small ints 0..9 so the literal is trivially valid jsonb with no quoting + * hazard. Returns the number of keys emitted via *nout. + */ +static void +draw_object_from(pg_prop_tc *tc, StringInfo buf, int base, int nmax, int *nout) +{ + int n = (int) pg_prop_draw_int(tc, 1, nmax); + int i; + + appendStringInfoString(buf, "'{"); + for (i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "\"k%d\": %d", + base + i, (int) pg_prop_draw_int(tc, 0, 9)); + } + appendStringInfoString(buf, "}'::jsonb"); + *nout = n; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operator errored or produced NULL on a valid input, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * concat_union_keys: (A || B) carries every key of A and every key of B. A + * and B are built with disjoint key ranges ([0, nA) and [100, 100+nB)), so k0 + * and k100 are always present, one from each side, and both must survive the + * union. + */ +PG_PROP(concat_union_keys) +{ + pg_prop_conn *conn; + char *err = NULL; + int nA, + nB; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_object_from(tc, &a, 0, 3, &nA); + draw_object_from(tc, &b, 100, 3, &nB); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s || %s) ? 'k0' AND (%s || %s) ? 'k100'", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * concat_right_overrides: on a key shared by both operands the right-hand + * value wins. Build the two single-key objects with the same key but + * different values and require the merged value to be the right one. + */ +PG_PROP(concat_right_overrides) +{ + pg_prop_conn *conn; + char *err = NULL; + int lv = (int) pg_prop_draw_int(tc, 0, 9); + int rv = (int) pg_prop_draw_int(tc, 0, 9); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (jsonb_build_object('k', %d) || " + "jsonb_build_object('k', %d)) -> 'k' = '%d'::jsonb", + lv, rv, rv); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * concat_empty_identity: '{}'::jsonb is a two-sided identity for ||, so + * A || '{}' = A and '{}' || A = A for any object A. + */ +PG_PROP(concat_empty_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + int nA; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_object_from(tc, &a, 0, 4, &nA); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s || '{}'::jsonb) = %s AND " + "('{}'::jsonb || %s) = %s", + a.data, a.data, a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * concat_associative_disjoint: for objects A, B, C with pairwise-disjoint key + * sets ([0,nA), [100,100+nB), [200,200+nC)), (A||B)||C = A||(B||C). Disjoint + * keys avoid override order affecting the result, so the two groupings must be + * byte-for-byte equal. + */ +PG_PROP(concat_associative_disjoint) +{ + pg_prop_conn *conn; + char *err = NULL; + int nA, + nB, + nC; + StringInfoData a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + initStringInfo(&c); + draw_object_from(tc, &a, 0, 3, &nA); + draw_object_from(tc, &b, 100, 3, &nB); + draw_object_from(tc, &c, 200, 3, &nC); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s || %s) || %s) = (%s || (%s || %s))", + a.data, b.data, c.data, a.data, b.data, c.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * concat_shallow_not_deep: || is SHALLOW -- a value shared under the same key + * is replaced wholesale, never deep-merged. Both operands have key "o" whose + * value is a nested object with disjoint inner keys; the result's "o" must be + * exactly the right operand's nested object, with no trace of the left inner + * key. + */ +PG_PROP(concat_shallow_not_deep) +{ + pg_prop_conn *conn; + char *err = NULL; + int la = (int) pg_prop_draw_int(tc, 0, 9); + int rb = (int) pg_prop_draw_int(tc, 0, 9); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('{\"o\": {\"a\": %d}}'::jsonb || " + "'{\"o\": {\"b\": %d}}'::jsonb) -> 'o' = " + "'{\"b\": %d}'::jsonb", + la, rb, rb); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_concat: concatenating two jsonb arrays appends element-wise, so + * [a0..] || [b0..] equals the array with all left elements followed by all + * right elements. Built from small ints with no quoting hazard. + */ +PG_PROP(array_concat) +{ + pg_prop_conn *conn; + char *err = NULL; + int x0 = (int) pg_prop_draw_int(tc, 0, 9); + int x1 = (int) pg_prop_draw_int(tc, 0, 9); + int y0 = (int) pg_prop_draw_int(tc, 0, 9); + int y1 = (int) pg_prop_draw_int(tc, 0, 9); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('[%d, %d]'::jsonb || '[%d, %d]'::jsonb) = " + "'[%d, %d, %d, %d]'::jsonb", + x0, x1, y0, y1, x0, x1, y0, y1); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * object_array_concat: concatenating an object with an array does not merge + * key-wise; it wraps into an array. We assert only the sound, documented + * consequence -- the result's type is 'array' -- not the exact element layout. + */ +PG_PROP(object_array_concat) +{ + pg_prop_conn *conn; + char *err = NULL; + int nA; + int e0 = (int) pg_prop_draw_int(tc, 0, 9); + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_object_from(tc, &a, 0, 3, &nA); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_typeof(%s || '[%d]'::jsonb) = 'array'", + a.data, e0); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * concat_key_count: for disjoint objects A (nA keys) and B (nB keys), the + * merged object has exactly nA + nB keys, since nothing is overwritten. + * jsonb_object_keys returns one row per key, so count(*) must equal nA + nB. + */ +PG_PROP(concat_key_count) +{ + pg_prop_conn *conn; + char *err = NULL; + int nA, + nB; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_object_from(tc, &a, 0, 4, &nA); + draw_object_from(tc, &b, 100, 4, &nB); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM " + "jsonb_object_keys(%s || %s)) = %d", + a.data, b.data, nA + nB); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 66e1ccbf98d5f..49c6dfbc0cfbf 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 5adccf367408d6dade2d7f437d19ecfeaf3c5c57 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:07 -0400 Subject: [PATCH 093/338] prop: add jsonb array element access, length, and concat property tests (jsonbarray) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonbarray.c | 383 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 387 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonbarray.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index d125219203f45..38dcdb5861673 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -108,7 +108,8 @@ OBJS = \ prop_jsonbcontain.o \ prop_jsonbdelete.o \ prop_jsonbexists.o \ - prop_jsonbmerge.o + prop_jsonbmerge.o \ + prop_jsonbarray.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index f4667d2d1f755..041c91b8c1519 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -106,6 +106,7 @@ prop_area_sources = files( 'prop_jsonbdelete.c', 'prop_jsonbexists.c', 'prop_jsonbmerge.c', + 'prop_jsonbarray.c', ) diff --git a/src/test/prop/prop_jsonbarray.c b/src/test/prop/prop_jsonbarray.c new file mode 100644 index 0000000000000..8aca92f1974e5 --- /dev/null +++ b/src/test/prop/prop_jsonbarray.c @@ -0,0 +1,383 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonbarray.c + * Property-based tests for jsonb array operations. + * + * jsonb arrays are an ordered sequence container with their own access path: + * jsonb_array_length, the -> integer element operator (including negative + * indices counting from the end), || concatenation, jsonb_array_elements set + * expansion, and jsonb_agg reconstruction. Each property builds a jsonb array + * of 0..6 known integers in C (computing the sum and contents on the C side) + * and pins the server's answer against that C-known oracle: length equals the + * built count, J -> i equals the i-th int, concatenation lengths add, the + * expanded element sum equals the C sum, and agg round-trips the array. A + * mismatch is a bug in the jsonb array code, not in the oracle. + * + * The oracle is the C-side known contents of each built array (length, each + * element, their sum), checked with no tolerance; disagreement is a failure. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonbarray.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonbarray" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* Element domain: small ints keep counterexamples legible. */ +#define ELEM_MIN (-5) +#define ELEM_MAX 5 +/* 0..6 elements per the area spec. */ +#define MAX_ELEMS 6 + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a jsonb array operation errored or disagreed with the + * C-known oracle on a well-formed input, which is itself a bug -- so either is + * a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Draw a jsonb array of m integers (0..MAX_ELEMS), each in [ELEM_MIN, + * ELEM_MAX]. Emits a cast literal ('[...]'::jsonb) into buf; stores each drawn + * element into vals[] and the count into *m. Sum is the caller's to compute + * from vals[] so it stays C-known. An empty array ('[]'::jsonb) is a valid + * draw and exercises the m = 0 boundary. + */ +static void +draw_array(pg_prop_tc *tc, StringInfo buf, int vals[MAX_ELEMS], int *m) +{ + int n = (int) pg_prop_draw_int(tc, 0, MAX_ELEMS); + int i; + + appendStringInfoString(buf, "('["); + for (i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + vals[i] = (int) pg_prop_draw_int(tc, ELEM_MIN, ELEM_MAX); + appendStringInfo(buf, "%d", vals[i]); + } + appendStringInfoString(buf, "]'::jsonb)"); + *m = n; +} + +/* Sum of the first m entries of vals[], computed in C. */ +static long long +sum_vals(const int vals[MAX_ELEMS], int m) +{ + long long s = 0; + int i; + + for (i = 0; i < m; i++) + s += vals[i]; + return s; +} + +/* + * length_matches: jsonb_array_length(A) = m for an array A built from m ints. + */ +PG_PROP(length_matches) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int vals[MAX_ELEMS]; + int m; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_array(tc, &a, vals, &m); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT jsonb_array_length(%s) = %d", a.data, m); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * element_access: A -> i = to_jsonb(the i-th int) for every i in 0..m-1. The + * integer -> element operator must return the element the C side put at i. + * A single SELECT bool-ANDs all m comparisons (true for the empty array). + */ +PG_PROP(element_access) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int vals[MAX_ELEMS]; + int m; + int i; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_array(tc, &a, vals, &m); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT true"); + for (i = 0; i < m; i++) + appendStringInfo(&q, " AND (%s -> %d) = to_jsonb(%d)", + a.data, i, vals[i]); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * negative_index: A -> -1 = to_jsonb(last int) for a non-empty A. Negative + * indices count from the end; -1 is the last element. Discard the empty draw, + * for which -1 is out of range. + */ +PG_PROP(negative_index) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int vals[MAX_ELEMS]; + int m; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_array(tc, &a, vals, &m); + + pg_prop_assume(tc, m > 0); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s -> -1) = to_jsonb(%d)", + a.data, vals[m - 1]); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * concat_length: jsonb_array_length(A || B) = jsonb_array_length(A) + + * jsonb_array_length(B). Concatenating two arrays yields an array whose length + * is the sum of the operand lengths. + */ +PG_PROP(concat_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + int avals[MAX_ELEMS]; + int bvals[MAX_ELEMS]; + int am; + int bm; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_array(tc, &a, avals, &am); + draw_array(tc, &b, bvals, &bm); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_array_length(%s || %s) " + "= jsonb_array_length(%s) + jsonb_array_length(%s)", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * elements_sum: the sum of jsonb_array_elements(A) equals the C-computed sum + * of the built ints. coalesce(sum(...), 0) makes the empty array sum to 0, + * matching the C sum of zero elements. + */ +PG_PROP(elements_sum) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int vals[MAX_ELEMS]; + int m; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_array(tc, &a, vals, &m); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT coalesce(sum(e::int), 0) " + "FROM jsonb_array_elements(%s) e) = %lld", + a.data, sum_vals(vals, m)); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * agg_roundtrip: aggregating jsonb_array_elements(A) back with jsonb_agg + * reconstructs A. jsonb_agg over 0 rows is NULL, so coalesce to '[]'::jsonb + * for the empty array, which must then equal A itself. + */ +PG_PROP(agg_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int vals[MAX_ELEMS]; + int m; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_array(tc, &a, vals, &m); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT coalesce(jsonb_agg(e), '[]'::jsonb) " + "FROM jsonb_array_elements(%s) e) = %s", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * oob_null: an index past either end of A yields SQL NULL. A -> 1000 and + * A -> -1000 are both out of range for any array of at most MAX_ELEMS + * elements, so both must be NULL (true for the empty array too). + */ +PG_PROP(oob_null) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int vals[MAX_ELEMS]; + int m; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_array(tc, &a, vals, &m); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s -> 1000) IS NULL AND (%s -> (-1000)) IS NULL", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * contains_element: a non-empty A @> to_jsonb(ARRAY[first int]) -- the jsonb + * containment operator must report that A contains its own first element as a + * single-element array. Discard the empty draw, which has no first element. + */ +PG_PROP(contains_element) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int vals[MAX_ELEMS]; + int m; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_array(tc, &a, vals, &m); + + pg_prop_assume(tc, m > 0); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT %s @> to_jsonb(ARRAY[%d])", + a.data, vals[0]); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 49c6dfbc0cfbf..f45a5e65e31a8 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From deec19c7c070cf3ce0950bdef1b503ce8e4a4036 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:07 -0400 Subject: [PATCH 094/338] prop: add jsonb value round-trips through text and pretty property tests (jsonbroundtrip) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonbroundtrip.c | 373 ++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 377 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonbroundtrip.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 38dcdb5861673..fd443b12cad25 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -109,7 +109,8 @@ OBJS = \ prop_jsonbdelete.o \ prop_jsonbexists.o \ prop_jsonbmerge.o \ - prop_jsonbarray.o + prop_jsonbarray.o \ + prop_jsonbroundtrip.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 041c91b8c1519..bd447761d0753 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -107,6 +107,7 @@ prop_area_sources = files( 'prop_jsonbexists.c', 'prop_jsonbmerge.c', 'prop_jsonbarray.c', + 'prop_jsonbroundtrip.c', ) diff --git a/src/test/prop/prop_jsonbroundtrip.c b/src/test/prop/prop_jsonbroundtrip.c new file mode 100644 index 0000000000000..36ba513751f69 --- /dev/null +++ b/src/test/prop/prop_jsonbroundtrip.c @@ -0,0 +1,373 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonbroundtrip.c + * Property-based tests for full jsonb value round-trips. + * + * The oracle is identity: a jsonb value J serialized to text (or pretty text, + * or concatenated with an empty object) and parsed back must equal J. Nested + * objects and arrays with integer values are built in C and substituted into + * SQL. Any "f" is a serialize/parse asymmetry in jsonb I/O -- key ordering, + * number canonicalization, whitespace, or structural handling -- which is a + * bug. Distinct from jsonformat/jsonbnum/jsonbtype: those check pretty-print + * dedup, number canonicalization, and type-of extraction respectively; this + * area asserts whole-value identity across text/pretty/concat/strip_nulls on + * built nested shapes. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonbroundtrip.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonbroundtrip" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the round-trip errored or produced NULL, which is itself + * a failure, reported with the SQL. (Same shape as prop_numeric.c.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Draw a signed integer and append it as a JSON number into buf. Negative + * values are not special inside a JSON object/array literal (no cast is + * applied here), but the matching to_jsonb() cases parenthesize so a leading + * '-' never misparses -- see draw_jsonb_scalar_arg. + */ +static void +append_json_int(pg_prop_tc *tc, StringInfo buf) +{ + int64 n = pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + + appendStringInfo(buf, INT64_FORMAT, n); +} + +/* + * Build a jsonb object literal with a-c keys and integer values, plus one + * nested array of integers, into buf as a single-quoted SQL text literal cast + * to jsonb. Shape A: {"a":,"b":,...,"arr":[,...]}. Keys are a fixed + * ASCII alphabet so no quoting hazard arises; values are plain JSON numbers so + * the literal is always valid JSON. + */ +static void +build_jsonb_object(pg_prop_tc *tc, StringInfo buf) +{ + static const char *const keys[] = {"a", "b", "c"}; + int nkeys = (int) pg_prop_draw_int(tc, 1, 3); + int narr = (int) pg_prop_draw_int(tc, 0, 4); + int i; + + appendStringInfoString(buf, "'{"); + for (i = 0; i < nkeys; i++) + { + if (i > 0) + appendStringInfoChar(buf, ','); + appendStringInfo(buf, "\"%s\":", keys[i]); + append_json_int(tc, buf); + } + /* one nested array of integers */ + if (nkeys > 0) + appendStringInfoChar(buf, ','); + appendStringInfoString(buf, "\"arr\":["); + for (i = 0; i < narr; i++) + { + if (i > 0) + appendStringInfoChar(buf, ','); + append_json_int(tc, buf); + } + appendStringInfoString(buf, "]}'::jsonb"); +} + +/* + * Build a jsonb array literal of integers into buf as a single-quoted SQL + * text literal cast to jsonb: '[,,...]'::jsonb. + */ +static void +build_jsonb_array(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 6); + int i; + + appendStringInfoString(buf, "'["); + for (i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoChar(buf, ','); + append_json_int(tc, buf); + } + appendStringInfoString(buf, "]'::jsonb"); +} + +/* + * J::text::jsonb = J for a built object. Serializing a jsonb object to text + * and parsing it back must reproduce the identical value. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + build_jsonb_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)::text::jsonb = (%s)", j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * jsonb_pretty(J)::jsonb = J. Pretty-printing adds whitespace/indentation + * only; reparsing must recover J exactly. + */ +PG_PROP(pretty_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + build_jsonb_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT jsonb_pretty(%s)::jsonb = (%s)", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * (J || '{}'::jsonb)::text::jsonb = J. Concatenating an empty object is a + * no-op on an object value, so the full text round-trip must still equal J. + */ +PG_PROP(concat_empty_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + build_jsonb_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s) || '{}'::jsonb)::text::jsonb = (%s)", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * For J = {"o":{"i":},"a":[1,2,3]}, J::text::jsonb = J. A deliberately + * nested object-in-object plus array value exercises recursive serialize and + * parse together. + */ +PG_PROP(nested_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + appendStringInfoString(&j, "'{\"o\":{\"i\":"); + append_json_int(tc, &j); + appendStringInfoString(&j, "},\"a\":[1,2,3]}'::jsonb"); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)::text::jsonb = (%s)", j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * For a jsonb array A, A::text::jsonb = A. Array order and element values + * must survive the text round-trip. + */ +PG_PROP(array_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + build_jsonb_array(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)::text::jsonb = (%s)", a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Scalar round-trips: to_jsonb()::text::jsonb = to_jsonb(), and the + * literal scalars 'true' and 'null' round-trip through text too. The integer + * argument is parenthesized so a negative value binds to to_jsonb, not to the + * surrounding cast. + */ +PG_PROP(scalar_roundtrips) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int64 n = pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_jsonb((" INT64_FORMAT "))::text::jsonb " + "= to_jsonb((" INT64_FORMAT ")) " + "AND 'true'::jsonb::text::jsonb = 'true'::jsonb " + "AND 'null'::jsonb::text::jsonb = 'null'::jsonb", + n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Double pretty: jsonb_pretty(jsonb_pretty(J)::jsonb) = jsonb_pretty(J). + * Pretty-printing then reparsing then pretty-printing again must be stable -- + * pretty output is a canonical rendering of the same value. + */ +PG_PROP(double_pretty) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + build_jsonb_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_pretty(jsonb_pretty(%s)::jsonb) " + "= jsonb_pretty(%s)", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * strip_nulls round-trip: jsonb_strip_nulls(J)::text::jsonb = + * jsonb_strip_nulls(J). Whatever structure survives null-stripping must + * itself round-trip through text unchanged. + */ +PG_PROP(strip_nulls_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + build_jsonb_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_strip_nulls(%s)::text::jsonb " + "= jsonb_strip_nulls(%s)", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index f45a5e65e31a8..701cc988c71cc 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 293e4f4be8df0dd366c57474fc630cd21681ab28 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:07 -0400 Subject: [PATCH 095/338] prop: add nested jsonb path navigation (#>, #>>) property tests (jsonnest) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonnest.c | 390 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 394 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonnest.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index fd443b12cad25..0abf532bb8b7f 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -110,7 +110,8 @@ OBJS = \ prop_jsonbexists.o \ prop_jsonbmerge.o \ prop_jsonbarray.o \ - prop_jsonbroundtrip.o + prop_jsonbroundtrip.o \ + prop_jsonnest.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index bd447761d0753..3f7e4a70a48e0 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -108,6 +108,7 @@ prop_area_sources = files( 'prop_jsonbmerge.c', 'prop_jsonbarray.c', 'prop_jsonbroundtrip.c', + 'prop_jsonnest.c', ) diff --git a/src/test/prop/prop_jsonnest.c b/src/test/prop/prop_jsonnest.c new file mode 100644 index 0000000000000..9daa5cb36e9a8 --- /dev/null +++ b/src/test/prop/prop_jsonnest.c @@ -0,0 +1,390 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonnest.c + * Property-based tests for nested jsonb path navigation (#> and #>>). + * + * Exercises the path-extraction operators #> (returns jsonb at a text[] path) + * and #>> (returns text at a path), plus jsonb_typeof and jsonb_set, against a + * jsonb value we build in C with a known shape: {"o": {"i": }, "arr": + * []}. Because we know the shape and the exact values that went in, the + * oracle is each operator's documented contract -- #> descends the named path + * step by step, a missing key yields SQL NULL, a negative array index counts + * from the end, composing two path steps equals the one combined path, typeof + * reports the container kind, and set-then-get returns the value set -- so any + * "f" (or a NULL/error where a value is expected) is a bug in path navigation, + * not a modelling artefact. (Distinct from areas "jsonb"/"jsonbops"/ + * "jsonbpath"/"jsonbagg2": those cover normalization, the raw ||/-/#- + * mutation operators, the jsonb_set/#- path FUNCTIONS over literals, and the + * builder/aggregate round-trips; this area is the #>/#>> READ navigation over + * a C-built nested value.) + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonnest.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonnest" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operator produced NULL/errored on finite valid input, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build a nested jsonb value {"o": {"i": ival}, "arr": [a0, a1, ...]} as a SQL + * expression text into "j". o.i is always present (one drawn int); arr holds + * 0..5 drawn ints. *iout receives the i value, *narr the array length, and + * arrout[] the array elements, so a property can assert against the exact + * values that went in. All ints are drawn from a small signed domain and + * substituted parenthesized as "(%lld)" so a negative value never misparses. + * + * The value is assembled with jsonb_build_object/jsonb_build_array so no + * hand-rolled JSON literal escaping is needed and the result is already a + * canonical jsonb. + */ +static void +build_nested(pg_prop_tc *tc, StringInfo j, long long *iout, + int *narr, long long arrout[6]) +{ + long long ival = pg_prop_draw_int(tc, -5, 5); + int n = (int) pg_prop_draw_int(tc, 0, 5); + StringInfoData arr; + int k; + + *iout = ival; + *narr = n; + + initStringInfo(&arr); + appendStringInfoString(&arr, "jsonb_build_array("); + for (k = 0; k < n; k++) + { + long long e = pg_prop_draw_int(tc, -5, 5); + + arrout[k] = e; + if (k > 0) + appendStringInfoString(&arr, ", "); + appendStringInfo(&arr, "(%lld)", e); + } + appendStringInfoChar(&arr, ')'); + + appendStringInfo(j, + "jsonb_build_object('o', jsonb_build_object('i', (%lld)), " + "'arr', %s)", + ival, arr.data); + + pfree(arr.data); +} + +/* + * 1. nested_get: J #> '{o,i}' descends o then i, yielding the i value as + * jsonb. Oracle: equals to_jsonb(ival). + */ +PG_PROP(nested_get) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + long long iv; + int n; + long long arr[6]; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + build_nested(tc, &j, &iv, &n, arr); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s) #> '{o,i}' = to_jsonb((%lld))", + j.data, iv); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. nested_get_text: J #>> '{o,i}' returns the i value as text; cast back to + * int it must equal the i value that went in. + */ +PG_PROP(nested_get_text) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + long long iv; + int n; + long long arr[6]; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + build_nested(tc, &j, &iv, &n, arr); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s) #>> '{o,i}')::int = (%lld)", + j.data, iv); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. missing_path_null: a path through a nonexistent key yields SQL NULL. + * o has only key "i", so J #> '{o,nope}' IS NULL must hold. + */ +PG_PROP(missing_path_null) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + long long iv; + int n; + long long arr[6]; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + build_nested(tc, &j, &iv, &n, arr); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s) #> '{o,nope}' IS NULL", + j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. array_index: when arr is non-empty, J #> '{arr,0}' is the first element. + * Oracle: equals to_jsonb(arr[0]). Guarded: an empty array has no element 0, + * so skip (nothing to assert) rather than assert an unsound equality. + */ +PG_PROP(array_index) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + long long iv; + int n; + long long arr[6]; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + build_nested(tc, &j, &iv, &n, arr); + + if (n == 0) + { + pfree(j.data); + pg_prop_disconnect(conn); + return; + } + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s) #> '{arr,0}' = to_jsonb((%lld))", + j.data, arr[0]); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. negative_array_index: a negative index counts from the end, so when arr + * is non-empty, J #> '{arr,-1}' is the last element. Oracle: equals + * to_jsonb(arr[n-1]). Guarded on empty. + */ +PG_PROP(negative_array_index) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + long long iv; + int n; + long long arr[6]; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + build_nested(tc, &j, &iv, &n, arr); + + if (n == 0) + { + pfree(j.data); + pg_prop_disconnect(conn); + return; + } + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s) #> '{arr,-1}' = to_jsonb((%lld))", + j.data, arr[n - 1]); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. path_depth_consistency: navigating in two steps equals one combined path. + * (J #> '{o}') #> '{i}' = J #> '{o,i}'. A differential oracle: both sides + * must agree for every input regardless of the drawn values. + */ +PG_PROP(path_depth_consistency) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + long long iv; + int n; + long long arr[6]; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + build_nested(tc, &j, &iv, &n, arr); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s) #> '{o}') #> '{i}' = (%s) #> '{o,i}'", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. jsonb_typeof: the container/scalar kinds are reported correctly -- o is + * an object, arr is an array, and o.i is a number. All three must hold. + */ +PG_PROP(jsonb_typeof) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + long long iv; + int n; + long long arr[6]; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + build_nested(tc, &j, &iv, &n, arr); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_typeof((%s)->'o') = 'object' " + "AND jsonb_typeof((%s)->'arr') = 'array' " + "AND jsonb_typeof((%s) #> '{o,i}') = 'number'", + j.data, j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 8. set_deep: jsonb_set replacing o.i with 777 makes J #> '{o,i}' read back + * as 777. Set-then-get at a deep path returns the value set. + */ +PG_PROP(set_deep) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + long long iv; + int n; + long long arr[6]; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + build_nested(tc, &j, &iv, &n, arr); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_set((%s), '{o,i}', '777') #> '{o,i}' " + "= '777'::jsonb", + j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 701cc988c71cc..f89200f3b5fa3 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From d23011116cf3434821eb249b01660686cdae3da4 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:07 -0400 Subject: [PATCH 096/338] prop: add jsonbuild property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonbuild.c | 429 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 433 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonbuild.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 0abf532bb8b7f..65eb3e31acdb3 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -111,7 +111,8 @@ OBJS = \ prop_jsonbmerge.o \ prop_jsonbarray.o \ prop_jsonbroundtrip.o \ - prop_jsonnest.o + prop_jsonnest.o \ + prop_jsonbuild.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 3f7e4a70a48e0..c3ddfefe398a0 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -109,6 +109,7 @@ prop_area_sources = files( 'prop_jsonbarray.c', 'prop_jsonbroundtrip.c', 'prop_jsonnest.c', + 'prop_jsonbuild.c', ) diff --git a/src/test/prop/prop_jsonbuild.c b/src/test/prop/prop_jsonbuild.c new file mode 100644 index 0000000000000..6620ff1bd0824 --- /dev/null +++ b/src/test/prop/prop_jsonbuild.c @@ -0,0 +1,429 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonbuild.c + * Property-based tests for jsonb construction/accessor functions + * (src/backend/utils/adt/jsonfuncs.c, jsonb.c). + * + * The jsonb builder and accessor functions in jsonfuncs.c have had ~14 fixes + * over their history. These properties check builder/accessor inverses -- + * building an object or array and reading an element back must return what + * went in -- and that the decomposition functions (jsonb_each, + * jsonb_object_keys) agree with construction: the set of keys and the member + * count of a built object must match the keys and count it was built from. + * + * The oracle is jsonb's own algebra of construction and extraction: + * jsonb_build_object / jsonb_build_array produce a value whose members are + * exactly their arguments, so -> , ->> , #> , jsonb_set, jsonb_each and + * jsonb_object_keys must recover those arguments exactly. Any "f" (or a + * NULL/error on finite valid input) is a bug in the builder or accessor, not + * a representational artefact. + * + * Distinct from prop_jsonb.c, which checks text round-trips, normalization + * idempotency, key dedup, build-vs-parse and containment. This area targets + * the jsonb_build_*, jsonb_object, jsonb_set, ->/->>, jsonb_each and + * jsonb_object_keys functions and their inverses. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonbuild.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonbuild" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Keys and string values are drawn only from [a-z]. Restricting the alphabet + * means a generated key or value never contains a single quote (or a + * backslash, or a double quote), so it can be dropped into a single-quoted SQL + * literal with no escaping hazard. + */ +#define KEY_ALPHABET "abcdefghijklmnopqrstuvwxyz" + +/* Draw a 1-3 letter [a-z] key (e.g. "ab") into buf, unquoted. */ +static void +draw_key(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 1, 3); + + for (int i = 0; i < n; i++) + { + int idx = (int) pg_prop_draw_int(tc, 0, (int) strlen(KEY_ALPHABET) - 1); + + appendStringInfoChar(buf, KEY_ALPHABET[idx]); + } +} + +/* + * Draw a small JSON scalar as a *typed jsonb expression* into buf: a small + * integer (to_jsonb(n::int)), a quoted [a-z] string ('"abc"'::jsonb), or one + * of true/false/null ('true'::jsonb). The result is a self-contained jsonb + * value usable both as a jsonb_build_* argument and as the expected value in + * an IS NOT DISTINCT FROM comparison. + */ +static void +draw_scalar(pg_prop_tc *tc, StringInfo buf) +{ + switch ((int) pg_prop_draw_int(tc, 0, 4)) + { + case 0: + appendStringInfo(buf, "to_jsonb(%d::int)", + (int) pg_prop_draw_int(tc, -1000, 1000)); + break; + case 1: + { + int n = (int) pg_prop_draw_int(tc, 1, 4); + + appendStringInfoString(buf, "'\""); + for (int i = 0; i < n; i++) + { + int idx = (int) pg_prop_draw_int(tc, 0, (int) strlen(KEY_ALPHABET) - 1); + + appendStringInfoChar(buf, KEY_ALPHABET[idx]); + } + appendStringInfoString(buf, "\"'::jsonb"); + break; + } + case 2: + appendStringInfoString(buf, "'true'::jsonb"); + break; + case 3: + appendStringInfoString(buf, "'false'::jsonb"); + break; + default: + appendStringInfoString(buf, "'null'::jsonb"); + break; + } +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a builder/accessor errored or produced NULL on finite + * valid input, which is itself a bug -- so either is a failure, reported with + * the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Builder/getter inverse for objects: jsonb_build_object(k, V) -> k equals V. + * Building a one-member object from a key and value, then fetching that key + * with ->, must recover the value exactly. + */ +PG_PROP(build_object_get_inverse) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData k, + v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&k); + initStringInfo(&v); + draw_key(tc, &k); + draw_scalar(tc, &v); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (jsonb_build_object('%s', %s) -> '%s') " + "IS NOT DISTINCT FROM (%s)", + k.data, v.data, k.data, v.data); + + require_true(tc, conn, q.data); + + pfree(k.data); + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Array length matches construction: a jsonb_build_array of n elements has + * jsonb_array_length n. n ranges over 0..5; the empty case jsonb_build_array() + * yields '[]' with length 0. + */ +PG_PROP(build_array_length) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 0, 5); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT jsonb_array_length(jsonb_build_array("); + for (int i = 0; i < n; i++) + { + StringInfoData e; + + if (i > 0) + appendStringInfoString(&q, ", "); + initStringInfo(&e); + draw_scalar(tc, &e); + appendStringInfoString(&q, e.data); + pfree(e.data); + } + appendStringInfo(&q, ")) = %d", n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Array element matches construction: the i-th element of a built array + * equals the i-th input argument. Draws n (1..5) scalars and an index i in + * 0..n-1, keeping each element's expression so the expected value can be + * compared with -> i. + */ +PG_PROP(array_element_matches_build) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 1, 5); + int idx = (int) pg_prop_draw_int(tc, 0, n - 1); + StringInfoData q; + StringInfoData elems[5]; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + for (int i = 0; i < n; i++) + { + initStringInfo(&elems[i]); + draw_scalar(tc, &elems[i]); + } + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT (jsonb_build_array("); + for (int i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(&q, ", "); + appendStringInfoString(&q, elems[i].data); + } + appendStringInfo(&q, ") -> %d) IS NOT DISTINCT FROM (%s)", + idx, elems[idx].data); + + require_true(tc, conn, q.data); + + for (int i = 0; i < n; i++) + pfree(elems[i].data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Set-then-get inverse: build an object with a known key, jsonb_set that key + * to a new value NV, then read it back with ->. Setting a path and then + * getting it must return the set value exactly. + */ +PG_PROP(set_then_get) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v0, + nv, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v0); + initStringInfo(&nv); + draw_scalar(tc, &v0); + draw_scalar(tc, &nv); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (jsonb_set(jsonb_build_object('k0', %s), " + "'{k0}', %s) -> 'k0') IS NOT DISTINCT FROM (%s)", + v0.data, nv.data, nv.data); + + require_true(tc, conn, q.data); + + pfree(v0.data); + pfree(nv.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Decomposition agrees with construction (keys): for an object built from + * distinct keys k0,k1,..., jsonb_object_keys returns exactly those keys as a + * set. Keys are generated by index (k0, k1, ...) so they are distinct and + * none are dropped by dedup; both sides are sorted so order does not matter. + */ +PG_PROP(object_keys_matches_build) +{ + pg_prop_conn *conn; + char *err = NULL; + int m = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData obj, + keys, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&obj); + initStringInfo(&keys); + appendStringInfoString(&obj, "jsonb_build_object("); + for (int i = 0; i < m; i++) + { + StringInfoData v; + + if (i > 0) + { + appendStringInfoString(&obj, ", "); + appendStringInfoString(&keys, ", "); + } + initStringInfo(&v); + draw_scalar(tc, &v); + appendStringInfo(&obj, "'k%d', %s", i, v.data); + appendStringInfo(&keys, "'k%d'", i); + pfree(v.data); + } + appendStringInfoString(&obj, ")"); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT array_agg(key ORDER BY key) " + "FROM jsonb_object_keys(%s) key) " + "= (SELECT array_agg(k ORDER BY k) " + "FROM unnest(ARRAY[%s]) k)", + obj.data, keys.data); + + require_true(tc, conn, q.data); + + pfree(obj.data); + pfree(keys.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Decomposition agrees with construction (count): an object built from m + * distinct keys yields exactly m rows from jsonb_each (one row per member). + * Keys are generated by index so they are distinct and none are dropped by + * dedup. + */ +PG_PROP(each_count_matches_size) +{ + pg_prop_conn *conn; + char *err = NULL; + int m = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData obj, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&obj); + appendStringInfoString(&obj, "jsonb_build_object("); + for (int i = 0; i < m; i++) + { + StringInfoData v; + + if (i > 0) + appendStringInfoString(&obj, ", "); + initStringInfo(&v); + draw_scalar(tc, &v); + appendStringInfo(&obj, "'k%d', %s", i, v.data); + pfree(v.data); + } + appendStringInfoString(&obj, ")"); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM jsonb_each(%s)) = %d", + obj.data, m); + + require_true(tc, conn, q.data); + + pfree(obj.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Path extraction agrees with chained arrow access: for a nested object + * J = {"k0": {"k1": V}}, J #> '{k0,k1}' equals (J -> 'k0') -> 'k1'. #> and -> + * are independent accessor paths and must land on the same value. + */ +PG_PROP(extract_path_matches_arrow) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + obj, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_scalar(tc, &v); + + initStringInfo(&obj); + appendStringInfo(&obj, + "jsonb_build_object('k0', jsonb_build_object('k1', %s))", + v.data); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s) #> '{k0,k1}') " + "IS NOT DISTINCT FROM (((%s) -> 'k0') -> 'k1')", + obj.data, obj.data); + + require_true(tc, conn, q.data); + + pfree(v.data); + pfree(obj.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index f89200f3b5fa3..de09291444627 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 0b830f5bcadb1485790538865f7a5c9fb6d491e5 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:07 -0400 Subject: [PATCH 097/338] prop: add jsonb_build_object/array construction property tests (jsonconstruct) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonconstruct.c | 268 +++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 272 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonconstruct.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 65eb3e31acdb3..e42ea0474181c 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -112,7 +112,8 @@ OBJS = \ prop_jsonbarray.o \ prop_jsonbroundtrip.o \ prop_jsonnest.o \ - prop_jsonbuild.o + prop_jsonbuild.o \ + prop_jsonconstruct.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index c3ddfefe398a0..0ea3071c27f0a 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -110,6 +110,7 @@ prop_area_sources = files( 'prop_jsonbroundtrip.c', 'prop_jsonnest.c', 'prop_jsonbuild.c', + 'prop_jsonconstruct.c', ) diff --git a/src/test/prop/prop_jsonconstruct.c b/src/test/prop/prop_jsonconstruct.c new file mode 100644 index 0000000000000..1e3134c62c1a7 --- /dev/null +++ b/src/test/prop/prop_jsonconstruct.c @@ -0,0 +1,268 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonconstruct.c + * Property-based tests for jsonb construction functions. + * + * Exercises jsonb_build_object, jsonb_build_array, jsonb_object(text[]) and + * to_jsonb() of a composite. The oracle is each function's own documented + * contract: a built object has exactly the keys passed, a built array + * preserves element order and length, a looked-up value round-trips through + * to_jsonb, and jsonb_object() pairs a flat text array into string values. + * Any "f" result is a construction bug, not a formatting artefact. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonconstruct.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonconstruct" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the construction errored or produced NULL on valid + * inputs, which is itself a bug -- so either is a failure, reported with the + * SQL. (Copied from prop_numeric.c's require_true.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Draw a short text value from a letters-only alphabet, single-quoted and + * safe to substitute into SQL (the alphabet contains no quote to double). + */ +static void +draw_str(pg_prop_tc *tc, StringInfo buf) +{ + char *s; + size_t len = 0; + + s = pg_prop_draw_from(tc, "abc", 0, 6, &len); + appendStringInfoChar(buf, '\''); + appendStringInfoString(buf, s); + appendStringInfoChar(buf, '\''); + pg_free(s); +} + +/* + * build_object_keys: jsonb_build_object('a',1,'b',2,'c',3) has exactly the + * three keys a, b, c -- so jsonb_object_keys returns 3 rows. + */ +PG_PROP(build_object_keys) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "jsonb_object_keys(jsonb_build_object('a',1,'b',2,'c',3))) = 3"); + + pg_prop_disconnect(conn); +} + +/* + * build_array_order: jsonb_build_array(3,1,2) preserves element order, so + * ->0, ->1, ->2 are '3', '1', '2' respectively. + */ +PG_PROP(build_array_order) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT jsonb_build_array(3,1,2) -> 0 = '3'::jsonb " + "AND jsonb_build_array(3,1,2) -> 1 = '1'::jsonb " + "AND jsonb_build_array(3,1,2) -> 2 = '2'::jsonb"); + + pg_prop_disconnect(conn); +} + +/* + * build_object_lookup: for a drawn int n, looking up the key in + * jsonb_build_object('k', n) yields to_jsonb(n). + */ +PG_PROP(build_object_lookup) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, -2147483648, 2147483647); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_build_object('k', (%d)::int) -> 'k' " + "= to_jsonb((%d)::int)", + n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * jsonb_object_from_array: jsonb_object() over a flat key/value text array + * builds an object whose values are the text elements (as JSON strings). + */ +PG_PROP(jsonb_object_from_array) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT jsonb_object(ARRAY['a','1','b','2']) " + "= '{\"a\":\"1\",\"b\":\"2\"}'::jsonb"); + + pg_prop_disconnect(conn); +} + +/* + * to_jsonb_composite: to_jsonb() of a one-row composite with named columns + * (a int, b text) produces an object whose 'a' member round-trips to + * to_jsonb(n). A record literal has no field names, so a named subquery row + * supplies them. + */ +PG_PROP(to_jsonb_composite) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, -2147483648, 2147483647); + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_str(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT to_jsonb(t) FROM " + "(SELECT (%d)::int AS a, %s::text AS b) t) -> 'a' " + "= to_jsonb((%d)::int)", + n, s.data, n); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * build_array_length: jsonb_array_length agrees with the number of elements + * passed to jsonb_build_array, including the empty array (length 0). + */ +PG_PROP(build_array_length) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT jsonb_array_length(jsonb_build_array(1,2,3,4,5)) = 5 " + "AND jsonb_array_length(jsonb_build_array()) = 0"); + + pg_prop_disconnect(conn); +} + +/* + * nested_build: a nested jsonb_build_object is addressable with a path, so + * jsonb_build_object('o', jsonb_build_object('i', n)) #> '{o,i}' = to_jsonb(n). + */ +PG_PROP(nested_build) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, -2147483648, 2147483647); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_build_object('o', " + "jsonb_build_object('i', (%d)::int)) #> '{o,i}' " + "= to_jsonb((%d)::int)", + n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * build_object_null_value: a NULL value in jsonb_build_object becomes JSON + * null (not a dropped key), so ->'k' = 'null'::jsonb. + */ +PG_PROP(build_object_null_value) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT jsonb_build_object('k', NULL::int) -> 'k' " + "= 'null'::jsonb"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index de09291444627..0ef13f7468389 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From e384af485f2869517d316fd26fe19a1c1f3458f0 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:07 -0400 Subject: [PATCH 098/338] prop: add jsontext property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsontext.c | 424 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 428 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsontext.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index e42ea0474181c..73b85d4ba2006 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -113,7 +113,8 @@ OBJS = \ prop_jsonbroundtrip.o \ prop_jsonnest.o \ prop_jsonbuild.o \ - prop_jsonconstruct.o + prop_jsonconstruct.o \ + prop_jsontext.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 0ea3071c27f0a..253d6ecf45299 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -111,6 +111,7 @@ prop_area_sources = files( 'prop_jsonnest.c', 'prop_jsonbuild.c', 'prop_jsonconstruct.c', + 'prop_jsontext.c', ) diff --git a/src/test/prop/prop_jsontext.c b/src/test/prop/prop_jsontext.c new file mode 100644 index 0000000000000..aff04894e0513 --- /dev/null +++ b/src/test/prop/prop_jsontext.c @@ -0,0 +1,424 @@ +/*------------------------------------------------------------------------- + * + * prop_jsontext.c + * Property-based tests for the text-preserving json type + * (src/backend/utils/adt/json*). + * + * Unlike jsonb, the json type stores the original text verbatim: insignificant + * whitespace, object key order, and duplicate keys are all preserved exactly, + * because json is validated-but-unparsed text rather than a normalized binary + * tree. These properties check that verbatim-preservation contract and the + * json accessor/builder functions (json_array_length, ->, json_each, + * json_build_array), using jsonb as the normalizing oracle where a value + * comparison is wanted so formatting differences do not cause false failures. + * + * The headline distinction from prop_jsonb.c: where jsonb strips whitespace + * and dedups keys "last wins", json keeps the bytes. Property 1 (whitespace) + * and property 7 (duplicate keys) exploit exactly that difference; the + * remaining properties pin the json functions against the jsonb oracle. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsontext.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsontext" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * JSON string/key contents are drawn only from this alphabet. Restricting to + * [a-zA-Z0-9] means a generated value never contains a single quote (or a + * backslash, or a double quote), so the whole JSON text can be dropped into a + * single-quoted SQL literal with no escaping hazard. + */ +#define JSON_ALPHABET \ + "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789" + +/* Draw a quoted JSON string of 0..4 safe characters into buf. */ +static void +draw_json_string(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 4); + + appendStringInfoChar(buf, '"'); + for (int i = 0; i < n; i++) + { + int idx = (int) pg_prop_draw_int(tc, 0, (int) strlen(JSON_ALPHABET) - 1); + + appendStringInfoChar(buf, JSON_ALPHABET[idx]); + } + appendStringInfoChar(buf, '"'); +} + +/* Draw one JSON scalar (integer, string, true, false, or null) into buf. */ +static void +draw_json_scalar(pg_prop_tc *tc, StringInfo buf) +{ + switch ((int) pg_prop_draw_int(tc, 0, 4)) + { + case 0: + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, -1000, 1000)); + break; + case 1: + draw_json_string(tc, buf); + break; + case 2: + appendStringInfoString(buf, "true"); + break; + case 3: + appendStringInfoString(buf, "false"); + break; + default: + appendStringInfoString(buf, "null"); + break; + } +} + +/* + * Draw a random, always-valid JSON text value into buf. + * + * The depth budget bounds nesting so a counterexample stays legible and the + * recursion always terminates: at depth <= 0, or ~1/3 of the time otherwise, + * emit a scalar; else emit an array or object with 0..3 children one level + * shallower. Separators are "exactly one space after each ':' and ','", so + * the text carries deliberate insignificant whitespace -- json preserves it + * verbatim (which the preservation property checks), while jsonb would strip + * it. Object keys are k0..k(n-1), always distinct within one object. + */ +static void +draw_json(pg_prop_tc *tc, StringInfo buf, int depth) +{ + int nchildren; + + if (depth <= 0 || pg_prop_draw_int(tc, 0, 2) == 0) + { + draw_json_scalar(tc, buf); + return; + } + + if (pg_prop_draw_bool(tc)) + { + /* array of 0..3 elements */ + nchildren = (int) pg_prop_draw_int(tc, 0, 3); + appendStringInfoChar(buf, '['); + for (int i = 0; i < nchildren; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + draw_json(tc, buf, depth - 1); + } + appendStringInfoChar(buf, ']'); + } + else + { + /* object of 0..3 members with distinct keys k0..k(n-1) */ + nchildren = (int) pg_prop_draw_int(tc, 0, 3); + appendStringInfoChar(buf, '{'); + for (int i = 0; i < nchildren; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "\"k%d\": ", i); + draw_json(tc, buf, depth - 1); + } + appendStringInfoChar(buf, '}'); + } +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the json code errored or produced NULL on finite valid + * JSON, which is itself a bug -- so either is a failure, reported with the + * context (the JSON text or expression under test) and the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql, + const char *context) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for %s\n sql: %s\n error: %s", + context, sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for %s\n sql: %s", + context, sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for %s\n sql: %s", + res, context, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * The json type stores text verbatim. draw_json injects exactly one space + * after every ':' and ',', so the generated text carries insignificant + * whitespace that jsonb would strip but json must keep. Casting the json + * value back to text must reproduce the original bytes exactly -- json does + * NOT reformat. Any "f" means json lost or rewrote the input text. + */ +PG_PROP(json_preserves_text_exactly) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_json(tc, &j, 3); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('%s'::json)::text = '%s'", + j.data, j.data); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Two routes to jsonb must agree as values. Parsing text as json and then + * casting that json to jsonb normalizes it; parsing the same text directly as + * jsonb normalizes it too. The intermediate json carries whitespace/order + * the direct jsonb does not, but once both are jsonb the values must be + * EQUAL. A difference means the json->jsonb cast and the jsonb parser build + * different normal forms from the same input. + */ +PG_PROP(json_vs_jsonb_normalization_differs_or_equal) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_json(tc, &j, 3); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('%s'::json::jsonb) = ('%s'::jsonb)", + j.data, j.data); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * json_array_length counts elements. Build an array literal of exactly n + * drawn scalars (n in 0..5) and assert json_array_length reports n. Spaces + * after commas are insignificant and must not change the count. + */ +PG_PROP(json_array_length) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 0, 5); + StringInfoData arr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&arr); + appendStringInfoChar(&arr, '['); + for (int i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(&arr, ", "); + draw_json_scalar(tc, &arr); + } + appendStringInfoChar(&arr, ']'); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT json_array_length('%s'::json) = %d", + arr.data, n); + + require_true(tc, conn, q.data, arr.data); + + pfree(arr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The json field accessor agrees with the jsonb one. Build an object with a + * known first key "k0" and a drawn value, then compare json's -> 'k0' (which + * returns json) cast to jsonb against jsonb's -> 'k0'. IS NOT DISTINCT FROM + * is used so a missing key (both NULL) also matches and formatting never + * trips the comparison -- only the normalized values are compared. + */ +PG_PROP(json_object_field_matches_jsonb) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + obj, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* object {"k0": } with a known key */ + initStringInfo(&v); + draw_json_scalar(tc, &v); + initStringInfo(&obj); + appendStringInfo(&obj, "{\"k0\": %s}", v.data); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (('%s'::json -> 'k0')::jsonb) " + "IS NOT DISTINCT FROM ('%s'::jsonb -> 'k0')", + obj.data, obj.data); + + require_true(tc, conn, q.data, obj.data); + + pfree(v.data); + pfree(obj.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The json and jsonb array builders agree. json_build_array and + * jsonb_build_array take the same scalar arguments; parsing the json builder's + * result as jsonb must equal the jsonb builder's result. Two independent + * construction paths land on the same normalized value. Arguments are drawn + * as plain integers so they are valid SQL constants on both sides. + */ +PG_PROP(json_build_roundtrips_through_jsonb) +{ + pg_prop_conn *conn; + char *err = NULL; + int nargs = (int) pg_prop_draw_int(tc, 0, 4); + StringInfoData args, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&args); + for (int i = 0; i < nargs; i++) + { + if (i > 0) + appendStringInfoString(&args, ", "); + appendStringInfo(&args, "%d", (int) pg_prop_draw_int(tc, -1000, 1000)); + } + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (json_build_array(%s)::jsonb) = " + "jsonb_build_array(%s)", + args.data, args.data); + + require_true(tc, conn, q.data, q.data); + + pfree(args.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * json_each yields one row per member. Build an object with m DISTINCT keys + * k0..k(m-1) (m in 0..5); json_each must return exactly m rows. Distinct keys + * keep the oracle simple -- the duplicate-key case (where json keeps both) is + * exercised separately below. + */ +PG_PROP(json_each_count_matches) +{ + pg_prop_conn *conn; + char *err = NULL; + int m = (int) pg_prop_draw_int(tc, 0, 5); + StringInfoData obj, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&obj); + appendStringInfoChar(&obj, '{'); + for (int i = 0; i < m; i++) + { + StringInfoData v; + + if (i > 0) + appendStringInfoString(&obj, ", "); + initStringInfo(&v); + draw_json_scalar(tc, &v); + appendStringInfo(&obj, "\"k%d\": %s", i, v.data); + pfree(v.data); + } + appendStringInfoChar(&obj, '}'); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM json_each('%s'::json)) = %d", + obj.data, m); + + require_true(tc, conn, q.data, obj.data); + + pfree(obj.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The headline json-vs-jsonb distinction: json preserves duplicate object + * keys verbatim, jsonb dedups them. For the fixed literal {"k": 1, "k": 2}, + * json_each returns BOTH members (2 rows) while jsonb_each keeps only the last + * (1 row). Fixed input, no draw -- this pins the behavioural contract itself. + */ +PG_PROP(json_duplicate_keys_preserved) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + q = "SELECT (SELECT count(*) FROM json_each('{\"k\": 1, \"k\": 2}'::json)) = 2 " + "AND (SELECT count(*) FROM jsonb_each('{\"k\": 1, \"k\": 2}'::jsonb)) = 1"; + + require_true(tc, conn, q, q); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 0ef13f7468389..97c9dc5c1ad11 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From e279a1fa03d9c69b09e132a7c879f3574683137b Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:08 -0400 Subject: [PATCH 099/338] prop: add JSON formatting, dedup, and normalization property tests (jsonformat) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonformat.c | 401 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 405 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonformat.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 73b85d4ba2006..6755c267adfc2 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -114,7 +114,8 @@ OBJS = \ prop_jsonnest.o \ prop_jsonbuild.o \ prop_jsonconstruct.o \ - prop_jsontext.o + prop_jsontext.o \ + prop_jsonformat.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 253d6ecf45299..13c810493b106 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -112,6 +112,7 @@ prop_area_sources = files( 'prop_jsonbuild.c', 'prop_jsonconstruct.c', 'prop_jsontext.c', + 'prop_jsonformat.c', ) diff --git a/src/test/prop/prop_jsonformat.c b/src/test/prop/prop_jsonformat.c new file mode 100644 index 0000000000000..c4605d8a9d33d --- /dev/null +++ b/src/test/prop/prop_jsonformat.c @@ -0,0 +1,401 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonformat.c + * Property-based tests for jsonb text formatting and canonicalization + * (src/backend/utils/adt/jsonb.c, json.c: jsonb_pretty, the jsonb parser's + * normal form, row_to_json, jsonb_build_array). + * + * jsonb stores a normalized binary tree, so its text output is a canonical + * rendering: pretty-printing is a pure formatting layer that must reparse to + * the identical value, and the parser must collapse duplicate keys (last + * wins) and impose a fixed key order regardless of input order. These + * properties check those formatting/canonicalization contracts as exact + * identities; any "f" is a bug in the renderer or the parser's normal form, + * not a tolerance artefact. This is distinct from prop_jsontext (verbatim + * json), prop_jsonb (jsonb value ops) and prop_jsonbuild (builder algebra): + * here the subject is the FORMATTING round-trip and canonical normal form. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonformat.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonformat" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * JSON string/key contents are drawn only from this alphabet (letters and + * digits), so a generated value never contains a single quote, double quote, + * or backslash and the whole JSON text drops into a single-quoted SQL literal + * with no escaping hazard. + */ +#define JSON_ALPHABET \ + "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789" + +/* Draw a quoted JSON string of 0..4 safe characters into buf. */ +static void +draw_json_string(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 4); + int i; + + appendStringInfoChar(buf, '"'); + for (i = 0; i < n; i++) + { + int idx = (int) pg_prop_draw_int(tc, 0, (int) strlen(JSON_ALPHABET) - 1); + + appendStringInfoChar(buf, JSON_ALPHABET[idx]); + } + appendStringInfoChar(buf, '"'); +} + +/* Draw one JSON scalar (integer, string, true, false, or null) into buf. */ +static void +draw_json_scalar(pg_prop_tc *tc, StringInfo buf) +{ + switch ((int) pg_prop_draw_int(tc, 0, 4)) + { + case 0: + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, -1000, 1000)); + break; + case 1: + draw_json_string(tc, buf); + break; + case 2: + appendStringInfoString(buf, "true"); + break; + case 3: + appendStringInfoString(buf, "false"); + break; + default: + appendStringInfoString(buf, "null"); + break; + } +} + +/* + * Draw a random, always-valid JSON text value into buf with DISTINCT object + * keys (k0..k(n-1)), so no draw ever introduces an incidental duplicate key. + * + * The depth budget bounds nesting so a counterexample stays legible and the + * recursion terminates: at depth <= 0, or ~1/3 of the time otherwise, emit a + * scalar; else an array or object with 0..3 children one level shallower. + * Separators carry a space after ':' and ',', which jsonb strips -- so the + * input is deliberately not already in jsonb's canonical form. + */ +static void +draw_json(pg_prop_tc *tc, StringInfo buf, int depth) +{ + int nchildren; + int i; + + if (depth <= 0 || pg_prop_draw_int(tc, 0, 2) == 0) + { + draw_json_scalar(tc, buf); + return; + } + + if (pg_prop_draw_bool(tc)) + { + /* array of 0..3 elements */ + nchildren = (int) pg_prop_draw_int(tc, 0, 3); + appendStringInfoChar(buf, '['); + for (i = 0; i < nchildren; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + draw_json(tc, buf, depth - 1); + } + appendStringInfoChar(buf, ']'); + } + else + { + /* object of 0..3 members with distinct keys k0..k(n-1) */ + nchildren = (int) pg_prop_draw_int(tc, 0, 3); + appendStringInfoChar(buf, '{'); + for (i = 0; i < nchildren; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "\"k%d\": ", i); + draw_json(tc, buf, depth - 1); + } + appendStringInfoChar(buf, '}'); + } +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the formatting/parsing code errored or produced NULL on + * finite valid JSON, which is itself a bug -- so either is a failure, + * reported with the context (the JSON text or expression under test). + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql, + const char *context) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for %s\n sql: %s\n error: %s", + context, sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for %s\n sql: %s", + context, sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for %s\n sql: %s", + res, context, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * 1. pretty_reparse: pretty-printing is a lossless formatting layer. + * jsonb_pretty(J) rewrites J with indentation and newlines; reparsing that + * text as jsonb must yield exactly J back. Any "f" means jsonb_pretty + * emitted text that no longer parses to the same value -- a renderer bug. + */ +PG_PROP(pretty_reparse) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_json(tc, &j, 3); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_pretty('%s'::jsonb)::jsonb = '%s'::jsonb", + j.data, j.data); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. jsonb_dedup_keys: a JSON object text with a duplicate key is normalized + * by jsonb to keep the LAST occurrence. Fixed input pins the contract: + * '{"a":1,"a":2}'::jsonb equals '{"a":2}'::jsonb. Any "f" means jsonb kept + * the wrong duplicate (or neither). + */ +PG_PROP(jsonb_dedup_keys) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + q = "SELECT '{\"a\":1,\"a\":2}'::jsonb = '{\"a\":2}'::jsonb"; + + require_true(tc, conn, q, q); + + pg_prop_disconnect(conn); +} + +/* + * 3. jsonb_key_order_normalized: jsonb imposes a canonical key order + * independent of input order, so two objects with the same members written in + * different orders are EQUAL. Fixed input: '{"b":1,"a":2}' = '{"a":2,"b":1}'. + * Any "f" means jsonb's normal form is sensitive to input key order. + */ +PG_PROP(jsonb_key_order_normalized) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + q = "SELECT '{\"b\":1,\"a\":2}'::jsonb = '{\"a\":2,\"b\":1}'::jsonb"; + + require_true(tc, conn, q, q); + + pg_prop_disconnect(conn); +} + +/* + * 4. json_preserves_vs_jsonb: parsing a json value and casting it to jsonb + * must land on the same value as parsing the identical text directly as + * jsonb. Both routes normalize to jsonb; the drawn integer n exercises the + * numeric path through json's text form into the canonical binary tree. Any + * "f" means the json->jsonb cast and the jsonb parser disagree on the value. + */ +PG_PROP(json_preserves_vs_jsonb) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, -1000000, 1000000); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('{\"x\": %d}'::json)::jsonb = ('{\"x\": %d}'::jsonb)", + n, n); + + require_true(tc, conn, q.data, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. build_array_length: jsonb_build_array builds an array whose length is the + * number of arguments. Three fixed args plus a drawn n gives a 4-element + * array, so jsonb_array_length must be 4 for every n. Any "f" means the + * builder dropped or duplicated an element. + */ +PG_PROP(build_array_length) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, -1000000, 1000000); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_array_length(jsonb_build_array(1, 2, 3, %d)) = 4", + n); + + require_true(tc, conn, q.data, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. row_to_json_roundtrip: row_to_json names anonymous ROW fields f1, f2, + * ...; casting that json to jsonb and extracting field 'f1' must equal the + * jsonb built directly from the first ROW value. Draw an int n (first field) + * and a short safe string s (second field, [a-c], no escaping hazard). Any + * "f" means row_to_json mislabeled or mis-rendered the first column. + */ +PG_PROP(row_to_json_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, -1000000, 1000000); + char *s; + size_t slen; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + s = pg_prop_draw_from(tc, "abc", 0, 4, &slen); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (row_to_json(ROW((%d), '%s'))::jsonb -> 'f1') " + "= to_jsonb((%d))", + n, s, n); + + require_true(tc, conn, q.data, q.data); + + pfree(q.data); + free(s); + pg_prop_disconnect(conn); +} + +/* + * 7. jsonb_scalar_roundtrip: a jsonb scalar survives a text round-trip. + * to_jsonb(n) rendered to text and reparsed as jsonb must equal to_jsonb(n). + * Draw an int n to exercise the numeric scalar's canonical text form. Any + * "f" means the scalar's text output does not reparse to itself. + */ +PG_PROP(jsonb_scalar_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, -1000000, 1000000); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (to_jsonb((%d)))::text::jsonb = to_jsonb((%d))", + n, n); + + require_true(tc, conn, q.data, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 8. pretty_idempotent_text: jsonb_pretty is stable under reparse at the TEXT + * level, not just the value level -- pretty-printing an already-pretty value + * yields byte-identical text. jsonb_pretty(J) = jsonb_pretty(jsonb_pretty(J) + * ::jsonb). Any "f" means the renderer is not idempotent (e.g. accumulates + * or drops indentation across a round-trip). + */ +PG_PROP(pretty_idempotent_text) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_json(tc, &j, 3); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_pretty('%s'::jsonb) " + "= jsonb_pretty(jsonb_pretty('%s'::jsonb)::jsonb)", + j.data, j.data); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 97c9dc5c1ad11..e8a931cd31881 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 6fd01bd424fcb9cae4ea29b13735b61f83f26d60 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:08 -0400 Subject: [PATCH 100/338] prop: add jsonpath property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonpath.c | 394 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 398 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonpath.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 6755c267adfc2..ce31f5d2c3e4e 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -115,7 +115,8 @@ OBJS = \ prop_jsonbuild.o \ prop_jsonconstruct.o \ prop_jsontext.o \ - prop_jsonformat.o + prop_jsonformat.o \ + prop_jsonpath.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 13c810493b106..1ab2ffac32ebe 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -113,6 +113,7 @@ prop_area_sources = files( 'prop_jsonconstruct.c', 'prop_jsontext.c', 'prop_jsonformat.c', + 'prop_jsonpath.c', ) diff --git a/src/test/prop/prop_jsonpath.c b/src/test/prop/prop_jsonpath.c new file mode 100644 index 0000000000000..9ad46cbb8e1cd --- /dev/null +++ b/src/test/prop/prop_jsonpath.c @@ -0,0 +1,394 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonpath.c + * Property-based tests for jsonpath (src/backend/utils/adt/jsonpath*). + * + * jsonpath has had ~22 correctness fixes; these check that jsonpath query + * operators agree with independent jsonb accessors and that + * predicate/existence semantics are consistent. + * + * The oracle is jsonb's own element/member accessors: the member accessor + * (->), the array-element accessor (-> int), and jsonb_array_elements reach + * the same values a jsonpath selects, by a code path jsonpath does not share. + * So jsonb_path_query_first against a known path must agree with ->, a + * numeric filter's match count must agree with an independent unnest count, + * and $ must select the whole document. Any "f" (or a NULL/error on finite + * valid JSON where the property does not tolerate it) is a jsonpath bug. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonpath.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonpath" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * JSON string/key contents are drawn only from this alphabet. Restricting to + * [a-zA-Z0-9] means a generated value never contains a single quote (or a + * backslash, or a double quote), so the whole JSON text can be dropped into a + * single-quoted SQL literal with no escaping hazard. The jsonpath argument is + * likewise passed as a bare single-quoted string. + */ +#define JSON_ALPHABET \ + "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789" + +/* Draw a quoted JSON string of 0..4 safe characters into buf. */ +static void +draw_json_string(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 4); + + appendStringInfoChar(buf, '"'); + for (int i = 0; i < n; i++) + { + int idx = (int) pg_prop_draw_int(tc, 0, (int) strlen(JSON_ALPHABET) - 1); + + appendStringInfoChar(buf, JSON_ALPHABET[idx]); + } + appendStringInfoChar(buf, '"'); +} + +/* Draw one JSON scalar (integer, string, true, false, or null) into buf. */ +static void +draw_json_scalar(pg_prop_tc *tc, StringInfo buf) +{ + switch ((int) pg_prop_draw_int(tc, 0, 4)) + { + case 0: + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, -1000, 1000)); + break; + case 1: + draw_json_string(tc, buf); + break; + case 2: + appendStringInfoString(buf, "true"); + break; + case 3: + appendStringInfoString(buf, "false"); + break; + default: + appendStringInfoString(buf, "null"); + break; + } +} + +/* + * Draw a random, always-valid JSON text value into buf. + * + * The depth budget bounds nesting so a counterexample stays legible and the + * recursion always terminates: at depth <= 0, or ~1/3 of the time otherwise, + * emit a scalar; else emit an array or object with 0..3 children one level + * shallower. Keys are "k0".."k2". + */ +static void +draw_json(pg_prop_tc *tc, StringInfo buf, int depth) +{ + int nchildren; + + if (depth <= 0 || pg_prop_draw_int(tc, 0, 2) == 0) + { + draw_json_scalar(tc, buf); + return; + } + + if (pg_prop_draw_bool(tc)) + { + /* array of 0..3 elements */ + nchildren = (int) pg_prop_draw_int(tc, 0, 3); + appendStringInfoChar(buf, '['); + for (int i = 0; i < nchildren; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + draw_json(tc, buf, depth - 1); + } + appendStringInfoChar(buf, ']'); + } + else + { + /* object of 0..3 members with keys "k0".."k2" */ + nchildren = (int) pg_prop_draw_int(tc, 0, 3); + appendStringInfoChar(buf, '{'); + for (int i = 0; i < nchildren; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "\"k%d\": ", i); + draw_json(tc, buf, depth - 1); + } + appendStringInfoChar(buf, '}'); + } +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means jsonpath errored or produced NULL on finite valid JSON, + * which is itself a bug -- so either is a failure, reported with the context + * (the JSON text or expression under test) and the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql, + const char *context) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for %s\n sql: %s\n error: %s", + context, sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for %s\n sql: %s", + context, sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for %s\n sql: %s", + res, context, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * $ selects the whole document, so for any JSON value J, + * jsonb_path_exists(J, '$') is true. This is the most basic jsonpath + * guarantee and must hold for every scalar, array, and object. + */ +PG_PROP(exists_root_always_true) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_json(tc, &j, 3); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_path_exists('%s'::jsonb, '$')", + j.data); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The member-selecting jsonpath agrees with the member accessor: for an object + * {"k0": V}, jsonb_path_query_first(J, '$.k0') equals J -> 'k0'. The accessor + * (->) reaches the member by a code path jsonpath does not share, so this + * cross-checks jsonpath's object key resolution. IS NOT DISTINCT FROM makes + * the comparison NULL-safe even though J always has the key here. + */ +PG_PROP(path_query_first_matches_accessor) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData v, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&v); + draw_json_scalar(tc, &v); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_path_query_first('{\"k0\": %s}'::jsonb, '$.k0') " + "IS NOT DISTINCT FROM ('{\"k0\": %s}'::jsonb -> 'k0')", + v.data, v.data); + + require_true(tc, conn, q.data, q.data); + + pfree(v.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The index-selecting jsonpath agrees with the array-element accessor: for an + * array of drawn scalars and an in-range index i, + * jsonb_path_query_first(J, '$[i]') equals J -> i. The accessor (-> int) + * reaches the element independently of jsonpath's subscript handling. + */ +PG_PROP(array_index_matches_accessor) +{ + pg_prop_conn *conn; + char *err = NULL; + int len = (int) pg_prop_draw_int(tc, 1, 5); + int idx; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + appendStringInfoChar(&j, '['); + for (int i = 0; i < len; i++) + { + if (i > 0) + appendStringInfoString(&j, ", "); + draw_json_scalar(tc, &j); + } + appendStringInfoChar(&j, ']'); + + idx = (int) pg_prop_draw_int(tc, 0, len - 1); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_path_query_first('%s'::jsonb, '$[%d]') " + "IS NOT DISTINCT FROM ('%s'::jsonb -> %d)", + j.data, idx, j.data, idx); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Existence is consistent with query: jsonb_path_exists(J, '$.k0') is true + * exactly when jsonb_path_query_first(J, '$.k0') is not SQL NULL. Note that a + * present JSON null value makes query_first return 'null'::jsonb (not SQL + * NULL) while exists is still true, so the two agree only against IS NOT NULL + * -- which is the point. J is an object that sometimes has k0 and sometimes + * does not, so both branches are exercised. + */ +PG_PROP(exists_consistent_with_query) +{ + pg_prop_conn *conn; + char *err = NULL; + bool present = pg_prop_draw_bool(tc); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + appendStringInfoChar(&j, '{'); + if (present) + { + appendStringInfoString(&j, "\"k0\": "); + draw_json_scalar(tc, &j); + } + appendStringInfoChar(&j, '}'); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_path_exists('%s'::jsonb, '$.k0') = " + "(jsonb_path_query_first('%s'::jsonb, '$.k0') IS NOT NULL)", + j.data, j.data); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A numeric filter selects correctly: for an array of small integers and a + * threshold t, the number of elements jsonpath keeps with '$[*] ? (@ > t)' + * equals the count computed independently by unnesting the array and filtering + * in SQL. jsonb_array_elements is the independent oracle; the array holds only + * integers so e::text::int is safe. This exercises jsonpath filter evaluation. + */ +PG_PROP(numeric_filter_selects_correctly) +{ + pg_prop_conn *conn; + char *err = NULL; + int len = (int) pg_prop_draw_int(tc, 1, 6); + int thresh = (int) pg_prop_draw_int(tc, -1000, 1000); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + appendStringInfoChar(&j, '['); + for (int i = 0; i < len; i++) + { + if (i > 0) + appendStringInfoString(&j, ", "); + appendStringInfo(&j, "%d", (int) pg_prop_draw_int(tc, -1000, 1000)); + } + appendStringInfoChar(&j, ']'); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM " + "jsonb_path_query('%s'::jsonb, '$[*] ? (@ > %d)') q) " + "= (SELECT count(*) FROM " + "jsonb_array_elements('%s'::jsonb) e " + "WHERE (e::text::int) > %d)", + j.data, thresh, j.data, thresh); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * lax and strict modes agree on a bare scalar selected by '$': for a scalar J, + * jsonb_path_query_first(J, 'strict $') equals jsonb_path_query_first(J, + * 'lax $'). The mode only changes how structural accessors treat missing + * members and non-arrays; selecting the whole document is identical either + * way. IS NOT DISTINCT FROM keeps the comparison NULL-safe. + */ +PG_PROP(lax_strict_consistent_on_scalar) +{ + pg_prop_conn *conn; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, -1000000, 1000000); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_path_query_first('%d'::jsonb, 'strict $') " + "IS NOT DISTINCT FROM " + "jsonb_path_query_first('%d'::jsonb, 'lax $')", + n, n); + + require_true(tc, conn, q.data, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index e8a931cd31881..17580be1de351 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From e077a74c79c4fa457daacf4928c75af3d7c40830 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:08 -0400 Subject: [PATCH 101/338] prop: add jsonb_path_query/exists/match operators property tests (jsonpathq) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonpathq.c | 321 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 325 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonpathq.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index ce31f5d2c3e4e..ae885ff9a0e54 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -116,7 +116,8 @@ OBJS = \ prop_jsonconstruct.o \ prop_jsontext.o \ prop_jsonformat.o \ - prop_jsonpath.o + prop_jsonpath.o \ + prop_jsonpathq.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 1ab2ffac32ebe..547ea82599373 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -114,6 +114,7 @@ prop_area_sources = files( 'prop_jsontext.c', 'prop_jsonformat.c', 'prop_jsonpath.c', + 'prop_jsonpathq.c', ) diff --git a/src/test/prop/prop_jsonpathq.c b/src/test/prop/prop_jsonpathq.c new file mode 100644 index 0000000000000..05a6b2dc2f405 --- /dev/null +++ b/src/test/prop/prop_jsonpathq.c @@ -0,0 +1,321 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonpathq.c + * Property-based tests for jsonb SQL/JSON path QUERY functions. + * + * These exercise the jsonpath query evaluators -- jsonb_path_exists, + * jsonb_path_query, jsonb_path_query_array, jsonb_path_match, and the @? / @@ + * operators -- as applied to small built-in jsonb objects. The oracle is + * each function's documented contract cross-checked against the plain jsonb + * accessors it must agree with: an existence query on '$.a' agrees with the + * key-exists operator ?, a single-match query returns the same value as ->, + * a filter query selecting '$.arr[*]' returns every array element, a numeric + * filter predicate '$.a > k' agrees with the plain integer comparison, the + * @? / @@ operators agree with their function spellings, and a value filter + * never selects more elements than the array holds. Any "f" is a bug in the + * jsonpath query evaluator, not a modelling slop on our side. (Distinct from + * areas "jsonpath" -- the jsonpath TYPE's parse/print -- and "jsonbpath" -- + * the text[]-path mutation functions jsonb_set/#-; this area is the jsonpath + * QUERY functions and operators.) + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonpathq.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonpathq" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a jsonb object literal of the fixed shape {"a": , "arr": [...]} + * as text '{"a": 3, "arr": [1, 4]}'::jsonb. Both members are always present: + * "a" is a small int 0..9, "arr" is 0..5 small ints 0..9. Values are tiny + * ints so the literal is trivially valid jsonb with no quoting hazard, and + * the jsonpath strings over it ('$.a', '$.arr[*]', ...) stay simple. + */ +static void +draw_object(pg_prop_tc *tc, StringInfo buf) +{ + int narr = (int) pg_prop_draw_int(tc, 0, 5); + int i; + + appendStringInfo(buf, "'{\"a\": %d, \"arr\": [", + (int) pg_prop_draw_int(tc, 0, 9)); + for (i = 0; i < narr; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, 0, 9)); + } + appendStringInfoString(buf, "]}'::jsonb"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on a valid input, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * path_exists_matches_contains_key: jsonb_path_exists(J, '$.a') agrees with + * the key-exists operator (J ? 'a'). "a" is always present in J, so both + * are true; the point is that the path evaluator and the accessor agree. + */ +PG_PROP(path_exists_matches_contains_key) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_path_exists(%s, '$.a') = (%s ? 'a')", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * path_query_returns_value: jsonb_path_query(J, '$.a') returns exactly the + * value J -> 'a'. '$.a' has a single match (one scalar member), so the + * set-returning function yields one row equal to the plain accessor. + */ +PG_PROP(path_query_returns_value) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_path_query(%s, '$.a') = (%s -> 'a')", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * path_match_filter: for the numeric member "a", jsonb_path_match with the + * predicate '$.a > k' agrees with the plain integer comparison + * ((J ->> 'a')::int > k) for a drawn k. k is drawn across and beyond the + * value range of "a" (0..9) so both the true and false sides are exercised. + */ +PG_PROP(path_match_filter) +{ + pg_prop_conn *conn; + char *err = NULL; + int k = (int) pg_prop_draw_int(tc, -2, 11); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_path_match(%s, '$.a > %d') " + "= ((%s ->> 'a')::int > %d)", + j.data, k, j.data, k); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_path_count: jsonb_path_query_array(J, '$.arr[*]') selects every + * element of "arr", so its jsonb_array_length equals the length of J -> 'arr'. + * This checks the wildcard array accessor returns the whole array, no more and + * no fewer. + */ +PG_PROP(array_path_count) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_array_length(" + "jsonb_path_query_array(%s, '$.arr[*]')) " + "= jsonb_array_length(%s -> 'arr')", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * exists_operator_consistency: the @? operator is the operator spelling of + * jsonb_path_exists, so (J @? '$.a') equals jsonb_path_exists(J, '$.a') for + * every J. + */ +PG_PROP(exists_operator_consistency) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s @? '$.a') = jsonb_path_exists(%s, '$.a')", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * match_operator_consistency: the @@ operator is the operator spelling of + * jsonb_path_match, so (J @@ '$.a > k') agrees with jsonb_path_match(J, + * '$.a > k'). IS NOT DISTINCT FROM so any NULL (should not arise for this + * shape, but is sound either way) compares equal on both sides. + */ +PG_PROP(match_operator_consistency) +{ + pg_prop_conn *conn; + char *err = NULL; + int k = (int) pg_prop_draw_int(tc, -2, 11); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s @@ '$.a > %d') " + "IS NOT DISTINCT FROM jsonb_path_match(%s, '$.a > %d')", + j.data, k, j.data, k); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * filter_selects_subset: a value filter can only drop elements, never add + * them, so jsonb_path_query_array(J, '$.arr[*] ? (@ > k)') is never longer + * than the whole array J -> 'arr'. A sound one-sided bound (<=) regardless + * of how many elements pass the filter for the drawn k. + */ +PG_PROP(filter_selects_subset) +{ + pg_prop_conn *conn; + char *err = NULL; + int k = (int) pg_prop_draw_int(tc, -2, 11); + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_array_length(" + "jsonb_path_query_array(%s, '$.arr[*] ? (@ > %d)')) " + "<= jsonb_array_length(%s -> 'arr')", + j.data, k, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 17580be1de351..e5e9f2b4812a6 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 58374d86ea03bdca469c986abd6c620db01a534e Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:08 -0400 Subject: [PATCH 102/338] prop: add jsonpath arithmetic and filter methods property tests (jsonpathmath) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonpathmath.c | 411 ++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 415 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonpathmath.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index ae885ff9a0e54..94f1b9a0ea036 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -117,7 +117,8 @@ OBJS = \ prop_jsontext.o \ prop_jsonformat.o \ prop_jsonpath.o \ - prop_jsonpathq.o + prop_jsonpathq.o \ + prop_jsonpathmath.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 547ea82599373..f609d4bdf12fe 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -115,6 +115,7 @@ prop_area_sources = files( 'prop_jsonformat.c', 'prop_jsonpath.c', 'prop_jsonpathq.c', + 'prop_jsonpathmath.c', ) diff --git a/src/test/prop/prop_jsonpathmath.c b/src/test/prop/prop_jsonpathmath.c new file mode 100644 index 0000000000000..244c7d7a06d0b --- /dev/null +++ b/src/test/prop/prop_jsonpathmath.c @@ -0,0 +1,411 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonpathmath.c + * Property-based tests for jsonpath arithmetic and filter methods. + * + * This area targets the jsonpath item methods and in-path arithmetic + * (.size(), .type(), .abs(), .double(), @ comparisons in filters, and + * expressions like '$.a + 1') rather than the bare member/index/existence + * selection that prop_jsonpath / prop_jsonbpath / prop_jsonpathq cover. + * + * The oracle is an independent jsonb accessor computing the same answer by a + * code path the jsonpath method evaluator does not share: jsonb_array_length, + * J->>'a' cast to int/float8, abs()/to_jsonb() in SQL, and a subquery over + * jsonb_array_elements for the filter counts. Any "f" (or a NULL/error on + * finite valid JSON where the property does not tolerate it) is a jsonpath + * method bug. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonpathmath.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonpathmath" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Integer domains for J. "a" is drawn across a wide signed range (so abs() + * crosses zero and the double() cast exercises large magnitudes), and array + * elements are drawn -6..6 (a mix of signs including zero) so the sign-filter + * properties see positive, negative, and zero elements. + */ +#define A_MIN (-1000000) +#define A_MAX 1000000 +#define ARR_MIN (-6) +#define ARR_MAX 6 + +/* + * Build the JSON text {"a": , "arr": [<0..6 ints>]} into buf, and report + * the drawn "a" value and the array through *a_out and the StringInfo arr_out + * (a comma-joined list of the element values) so callers can form the oracle. + * All values are integers rendered with %d, so the text is safe to drop into a + * single-quoted SQL literal with no escaping. + */ +static void +draw_doc(pg_prop_tc *tc, StringInfo buf, int *a_out, StringInfo arr_out) +{ + int a = (int) pg_prop_draw_int(tc, A_MIN, A_MAX); + int len = (int) pg_prop_draw_int(tc, 0, 6); + + *a_out = a; + + appendStringInfo(buf, "{\"a\": %d, \"arr\": [", a); + for (int i = 0; i < len; i++) + { + int e = (int) pg_prop_draw_int(tc, ARR_MIN, ARR_MAX); + + if (i > 0) + { + appendStringInfoString(buf, ", "); + appendStringInfoString(arr_out, ", "); + } + appendStringInfo(buf, "%d", e); + appendStringInfo(arr_out, "%d", e); + } + appendStringInfoString(buf, "]}"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a jsonpath method errored or produced NULL on finite valid + * JSON, which is itself a bug -- so either is a failure, reported with the + * JSON text under test and the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql, + const char *context) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for %s\n sql: %s\n error: %s", + context, sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for %s\n sql: %s", + context, sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for %s\n sql: %s", + res, context, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * .size() on an array equals its length: jsonb_path_query_first(J, '$.arr.size()') + * equals to_jsonb(jsonb_array_length(J->'arr')). jsonb_array_length reaches + * the length by a code path the .size() method does not share. + */ +PG_PROP(size_method) +{ + pg_prop_conn *conn; + char *err = NULL; + int a; + StringInfoData j, + arr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + initStringInfo(&arr); + draw_doc(tc, &j, &a, &arr); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_path_query_first('%s'::jsonb, '$.arr.size()') " + "= to_jsonb(jsonb_array_length('%s'::jsonb -> 'arr'))", + j.data, j.data); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(arr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * .type() reports the item kind: '$.a.type()' is '"number"' and + * '$.arr.type()' is '"array"'. "a" is always an integer and "arr" always an + * array in the generated document, so both are exact constants. + */ +PG_PROP(type_method) +{ + pg_prop_conn *conn; + char *err = NULL; + int a; + StringInfoData j, + arr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + initStringInfo(&arr); + draw_doc(tc, &j, &a, &arr); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_path_query_first('%s'::jsonb, '$.a.type()') " + "= '\"number\"'::jsonb " + "AND jsonb_path_query_first('%s'::jsonb, '$.arr.type()') " + "= '\"array\"'::jsonb", + j.data, j.data); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(arr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Arithmetic in a path: '$.a + 1' equals to_jsonb(a + 1) where a is the drawn + * integer. A_MAX + 1 is far below int4 overflow, so (a + 1) is computed in C + * and compared against jsonpath's own addition. + */ +PG_PROP(arithmetic_in_path) +{ + pg_prop_conn *conn; + char *err = NULL; + int a; + StringInfoData j, + arr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + initStringInfo(&arr); + draw_doc(tc, &j, &a, &arr); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_path_query_first('%s'::jsonb, '$.a + 1') " + "= to_jsonb((%d))", + j.data, a + 1); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(arr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A '> 0' filter keeps exactly the positive elements: the length of + * jsonb_path_query_array(J, '$.arr[*] ? (@ > 0)') equals the count of positive + * array elements, computed independently by a subquery over + * jsonb_array_elements. Elements are integers so e::text::int is safe. + */ +PG_PROP(filter_gt) +{ + pg_prop_conn *conn; + char *err = NULL; + int a; + StringInfoData j, + arr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + initStringInfo(&arr); + draw_doc(tc, &j, &a, &arr); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_array_length(" + "jsonb_path_query_array('%s'::jsonb, '$.arr[*] ? (@ > 0)')) " + "= (SELECT count(*) FROM " + "jsonb_array_elements('%s'::jsonb -> 'arr') e " + "WHERE (e::text::int) > 0)", + j.data, j.data); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(arr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * An equality filter's existence agrees with membership: for a drawn k, + * jsonb_path_exists(J, '$.arr[*] ? (@ == k)') is true exactly when k is an + * element of arr, determined independently via jsonb_array_elements. k is + * drawn from the same -6..6 domain so both the present and absent branches are + * exercised. + */ +PG_PROP(exists_filter) +{ + pg_prop_conn *conn; + char *err = NULL; + int a; + int k = (int) pg_prop_draw_int(tc, ARR_MIN, ARR_MAX); + StringInfoData j, + arr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + initStringInfo(&arr); + draw_doc(tc, &j, &a, &arr); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_path_exists('%s'::jsonb, '$.arr[*] ? (@ == %d)') " + "= EXISTS (SELECT 1 FROM " + "jsonb_array_elements('%s'::jsonb -> 'arr') e " + "WHERE (e::text::int) = %d)", + j.data, k, j.data, k); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(arr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * .abs() agrees with SQL abs(): '$.a.abs()' equals to_jsonb(abs(a)). "a" is + * drawn in [-1000000, 1000000], well inside int4, so abs() does not overflow + * and the comparison is exact. + */ +PG_PROP(abs_method) +{ + pg_prop_conn *conn; + char *err = NULL; + int a; + StringInfoData j, + arr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + initStringInfo(&arr); + draw_doc(tc, &j, &a, &arr); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_path_query_first('%s'::jsonb, '$.a.abs()') " + "= to_jsonb(abs((%d)))", + j.data, a); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(arr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * .double() agrees with the float8 cast: '$.a.double()' equals + * to_jsonb(a::float8). "a" is a small integer, exactly representable in + * float8, so both sides render the same numeric value. + */ +PG_PROP(double_method) +{ + pg_prop_conn *conn; + char *err = NULL; + int a; + StringInfoData j, + arr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + initStringInfo(&arr); + draw_doc(tc, &j, &a, &arr); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_path_query_first('%s'::jsonb, '$.a.double()') " + "= to_jsonb((%d)::float8)", + j.data, a); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(arr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A filter never grows the array: the length of + * jsonb_path_query_array(J, '$.arr[*] ? (@ >= 0)') is at most the length of + * arr. This is a sound one-sided bound for any array -- a filter can only + * select a subset -- and holds even when arr is empty (0 <= 0). + */ +PG_PROP(min_filter_count_le) +{ + pg_prop_conn *conn; + char *err = NULL; + int a; + StringInfoData j, + arr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + initStringInfo(&arr); + draw_doc(tc, &j, &a, &arr); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT jsonb_array_length(" + "jsonb_path_query_array('%s'::jsonb, '$.arr[*] ? (@ >= 0)')) " + "<= jsonb_array_length('%s'::jsonb -> 'arr')", + j.data, j.data); + + require_true(tc, conn, q.data, j.data); + + pfree(j.data); + pfree(arr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index e5e9f2b4812a6..8cb03d014cf7e 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 7b0ef7b584476137839b84a5f9cd028c8ec00e2c Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:08 -0400 Subject: [PATCH 103/338] prop: add jsonb_each, object_keys, and element round-trips property tests (jsoneach) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsoneach.c | 364 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 368 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsoneach.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 94f1b9a0ea036..efdd579dccfb0 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -118,7 +118,8 @@ OBJS = \ prop_jsonformat.o \ prop_jsonpath.o \ prop_jsonpathq.o \ - prop_jsonpathmath.o + prop_jsonpathmath.o \ + prop_jsoneach.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index f609d4bdf12fe..38fb0eb211833 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -116,6 +116,7 @@ prop_area_sources = files( 'prop_jsonpath.c', 'prop_jsonpathq.c', 'prop_jsonpathmath.c', + 'prop_jsoneach.c', ) diff --git a/src/test/prop/prop_jsoneach.c b/src/test/prop/prop_jsoneach.c new file mode 100644 index 0000000000000..4ab9048b02ff4 --- /dev/null +++ b/src/test/prop/prop_jsoneach.c @@ -0,0 +1,364 @@ +/*------------------------------------------------------------------------- + * + * prop_jsoneach.c + * Property-based tests for jsonb set-returning expansion functions. + * + * Exercises jsonb_each / jsonb_object_keys / jsonb_each_text / + * jsonb_array_elements and the build/aggregate inverses (jsonb_object_agg, + * jsonb_agg, jsonb_build_object). The oracle in every case is a round-trip + * or a count identity: expanding a jsonb object or array into rows and + * re-aggregating must reproduce the original value, and the number of keys or + * elements expanded must equal what was put in. A broken expansion (dropped, + * duplicated, or mis-ordered key/element) breaks one of these identities, so + * any "f" result is a bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsoneach.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsoneach" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the expansion errored or produced NULL on a well-formed + * jsonb value, which is itself a bug -- so either is a failure, reported with + * the SQL. (Same contract as prop_numeric.c's require_true.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build a jsonb object literal with n distinct keys k0..k(n-1), each mapped to + * a random small integer, into buf. n in [0, 5]. Keys are ASCII "k" so + * they are distinct by construction and need no quote-escaping. Returns n so + * the caller can gate key-dependent properties (e.g. k0 present only n >= 1). + */ +static int +draw_object(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 5); + + appendStringInfoString(buf, "'{"); + for (int i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(buf, ","); + appendStringInfo(buf, "\"k%d\": %d", + i, (int) pg_prop_draw_int(tc, -5, 5)); + } + appendStringInfoString(buf, "}'::jsonb"); + return n; +} + +/* + * Build a jsonb array literal with m random small-integer elements, m in + * [0, 5], into buf. Returns m. + */ +static int +draw_array(pg_prop_tc *tc, StringInfo buf) +{ + int m = (int) pg_prop_draw_int(tc, 0, 5); + + appendStringInfoString(buf, "'["); + for (int i = 0; i < m; i++) + { + if (i > 0) + appendStringInfoString(buf, ","); + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, -5, 5)); + } + appendStringInfoString(buf, "]'::jsonb"); + return m; +} + +/* + * Property 1 -- object_keys_count: jsonb_object_keys emits exactly one row per + * key, so count(*) over it equals the number of keys n in the object. + */ +PG_PROP(object_keys_count) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + n = draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM jsonb_object_keys(%s)) = %d", + j.data, n); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 2 -- each_reconstructs: expanding an object with jsonb_each and + * re-aggregating with jsonb_object_agg(key, value) reproduces the original + * object. The empty object has no rows, so jsonb_object_agg is NULL there; + * coalesce back to '{}' so the empty case round-trips soundly. + */ +PG_PROP(each_reconstructs) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + (void) draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(" + "(SELECT jsonb_object_agg(key, value) " + "FROM jsonb_each(%s)), '{}'::jsonb) = %s", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 3 -- each_text_values: for key k0, the text value jsonb_each_text + * reports for that key equals the object's ->> 'k0'. Only meaningful when the + * object has at least one key (k0 present); for the empty object the property + * is vacuously true, so guard with n >= 1. + */ +PG_PROP(each_text_values) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + n = draw_object(tc, &j); + if (n < 1) + { + /* no k0 to probe; discard this case */ + pg_prop_assume(tc, false); + pfree(j.data); + pg_prop_disconnect(conn); + return; + } + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT value FROM jsonb_each_text(%s) " + "WHERE key = 'k0') IS NOT DISTINCT FROM (%s ->> 'k0')", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 4 -- keys_are_subset: every key reported by jsonb_object_keys is + * actually present in the object (J ? key). bool_and over all keys must be + * true; coalesce(..., true) handles the empty object (no rows, NULL agg). + */ +PG_PROP(keys_are_subset) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + (void) draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(bool_and(%s ? k), true) " + "FROM jsonb_object_keys(%s) k", + j.data, j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 5 -- object_keys_distinct: a jsonb object has no duplicate keys, so + * jsonb_object_keys emits each key once -- count(*) equals count(DISTINCT key). + * The empty object gives 0 = 0, which holds. + */ +PG_PROP(object_keys_distinct) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&j); + (void) draw_object(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT count(*) = count(DISTINCT key) " + "FROM jsonb_object_keys(%s) AS key", + j.data); + + require_true(tc, conn, q.data); + + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 6 -- array_elements_count: jsonb_array_elements emits exactly one + * row per element, so count(*) over it equals the array length m. + */ +PG_PROP(array_elements_count) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int m; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + m = draw_array(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM jsonb_array_elements(%s)) = %d", + a.data, m); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 7 -- array_elements_roundtrip: expanding an array with + * jsonb_array_elements and re-aggregating with jsonb_agg (which preserves + * element order) reproduces the original array. The empty array has no rows, + * so jsonb_agg is NULL; coalesce back to '[]' so the empty case round-trips. + */ +PG_PROP(array_elements_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + (void) draw_array(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(" + "(SELECT jsonb_agg(e) FROM jsonb_array_elements(%s) e), " + "'[]'::jsonb) = %s", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 8 -- build_object_each: jsonb_build_object('a',1,'b',2) builds an + * object with exactly the keys a and b. Fixed-input sanity check that the + * build path and the key-expansion path agree: two keys, both present. + */ +PG_PROP(build_object_each) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "jsonb_object_keys(jsonb_build_object('a',1,'b',2))) = 2 " + "AND jsonb_build_object('a',1,'b',2) ? 'a' " + "AND jsonb_build_object('a',1,'b',2) ? 'b'"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 8cb03d014cf7e..f43928e8973b2 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From d87e8d09f7af3f9ae8328d32139a1969ba448692 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:09 -0400 Subject: [PATCH 104/338] prop: add array property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_array.c | 307 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 311 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_array.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index efdd579dccfb0..574b659282194 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -119,7 +119,8 @@ OBJS = \ prop_jsonpath.o \ prop_jsonpathq.o \ prop_jsonpathmath.o \ - prop_jsoneach.o + prop_jsoneach.o \ + prop_array.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 38fb0eb211833..b3b19ce75b6ed 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -117,6 +117,7 @@ prop_area_sources = files( 'prop_jsonpathq.c', 'prop_jsonpathmath.c', 'prop_jsoneach.c', + 'prop_array.c', ) diff --git a/src/test/prop/prop_array.c b/src/test/prop/prop_array.c new file mode 100644 index 0000000000000..6cba18bcbcb68 --- /dev/null +++ b/src/test/prop/prop_array.c @@ -0,0 +1,307 @@ +/*------------------------------------------------------------------------- + * + * prop_array.c + * Property-based tests for PostgreSQL array types and functions. + * + * Array in/out, subscripting, cardinality, and aggregation have had assorted + * fixes; these check round-trips and the aggregate/unnest inverse over + * integer arrays. + * + * The oracle is the array machinery's own algebra against SQL: text output + * parsed back must equal the array, cardinality must agree with array_length + * on a 1-D array, concatenation length is additive, an array contains itself, + * element membership agrees with an explicit unnest scan, and aggregating the + * unnest of an array (then unnesting and re-aggregating) yields the same + * sorted array. Any "f" (or an unexpected NULL/error) is a bug in the array + * code, not a modelling slip -- the properties are stated so NULL and empty + * edges stay crisp via coalesce()/IS NOT DISTINCT FROM. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_array.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "array" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a random SQL integer-array literal into buf as an ARRAY[...] expression + * typed ::int[]. n is drawn in 0..8 elements, each a small int in -20..20; + * with include_null set, elements may be SQL NULL (ARRAY[1, NULL, 3]). The + * ::int[] cast is always emitted so an empty array (ARRAY[]::int[]) and an + * all-NULL array still carry a type. Returns the element count n in case the + * caller wants it. + */ +static int +draw_arr_opt(pg_prop_tc *tc, StringInfo buf, bool include_null) +{ + int n = (int) pg_prop_draw_int(tc, 0, 8); + + appendStringInfoString(buf, "ARRAY["); + for (int i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + if (include_null && pg_prop_draw_int(tc, 0, 4) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, -20, 20)); + } + appendStringInfoString(buf, "]::int[]"); + + return n; +} + +/* Draw an integer array that may contain NULL elements. */ +static int +draw_arr(pg_prop_tc *tc, StringInfo buf) +{ + return draw_arr_opt(tc, buf, true); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the array operation errored or produced NULL on valid + * input, which is itself a bug -- so either is a failure, reported with the + * SQL and any error text. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * cardinality(A) = coalesce(array_length(A, 1), 0). For a 1-D array these + * agree: array_length is NULL for an empty array while cardinality is 0, so + * the coalesce reconciles the empty case. + */ +PG_PROP(cardinality_matches_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cardinality(%s) = coalesce(array_length(%s, 1), 0)", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Aggregate/unnest inverse, order-insensitively: aggregating the unnest of A, + * then unnesting that and re-aggregating, yields the same sorted array. Both + * sides sort with ORDER BY x, and IS NOT DISTINCT FROM makes the all-NULL and + * empty cases (where either side can be NULL) compare equal. unnest preserves + * NULL elements, so the multiset of elements is what is being compared. + */ +PG_PROP(agg_unnest_roundtrip_sorted) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT array_agg(x ORDER BY x) FROM unnest(%s) AS x) " + "IS NOT DISTINCT FROM " + "(SELECT array_agg(y ORDER BY y) " + " FROM unnest((SELECT array_agg(x) FROM unnest(%s) AS x)) AS y)", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Concatenation length is additive: + * coalesce(cardinality(A || B), 0) + * = coalesce(cardinality(A), 0) + coalesce(cardinality(B), 0). + * cardinality is already 0 for an empty array, but the coalesce guards the + * NULL that cardinality() returns if a concatenation ever yields NULL. + */ +PG_PROP(concat_length_additive) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_arr(tc, &a); + draw_arr(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(cardinality(%s || %s), 0) " + "= coalesce(cardinality(%s), 0) + coalesce(cardinality(%s), 0)", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Text round-trip: A::text::int[] IS NOT DISTINCT FROM A. Array output parsed + * back must equal the array; IS NOT DISTINCT FROM handles the all-NULL and + * empty edges where a plain = would yield NULL. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s)::text::int[] IS NOT DISTINCT FROM (%s)", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Containment is reflexive: A @> A. Every array contains itself, including + * the empty array. + */ +PG_PROP(contains_self) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + + /* + * Containment uses element equality, and SQL NULL is never equal to NULL, + * so an array containing a NULL does not even contain itself. @> is + * reflexive only for NULL-free arrays; draw one. + */ + draw_arr_opt(tc, &a, false); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s) @> (%s)", a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Element membership agrees with an explicit unnest scan: + * (e = ANY (A)) = EXISTS (SELECT 1 FROM unnest(A) u WHERE u = e). + * A is drawn WITHOUT NULL elements here so e = ANY(A) is a crisp true/false + * (a NULL element would let ANY return NULL on a no-match), keeping the + * equality exact; e is a concrete small int. + */ +PG_PROP(element_membership_consistent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int e; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arr_opt(tc, &a, false); /* no NULL elements for a crisp ANY */ + e = (int) pg_prop_draw_int(tc, -20, 20); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%d = ANY (%s)) " + "= EXISTS (SELECT 1 FROM unnest(%s) u WHERE u = %d)", + e, a.data, a.data, e); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index f43928e8973b2..e9c15d953149a 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From ca705a4a5e835ea2050dbf3d014cfbe870526df7 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:09 -0400 Subject: [PATCH 105/338] prop: add array concat, position, remove, and round-trips property tests (arrayops) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_arrayops.c | 402 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 406 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_arrayops.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 574b659282194..8897fa8fb4531 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -120,7 +120,8 @@ OBJS = \ prop_jsonpathq.o \ prop_jsonpathmath.o \ prop_jsoneach.o \ - prop_array.o + prop_array.o \ + prop_arrayops.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index b3b19ce75b6ed..6320102aa054a 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -118,6 +118,7 @@ prop_area_sources = files( 'prop_jsonpathmath.c', 'prop_jsoneach.c', 'prop_array.c', + 'prop_arrayops.c', ) diff --git a/src/test/prop/prop_arrayops.c b/src/test/prop/prop_arrayops.c new file mode 100644 index 0000000000000..2d75d1ce16f8c --- /dev/null +++ b/src/test/prop/prop_arrayops.c @@ -0,0 +1,402 @@ +/*------------------------------------------------------------------------- + * + * prop_arrayops.c + * Property-based tests for array modification and conversion functions. + * + * Covers the array builtins not exercised by the "array" and "arrayslice" + * areas: array_cat (||), array_append/array_prepend, array_position/ + * array_positions, array_remove, array_replace, cardinality, + * array_to_string/string_to_array, and the array_agg/unnest round trip. + * The oracle is each function's own contract -- length arithmetic, where an + * appended/prepended element lands, that remove/replace actually eliminate a + * value, and that string and aggregate conversions round-trip -- so any "f" + * result is a wrong-results bug in one of these functions. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_arrayops.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "arrayops" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a NULL-free integer-array literal into buf as ARRAY[...]::int[], with + * nmin..nmax elements each drawn from -5..5. The cast makes an empty array + * (ARRAY[]::int[]) carry a type. Negative elements are parenthesised as + * "(%d)" so a leading minus never misparses inside the ARRAY[] list. No NULL + * elements: the ordered/position properties below rely on crisp equality and + * a known-absent sentinel. + */ +static void +draw_intarr(pg_prop_tc *tc, StringInfo buf, int nmin, int nmax) +{ + int n = (int) pg_prop_draw_int(tc, nmin, nmax); + int i; + + appendStringInfoString(buf, "ARRAY["); + for (i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "(%d)", (int) pg_prop_draw_int(tc, -5, 5)); + } + appendStringInfoString(buf, "]::int[]"); +} + +/* + * Draw an integer-array literal that may contain SQL NULL elements (~20% of + * the time), for the NULL-aware properties. Elements otherwise drawn from + * -5..5 and parenthesised as above. nmin..nmax bound the length. + */ +static void +draw_intarr_null(pg_prop_tc *tc, StringInfo buf, int nmin, int nmax) +{ + int n = (int) pg_prop_draw_int(tc, nmin, nmax); + int i; + + appendStringInfoString(buf, "ARRAY["); + for (i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + if (pg_prop_draw_int(tc, 0, 4) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "(%d)", (int) pg_prop_draw_int(tc, -5, 5)); + } + appendStringInfoString(buf, "]::int[]"); +} + +/* + * Draw a NULL-free, non-negative integer-array literal (elements 0..5) into + * buf, nmin..nmax elements. Used by the array_to_string round trip so no '-' + * sign ever appears in the delimited text. + */ +static void +draw_intarr_nonneg(pg_prop_tc *tc, StringInfo buf, int nmin, int nmax) +{ + int n = (int) pg_prop_draw_int(tc, nmin, nmax); + int i; + + appendStringInfoString(buf, "ARRAY["); + for (i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, 0, 5)); + } + appendStringInfoString(buf, "]::int[]"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on a well-formed + * input, which is itself a bug -- so either is a failure, reported with the + * SQL. (Copied from prop_numeric.c's require_true.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * cardinality(a || b) = cardinality(a) + cardinality(b). "||" on two arrays + * is array_cat; concatenation adds lengths exactly, including the empty-array + * cases (cardinality of empty is 0). + */ +PG_PROP(cat_length_additive) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_intarr(tc, &a, 0, 6); + draw_intarr(tc, &b, 0, 6); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cardinality(%s || %s) = " + "cardinality(%s) + cardinality(%s)", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_append(a, 7) grows the length by one and places 7 at the new last + * position: cardinality(array_append(a, 7)) = cardinality(a) + 1 and + * array_append(a, 7)[cardinality(a) + 1] = 7. (int arrays are 1-based, so + * the appended element is at index cardinality(a) + 1.) + */ +PG_PROP(append_increments) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a, 0, 6); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cardinality(array_append(%s, 7)) = cardinality(%s) + 1 " + "AND (array_append(%s, 7))[cardinality(%s) + 1] = 7", + a.data, a.data, a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_prepend(7, a) grows the length by one and places 7 at the first + * position: cardinality(array_prepend(7, a)) = cardinality(a) + 1 and + * array_prepend(7, a)[1] = 7. (int arrays are 1-based.) + */ +PG_PROP(prepend_increments) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a, 0, 6); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cardinality(array_prepend(7, %s)) = cardinality(%s) + 1 " + "AND (array_prepend(7, %s))[1] = 7", + a.data, a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_position(array_append(a, 99), 99) = cardinality(a) + 1. 99 cannot + * occur in a (drawn from -5..5), so the only 99 is the appended one, which + * sits at the new last position -- array_position finds exactly it. + */ +PG_PROP(position_finds_appended) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a, 0, 6); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT array_position(array_append(%s, 99), 99) = " + "cardinality(%s) + 1", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_remove deletes every occurrence of its argument: after appending the + * known-absent sentinel 99 and then removing 99, no 99 remains, so + * array_position(..., 99) IS NULL. (a is drawn from -5..5, so the only 99 + * is the one we appended.) + */ +PG_PROP(remove_eliminates) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a, 0, 6); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT array_position(" + "array_remove(array_append(%s, 99), 99), 99) IS NULL", + a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_replace swaps one value for another: after appending the known-absent + * sentinel 99 and replacing 99 with 42, the 99 is gone and a 42 is present. + * Both 99 and 42 are outside -5..5, so a contributes neither; the single 99 + * we appended becomes the single 42. + */ +PG_PROP(replace_swaps) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a, 0, 6); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT array_position(" + "array_replace(array_append(%s, 99), 99, 42), 99) IS NULL " + "AND array_position(" + "array_replace(array_append(%s, 99), 99, 42), 42) IS NOT NULL", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_to_string / string_to_array round trip: for a non-empty array of + * non-negative ints, string_to_array(array_to_string(a, ','), ',')::int[] = a. + * Elements are 0..5 so no '-' complicates the delimited text, and the array + * is non-empty (1..6 elements) so the empty-array corner (array_to_string of + * empty is '', and string_to_array('', ',') is {''}) never arises. + */ +PG_PROP(to_string_from_string_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr_nonneg(tc, &a, 1, 6); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT string_to_array(array_to_string(%s, ','), ',')::int[] = %s", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_agg over unnest WITH ORDINALITY reconstructs the array: + * array_agg(x ORDER BY ord) FROM unnest(a) WITH ORDINALITY AS u(x, ord) + * returns a exactly, since ORDINALITY numbers the elements in array order and + * array_agg(... ORDER BY ord) reassembles them in that order -- including + * NULL elements, which unnest preserves. Drawn with 1..6 elements so a is + * non-empty (array_agg over 0 rows is NULL); draw_intarr_null exercises the + * NULL-preserving path. + */ +PG_PROP(unnest_agg_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr_null(tc, &a, 1, 6); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT array_agg(x ORDER BY ord) " + "FROM unnest(%s) WITH ORDINALITY AS u(x, ord)) " + "IS NOT DISTINCT FROM %s", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index e9c15d953149a..c57018d3d8cf4 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 4095c1866ecfc51e615265313e51209955c18fe4 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:09 -0400 Subject: [PATCH 106/338] prop: add arrayslice property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_arrayslice.c | 359 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 363 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_arrayslice.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 8897fa8fb4531..0e6000e8cb896 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -121,7 +121,8 @@ OBJS = \ prop_jsonpathmath.o \ prop_jsoneach.o \ prop_array.o \ - prop_arrayops.o + prop_arrayops.o \ + prop_arrayslice.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 6320102aa054a..7b8235d373f80 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -119,6 +119,7 @@ prop_area_sources = files( 'prop_jsoneach.c', 'prop_array.c', 'prop_arrayops.c', + 'prop_arrayslice.c', ) diff --git a/src/test/prop/prop_arrayslice.c b/src/test/prop/prop_arrayslice.c new file mode 100644 index 0000000000000..869d1e2a705b4 --- /dev/null +++ b/src/test/prop/prop_arrayslice.c @@ -0,0 +1,359 @@ +/*------------------------------------------------------------------------- + * + * prop_arrayslice.c + * Property-based tests for PostgreSQL array slicing and dimensions. + * + * Array slicing and the dimension accessors have subtle 1-based/bound + * semantics; these check slice length, bound consistency, and slice/concat + * composition over generated 1-D integer arrays. + * + * Distinct from the core "array" and "intarray" areas: this covers + * array_dims, array_upper/lower, slicing a[m:n], and slice/array_cat + * composition rather than cardinality/unnest/concat/text/contains/membership. + * + * The oracle is the array machinery's own slicing algebra against SQL: for a + * 1-D array A of length n the bounds are [1:n]; the full slice A[1:n] is A; a + * slice A[m:p] has exactly p-m+1 elements; splitting and re-concatenating the + * two halves A[1:k] || A[k+1:n] rebuilds A; array_dims renders '[1:n]'; + * slices preserve the original subscripts so (A[i:i])[i] = A[i]; and element + * access outside the bounds returns NULL rather than erroring. Any "f" (or + * an unexpected NULL/error) is a bug in the slicing code, not a modelling + * slip -- elements are drawn NULL-free so every = comparison stays exact. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_arrayslice.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "arrayslice" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a random SQL integer-array literal into buf as an ARRAY[...] expression + * typed ::int[]. n is drawn in 0..10 elements, each a small int in -20..20, + * with NO NULL elements so every = comparison and slice reconstruction stays + * exact. The ::int[] cast is always emitted so an empty array + * (ARRAY[]::int[]) still carries a type. *n receives the element count. + */ +static void +draw_arr(pg_prop_tc *tc, StringInfo buf, int *n) +{ + int len = (int) pg_prop_draw_int(tc, 0, 10); + + appendStringInfoString(buf, "ARRAY["); + for (int i = 0; i < len; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, -20, 20)); + } + appendStringInfoString(buf, "]::int[]"); + + *n = len; +} + +/* + * Draw an array like draw_arr but forced non-empty (n in 1..10), for the + * bound/dims/slice-length properties: an empty 1-D array has NULL bounds and + * NULL array_dims, so those identities only hold for n >= 1. + */ +static void +draw_arr_nonempty(pg_prop_tc *tc, StringInfo buf, int *n) +{ + int len = (int) pg_prop_draw_int(tc, 1, 10); + + appendStringInfoString(buf, "ARRAY["); + for (int i = 0; i < len; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, -20, 20)); + } + appendStringInfoString(buf, "]::int[]"); + + *n = len; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the slice operation errored or produced NULL on valid + * input, which is itself a bug -- so either is a failure, reported with the + * SQL and any error text. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * For a 1-D array A of length n >= 1, the default bounds are [1:n]: + * array_lower(A, 1) = 1 AND array_upper(A, 1) = n AND cardinality(A) = n. + * Empty arrays have NULL bounds, so n is drawn >= 1 here. + */ +PG_PROP(upper_lower_match_cardinality) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arr_nonempty(tc, &a, &n); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT array_lower(%s, 1) = 1 " + "AND array_upper(%s, 1) = %d " + "AND cardinality(%s) = %d", + a.data, a.data, n, a.data, n); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The full slice is the whole array: A[1:n] = A for n = cardinality. n is + * drawn >= 1 so the slice bounds are well-defined (A[1:0] would be empty). + */ +PG_PROP(full_slice_is_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arr_nonempty(tc, &a, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)[1:%d] = (%s)", + a.data, n, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A slice A[m:p] with 1 <= m <= p <= n has exactly p-m+1 elements: + * cardinality(A[m:p]) = p - m + 1. + * m and p are drawn in 1..n and ordered in C so m <= p; n is drawn >= 1. + */ +PG_PROP(slice_length_matches) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int n; + int m, + p; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arr_nonempty(tc, &a, &n); + + m = (int) pg_prop_draw_int(tc, 1, n); + p = (int) pg_prop_draw_int(tc, 1, n); + if (m > p) + { + int t = m; + + m = p; + p = t; + } + + initStringInfo(&q); + appendStringInfo(&q, "SELECT cardinality((%s)[%d:%d]) = (%d - %d + 1)", + a.data, m, p, p, m); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Splitting at k and re-concatenating rebuilds the array: + * A[1:k] || A[k+1:n] = A for a split k in 0..n. + * The edge splits are empty slices (A[1:0] and A[n+1:n]), handled by slice + * semantics, so a plain = still holds across the whole range. + */ +PG_PROP(slice_concat_reconstructs) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int n; + int k; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arr_nonempty(tc, &a, &n); + + k = (int) pg_prop_draw_int(tc, 0, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s)[1:%d] || (%s)[%d+1:%d]) = (%s)", + a.data, k, a.data, k, n, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_dims of a 1-D non-empty array is the text '[1:n]': + * array_dims(A) = '[1:' || n || ']'. + * array_dims is NULL for an empty array, so n is drawn >= 1. + */ +PG_PROP(array_dims_matches_bounds) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arr_nonempty(tc, &a, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT array_dims(%s) = '[1:' || %d || ']'", + a.data, n); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Slices preserve original subscripts, so a single-element slice A[i:i] + * keeps index i and (A[i:i])[i] = A[i] for i in 1..n. + */ +PG_PROP(element_access_in_slice) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int n; + int i; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arr_nonempty(tc, &a, &n); + + i = (int) pg_prop_draw_int(tc, 1, n); + + /* + * A single-element slice arr[i:i] contains exactly the i-th element. A + * slice is re-based so its own subscript is not i in general; compare the + * element it unnests to against arr[i] rather than subscripting the slice. + */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s)[%d] = (SELECT x FROM unnest((%s)[%d:%d]) x)", + a.data, i, a.data, i, i); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Subscripts outside the array bounds return NULL rather than erroring: + * A[n+1] IS NULL AND A[0] IS NULL. + * n is drawn >= 1 so n+1 and 0 are both genuinely out of the [1:n] range. + */ +PG_PROP(out_of_bounds_is_null) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arr_nonempty(tc, &a, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)[%d] IS NULL AND (%s)[0] IS NULL", + a.data, n + 1, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index c57018d3d8cf4..5e637afa3faf0 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From d069ac5f9d833742106387359f7ad0c2b66ab1d8 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:09 -0400 Subject: [PATCH 107/338] prop: add array containment, overlap, and comparison property tests (arraycmp) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_arraycmp.c | 399 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 403 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_arraycmp.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 0e6000e8cb896..fe4ad3b2e5927 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -122,7 +122,8 @@ OBJS = \ prop_jsoneach.o \ prop_array.o \ prop_arrayops.o \ - prop_arrayslice.o + prop_arrayslice.o \ + prop_arraycmp.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 7b8235d373f80..576712d7f10e1 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -120,6 +120,7 @@ prop_area_sources = files( 'prop_array.c', 'prop_arrayops.c', 'prop_arrayslice.c', + 'prop_arraycmp.c', ) diff --git a/src/test/prop/prop_arraycmp.c b/src/test/prop/prop_arraycmp.c new file mode 100644 index 0000000000000..39f1b7aa976f9 --- /dev/null +++ b/src/test/prop/prop_arraycmp.c @@ -0,0 +1,399 @@ +/*------------------------------------------------------------------------- + * + * prop_arraycmp.c + * Property-based tests for array comparison, containment, and overlap. + * + * Exercises the built-in anyarray operators @> (contains), <@ (contained by), + * && (overlaps), = (equality), < (lexicographic order), and || (concat) over + * integer arrays. The oracle is each operator's own definition expressed a + * second way: containment is element-set membership, overlap is a shared + * element, equality is element-wise agreement, and lexicographic order has + * fixed-constant anchors. Any "f" (or an unexpected NULL/error) is a bug in + * the array operator code, not a modelling slip -- NULL-array semantics are + * subtle so the overlap/equality properties draw NULL-free elements. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_arraycmp.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "arraycmp" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a random SQL integer-array literal into buf as ARRAY[...]::int[]. n is + * drawn in 0..6 elements, each a small int in -5..5; with include_null set, + * an element may be SQL NULL (ARRAY[1, NULL, 3]). The ::int[] cast is always + * emitted so an empty array (ARRAY[]::int[]) still carries a type. Returns + * the element count n. + */ +static int +draw_arr_opt(pg_prop_tc *tc, StringInfo buf, bool include_null) +{ + int n = (int) pg_prop_draw_int(tc, 0, 6); + + appendStringInfoString(buf, "ARRAY["); + for (int i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + if (include_null && pg_prop_draw_int(tc, 0, 4) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, -5, 5)); + } + appendStringInfoString(buf, "]::int[]"); + + return n; +} + +/* + * Draw a fixed-length integer-array literal of exactly m non-NULL elements. + * Used where two arrays must share a length (unnest of two arrays) or where + * NULL elements would make the operator return NULL instead of a crisp bool. + */ +static void +draw_arr_fixed(pg_prop_tc *tc, StringInfo buf, int m) +{ + appendStringInfoString(buf, "ARRAY["); + for (int i = 0; i < m; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, -5, 5)); + } + appendStringInfoString(buf, "]::int[]"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the array operation errored or produced NULL on valid + * input, which is itself a bug -- so either is a failure, reported with the + * SQL and any error text. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Containment is reflexive and symmetric at the ends: A @> A AND A <@ A. + * Containment is set membership, and SQL NULL is never equal to NULL, so an + * array holding a NULL would not even contain itself; draw NULL-free so both + * directions are crisp true. + */ +PG_PROP(contains_self) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_arr_opt(tc, &a, false); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s) @> (%s) AND (%s) <@ (%s)", + a.data, a.data, a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Every array contains any subset of its own elements, including the empty + * array: A @> ARRAY[]::int[], and A @> (the first k elements of A). The + * subset is built with unnest + LIMIT k; @> is element-set containment, so + * the order the subset comes out in does not matter and no ORDER BY is needed + * for soundness. A is drawn NULL-free so the elements compare crisply. + */ +PG_PROP(contains_subset) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int n; + int k; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + n = draw_arr_opt(tc, &a, false); + k = (n > 0) ? (int) pg_prop_draw_int(tc, 0, n) : 0; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s) @> ARRAY[]::int[] " + "AND (%s) @> (ARRAY(SELECT unnest(%s) LIMIT %d))", + a.data, a.data, a.data, k); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Overlap is symmetric: (A && B) = (B && A). Draw NULL-free elements so && + * returns a crisp bool (a NULL element can make && return NULL on a + * no-overlap), keeping the equality exact. + */ +PG_PROP(overlap_symmetric) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_arr_opt(tc, &a, false); + draw_arr_opt(tc, &b, false); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s) && (%s)) = ((%s) && (%s))", + a.data, b.data, b.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Overlap holds iff the two arrays share an element: + * (A && B) = EXISTS (SELECT 1 FROM unnest(A) x WHERE x = ANY (B)). + * Both A and B are drawn NULL-free so && and the ANY scan agree with no + * three-valued-logic slack. + */ +PG_PROP(overlap_iff_common) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_arr_opt(tc, &a, false); + draw_arr_opt(tc, &b, false); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s) && (%s)) " + "= EXISTS (SELECT 1 FROM unnest(%s) x WHERE x = ANY (%s))", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Equality is element-wise, for two equal-length arrays: + * (A = B) = NOT EXISTS (SELECT 1 FROM unnest(A, B) AS u(x, y) + * WHERE x IS DISTINCT FROM y). + * unnest of two arrays requires equal length (otherwise it pads with NULL), + * so A and B are drawn with the same length m of non-NULL elements; IS + * DISTINCT FROM makes the comparison total. + */ +PG_PROP(equality_elementwise) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + int m; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + m = (int) pg_prop_draw_int(tc, 0, 6); + + initStringInfo(&a); + initStringInfo(&b); + draw_arr_fixed(tc, &a, m); + draw_arr_fixed(tc, &b, m); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s) = (%s)) " + "= NOT EXISTS (SELECT 1 FROM unnest(%s, %s) AS u(x, y) " + "WHERE x IS DISTINCT FROM y)", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Concatenation contains both operands' elements: + * (A || B) @> A AND (A || B) @> B. + * Containment is element-set membership and A, B are NULL-free so both @> + * tests are crisp; concat places every element of A and of B into the + * result, so each is contained. + */ +PG_PROP(concat_contains_both) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_arr_opt(tc, &a, false); + draw_arr_opt(tc, &b, false); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s) || (%s)) @> (%s) AND ((%s) || (%s)) @> (%s)", + a.data, b.data, a.data, a.data, b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Array < is lexicographic: fixed anchors ARRAY[1,2] < ARRAY[1,3] (first + * differing element decides) and ARRAY[1] < ARRAY[1,2] (a proper prefix is + * less than its extension), plus a drawn case where P is a strict prefix of + * P || [tail] with tail a non-empty NULL-free suffix, so P < P || tail must + * hold. All elements are NULL-free so the b-tree comparator sees total order. + */ +PG_PROP(array_lt_lexicographic) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData p, + tail, + q; + int plen; + int tlen; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + plen = (int) pg_prop_draw_int(tc, 0, 4); + tlen = (int) pg_prop_draw_int(tc, 1, 3); /* non-empty tail */ + + initStringInfo(&p); + initStringInfo(&tail); + draw_arr_fixed(tc, &p, plen); + draw_arr_fixed(tc, &tail, tlen); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ARRAY[1,2]::int[] < ARRAY[1,3]::int[] " + "AND ARRAY[1]::int[] < ARRAY[1,2]::int[] " + "AND (%s) < ((%s) || (%s))", + p.data, p.data, tail.data); + + require_true(tc, conn, q.data); + + pfree(p.data); + pfree(tail.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The empty array is contained in every array: ARRAY[]::int[] <@ A. <@ is + * the mirror of @>, and containment of the empty set is vacuously true, so + * this holds even when A itself contains NULL elements. + */ +PG_PROP(empty_contained_in_all) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + /* <@ errors on NULL elements, so draw a NULL-free array of 0..6 elements */ + draw_arr_fixed(tc, &a, (int) pg_prop_draw_int(tc, 0, 6)); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ARRAY[]::int[] <@ (%s)", a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 5e637afa3faf0..6c8ffaa7cce50 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From e778ba5bdc00d5079a196efaf05145243e7204c5 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:09 -0400 Subject: [PATCH 108/338] prop: add ARRAY constructor, fill, and dimensions property tests (arrayconstruct) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_arrayconstruct.c | 261 ++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 265 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_arrayconstruct.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index fe4ad3b2e5927..8b82ef65dd07c 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -123,7 +123,8 @@ OBJS = \ prop_array.o \ prop_arrayops.o \ prop_arrayslice.o \ - prop_arraycmp.o + prop_arraycmp.o \ + prop_arrayconstruct.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 576712d7f10e1..0873457b20b44 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -121,6 +121,7 @@ prop_area_sources = files( 'prop_arrayops.c', 'prop_arrayslice.c', 'prop_arraycmp.c', + 'prop_arrayconstruct.c', ) diff --git a/src/test/prop/prop_arrayconstruct.c b/src/test/prop/prop_arrayconstruct.c new file mode 100644 index 0000000000000..1bc1a89a98518 --- /dev/null +++ b/src/test/prop/prop_arrayconstruct.c @@ -0,0 +1,261 @@ +/*------------------------------------------------------------------------- + * + * prop_arrayconstruct.c + * Property-based tests for array construction (constructors, dims, fill). + * + * Covers the array-construction surface not exercised by array/arrayops/ + * arrayslice/arraycmp: the ARRAY[...] constructor, ARRAY(subquery), array_fill, + * array_dims/array_ndims/array_length, cardinality over multidim arrays, and + * the dimensional effect of array_append. The oracle is each construct's own + * contract -- a constructed array equals the equivalent literal, array_fill + * produces exactly the requested shape and value, and the dim/length/ + * cardinality functions report the true shape -- so any "f" result is a + * wrong-results bug in array construction or introspection. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_arrayconstruct.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "arrayconstruct" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the construct errored or produced NULL on finite inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * ARRAY(subquery) materialises its ordered result in order, so + * ARRAY(SELECT v FROM (VALUES ...) ORDER BY v) over 1,2,3 equals the literal + * constructor ARRAY[1,2,3]. Checks the subquery-array path against the plain + * constructor. + */ +PG_PROP(array_subquery_eq_agg) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT ARRAY(SELECT v FROM (VALUES (1),(2),(3)) t(v) " + "ORDER BY v) = ARRAY[1,2,3]"); + + pg_prop_disconnect(conn); +} + +/* + * array_fill(7, ARRAY[k]) builds a 1-D array of exactly k copies of 7, so its + * cardinality is k and every element (checked at index 1) is 7. Drawn k in + * 1..6. + */ +PG_PROP(array_fill_dims) +{ + pg_prop_conn *conn; + char *err = NULL; + int k = (int) pg_prop_draw_int(tc, 1, 6); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cardinality(array_fill(7, ARRAY[%d])) = %d " + "AND (array_fill(7, ARRAY[%d]))[1] = 7", + k, k, k); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A 1-D ARRAY[...] of m >= 1 elements has dimension string '[1:m]', since the + * constructor assigns the default lower bound 1 and the length is m. Elements + * are drawn ints (parenthesised negatives); their values do not affect dims. + */ +PG_PROP(array_dims_1d) +{ + pg_prop_conn *conn; + char *err = NULL; + int m = (int) pg_prop_draw_int(tc, 1, 6); + int i; + StringInfoData arr; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&arr); + appendStringInfoString(&arr, "ARRAY["); + for (i = 0; i < m; i++) + { + if (i > 0) + appendStringInfoString(&arr, ", "); + appendStringInfo(&arr, "(%d)", (int) pg_prop_draw_int(tc, -5, 5)); + } + appendStringInfoString(&arr, "]"); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT array_dims(%s) = '[1:' || %d || ']'", + arr.data, m); + + require_true(tc, conn, q.data); + + pfree(arr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_ndims reports the number of dimensions: 1 for a flat constructor and 2 + * for a nested one. Constant checks -- the contract is about the shape of the + * literal, independent of element values. + */ +PG_PROP(array_ndims) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT array_ndims(ARRAY[1,2,3]) = 1 " + "AND array_ndims(ARRAY[[1,2],[3,4]]) = 2"); + + pg_prop_disconnect(conn); +} + +/* + * For a 2x3 nested constructor, array_length(.,1) is the outer dimension (2) + * and array_length(.,2) is the inner dimension (3). Constant check against a + * known literal shape. + */ +PG_PROP(array_length_each_dim) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT array_length(ARRAY[[1,2,3],[4,5,6]], 1) = 2 " + "AND array_length(ARRAY[[1,2,3],[4,5,6]], 2) = 3"); + + pg_prop_disconnect(conn); +} + +/* + * The ARRAY[a,b,c] constructor and the text literal '{a,b,c}'::int[] denote the + * same array, so they compare equal. Negatives are parenthesised in the + * constructor ("(%d)"); the literal form writes '-' directly inside the braces. + */ +PG_PROP(array_constructor_eq_literal) +{ + pg_prop_conn *conn; + char *err = NULL; + int a = (int) pg_prop_draw_int(tc, -5, 5); + int b = (int) pg_prop_draw_int(tc, -5, 5); + int c = (int) pg_prop_draw_int(tc, -5, 5); + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ARRAY[(%d),(%d),(%d)] = " + "('{' || %d || ',' || %d || ',' || %d || '}')::int[]", + a, b, c, a, b, c); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * cardinality counts every element across all dimensions, so a 3x2 nested + * constructor has cardinality 6 (not its outer length 3). Constant check. + */ +PG_PROP(cardinality_multidim) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT cardinality(ARRAY[[1,2],[3,4],[5,6]]) = 6"); + + pg_prop_disconnect(conn); +} + +/* + * array_append extends a 1-D array by exactly one element, so appending to a + * length-3 array yields array_length(.,1) = 4. Constant check. + */ +PG_PROP(array_append_dim) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT array_length(array_append(ARRAY[1,2,3], 4), 1) = 4"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 6c8ffaa7cce50..19c4b66e6eb94 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From da1244ce622bccc39cb0bb7055292f9be31c56fe Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:09 -0400 Subject: [PATCH 109/338] prop: add array_positions, trim_array, and reduction property tests (arrayreduce) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_arrayreduce.c | 383 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 387 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_arrayreduce.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 8b82ef65dd07c..cf0403aca3917 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -124,7 +124,8 @@ OBJS = \ prop_arrayops.o \ prop_arrayslice.o \ prop_arraycmp.o \ - prop_arrayconstruct.o + prop_arrayconstruct.o \ + prop_arrayreduce.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 0873457b20b44..936322d9dcf68 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -122,6 +122,7 @@ prop_area_sources = files( 'prop_arrayslice.c', 'prop_arraycmp.c', 'prop_arrayconstruct.c', + 'prop_arrayreduce.c', ) diff --git a/src/test/prop/prop_arrayreduce.c b/src/test/prop/prop_arrayreduce.c new file mode 100644 index 0000000000000..c158e3d7ddefa --- /dev/null +++ b/src/test/prop/prop_arrayreduce.c @@ -0,0 +1,383 @@ +/*------------------------------------------------------------------------- + * + * prop_arrayreduce.c + * Property-based tests for array aggregation and reduction functions. + * + * Covers the length/position/trim/aggregate reductions not exercised by the + * "array", "arrayops", "arraycmp", "arrayconstruct" and "arrayslice" areas: + * array_positions, trim_array, array_agg over unnest WITH ORDINALITY, the + * array_length-is-NULL-on-empty versus cardinality-is-0 distinction, and + * array_remove. The oracle is each function's own contract -- where + * positions land, that trim removes exactly k elements from the end, that + * agg/unnest round-trips, and the empty/NULL length semantics -- so any "f" + * result is a wrong-results bug in one of these reductions. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_arrayreduce.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "arrayreduce" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a NULL-free integer-array literal into buf as ARRAY[...]::int[], with + * 0..6 elements each drawn from 0..5. The cast gives an empty array + * (ARRAY[]::int[]) a type. All elements are in 0..5, so 7, 9 and 99 are + * guaranteed-absent sentinels the properties below rely on; and because no + * element is negative there is never a leading '-' to misparse in the list. + * No NULL elements: the position/remove/aggregate properties rely on crisp + * equality and a known-absent sentinel. + */ +static void +draw_intarr(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 6); + int i; + + appendStringInfoString(buf, "ARRAY["); + for (i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, 0, 5)); + } + appendStringInfoString(buf, "]::int[]"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on a well-formed + * input, which is itself a bug -- so either is a failure, reported with the + * SQL. (Copied from prop_numeric.c's require_true.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * array_positions finds every occurrence. Appending ARRAY[9,9] to a (9 is a + * guaranteed-absent sentinel, elements of a are 0..5) places the only two 9s + * at the last two positions, so array_positions must report exactly + * {cardinality(a)+1, cardinality(a)+2}. + */ +PG_PROP(positions_finds_all) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT array_positions(%s || ARRAY[9,9], 9) " + "= ARRAY[cardinality(%s) + 1, cardinality(%s) + 2]", + a.data, a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_positions of an absent value is the empty array (not NULL). 99 is a + * guaranteed-absent sentinel, so searching for it must yield ARRAY[]::int[]. + */ +PG_PROP(positions_empty_when_absent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT array_positions(%s, 99) = ARRAY[]::int[]", + a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * trim_array shortens by exactly k: cardinality(trim_array(a, k)) = + * cardinality(a) - k for every valid k in 0..cardinality(a). k is drawn + * after a's length is known so it never exceeds the array (trim_array errors + * if k is out of range). + */ +PG_PROP(trim_array_shortens) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int card; + int k; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a); + + /* learn the cardinality so k stays in trim_array's legal range */ + { + bool failed = false; + char *cerr = NULL; + char *cres; + StringInfoData cq; + + initStringInfo(&cq); + appendStringInfo(&cq, "SELECT cardinality(%s)", a.data); + cres = pg_prop_sql_scalar(conn, cq.data, &failed, &cerr); + if (failed || cres == NULL) + pg_prop_fail(tc, "cardinality probe failed: %s\n SQL: %s", + cerr ? cerr : "(null)", cq.data); + card = atoi(cres); + pg_free(cres); + if (cerr) + pg_free(cerr); + pfree(cq.data); + } + + k = (int) pg_prop_draw_int(tc, 0, card); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cardinality(trim_array(%s, %d)) " + "= cardinality(%s) - %d", + a.data, k, a.data, k); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * trim_array removes from the END: trim_array(a, k) equals the prefix slice + * a[1 : cardinality(a) - k]. k is drawn in range as above. IS NOT DISTINCT + * FROM covers the k = cardinality(a) case cleanly (both sides are the empty + * array; neither is NULL, but the operator is sound regardless). + */ +PG_PROP(trim_array_prefix) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int card; + int k; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a); + + { + bool failed = false; + char *cerr = NULL; + char *cres; + StringInfoData cq; + + initStringInfo(&cq); + appendStringInfo(&cq, "SELECT cardinality(%s)", a.data); + cres = pg_prop_sql_scalar(conn, cq.data, &failed, &cerr); + if (failed || cres == NULL) + pg_prop_fail(tc, "cardinality probe failed: %s\n SQL: %s", + cerr ? cerr : "(null)", cq.data); + card = atoi(cres); + pg_free(cres); + if (cerr) + pg_free(cerr); + pfree(cq.data); + } + + k = (int) pg_prop_draw_int(tc, 0, card); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT trim_array(%s, %d) " + "IS NOT DISTINCT FROM (%s)[1 : cardinality(%s) - %d]", + a.data, k, a.data, a.data, k); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_agg(x ORDER BY ord) over unnest(a) WITH ORDINALITY reconstructs a. + * The aggregate over 0 rows is SQL NULL, so coalesce it to '{}'::int[] for + * the empty-array case; the comparison then holds for every a. + */ +PG_PROP(array_agg_unnest_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(" + "(SELECT array_agg(x ORDER BY ord) " + "FROM unnest(%s) WITH ORDINALITY AS u(x, ord)), " + "'{}'::int[]) = %s", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * An empty array has a NULL length in each dimension: array_length('{}', 1) + * IS NULL. This is a fixed, input-independent fact; it documents the + * empty-array length semantics the next property contrasts against + * cardinality. + */ +PG_PROP(array_length_null_empty) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT array_length('{}'::int[], 1) IS NULL"); + + pg_prop_disconnect(conn); +} + +/* + * cardinality is 0 for the empty array where array_length is NULL: + * cardinality(a) = coalesce(array_length(a, 1), 0). For a non-empty 1-D + * array both equal the length; for the empty array cardinality is 0 and + * array_length is NULL, which the coalesce turns into 0. + */ +PG_PROP(cardinality_vs_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cardinality(%s) = coalesce(array_length(%s, 1), 0)", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_remove deletes every matching element. Appending ARRAY[7] (7 is a + * guaranteed-absent sentinel) adds exactly one 7; removing 7 must therefore + * leave cardinality(a) elements -- the one appended 7 and nothing else is + * removed. + */ +PG_PROP(array_remove_count) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cardinality(array_remove(%s || ARRAY[7], 7)) " + "= cardinality(%s)", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 19c4b66e6eb94..0362de82e2ef8 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 637276ac407059bb20882e88ea7e8c5fc64e8f60 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:10 -0400 Subject: [PATCH 110/338] prop: add array sorting via unnest and array_agg property tests (arraysort) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_arraysort.c | 346 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 350 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_arraysort.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index cf0403aca3917..a147ed49e7be5 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -125,7 +125,8 @@ OBJS = \ prop_arrayslice.o \ prop_arraycmp.o \ prop_arrayconstruct.o \ - prop_arrayreduce.o + prop_arrayreduce.o \ + prop_arraysort.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 936322d9dcf68..e2cad8297824f 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -123,6 +123,7 @@ prop_area_sources = files( 'prop_arraycmp.c', 'prop_arrayconstruct.c', 'prop_arrayreduce.c', + 'prop_arraysort.c', ) diff --git a/src/test/prop/prop_arraysort.c b/src/test/prop/prop_arraysort.c new file mode 100644 index 0000000000000..11c9937856dc2 --- /dev/null +++ b/src/test/prop/prop_arraysort.c @@ -0,0 +1,346 @@ +/*------------------------------------------------------------------------- + * + * prop_arraysort.c + * Property-based tests for array sorting and ordering. + * + * Sorting is expressed as array_agg(x ORDER BY x) over unnest(a): the ordered + * aggregate materialises a sorted copy of the multiset of elements. The + * oracle is what a correct sort must satisfy -- the result is non-decreasing, + * is a permutation of the input (same cardinality and sum), is idempotent, + * DESC is the reverse of ASC, the first/last sorted elements are the min/max, + * and lexicographic array ordering is consistent with element ordering. Any + * "f" result is a wrong-results bug in the ordered aggregate or in array + * comparison, a concern distinct from the arrayops/arraycmp/arrayreduce/ + * arrayconstruct areas. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_arraysort.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "arraysort" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a NULL-free integer-array literal into buf as ARRAY[...]::int[], with + * 0..6 elements each drawn from -5..5. Each element is emitted parenthesized + * as "(%d)" so a negative literal never misparses when substituted into a + * larger expression (bare -N::int4 would parse as -(N::int4)). The cast gives + * an empty array (ARRAY[]::int[]) a type. No NULL elements: the ordering + * properties below rely on crisp comparison with no NULL sort placement. + */ +static void +draw_intarr(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 6); + int i; + + appendStringInfoString(buf, "ARRAY["); + for (i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "(%d)", (int) pg_prop_draw_int(tc, -5, 5)); + } + appendStringInfoString(buf, "]::int[]"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the ordered aggregate or array comparison errored or + * produced NULL on a well-formed input, which is itself a bug -- so either is + * a failure, reported with the SQL. (Copied from prop_numeric.c's + * require_true.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * The sorted array is non-decreasing. s = array_agg(x ORDER BY x) over + * unnest(a); bool_and over every adjacent pair s[i] <= s[i+1] must hold. The + * aggregate over 0 rows (empty a) makes s NULL and the subscripts set empty, + * so coalesce the pairwise AND to true: a 0- or 1-element array is trivially + * ordered. + */ +PG_PROP(sort_is_ordered) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(" + "(SELECT bool_and(s[i] <= s[i+1]) " + "FROM generate_subscripts(s, 1) i " + "WHERE i < cardinality(s)), true) " + "FROM (SELECT array_agg(x ORDER BY x) s " + "FROM unnest(%s) x) q", + a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Sorting preserves the multiset: the sorted array is a permutation of a, so + * it has the same cardinality and the same element sum. (Cardinality and sum + * together do not prove a permutation in general, but combined with + * sort_is_ordered they are a strong, always-sound check against elements being + * dropped, duplicated, or altered by the sort.) coalesce handles the empty + * array: cardinality 0 and sum NULL -> 0 on both sides. + */ +PG_PROP(sort_permutation) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cardinality(s) = cardinality(%s) " + "AND coalesce((SELECT sum(e) FROM unnest(s) e), 0) " + "= coalesce((SELECT sum(e) FROM unnest(%s) e), 0) " + "FROM (SELECT coalesce(" + "(SELECT array_agg(x ORDER BY x) FROM unnest(%s) x), " + "'{}'::int[]) s) q", + a.data, a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Sorting is idempotent: sorting an already-sorted array yields the same + * array. s = sorted(a); sorted(s) must equal s. Both aggregates are + * coalesced to '{}'::int[] so the empty case compares two empty arrays. + */ +PG_PROP(sort_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(" + "(SELECT array_agg(y ORDER BY y) FROM unnest(s) y), " + "'{}'::int[]) = s " + "FROM (SELECT coalesce(" + "(SELECT array_agg(x ORDER BY x) FROM unnest(%s) x), " + "'{}'::int[]) s) q", + a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * DESC is the reverse of ASC: with asc = array_agg(x ORDER BY x) and desc = + * array_agg(x ORDER BY x DESC), element i of desc equals element (card-i+1) of + * asc for every i. bool_and over all subscripts of desc checks the reversal; + * coalesce to true covers the empty array (no subscripts). + */ +PG_PROP(desc_is_reverse) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(" + "(SELECT bool_and(desc_a[i] = asc_a[cardinality(asc_a) - i + 1]) " + "FROM generate_subscripts(desc_a, 1) i), true) " + "FROM (SELECT " + "coalesce((SELECT array_agg(x ORDER BY x) " + "FROM unnest(%s) x), '{}'::int[]) asc_a, " + "coalesce((SELECT array_agg(x ORDER BY x DESC) " + "FROM unnest(%s) x), '{}'::int[]) desc_a) q", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The first element of the ascending sort is the minimum element. Guarded to + * non-empty a (cardinality > 0): over an empty array both the sorted array and + * min() are NULL and the subscript [1] is undefined, so the property is + * vacuously true -- the CASE returns true without indexing. + */ +PG_PROP(min_is_first_sorted) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN cardinality(%s) = 0 THEN true " + "ELSE (SELECT array_agg(x ORDER BY x) FROM unnest(%s) x)[1] " + "= (SELECT min(e) FROM unnest(%s) e) END", + a.data, a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The last element of the ascending sort is the maximum element. Guarded to + * non-empty a as in min_is_first_sorted; the last index is cardinality(a). + */ +PG_PROP(max_is_last_sorted) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN cardinality(%s) = 0 THEN true " + "ELSE (SELECT array_agg(x ORDER BY x) FROM unnest(%s) x)" + "[cardinality(%s)] " + "= (SELECT max(e) FROM unnest(%s) e) END", + a.data, a.data, a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Lexicographic array ordering is consistent with the first-differing + * element: appending a strictly-larger element to a sorted array makes it + * compare strictly greater. s = sorted(a); 6 is a guaranteed-absent sentinel + * (elements are -5..5), so s is a prefix of s || ARRAY[6] whose next element + * (6) is larger than any element of s -- therefore s < s || ARRAY[6] always, + * and s <= s || ARRAY[6] holds for every a (including the empty array, where + * '{}' < ARRAY[6]). Checking <= keeps the property sound at the empty + * boundary while still exercising the comparison. + */ +PG_PROP(array_lt_consistent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT s <= s || ARRAY[6] " + "FROM (SELECT coalesce(" + "(SELECT array_agg(x ORDER BY x) FROM unnest(%s) x), " + "'{}'::int[]) s) q", + a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 0362de82e2ef8..859000cc0d678 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From f47e9f3aec450e0b0750ed5897f51c01852d7ce1 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:10 -0400 Subject: [PATCH 111/338] prop: add arrays with NULL elements and NULL-aware ops property tests (arraynull) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_arraynull.c | 325 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 329 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_arraynull.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index a147ed49e7be5..7357fbb60b1ac 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -126,7 +126,8 @@ OBJS = \ prop_arraycmp.o \ prop_arrayconstruct.o \ prop_arrayreduce.o \ - prop_arraysort.o + prop_arraysort.o \ + prop_arraynull.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index e2cad8297824f..f5a70e4cbb239 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -124,6 +124,7 @@ prop_area_sources = files( 'prop_arrayconstruct.c', 'prop_arrayreduce.c', 'prop_arraysort.c', + 'prop_arraynull.c', ) diff --git a/src/test/prop/prop_arraynull.c b/src/test/prop/prop_arraynull.c new file mode 100644 index 0000000000000..80ec8a14e7e48 --- /dev/null +++ b/src/test/prop/prop_arraynull.c @@ -0,0 +1,325 @@ +/*------------------------------------------------------------------------- + * + * prop_arraynull.c + * Property-based tests for arrays containing NULL elements and the + * NULL-aware array operations. + * + * SQL arrays carry NULL as a first-class element value, distinct from an + * empty array and from a NULL array. The oracle here is the documented + * NULL semantics of the array machinery: cardinality counts NULL elements, + * array_remove(a, NULL) drops exactly the NULLs, array_position(a, NULL) + * finds a NULL element, unnest yields one row per element (NULLs included), + * array_to_string omits NULLs (or substitutes its third argument), coalesce + * rewrites NULL elements in place, and ARRAY[NULL] is a one-element array + * that is not the empty array. Expected non-NULL counts are computed in C + * from the known NULL positions, so a wrong count or a dropped/duplicated + * NULL row is a bug in the array code, not an artefact of the test -- a + * concern distinct from array/arrayops/arraycmp/arraysort. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_arraynull.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "arraynull" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw an int-array literal into buf as ARRAY[...]::int[], with 0..6 elements. + * Each element is NULL with ~25% probability (drawing 0 out of 0..3), else an + * integer from -5..5 emitted parenthesized as "(%d)" so a negative literal is + * not misparsed when substituted into a larger expression. The cast gives an + * all-NULL or empty array a concrete element type. + * + * *total receives the element count and *nonnull the number of non-NULL + * elements, so a property can compute the expected cardinality after NULL + * removal in C rather than trusting a second SQL expression. + */ +static void +draw_intarr_null(pg_prop_tc *tc, StringInfo buf, int *total, int *nonnull) +{ + int n = (int) pg_prop_draw_int(tc, 0, 6); + int i; + int nn = 0; + + appendStringInfoString(buf, "ARRAY["); + for (i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + if (pg_prop_draw_int(tc, 0, 3) == 0) + { + appendStringInfoString(buf, "NULL"); + } + else + { + appendStringInfo(buf, "(%d)", (int) pg_prop_draw_int(tc, -5, 5)); + nn++; + } + } + appendStringInfoString(buf, "]::int[]"); + + *total = n; + *nonnull = nn; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the array operation errored or produced NULL on a + * well-formed input, which is itself a bug -- so either is a failure, reported + * with the SQL. (Copied from prop_numeric.c's require_true.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * cardinality counts NULL elements: an array with known NULL positions has + * cardinality equal to its total element count, NULLs included. The expected + * total is computed in C from the draw. + */ +PG_PROP(cardinality_counts_nulls) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int total, + nonnull; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr_null(tc, &a, &total, &nonnull); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT cardinality(%s) = %d", a.data, total); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_remove(a, NULL) removes exactly the NULL elements: the cardinality of + * the result equals the number of non-NULL elements, computed in C. (NULLs + * removed; non-NULLs preserved.) + */ +PG_PROP(array_remove_nulls) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int total, + nonnull; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr_null(tc, &a, &total, &nonnull); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cardinality(array_remove(%s, NULL)) = %d", + a.data, nonnull); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_position finds a NULL element: for the fixed array ARRAY[1,NULL,2] + * the first NULL is at position 2. array_position accepts a NULL needle and + * locates the NULL element rather than returning NULL or erroring. + */ +PG_PROP(contains_null_semantics) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT array_position(ARRAY[1,NULL,2]::int[], NULL) = 2"); + + pg_prop_disconnect(conn); +} + +/* + * unnest yields one row per element, NULLs included: the number of rows from + * unnest(a) equals cardinality(a). Expressed as a scalar oracle so the empty + * array yields count 0 (not a zero-row result): count(*) over the empty + * unnest is 0, and cardinality of the empty array is 0. + */ +PG_PROP(unnest_preserves_nulls) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int total, + nonnull; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr_null(tc, &a, &total, &nonnull); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM unnest(%s) e) " + "= cardinality(%s)", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array_to_string omits NULL elements: for the fixed text array + * ARRAY['a',NULL,'b'] joined on ',', the NULL is dropped and the result is + * 'a,b'. + */ +PG_PROP(array_to_string_skips_null) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT array_to_string(ARRAY['a',NULL,'b']::text[], ',') " + "= 'a,b'"); + + pg_prop_disconnect(conn); +} + +/* + * array_to_string with a third argument substitutes it for NULL elements + * rather than omitting them: ARRAY['a',NULL,'b'] joined on ',' with null-text + * 'X' is 'a,X,b'. + */ +PG_PROP(array_to_string_null_param) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT array_to_string(ARRAY['a',NULL,'b']::text[], ',', 'X') " + "= 'a,X,b'"); + + pg_prop_disconnect(conn); +} + +/* + * coalesce rewrites each NULL element in place while preserving order and + * length: array_agg(coalesce(e, -1) ORDER BY ord) over unnest WITH ORDINALITY + * yields an array whose cardinality equals the input's and that contains no + * NULLs. The empty array makes array_agg return NULL, so coalesce the + * cardinality to 0 and treat the no-element case as having zero NULLs. + */ +PG_PROP(coalesce_each) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int total, + nonnull; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_intarr_null(tc, &a, &total, &nonnull); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(cardinality(c), 0) = %d " + "AND coalesce(array_position(c, NULL) IS NULL, true) " + "FROM (SELECT array_agg(coalesce(e, -1) ORDER BY ord) c " + "FROM unnest(%s) WITH ORDINALITY u(e,ord)) s", + total, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * An array holding one NULL is not the empty array: cardinality of + * ARRAY[NULL]::int[] is 1 and cardinality of '{}'::int[] is 0. (The two are + * distinct values; ARRAY[NULL] is a one-element array.) + */ +PG_PROP(null_array_vs_empty) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT cardinality(ARRAY[NULL]::int[]) = 1 " + "AND cardinality('{}'::int[]) = 0"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 859000cc0d678..b7b0acb325415 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From ee179982c86e2ce52e1ce41933f5bd57a8bab6a2 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:10 -0400 Subject: [PATCH 112/338] prop: add = ANY / = ALL over generated arrays property tests (anyall) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_anyall.c | 375 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 379 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_anyall.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 7357fbb60b1ac..5a534a718764f 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -127,7 +127,8 @@ OBJS = \ prop_arrayconstruct.o \ prop_arrayreduce.o \ prop_arraysort.o \ - prop_arraynull.o + prop_arraynull.o \ + prop_anyall.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index f5a70e4cbb239..cfffad4ff5671 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -125,6 +125,7 @@ prop_area_sources = files( 'prop_arrayreduce.c', 'prop_arraysort.c', 'prop_arraynull.c', + 'prop_anyall.c', ) diff --git a/src/test/prop/prop_anyall.c b/src/test/prop/prop_anyall.c new file mode 100644 index 0000000000000..f0d5e3c162ba5 --- /dev/null +++ b/src/test/prop/prop_anyall.c @@ -0,0 +1,375 @@ +/*------------------------------------------------------------------------- + * + * prop_anyall.c + * Property-based tests for scalar = ANY / = ALL array operators. + * + * These exercise "op ANY(array)" and "op ALL(array)" (ScalarArrayOpExpr) + * against generated integer arrays, which is a different executor path than + * IN-list or array-comparison lowering: the oracle rewrites each quantified + * comparison as an equivalent unnest + EXISTS/aggregate, so a wrong answer in + * the ANY/ALL evaluation (empty-array identity, min/max bound, membership) is + * a bug. Array elements are non-NULL (SQL NULL in ANY/ALL is a separate 3VL + * concern) and empty arrays are handled explicitly. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_anyall.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "anyall" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* Integer element domain: small, straddling 0 and both signs. */ +#define EL_MIN (-5) +#define EL_MAX 5 + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the comparison errored or produced NULL on inputs where + * the property is defined to be true/false, which is itself a bug. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Draw a non-NULL int element in [EL_MIN, EL_MAX] into buf, parenthesized and + * cast so a negative value never misparses when substituted into an array. + */ +static void +draw_el(pg_prop_tc *tc, StringInfo buf) +{ + appendStringInfo(buf, "(%d)", (int) pg_prop_draw_int(tc, EL_MIN, EL_MAX)); +} + +/* + * Build an int4[] literal "ARRAY[...]::int[]" with 0..8 non-NULL elements. + * An empty draw yields "ARRAY[]::int[]", exercising the empty-array path. + */ +static void +draw_array(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 8); + + appendStringInfoString(buf, "ARRAY["); + for (int i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + draw_el(tc, buf); + } + appendStringInfoString(buf, "]::int[]"); +} + +/* Draw a non-NULL scalar int in [EL_MIN, EL_MAX] into buf. */ +static void +draw_scalar(pg_prop_tc *tc, StringInfo buf) +{ + appendStringInfo(buf, "(%d)", (int) pg_prop_draw_int(tc, EL_MIN, EL_MAX)); +} + +/* + * 1. (x = ANY(a)) agrees with set membership computed via unnest + EXISTS. + * Both are false for an empty array, so no empty guard is needed. + */ +PG_PROP(eq_any_is_member) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&x); + draw_array(tc, &a); + draw_scalar(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s = ANY(%s)) " + "= EXISTS(SELECT 1 FROM unnest(%s) e WHERE e = %s)", + x.data, a.data, a.data, x.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. (x > ALL(a)) equals (x > max(a)) for non-empty a; for empty a, x > ALL + * is vacuously true. The CASE keeps the empty case sound (max() over 0 + * rows is NULL, so comparing against it would be unusable there). + */ +PG_PROP(gt_all_is_gt_max) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&x); + draw_array(tc, &a); + draw_scalar(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN cardinality(%s) = 0 " + "THEN (%s > ALL(%s)) " + "ELSE (%s > ALL(%s)) " + "= (%s > (SELECT max(e) FROM unnest(%s) e)) END", + a.data, + x.data, a.data, + x.data, a.data, + x.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. (x < ANY(a)) equals (x < max(a)) for non-empty a; for empty a, x < ANY + * is false (there is nothing to beat). Guarded by cardinality. + */ +PG_PROP(lt_any_is_lt_max) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&x); + draw_array(tc, &a); + draw_scalar(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN cardinality(%s) = 0 " + "THEN (%s < ANY(%s)) = false " + "ELSE (%s < ANY(%s)) " + "= (%s < (SELECT max(e) FROM unnest(%s) e)) END", + a.data, + x.data, a.data, + x.data, a.data, + x.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. (c = ALL(array of k copies of c)) is true: every element equals c. + * Build the array as k >= 1 copies of a single drawn value c. + */ +PG_PROP(eq_all_single_value) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + int c = (int) pg_prop_draw_int(tc, EL_MIN, EL_MAX); + int k = (int) pg_prop_draw_int(tc, 1, 8); + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + appendStringInfoString(&a, "ARRAY["); + for (int i = 0; i < k; i++) + { + if (i > 0) + appendStringInfoString(&a, ", "); + appendStringInfo(&a, "(%d)", c); + } + appendStringInfoString(&a, "]::int[]"); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%d) = ALL(%s)) = true", c, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. (x <> ALL(a)) = NOT (x = ANY(a)): non-membership is the negation of + * membership. Holds for empty a too (both sides are true/true). + */ +PG_PROP(ne_all_absent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&x); + draw_array(tc, &a); + draw_scalar(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s <> ALL(%s)) = NOT (%s = ANY(%s))", + x.data, a.data, x.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. (x = ANY(ARRAY[]::int[])) is false: nothing to match in an empty array. + */ +PG_PROP(any_empty_false) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_scalar(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s = ANY(ARRAY[]::int[])) = false", x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. (x = ALL(ARRAY[]::int[])) is true: vacuously, all zero elements match. + */ +PG_PROP(all_empty_true) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_scalar(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s = ALL(ARRAY[]::int[])) = true", x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 8. (x >= ANY(a)) equals (x >= min(a)) for non-empty a; for empty a, x >= ANY + * is false. Guarded by cardinality (min() over 0 rows is NULL). + */ +PG_PROP(ge_any_bound) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&x); + draw_array(tc, &a); + draw_scalar(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN cardinality(%s) = 0 " + "THEN (%s >= ANY(%s)) = false " + "ELSE (%s >= ANY(%s)) " + "= (%s >= (SELECT min(e) FROM unnest(%s) e)) END", + a.data, + x.data, a.data, + x.data, a.data, + x.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index b7b0acb325415..bcda39f4d3479 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From cbac348e2b79a342a58719262eb075ff1d4e7143 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:10 -0400 Subject: [PATCH 113/338] prop: add IN/NOT IN/ANY/ALL consistency and NULL semantics property tests (inlist) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_inlist.c | 384 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 388 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_inlist.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 5a534a718764f..e2c94e0d9c066 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -128,7 +128,8 @@ OBJS = \ prop_arrayreduce.o \ prop_arraysort.o \ prop_arraynull.o \ - prop_anyall.o + prop_anyall.o \ + prop_inlist.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index cfffad4ff5671..ddd63a1f88382 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -126,6 +126,7 @@ prop_area_sources = files( 'prop_arraysort.c', 'prop_arraynull.c', 'prop_anyall.c', + 'prop_inlist.c', ) diff --git a/src/test/prop/prop_inlist.c b/src/test/prop/prop_inlist.c new file mode 100644 index 0000000000000..a7c14d2f9d9b9 --- /dev/null +++ b/src/test/prop/prop_inlist.c @@ -0,0 +1,384 @@ +/*------------------------------------------------------------------------- + * + * prop_inlist.c + * Property-based tests for IN / NOT IN / ANY / ALL list-membership algebra. + * + * The row-expression and scalar-array operators (IN, NOT IN, = ANY, > ALL, + * > ANY) must all agree with their primitive definitions: IN is an OR-of- + * equals, NOT IN is an AND-of-not-equals, = ANY(array) is the array form of + * IN, and the ordered ANY/ALL forms reduce to a comparison against the + * least/greatest list element. A NULL in the list still produces the classic + * three-valued-logic unknown rather than false. A "f", NULL, or SQL error on + * these scalar boolean oracles is a bug in ScalarArrayOpExpr / ArrayExpr + * evaluation, not a tolerance artefact. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_inlist.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "inlist" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a non-null int literal from a small domain, parenthesized so a leading + * sign never misparses when substituted into a larger expression. + */ +static void +draw_int(pg_prop_tc *tc, StringInfo buf) +{ + appendStringInfo(buf, "(%d)", (int) pg_prop_draw_int(tc, -5, 5)); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the oracle produced unknown or errored on inputs that must + * yield true, which is itself a failure -- reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * IN equals = ANY(array): for a non-null x and a non-null list, (x IN (a,b,c)) + * is equivalent to (x = ANY(ARRAY[a,b,c])). Both sides are non-NULL booleans + * (no NULLs anywhere), so a plain = is sound and the result must be "t". + */ +PG_PROP(in_eq_any_array) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&a); + initStringInfo(&b); + initStringInfo(&c); + draw_int(tc, &x); + draw_int(tc, &a); + draw_int(tc, &b); + draw_int(tc, &c); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s IN (%s, %s, %s)) = " + "(%s = ANY(ARRAY[%s, %s, %s]))", + x.data, a.data, b.data, c.data, + x.data, a.data, b.data, c.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * IN is the OR-of-equals: for non-null x, a, b, (x IN (a, b)) is equivalent to + * (x = a OR x = b). No NULLs, so both sides are non-NULL booleans and a plain + * = must yield "t". + */ +PG_PROP(in_eq_or_of_equals) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&a); + initStringInfo(&b); + draw_int(tc, &x); + draw_int(tc, &a); + draw_int(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s IN (%s, %s)) = " + "((%s = %s) OR (%s = %s))", + x.data, a.data, b.data, + x.data, a.data, x.data, b.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * NOT IN is the AND-of-not-equals: for non-null x, a, b, (x NOT IN (a, b)) is + * equivalent to (x <> a AND x <> b). No NULLs, so a plain = must yield "t". + * The NOT IN is parenthesized so it binds as the left operand of =. + */ +PG_PROP(not_in_eq_and_of_ne) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&a); + initStringInfo(&b); + draw_int(tc, &x); + draw_int(tc, &a); + draw_int(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s NOT IN (%s, %s)) = " + "((%s <> %s) AND (%s <> %s))", + x.data, a.data, b.data, + x.data, a.data, x.data, b.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * > ALL reduces to a comparison against the greatest element: x is greater + * than every list member iff it is greater than their maximum, so + * (x > ALL(ARRAY[a,b,c])) = (x > greatest(a,b,c)) for non-null values. All + * non-NULL, so a plain = must yield "t". + */ +PG_PROP(all_gt) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&a); + initStringInfo(&b); + initStringInfo(&c); + draw_int(tc, &x); + draw_int(tc, &a); + draw_int(tc, &b); + draw_int(tc, &c); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s > ALL(ARRAY[%s, %s, %s])) = " + "(%s > greatest(%s, %s, %s))", + x.data, a.data, b.data, c.data, + x.data, a.data, b.data, c.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * > ANY reduces to a comparison against the least element: x is greater than + * some list member iff it is greater than their minimum, so + * (x > ANY(ARRAY[a,b,c])) = (x > least(a,b,c)) for non-null values. All + * non-NULL, so a plain = must yield "t". + */ +PG_PROP(any_gt) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + initStringInfo(&a); + initStringInfo(&b); + initStringInfo(&c); + draw_int(tc, &x); + draw_int(tc, &a); + draw_int(tc, &b); + draw_int(tc, &c); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s > ANY(ARRAY[%s, %s, %s])) = " + "(%s > least(%s, %s, %s))", + x.data, a.data, b.data, c.data, + x.data, a.data, b.data, c.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The IN-with-NULL unknown case: when x matches no non-NULL list item and the + * list contains NULL, IN yields unknown (NULL), not false. We build x and a + * with x <> a via a non-zero offset (x = a + drawn 1..10), avoiding a reject + * loop, so the only non-NULL list element a never equals x; then + * (x IN (a, NULL)) IS NULL is itself always a non-NULL boolean, so a plain + * comparison to "t" is sound. + */ +PG_PROP(in_null_unknown) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + x; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (int) pg_prop_draw_int(tc, -5, 5); + /* x != a without a reject loop: add a non-zero offset */ + x = a + (int) pg_prop_draw_int(tc, 1, 10); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%d) IN ((%d), NULL)) IS NULL", + x, a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The NOT IN-with-NULL unknown case: same construction as in_null_unknown, but + * for NOT IN. When x matches no non-NULL item and the list has a NULL, + * (x NOT IN (a, NULL)) is unknown (NULL), not true. With x <> a built via a + * non-zero offset, (x NOT IN (a, NULL)) IS NULL is a non-NULL boolean and must + * be "t". + */ +PG_PROP(not_in_null_unknown) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + x; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (int) pg_prop_draw_int(tc, -5, 5); + /* x != a without a reject loop: add a non-zero offset */ + x = a + (int) pg_prop_draw_int(tc, 1, 10); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%d) NOT IN ((%d), NULL)) IS NULL", + x, a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Empty scalar-array operators: = ANY over an empty array is vacuously false + * (no element to match), and <> ALL over an empty array is vacuously true (the + * universally-quantified claim over no elements holds). No drawn inputs; a + * fixed sanity oracle guarding the empty-array boundary of ScalarArrayOpExpr. + */ +PG_PROP(empty_any_false) +{ + pg_prop_conn *conn; + char *err = NULL; + int x; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x = (int) pg_prop_draw_int(tc, -5, 5); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%d) = ANY(ARRAY[]::int[])) = false " + "AND ((%d) <> ALL(ARRAY[]::int[])) = true", + x, x); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index bcda39f4d3479..e83c3373898de 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 563d77558f77f36b00f49821a0dcdf192b146674 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:10 -0400 Subject: [PATCH 114/338] prop: add range property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_range.c | 317 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 321 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_range.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index e2c94e0d9c066..0fbe4c50d5e21 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -129,7 +129,8 @@ OBJS = \ prop_arraysort.o \ prop_arraynull.o \ prop_anyall.o \ - prop_inlist.o + prop_inlist.o \ + prop_range.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index ddd63a1f88382..03778f9bb7c80 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -127,6 +127,7 @@ prop_area_sources = files( 'prop_arraynull.c', 'prop_anyall.c', 'prop_inlist.c', + 'prop_range.c', ) diff --git a/src/test/prop/prop_range.c b/src/test/prop/prop_range.c new file mode 100644 index 0000000000000..d4ef8ed543159 --- /dev/null +++ b/src/test/prop/prop_range.c @@ -0,0 +1,317 @@ +/*------------------------------------------------------------------------- + * + * prop_range.c + * Property-based tests for range types (int4range). + * + * Range types have had many correctness fixes over the years -- in bound + * comparison, empty-range handling, and the canonicalization of discrete + * ranges (int4range is canonicalized to a half-open [lo, hi) form). These + * properties check the operator algebra (@>, <@, &&, -|-, +, *) against set + * semantics using int4range, whose canonical form makes equality exact: two + * int4ranges are equal iff they denote the same set, so an "=" oracle needs + * no tolerance. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_range.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "range" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small element domain keeps counterexamples legible and makes overlap and + * adjacency common enough that the guarded-union property exercises its + * interesting branch regularly. + */ +#define ELEM_MIN (-8) +#define ELEM_MAX 8 + +/* + * Draw a random int4range literal as a SQL expression into buf. Four shapes, + * chosen at random: + * + * - a normal bounded range int4range(lo, hi). int4range(a, b) is the + * half-open set [a, b); when a >= b this yields the empty range, which is + * a legal and interesting input, so we do not force lo < hi. + * - an explicitly empty range 'empty'::int4range. + * - unbounded below: int4range(NULL, hi). + * - unbounded above: int4range(lo, NULL). + * + * Every shape is a valid construction, so a SQL error on any of them is a bug + * the caller reports, not an input to discard. + */ +static void +draw_range(pg_prop_tc *tc, StringInfo buf) +{ + int shape = (int) pg_prop_draw_int(tc, 0, 3); + int lo; + int hi; + + switch (shape) + { + case 0: /* explicitly empty */ + appendStringInfoString(buf, "'empty'::int4range"); + break; + case 1: /* unbounded below */ + hi = (int) pg_prop_draw_int(tc, ELEM_MIN, ELEM_MAX); + appendStringInfo(buf, "int4range(NULL, %d)", hi); + break; + case 2: /* unbounded above */ + lo = (int) pg_prop_draw_int(tc, ELEM_MIN, ELEM_MAX); + appendStringInfo(buf, "int4range(%d, NULL)", lo); + break; + default: /* normal bounded; order the ends so lo <= hi + * (int4range(a, b) raises an error when a > b; + * a == b is the empty range, which is fine) */ + lo = (int) pg_prop_draw_int(tc, ELEM_MIN, ELEM_MAX); + hi = (int) pg_prop_draw_int(tc, ELEM_MIN, ELEM_MAX); + if (lo > hi) + { + int tmp = lo; + + lo = hi; + hi = tmp; + } + appendStringInfo(buf, "int4range(%d, %d)", lo, hi); + break; + } +} + +/* + * Draw an element value for membership tests, drawn one past each bound so + * values just outside the range domain are exercised too. + */ +static int +draw_elem(pg_prop_tc *tc) +{ + return (int) pg_prop_draw_int(tc, ELEM_MIN - 1, ELEM_MAX + 1); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operator produced NULL or errored on a valid range + * construction, which is itself a bug -- so either is a failure, reported with + * the SQL that triggered it. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Union contains both operands. The union (+) of two ranges errors if the + * result "would not be contiguous", so the CASE restricts the real check to + * the overlapping (&&) or adjacent (-|-) cases where the union is a single + * range, and returns true otherwise. Within that guard, (R1 + R2) must + * contain (@>) each of R1 and R2. + */ +PG_PROP(union_contains_both) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData r1, + r2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&r1); + initStringInfo(&r2); + draw_range(tc, &r1); + draw_range(tc, &r2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN %s && %s OR %s -|- %s " + "THEN (%s + %s) @> %s AND (%s + %s) @> %s " + "ELSE true END", + r1.data, r2.data, r1.data, r2.data, + r1.data, r2.data, r1.data, + r1.data, r2.data, r2.data); + + require_true(tc, conn, q.data); + + pfree(r1.data); + pfree(r2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Intersection is contained in both operands: (R1 * R2) <@ R1 AND <@ R2. + * Intersection never errors (unlike union, it is always a single range), and + * when the operands are disjoint the result is the empty range, which is + * contained in everything -- so this holds for all drawn R1, R2. + */ +PG_PROP(intersection_contained_in_both) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData r1, + r2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&r1); + initStringInfo(&r2); + draw_range(tc, &r1); + draw_range(tc, &r2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s * %s) <@ %s AND (%s * %s) <@ %s", + r1.data, r2.data, r1.data, + r1.data, r2.data, r2.data); + + require_true(tc, conn, q.data); + + pfree(r1.data); + pfree(r2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Element containment agrees with singleton-range containment: for a range R + * and an element x, (R @> x) must equal (R @> int4range(x, x+1)). The range + * int4range(x, x+1) canonicalizes to the half-open [x, x+1), the singleton + * set containing exactly x, so this cross-checks the scalar "@> element" + * operator against the range "@> range" operator -- two code paths that must + * agree. + */ +PG_PROP(membership_matches_bounds) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData r, + q; + int x; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&r); + draw_range(tc, &r); + x = draw_elem(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s @> %d) = (%s @> int4range(%d, %d))", + r.data, x, r.data, x, x + 1); + + require_true(tc, conn, q.data); + + pfree(r.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The empty range is the bottom element and the identity for union / zero for + * intersection: for any R, 'empty' <@ R, R + 'empty' = R, and + * R * 'empty' = 'empty'. (R + 'empty' never triggers the non-contiguous + * error because adding the empty range leaves R unchanged.) + */ +PG_PROP(empty_is_identity_and_bottom) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData r, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&r); + draw_range(tc, &r); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('empty'::int4range <@ %s) " + "AND (%s + 'empty'::int4range = %s) " + "AND (%s * 'empty'::int4range = 'empty'::int4range)", + r.data, r.data, r.data, r.data); + + require_true(tc, conn, q.data); + + pfree(r.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Containment is reflexive and antisymmetric: R1 @> R1 always, and mutual + * containment (R1 @> R2 AND R2 @> R1) holds exactly when R1 = R2. The + * antisymmetry check relies on int4range's canonical equality: two ranges are + * equal iff they denote the same set, so "=" is the right-hand oracle. + */ +PG_PROP(contains_is_reflexive_and_antisymmetric) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData r1, + r2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&r1); + initStringInfo(&r2); + draw_range(tc, &r1); + draw_range(tc, &r2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s @> %s) " + "AND ((%s @> %s AND %s @> %s) = (%s = %s))", + r1.data, r1.data, + r1.data, r2.data, r2.data, r1.data, + r1.data, r2.data); + + require_true(tc, conn, q.data); + + pfree(r1.data); + pfree(r2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index e83c3373898de..94be0af983b36 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From b82f5f4fa863d4b8608e1572bf5ac5250a023db2 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:11 -0400 Subject: [PATCH 115/338] prop: add int4range merge, difference, and adjacency property tests (rangeops) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_rangeops.c | 381 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 385 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_rangeops.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 0fbe4c50d5e21..42b1c2e18402d 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -130,7 +130,8 @@ OBJS = \ prop_arraynull.o \ prop_anyall.o \ prop_inlist.o \ - prop_range.o + prop_range.o \ + prop_rangeops.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 03778f9bb7c80..dd194faa471de 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -128,6 +128,7 @@ prop_area_sources = files( 'prop_anyall.c', 'prop_inlist.c', 'prop_range.c', + 'prop_rangeops.c', ) diff --git a/src/test/prop/prop_rangeops.c b/src/test/prop/prop_rangeops.c new file mode 100644 index 0000000000000..a205a2dc62ef6 --- /dev/null +++ b/src/test/prop/prop_rangeops.c @@ -0,0 +1,381 @@ +/*------------------------------------------------------------------------- + * + * prop_rangeops.c + * Property-based tests for int4range operator and function algebra. + * + * Target: the range type operator/function surface that the existing "range" + * and "multirange" areas do not cover -- containment of bounds under '[)' + * inclusivity, overlap (&&), adjacency (-|-), union (+), intersection (*), + * difference (-), range_merge(), and lower()/upper() extraction. The oracle + * is the type's own algebra: every identity below is true for every input, so + * an "f" result is a bug in the range code, not a modelling artefact. Union + * and difference are guarded with CASE because + over disjoint, non-adjacent + * ranges raises an error rather than returning a value. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_rangeops.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "rangeops" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw an ordered pair lo <= hi in -10..10 and emit the int4range(lo, hi) + * constructor text into buf (default '[)': lower inclusive, upper exclusive). + * *lo and *hi receive the drawn bounds so a property can guard on lo < hi + * (non-empty) versus lo == hi (empty). + */ +static void +draw_int4range(pg_prop_tc *tc, StringInfo buf, int *lo, int *hi) +{ + int a = (int) pg_prop_draw_int(tc, -10, 10); + int b = (int) pg_prop_draw_int(tc, -10, 10); + + if (a > b) + { + int t = a; + + a = b; + b = t; + } + *lo = a; + *hi = b; + appendStringInfo(buf, "int4range((%d), (%d))", a, b); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the range algebra produced NULL or errored on inputs for + * which the asserted identity should hold, which is itself a bug -- so either + * is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * contains_lower: under '[)' the lower bound is included. For a non-empty + * range (lo < hi), R @> lo. When lo == hi the range is empty and contains + * nothing, so we instead assert NOT (R @> lo): an empty range contains no + * element. Both halves are sound for every drawn pair. + */ +PG_PROP(contains_lower) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData r, + q; + int lo, + hi; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&r); + draw_int4range(tc, &r, &lo, &hi); + + initStringInfo(&q); + if (lo < hi) + appendStringInfo(&q, "SELECT %s @> (%d)", r.data, lo); + else + appendStringInfo(&q, "SELECT NOT (%s @> (%d))", r.data, lo); + + require_true(tc, conn, q.data); + + pfree(r.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * excludes_upper: under '[)' the upper bound is excluded, so a non-empty range + * does NOT contain hi: NOT (R @> hi). When lo == hi the range is empty and + * also contains nothing, so NOT (R @> hi) holds in that case too -- the + * assertion is sound for every drawn pair without a guard. + */ +PG_PROP(excludes_upper) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData r, + q; + int lo, + hi; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&r); + draw_int4range(tc, &r, &lo, &hi); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT NOT (%s @> (%d))", r.data, hi); + + require_true(tc, conn, q.data); + + pfree(r.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * intersection_contained: when A && B, their intersection A * B is contained + * in each operand: (A * B) <@ A AND (A * B) <@ B. Guarded with CASE on A && B + * because * of non-overlapping ranges yields the empty range, which is <@ + * everything but is not the interesting case; the ELSE branch is trivially + * true so the oracle is sound for every pair. + */ +PG_PROP(intersection_contained) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + int lo, + hi; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_int4range(tc, &a, &lo, &hi); + draw_int4range(tc, &b, &lo, &hi); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN %s && %s " + "THEN (%s * %s) <@ %s AND (%s * %s) <@ %s " + "ELSE true END", + a.data, b.data, + a.data, b.data, a.data, + a.data, b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * union_defined_when_overlap_or_adjacent: + is defined exactly when the + * operands overlap or are adjacent; the result then covers both operands. + * CASE WHEN (A && B OR A -|- B) THEN (A + B) @> A AND (A + B) @> B ELSE true. + * The guard is required because + over disjoint, non-adjacent ranges raises an + * error; the ELSE branch keeps the oracle sound for those pairs. + */ +PG_PROP(union_defined_when_overlap_or_adjacent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + int lo, + hi; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_int4range(tc, &a, &lo, &hi); + draw_int4range(tc, &b, &lo, &hi); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN (%s && %s OR %s -|- %s) " + "THEN (%s + %s) @> %s AND (%s + %s) @> %s " + "ELSE true END", + a.data, b.data, a.data, b.data, + a.data, b.data, a.data, + a.data, b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * difference_removes: when B @> A (B contains all of A), A - B is empty. + * CASE WHEN B @> A THEN isempty(A - B) ELSE true END. The guard confines the + * assertion to the case where - is well-defined and the result is predictable + * (A - B can error or split otherwise); the ELSE branch keeps it sound. + */ +PG_PROP(difference_removes) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + int lo, + hi; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_int4range(tc, &a, &lo, &hi); + draw_int4range(tc, &b, &lo, &hi); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN %s @> %s " + "THEN isempty(%s - %s) ELSE true END", + b.data, a.data, + a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * merge_contains_both: range_merge(A, B) is the smallest range covering both + * operands, so it always contains each: range_merge(A, B) @> A AND + * range_merge(A, B) @> B. range_merge is total (never errors on disjoint + * ranges, unlike +), so no guard is needed. + */ +PG_PROP(merge_contains_both) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + int lo, + hi; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_int4range(tc, &a, &lo, &hi); + draw_int4range(tc, &b, &lo, &hi); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT range_merge(%s, %s) @> %s " + "AND range_merge(%s, %s) @> %s", + a.data, b.data, a.data, + a.data, b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * elem_count_matches_width: for a non-empty '[)' range, the number of integers + * it holds is upper(R) - lower(R), which must equal the drawn hi - lo. This + * is trivial but exercises the lower()/upper() bound extractors. Guarded on + * lo < hi because lower()/upper() of an empty range are NULL. + */ +PG_PROP(elem_count_matches_width) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData r, + q; + int lo, + hi; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&r); + draw_int4range(tc, &r, &lo, &hi); + + initStringInfo(&q); + if (lo < hi) + appendStringInfo(&q, + "SELECT (upper(%s) - lower(%s)) = (%d)", + r.data, r.data, hi - lo); + else + appendStringInfoString(&q, "SELECT true"); + + require_true(tc, conn, q.data); + + pfree(r.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * empty_when_lo_eq_hi: a '[)' range with equal bounds is empty. For a drawn k, + * isempty(int4range(k, k)). This nails the degenerate boundary that the other + * properties guard against. + */ +PG_PROP(empty_when_lo_eq_hi) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int k = (int) pg_prop_draw_int(tc, -10, 10); + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT isempty(int4range((%d), (%d)))", k, k); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 94be0af983b36..bbee47dc54472 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 62b0480c010e2a205e760b6568b36b1ff93523f4 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:11 -0400 Subject: [PATCH 116/338] prop: add range_agg and multirange aggregation property tests (rangeagg) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_rangeagg.c | 288 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 292 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_rangeagg.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 42b1c2e18402d..4d64fbc61b35c 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -131,7 +131,8 @@ OBJS = \ prop_anyall.o \ prop_inlist.o \ prop_range.o \ - prop_rangeops.o + prop_rangeops.o \ + prop_rangeagg.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index dd194faa471de..c5eef49ee2b41 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -129,6 +129,7 @@ prop_area_sources = files( 'prop_inlist.c', 'prop_range.c', 'prop_rangeops.c', + 'prop_rangeagg.c', ) diff --git a/src/test/prop/prop_rangeagg.c b/src/test/prop/prop_rangeagg.c new file mode 100644 index 0000000000000..af5ca6465838a --- /dev/null +++ b/src/test/prop/prop_rangeagg.c @@ -0,0 +1,288 @@ +/*------------------------------------------------------------------------- + * + * prop_rangeagg.c + * Property-based tests for the range aggregates range_agg / range_merge + * and multirange containment. + * + * range_agg folds a set of input ranges into one normalized multirange, and + * range_merge collapses a multirange to its spanning range. Both are set + * operations whose results are exact: the aggregated multirange must contain + * every member range (and every point inside a member), its merge span must + * match min(lower)/max(upper) of the inputs, and merging can only reduce the + * component count. A failure of any of these is a bug in the aggregate merge + * path, not a rounding artefact -- there is no slack to tolerate. + * + * This exercises range_agg / range_merge over a built table (distinct from + * the pure-type algebra in prop_range / prop_multirange / prop_rangeops), + * including the empty-input edge case where range_agg returns SQL NULL. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_rangeagg.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "rangeagg" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Integer bounds for lo/hi. Kept small so the ranges overlap, abut and nest + * densely, which is what makes range_agg actually merge components rather than + * emit one component per row. + */ +#define VAL_MIN (-10) +#define VAL_MAX 10 + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the aggregate errored or produced NULL on inputs where the + * property guarded it with coalesce/CASE, which is itself a bug -- so either is + * a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build the rg_t table: id PRIMARY KEY, lo, hi with 0..15 rows and lo <= hi. + * The lo <= hi ordering is enforced in C (swap when needed) so the generated + * int4range(lo, hi, ...) literals never raise "range lower bound must be less + * than or equal to range upper bound". Rebuilt fresh in every property. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + char *err = NULL; + StringInfoData ddl; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS rg_t; " + "CREATE TABLE rg_t(id int PRIMARY KEY, lo int, hi int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 15); + for (int i = 0; i < nrows; i++) + { + int a = (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX); + int b = (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX); + int lo = a < b ? a : b; + int hi = a < b ? b : a; + + appendStringInfo(&ddl, + "INSERT INTO rg_t VALUES (%d, (%d), (%d)); ", + i, lo, hi); + } + appendStringInfoString(&ddl, "ANALYZE rg_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n SQL: %s", + err ? err : "", ddl.data); + + pfree(ddl.data); + if (err) + pg_free(err); +} + +/* + * The aggregated multirange contains every member range: for each row r, + * range_agg(all rows) @> int4range(r.lo, r.hi) must hold. bool_and over all + * rows, coalesced to true for the empty table (range_agg is NULL then and the + * row-driven SELECT returns no rows). The subselect names the aggregated + * table "t" so each member is tested against the whole aggregate. + */ +PG_PROP(range_agg_covers) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(" + "(SELECT range_agg(int4range(t.lo, t.hi)) FROM rg_t t) " + "@> int4range(r.lo, r.hi)), true) FROM rg_t r"); + + pg_prop_disconnect(conn); +} + +/* + * A point inside some input range is contained in the aggregated multirange. + * Pick one row (the min id) and a value lo <= p <= hi inside it; the half-open + * int4range(lo, hi) excludes hi, so test p = lo which is always a member. The + * CASE guards the empty table so the property passes soundly when there is no + * row to pick. + */ +PG_PROP(multirange_contains_point) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + /* + * Use the inclusive '[]' aggregate so hi is a member too; pick p = lo of + * the lowest-id row. int4range(lo, hi, '[]') with lo <= hi is never empty, + * so p = lo is always contained. + */ + require_true(tc, conn, + "SELECT CASE WHEN count(*) = 0 THEN true ELSE " + "(SELECT range_agg(int4range(lo, hi, '[]')) FROM rg_t) " + "@> (SELECT lo FROM rg_t ORDER BY id LIMIT 1) END " + "FROM rg_t"); + + pg_prop_disconnect(conn); +} + +/* + * range_merge of the aggregated multirange is its spanning range, so its lower + * bound equals min(lo) over the inputs. Inclusive '[]' ranges are used so the + * span's lower bound is exactly min(lo) (a half-open literal would normalize + * the lower bound unchanged, but '[]' keeps it inclusive and unambiguous). + * CASE guards the empty table. + */ +PG_PROP(range_merge_agg_bounds) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN count(*) = 0 THEN true ELSE " + "lower(range_merge(" + "(SELECT range_agg(int4range(lo, hi, '[]')) FROM rg_t t))) " + "= (SELECT min(lo) FROM rg_t) END " + "FROM rg_t"); + + pg_prop_disconnect(conn); +} + +/* + * range_agg over an empty row set is SQL NULL. WHERE false forces the empty + * input regardless of how many rows the table holds; the result must be NULL. + */ +PG_PROP(multirange_is_empty_when_no_rows) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT range_agg(int4range(t.lo, t.hi)) " + "FROM rg_t t WHERE false) IS NULL"); + + pg_prop_disconnect(conn); +} + +/* + * unnest(range_agg(...)) yields the multirange's disjoint, non-adjacent + * component ranges; merging can only reduce count, so the number of components + * is at most the number of input rows. coalesce guards the empty table where + * range_agg is NULL and unnest yields no rows (count 0 <= 0 holds anyway). + */ +PG_PROP(unnest_multirange_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(" + "(SELECT count(*) FROM unnest(" + "(SELECT range_agg(int4range(t.lo, t.hi)) FROM rg_t t))), 0) " + "<= (SELECT count(*) FROM rg_t)"); + + pg_prop_disconnect(conn); +} + +/* + * Every integer in some input [lo, hi] (inclusive) is contained in the + * inclusive '[]' aggregated multirange. Draw an offset and clamp it to pick a + * value v in [lo, hi] of the lowest-id row; the '[]' aggregate must contain v. + * CASE guards the empty table. The clamp (greatest/least against lo/hi) keeps + * v inside the chosen row regardless of the drawn offset, so the assertion is + * always true. + */ +PG_PROP(multirange_contains_all_elements) +{ + pg_prop_conn *conn; + char *err = NULL; + int off = (int) pg_prop_draw_int(tc, 0, 2 * (VAL_MAX - VAL_MIN)); + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + { + StringInfoData q; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN count(*) = 0 THEN true ELSE " + "(SELECT range_agg(int4range(lo, hi, '[]')) FROM rg_t) @> " + "(SELECT least(greatest(lo + (%d), lo), hi) " + "FROM rg_t ORDER BY id LIMIT 1) END " + "FROM rg_t", + off); + + require_true(tc, conn, q.data); + pfree(q.data); + } + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index bbee47dc54472..c842af8d98542 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 4dea0e45306b627c805bf1b84fac3e59133e76e0 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:11 -0400 Subject: [PATCH 117/338] prop: add multirange property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_multirange.c | 376 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 380 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_multirange.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 4d64fbc61b35c..5257b6cb542f1 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -132,7 +132,8 @@ OBJS = \ prop_inlist.o \ prop_range.o \ prop_rangeops.o \ - prop_rangeagg.o + prop_rangeagg.o \ + prop_multirange.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index c5eef49ee2b41..8a6b59d34f59a 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -130,6 +130,7 @@ prop_area_sources = files( 'prop_range.c', 'prop_rangeops.c', 'prop_rangeagg.c', + 'prop_multirange.c', ) diff --git a/src/test/prop/prop_multirange.c b/src/test/prop/prop_multirange.c new file mode 100644 index 0000000000000..e0c92627e05c8 --- /dev/null +++ b/src/test/prop/prop_multirange.c @@ -0,0 +1,376 @@ +/*------------------------------------------------------------------------- + * + * prop_multirange.c + * Property-based tests for multirange types (int4multirange). + * + * Multiranges have accumulated many correctness fixes over the years in the + * code that merges, normalizes, and combines them (union, intersection, + * minus). The defining invariant of the type is strong: a multirange is a + * normalized, ordered set of non-overlapping, non-adjacent ranges. Because + * that canonical form is a true set, the set identities over int4multirange + * are exact -- union commutes and is idempotent, intersection is contained in + * both operands, difference removes exactly the overlap -- and so set algebra + * is the natural property to assert against the type itself. + * + * The oracle here is the type's own set algebra. The server normalizes and + * merges on input, so overlapping or adjacent component ranges in a literal + * get folded together; feeding deliberately overlapping components therefore + * exercises the merge path, and the identities must still hold with no slack. + * Unlike single ranges, multirange + * - never raise a "non-contiguous" + * error -- representing a gap is the whole point of the type -- so these need + * no CASE guards and are cleaner to state than their single-range cousins. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_multirange.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "multirange" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small integer bounds keep each generated multirange legible in a + * counterexample and keep the overlap/adjacency interactions dense: with + * bounds in [-8, 8] and up to three components, drawn ranges routinely + * overlap, abut, and nest, which is exactly what the input normalizer and the + * set operators must fold correctly. + */ +#define BND_MIN (-8) +#define BND_MAX 8 + +/* + * Draw a random int4multirange literal into buf, e.g. + * '{[1,5), [8,10)}'::int4multirange + * or the empty multirange '{}'::int4multirange. + * + * Draw 0..3 component ranges with small bounds. A component is normally the + * half-open range [lo, hi) with lo < hi, but occasionally an explicitly empty + * component -- which the server drops on normalization, exercising that path. + * Components are NOT forced disjoint: overlapping and adjacent inputs are + * legal and get merged, so this deliberately feeds the merge logic. + */ +static void +draw_mr(pg_prop_tc *tc, StringInfo buf) +{ + int ncomp = (int) pg_prop_draw_int(tc, 0, 3); + bool first = true; + + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, '{'); + for (int i = 0; i < ncomp; i++) + { + if (!first) + appendStringInfoString(buf, ", "); + first = false; + + /* ~1 in 5 components is an empty range, dropped on normalization. */ + if (pg_prop_draw_int(tc, 0, 4) == 0) + { + appendStringInfoString(buf, "empty"); + } + else + { + int lo = (int) pg_prop_draw_int(tc, BND_MIN, BND_MAX - 1); + int hi = (int) pg_prop_draw_int(tc, lo + 1, BND_MAX); + + appendStringInfo(buf, "[%d,%d)", lo, hi); + } + } + appendStringInfoChar(buf, '}'); + appendStringInfoChar(buf, '\''); + appendStringInfoString(buf, "::int4multirange"); +} + +/* + * Draw a random int4range literal into buf, e.g. '[1,5)'::int4range or + * 'empty'::int4range. Used by the range-to-multirange roundtrip test. + */ +static void +draw_r(pg_prop_tc *tc, StringInfo buf) +{ + appendStringInfoChar(buf, '\''); + if (pg_prop_draw_int(tc, 0, 5) == 0) + { + appendStringInfoString(buf, "empty"); + } + else + { + int lo = (int) pg_prop_draw_int(tc, BND_MIN, BND_MAX - 1); + int hi = (int) pg_prop_draw_int(tc, lo + 1, BND_MAX); + + appendStringInfo(buf, "[%d,%d)", lo, hi); + } + appendStringInfoChar(buf, '\''); + appendStringInfoString(buf, "::int4range"); +} + +/* + * Draw a scalar int4 element into buf, in a range a little wider than the + * component bounds so it falls both inside and outside drawn ranges. + */ +static void +draw_elem(pg_prop_tc *tc, StringInfo buf) +{ + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, BND_MIN - 2, BND_MAX + 2)); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a set operator errored or produced NULL on valid finite + * input, which is itself a bug -- so either is a failure, reported with the + * SQL that triggered it. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Union commutes and is idempotent: (M1 + M2 = M2 + M1) AND (M1 + M1 = M1). + * Both follow from the canonical set form; a merge/ordering bug would break + * commutativity, and a failure to collapse equal operands would break + * idempotence. + */ +PG_PROP(union_is_commutative_and_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData m1, + m2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&m1); + initStringInfo(&m2); + draw_mr(tc, &m1); + draw_mr(tc, &m2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s + %s = %s + %s) AND (%s + %s = %s)", + m1.data, m2.data, m2.data, m1.data, + m1.data, m1.data, m1.data); + + require_true(tc, conn, q.data); + + pfree(m1.data); + pfree(m2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Intersection is contained in both operands: (M1 * M2) <@ M1 AND (M1 * M2) + * <@ M2. Multirange intersection never errors (unlike single-range union), + * so there is nothing to guard. + */ +PG_PROP(intersection_subset_of_both) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData m1, + m2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&m1); + initStringInfo(&m2); + draw_mr(tc, &m1); + draw_mr(tc, &m2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s * %s) <@ %s AND (%s * %s) <@ %s", + m1.data, m2.data, m1.data, + m1.data, m2.data, m2.data); + + require_true(tc, conn, q.data); + + pfree(m1.data); + pfree(m2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Union is always defined and contains both operands: (M1 + M2) @> M1 AND + * (M1 + M2) @> M2. Single-range union errors when the result would be + * non-contiguous; multirange union never does, which is the point of the + * type. + */ +PG_PROP(union_contains_both) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData m1, + m2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&m1); + initStringInfo(&m2); + draw_mr(tc, &m1); + draw_mr(tc, &m2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s + %s) @> %s AND (%s + %s) @> %s", + m1.data, m2.data, m1.data, + m1.data, m2.data, m2.data); + + require_true(tc, conn, q.data); + + pfree(m1.data); + pfree(m2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Difference removes exactly the overlap: NOT ((M1 - M2) && M2). After + * subtracting M2 from M1, the result shares no element with M2, so it does + * not overlap M2. A merge/split bug in minus would leave a sliver that still + * overlaps. + */ +PG_PROP(minus_then_intersection_empty) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData m1, + m2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&m1); + initStringInfo(&m2); + draw_mr(tc, &m1); + draw_mr(tc, &m2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT NOT ((%s - %s) && %s)", + m1.data, m2.data, m2.data); + + require_true(tc, conn, q.data); + + pfree(m1.data); + pfree(m2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A counting-free inclusion/exclusion identity: ((M1 + M2) - M2) <@ M1 AND + * M1 <@ (M1 + M2). Removing M2 from the union leaves only elements that were + * in M1, and M1 is always a subset of the union. This ties union and + * difference together without appealing to cardinality (which multiranges do + * not expose directly). + */ +PG_PROP(inclusion_exclusion_cardinality_free) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData m1, + m2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&m1); + initStringInfo(&m2); + draw_mr(tc, &m1); + draw_mr(tc, &m2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s + %s) - %s) <@ %s AND %s <@ (%s + %s)", + m1.data, m2.data, m2.data, m1.data, + m1.data, m1.data, m2.data); + + require_true(tc, conn, q.data); + + pfree(m1.data); + pfree(m2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A single range, lifted into a one-element multirange, contains exactly the + * same elements: (R @> x) = (multirange(R) @> x) for a drawn element x. + * multirange(anyrange) wraps one range as a multirange; the containment of + * any point must be preserved by the lift. + */ +PG_PROP(range_roundtrip_into_multirange) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData r, + x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&r); + initStringInfo(&x); + draw_r(tc, &r); + draw_elem(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s @> %s) = (multirange(%s) @> %s)", + r.data, x.data, r.data, x.data); + + require_true(tc, conn, q.data); + + pfree(r.data); + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index c842af8d98542..24e4bc50cf12c 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From f23cd921dfa03ff82db3a5720f15c9ac36d129f9 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:11 -0400 Subject: [PATCH 118/338] prop: add inet property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_inet.c | 302 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 306 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_inet.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 5257b6cb542f1..42e948bf0adf4 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -133,7 +133,8 @@ OBJS = \ prop_range.o \ prop_rangeops.o \ prop_rangeagg.o \ - prop_multirange.o + prop_multirange.o \ + prop_inet.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 8a6b59d34f59a..289855803b307 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -131,6 +131,7 @@ prop_area_sources = files( 'prop_rangeops.c', 'prop_rangeagg.c', 'prop_multirange.c', + 'prop_inet.c', ) diff --git a/src/test/prop/prop_inet.c b/src/test/prop/prop_inet.c new file mode 100644 index 0000000000000..f709930d8d932 --- /dev/null +++ b/src/test/prop/prop_inet.c @@ -0,0 +1,302 @@ +/*------------------------------------------------------------------------- + * + * prop_inet.c + * Property-based tests for the inet/cidr network types. + * + * inet/cidr parsing, the containment operators, and host/network/masklen + * accessors have had correctness fixes; these check text round-trips and the + * subnet containment order over generated IPv4 and IPv6 values. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_inet.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "inet" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a valid inet literal into buf as a quoted SQL literal cast to inet, + * e.g. "'10.0.0.1/24'::inet". + * + * IPv4 is drawn ~70% of the time (bias toward the common family while still + * exercising IPv6). IPv4 is "A.B.C.D" with a masklen 0..32, and the "/M" is + * sometimes omitted to generate a host (no mask). IPv6 uses the full 8-group + * form "a:b:c:d:e:f:g:h/M" with each hextet drawn 0..65535 and M in 0..128; + * writing all eight groups avoids any "::" compression ambiguity in the test + * generator itself. + */ +static void +draw_inet(pg_prop_tc *tc, StringInfo buf) +{ + bool ipv4 = (pg_prop_draw_int(tc, 0, 9) < 7); + + appendStringInfoChar(buf, '\''); + if (ipv4) + { + int a = (int) pg_prop_draw_int(tc, 0, 255); + int b = (int) pg_prop_draw_int(tc, 0, 255); + int c = (int) pg_prop_draw_int(tc, 0, 255); + int d = (int) pg_prop_draw_int(tc, 0, 255); + + appendStringInfo(buf, "%d.%d.%d.%d", a, b, c, d); + if (pg_prop_draw_bool(tc)) + appendStringInfo(buf, "/%d", (int) pg_prop_draw_int(tc, 0, 32)); + } + else + { + for (int i = 0; i < 8; i++) + { + if (i > 0) + appendStringInfoChar(buf, ':'); + appendStringInfo(buf, "%x", (unsigned) pg_prop_draw_int(tc, 0, 65535)); + } + appendStringInfo(buf, "/%d", (int) pg_prop_draw_int(tc, 0, 128)); + } + appendStringInfoString(buf, "'::inet"); +} + +/* + * Draw a valid cidr value into buf. + * + * A cidr literal errors if any host bit is set, so instead of writing + * 'A.B.C.D/M'::cidr directly we let the server clear the host bits: network() + * of an inet returns the cidr of the containing network, which is always a + * valid cidr with host bits zero. So this emits "network('...'::inet)". + */ +static void +draw_cidr(pg_prop_tc *tc, StringInfo buf) +{ + appendStringInfoString(buf, "network("); + draw_inet(tc, buf); + appendStringInfoChar(buf, ')'); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a validly-generated value parsed to NULL or errored, which + * is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Text round-trip: an inet rendered to text and parsed back equals the value. + * This exercises both inet_out and inet_in on every generated value. + */ +PG_PROP(text_roundtrip_inet) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_inet(tc, &i); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)::text::inet = %s", i.data, i.data); + + require_true(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Splitting an inet into its host address plus its masklen and recombining + * returns the original: set_masklen(host(I)::inet, masklen(I)) = I. host() + * drops the mask; set_masklen reattaches it. + */ +PG_PROP(host_masklen_reconstruct) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_inet(tc, &i); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT set_masklen(host(%s)::inet, masklen(%s)) = %s", + i.data, i.data, i.data); + + require_true(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The network of an inet contains the inet (>>= is contains-or-equals), and + * network() yields a valid cidr: round-tripping it through cidr and back to + * inet is a no-op. + */ +PG_PROP(network_contains_host) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_inet(tc, &i); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT network(%s) >>= %s " + "AND network(%s) = network(%s)::cidr::inet", + i.data, i.data, i.data, i.data); + + require_true(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Containment is a partial order consistent with masklen: a network that + * strictly contains another must have a shorter prefix. So if C1 >>= C2 and + * C1 <> C2 then masklen(C1) < masklen(C2); otherwise the implication is + * vacuously true. + */ +PG_PROP(containment_order_consistent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData c1, + c2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&c1); + initStringInfo(&c2); + draw_cidr(tc, &c1); + draw_cidr(tc, &c2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN %s >>= %s AND %s <> %s " + "THEN masklen(%s) < masklen(%s) ELSE true END", + c1.data, c2.data, c1.data, c2.data, + c1.data, c2.data); + + require_true(tc, conn, q.data); + + pfree(c1.data); + pfree(c2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The broadcast address of an inet lies within its network. + */ +PG_PROP(broadcast_in_network) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_inet(tc, &i); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT network(%s) >>= broadcast(%s)", + i.data, i.data); + + require_true(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * family() is 4 or 6, and it is 4 exactly when the address is within the IPv4 + * space (contained-or-equal in 0.0.0.0/0). + */ +PG_PROP(family_matches_version) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_inet(tc, &i); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT family(%s) IN (4, 6) " + "AND (family(%s) = 4) = (%s <<= '0.0.0.0/0'::inet)", + i.data, i.data, i.data); + + require_true(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 24e4bc50cf12c..4b25de842514a 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 05d2b93689ab69869878f911ee8a0ef8051f9e13 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:11 -0400 Subject: [PATCH 119/338] prop: add inetarith property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_inetarith.c | 328 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 332 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_inetarith.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 42e948bf0adf4..3860dcbbf3b91 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -134,7 +134,8 @@ OBJS = \ prop_rangeops.o \ prop_rangeagg.o \ prop_multirange.o \ - prop_inet.o + prop_inet.o \ + prop_inetarith.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 289855803b307..745a6e7d4df9e 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -132,6 +132,7 @@ prop_area_sources = files( 'prop_rangeagg.c', 'prop_multirange.c', 'prop_inet.c', + 'prop_inetarith.c', ) diff --git a/src/test/prop/prop_inetarith.c b/src/test/prop/prop_inetarith.c new file mode 100644 index 0000000000000..a31a0c69d3e19 --- /dev/null +++ b/src/test/prop/prop_inetarith.c @@ -0,0 +1,328 @@ +/*------------------------------------------------------------------------- + * + * prop_inetarith.c + * Property-based tests for inet arithmetic (inet +/- int, inet - inet). + * + * inet supports adding/subtracting an integer offset and subtracting two + * inets to get the integer distance; these check the add/subtract inverse and + * the distance/order consistency, skipping legitimate out-of-range. + * + * This is distinct from the "inet" area (prop_inet.c), which covers parsing, + * containment, and accessors; here the target is the +/- arithmetic. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_inetarith.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "inetarith" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw an IPv4 host inet literal into buf as a quoted SQL literal cast to + * inet, e.g. "'10.0.0.1'::inet". No mask is attached (a bare host), so that + * inet - inet is a well-defined integer address distance and inet +/- int + * arithmetic stays within the 32-bit IPv4 address space. + */ +static void +draw_host_inet(pg_prop_tc *tc, StringInfo buf) +{ + int a = (int) pg_prop_draw_int(tc, 0, 255); + int b = (int) pg_prop_draw_int(tc, 0, 255); + int c = (int) pg_prop_draw_int(tc, 0, 255); + int d = (int) pg_prop_draw_int(tc, 0, 255); + + appendStringInfo(buf, "'%d.%d.%d.%d'::inet", a, b, c, d); +} + +/* + * True if a SQL error message describes a legitimate out-of-range or + * overflow condition -- adding/subtracting an offset near 0.0.0.0 or + * 255.255.255.255 can run off the end of the address space, which is a + * correct error, not a bug. Such cases are skipped, not failed. + */ +static bool +is_overflow(const char *msg) +{ + if (msg == NULL) + return false; + return strstr(msg, "out of range") != NULL || + strstr(msg, "would overflow") != NULL || + strstr(msg, "cannot") != NULL || + strstr(msg, "result is out of range") != NULL; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a validly-generated value errored or produced NULL, which + * is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Like require_true, but a SQL error that is_overflow() recognises as a + * legitimate out-of-range/overflow is skipped (the case is assumed away) + * rather than failed. Any other error, a NULL, or a non-"t" result fails. + */ +static void +require_true_or_skip(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + if (is_overflow(err)) + { + if (err) + pg_free(err); + pg_prop_assume(tc, false); + return; /* not reached */ + } + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Adding then subtracting an integer offset returns the address: + * (I + k) - k = I. Near 0.0.0.0 or 255.255.255.255 the intermediate I + k + * can run off the end of the address space, a legitimate out-of-range error + * that is skipped rather than failed. + */ +PG_PROP(add_sub_inverse) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 k; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_host_inet(tc, &i); + k = pg_prop_draw_int(tc, -1000000, 1000000); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s + %lld) - %lld = %s", + i.data, (long long) k, (long long) k, i.data); + + require_true_or_skip(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * inet - inet yields the integer distance; adding it back to I gives J: + * I + (J - I) = J. Both are IPv4 hosts, so the difference is always defined + * and no out-of-range should occur; still skip overflow defensively. + */ +PG_PROP(diff_then_add) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData i, + j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + initStringInfo(&j); + draw_host_inet(tc, &i); + draw_host_inet(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s + (%s - %s) = %s", + i.data, j.data, i.data, j.data); + + require_true_or_skip(tc, conn, q.data); + + pfree(i.data); + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The integer difference's sign agrees with inet ordering: + * (J - I) > 0 iff I < J, and (J - I) = 0 iff I = J. + */ +PG_PROP(diff_sign_matches_order) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData i, + j, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + initStringInfo(&j); + draw_host_inet(tc, &i); + draw_host_inet(tc, &j); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s - %s) > 0) = (%s < %s) " + "AND ((%s - %s) = 0) = (%s = %s)", + j.data, i.data, i.data, j.data, + j.data, i.data, i.data, j.data); + + require_true(tc, conn, q.data); + + pfree(i.data); + pfree(j.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Adding a positive offset yields a greater address: I + k > I for k > 0, + * when no overflow. Near 255.255.255.255 the addition can run off the end, + * a legitimate out-of-range error that is skipped rather than failed. + */ +PG_PROP(add_monotonic) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 k; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_host_inet(tc, &i); + k = pg_prop_draw_int(tc, 1, 1000000); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s + %lld) > %s", + i.data, (long long) k, i.data); + + require_true_or_skip(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Adding zero is the identity: I + 0 = I. + */ +PG_PROP(add_zero_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_host_inet(tc, &i); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s + 0 = %s", i.data, i.data); + + require_true(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Addition associates with integer sum: (I + a) + b = I + (a + b), when no + * overflow. Either side can run off the address space near the ends, a + * legitimate out-of-range error that is skipped rather than failed. + */ +PG_PROP(add_associates_with_sum) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a, + b; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_host_inet(tc, &i); + a = pg_prop_draw_int(tc, -100000, 100000); + b = pg_prop_draw_int(tc, -100000, 100000); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s + %lld) + %lld) = (%s + (%lld + %lld))", + i.data, (long long) a, (long long) b, + i.data, (long long) a, (long long) b); + + require_true_or_skip(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 4b25de842514a..088c334bffa50 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From c1e0f79ddce92d26572d44b643c33646ae94dcc7 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:11 -0400 Subject: [PATCH 120/338] prop: add inet/cidr host, network, and masklen property tests (networkops) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_networkops.c | 266 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 270 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_networkops.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 3860dcbbf3b91..e657c9d572a6a 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -135,7 +135,8 @@ OBJS = \ prop_rangeagg.o \ prop_multirange.o \ prop_inet.o \ - prop_inetarith.o + prop_inetarith.o \ + prop_networkops.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 745a6e7d4df9e..6217073d2a6d0 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -133,6 +133,7 @@ prop_area_sources = files( 'prop_multirange.c', 'prop_inet.c', 'prop_inetarith.c', + 'prop_networkops.c', ) diff --git a/src/test/prop/prop_networkops.c b/src/test/prop/prop_networkops.c new file mode 100644 index 0000000000000..099fcc0267c69 --- /dev/null +++ b/src/test/prop/prop_networkops.c @@ -0,0 +1,266 @@ +/*------------------------------------------------------------------------- + * + * prop_networkops.c + * Property-based tests for inet/cidr network functions. + * + * Exercises the network-address functions distinct from the inet/inetarith + * areas: host(), network(), broadcast(), netmask(), masklen(), + * set_masklen(), family(), the text round-trip, and the containment + * operators (<<=, >>=). The oracle is each function's defining law -- + * set_masklen reports back the mask it set, network() has the host's own + * masklen, an IPv4 address is family 4, broadcast is never below network, + * text output re-parses to the same value, and every value contains itself -- + * so any "f" result is a bug in network.c, not a tolerance. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_networkops.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "networkops" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw an IPv4 host literal "a.b.c.d/len" into buf, each octet 0..255 and the + * mask length 0..32. No quoting is needed since the characters are all + * digits and dots/slash; the caller wraps it in inet '...'. + */ +static void +draw_inet(pg_prop_tc *tc, StringInfo buf) +{ + int a = (int) pg_prop_draw_int(tc, 0, 255); + int b = (int) pg_prop_draw_int(tc, 0, 255); + int c = (int) pg_prop_draw_int(tc, 0, 255); + int d = (int) pg_prop_draw_int(tc, 0, 255); + int len = (int) pg_prop_draw_int(tc, 0, 32); + + appendStringInfo(buf, "%d.%d.%d.%d/%d", a, b, c, d, len); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on valid inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * set_masklen reports back the mask it set: masklen(set_masklen(X, m)) = m, + * for a drawn m in 0..32. The host part of X is irrelevant to this, so m is + * drawn independently of X's own mask. + */ +PG_PROP(host_in_its_network) +{ + pg_prop_conn *conn; + char *err = NULL; + int m; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + m = (int) pg_prop_draw_int(tc, 0, 32); + initStringInfo(&x); + draw_inet(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT masklen(set_masklen(inet '%s', %d)) = %d", + x.data, m, m); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * network() keeps the host's mask length: masklen(network(inet 'a.b.c.d/len')) + * = len. network() zeroes the host bits but does not change the mask width. + */ +PG_PROP(network_masklen) +{ + pg_prop_conn *conn; + char *err = NULL; + int a = 0, + b = 0, + c = 0, + d = 0, + len = 0; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (int) pg_prop_draw_int(tc, 0, 255); + b = (int) pg_prop_draw_int(tc, 0, 255); + c = (int) pg_prop_draw_int(tc, 0, 255); + d = (int) pg_prop_draw_int(tc, 0, 255); + len = (int) pg_prop_draw_int(tc, 0, 32); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT masklen(network(inet '%d.%d.%d.%d/%d')) = %d", + a, b, c, d, len, len); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Every IPv4 inet is family 4: family(inet 'a.b.c.d/len') = 4. + */ +PG_PROP(family_is_4) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_inet(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT family(inet '%s') = 4", x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The broadcast address is never below the network address: + * broadcast(X) >= network(X) as an inet comparison. broadcast sets the host + * bits to 1 and network sets them to 0, so broadcast is always at or above. + */ +PG_PROP(broadcast_ge_network) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_inet(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT broadcast(inet '%s') >= network(inet '%s')", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Text round-trip: (inet 'a.b.c.d/len')::text::inet = inet 'a.b.c.d/len'. + * The output text must re-parse to exactly the same value. + */ +PG_PROP(abbrev_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_inet(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (inet '%s')::text::inet = inet '%s'", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Containment is reflexive: inet X <<= X and X >>= X both hold. Every + * network/host value contains and is contained by itself. + */ +PG_PROP(contains_self) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&x); + draw_inet(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (inet '%s' <<= inet '%s') " + "AND (inet '%s' >>= inet '%s')", + x.data, x.data, x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 088c334bffa50..478c1a0b277bf 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 47f067eec86bf1e95f77a850a7a134ca7e357ca5 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:11 -0400 Subject: [PATCH 121/338] prop: add cidr normalization and network containment property tests (cidrops) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_cidrops.c | 345 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 349 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_cidrops.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index e657c9d572a6a..cc878e97a853d 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -136,7 +136,8 @@ OBJS = \ prop_multirange.o \ prop_inet.o \ prop_inetarith.o \ - prop_networkops.o + prop_networkops.o \ + prop_cidrops.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 6217073d2a6d0..9ebf379cbd55b 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -134,6 +134,7 @@ prop_area_sources = files( 'prop_inet.c', 'prop_inetarith.c', 'prop_networkops.c', + 'prop_cidrops.c', ) diff --git a/src/test/prop/prop_cidrops.c b/src/test/prop/prop_cidrops.c new file mode 100644 index 0000000000000..aac1db09666a0 --- /dev/null +++ b/src/test/prop/prop_cidrops.c @@ -0,0 +1,345 @@ +/*------------------------------------------------------------------------- + * + * prop_cidrops.c + * Property-based tests for the cidr type: normalization and containment. + * + * The cidr type differs from inet in that it zeroes host bits (a cidr is a + * network, not an address): 'a.b.c.d/len'::cidr normalizes to the /len + * network of a.b.c.d. These properties check that normalization, the network + * containment hierarchy (a /len contains the matching /(len+1) and every host + * in the network), masklen/family accessors, text round-trip, and the + * network<=broadcast ordering. A failure means cidr I/O, network(), + * set_masklen(), masklen(), or the containment operators mis-handle an IPv4 + * address/masklen pair -- a correctness bug in the network type. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_cidrops.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "cidrops" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw an IPv4 "a.b.c.d/len" string (no quotes, no cast) into buf, with each + * octet 0..255 and len 0..32. Also return len via *len so properties that + * need the numeric prefix length (len, len+1) can build their SQL. + */ +static void +draw_ipslash(pg_prop_tc *tc, StringInfo buf, int *len) +{ + int a = (int) pg_prop_draw_int(tc, 0, 255); + int b = (int) pg_prop_draw_int(tc, 0, 255); + int c = (int) pg_prop_draw_int(tc, 0, 255); + int d = (int) pg_prop_draw_int(tc, 0, 255); + int l = (int) pg_prop_draw_int(tc, 0, 32); + + appendStringInfo(buf, "%d.%d.%d.%d/%d", a, b, c, d, l); + *len = l; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a validly-generated value parsed to NULL or errored, which + * is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * A cidr with host bits set normalizes to its /len network: casting the + * ip/len string to cidr equals rebuilding the cidr from network() of the same + * value as an inet. This is the defining property of cidr -- it zeroes host + * bits -- so both sides must denote the same network. + */ +PG_PROP(cidr_normalizes) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData ip, + q; + int len; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&ip); + draw_ipslash(tc, &ip, &len); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT set_masklen(('%s')::inet, %d)::cidr = " + "network(set_masklen(('%s')::inet, %d))::cidr", + ip.data, len, ip.data, len); + + require_true(tc, conn, q.data); + + pfree(ip.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The /len network contains the host it was built from: the normalized cidr + * contains-or-equals ('ip/32')::inet (the host address with a full mask). + */ +PG_PROP(cidr_contains_hosts) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData ip, + q; + int len; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&ip); + draw_ipslash(tc, &ip, &len); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT set_masklen(('%s')::inet, %d)::cidr >>= " + "(split_part('%s', '/', 1) || '/32')::inet", + ip.data, len, ip.data); + + require_true(tc, conn, q.data); + + pfree(ip.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * masklen() of the normalized cidr equals the drawn prefix length. + */ +PG_PROP(masklen_matches) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData ip, + q; + int len; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&ip); + draw_ipslash(tc, &ip, &len); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT masklen(set_masklen(('%s')::inet, %d)::cidr) = %d", + ip.data, len, len); + + require_true(tc, conn, q.data); + + pfree(ip.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A /len network contains the matching /(len+1) sub-network: the two cidrs + * built from the same address by set_masklen to len and len+1 are in the + * containment relation (>>=) for every len < 32. (At len = 32 there is no + * narrower network, so the property is vacuously skipped by regenerating a + * len in 0..31 for this check.) + */ +PG_PROP(broader_contains_narrower) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData ip, + q; + int a, + b, + c, + d, + len; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* draw len in 0..31 so len+1 is a valid IPv4 prefix (<= 32) */ + a = (int) pg_prop_draw_int(tc, 0, 255); + b = (int) pg_prop_draw_int(tc, 0, 255); + c = (int) pg_prop_draw_int(tc, 0, 255); + d = (int) pg_prop_draw_int(tc, 0, 255); + len = (int) pg_prop_draw_int(tc, 0, 31); + + initStringInfo(&ip); + appendStringInfo(&ip, "%d.%d.%d.%d/%d", a, b, c, d, len); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT set_masklen(('%s')::inet, %d)::cidr " + ">>= set_masklen(('%s')::inet, %d)::cidr", + ip.data, len, ip.data, len + 1); + + require_true(tc, conn, q.data); + + pfree(ip.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Text round-trip: a cidr rendered to text and parsed back equals the value. + * Exercises both cidr_out and cidr_in on every generated value. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData ip, + q; + int len; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&ip); + draw_ipslash(tc, &ip, &len); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (set_masklen(('%s')::inet, %d)::cidr)::text::cidr = " + "set_masklen(('%s')::inet, %d)::cidr", + ip.data, len, ip.data, len); + + require_true(tc, conn, q.data); + + pfree(ip.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * family() of an IPv4 cidr is 4. + */ +PG_PROP(family_4) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData ip, + q; + int len; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&ip); + draw_ipslash(tc, &ip, &len); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT family(set_masklen(('%s')::inet, %d)::cidr) = 4", + ip.data, len); + + require_true(tc, conn, q.data); + + pfree(ip.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The cidr's ordering as inet is consistent: the network address is <= the + * broadcast address of the same network. + */ +PG_PROP(cidr_le_inet_order) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData ip, + q; + int len; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&ip); + draw_ipslash(tc, &ip, &len); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT network(('%s')::inet) <= broadcast(('%s')::inet)", + ip.data, ip.data); + + require_true(tc, conn, q.data); + + pfree(ip.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Host-count/masklen bound: the number of addresses in a /len network, + * 2^(32-len), is at least 1, and forcing the mask to /32 with set_masklen + * yields a single-host network whose masklen is 32. + */ +PG_PROP(host_count_bound) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData ip, + q; + int len; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&ip); + draw_ipslash(tc, &ip, &len); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (2::numeric ^ (32 - %d)) >= 1 " + "AND masklen(set_masklen(('%s')::inet, 32)) = 32", + len, ip.data); + + require_true(tc, conn, q.data); + + pfree(ip.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 478c1a0b277bf..92a42d713f97b 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From caec585eab5f9afd49a8b524b0aea9e3562f4eba Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:12 -0400 Subject: [PATCH 122/338] prop: add IPv6 inet family, masklen, and containment property tests (inet6) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_inet6.c | 320 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 324 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_inet6.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index cc878e97a853d..c5ca740012c1e 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -137,7 +137,8 @@ OBJS = \ prop_inet.o \ prop_inetarith.o \ prop_networkops.o \ - prop_cidrops.o + prop_cidrops.o \ + prop_inet6.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 9ebf379cbd55b..85b1a7b9e9169 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -135,6 +135,7 @@ prop_area_sources = files( 'prop_inetarith.c', 'prop_networkops.c', 'prop_cidrops.c', + 'prop_inet6.c', ) diff --git a/src/test/prop/prop_inet6.c b/src/test/prop/prop_inet6.c new file mode 100644 index 0000000000000..fa2b74b8e956a --- /dev/null +++ b/src/test/prop/prop_inet6.c @@ -0,0 +1,320 @@ +/*------------------------------------------------------------------------- + * + * prop_inet6.c + * Property-based tests for IPv6 inet operations. + * + * These exercise the IPv6 path of the inet type specifically: the 128-bit + * family/masklen accessors, inet_in/inet_out round-tripping of the full + * eight-group form, the abbreviated ("::"-compressed) rendering parsing back + * to the same value, subnet containment, and the family partition that keeps + * IPv6 values from ever comparing equal to an IPv4 value. A failure means + * the 128-bit code mishandled an address, mask, or comparison -- a bug. + * + * This area is distinct from inet/inetarith/networkops/cidrops: every value + * here is IPv6, built from eight drawn 16-bit groups so no "::" compression + * ambiguity leaks into the generator. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_inet6.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "inet6" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw an IPv6 inet value as a quoted SQL literal cast to inet, e.g. + * "inet '1:2:3:4:5:6:7:8/64'". All eight 16-bit groups are always emitted in + * lowercase hex and joined by ':', so the generator never produces the "::" + * compressed form and there is no compression ambiguity in the input. The + * masklen is drawn across the full 0..128 range. *len receives the drawn + * masklen so a property can reference it. + */ +static void +draw_inet6(pg_prop_tc *tc, StringInfo buf, int *len) +{ + int masklen = (int) pg_prop_draw_int(tc, 0, 128); + + appendStringInfoString(buf, "inet '"); + for (int i = 0; i < 8; i++) + { + if (i > 0) + appendStringInfoChar(buf, ':'); + appendStringInfo(buf, "%x", (unsigned) pg_prop_draw_int(tc, 0, 65535)); + } + appendStringInfo(buf, "/%d'", masklen); + + *len = masklen; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a validly-generated IPv6 value parsed to NULL or errored, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * family() of any IPv6 value is 6. + */ +PG_PROP(family_6) +{ + pg_prop_conn *conn; + char *err = NULL; + int len; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_inet6(tc, &i, &len); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT family(%s) = 6", i.data); + + require_true(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * masklen() returns the prefix length the value was built with. + */ +PG_PROP(masklen) +{ + pg_prop_conn *conn; + char *err = NULL; + int len; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_inet6(tc, &i, &len); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT masklen(%s) = %d", i.data, len); + + require_true(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Text round-trip: an IPv6 inet rendered to text and parsed back equals the + * value. This exercises inet_out and inet_in on every generated value. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int len; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_inet6(tc, &i, &len); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)::text::inet = %s", i.data, i.data); + + require_true(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Forcing the mask to the full 128 bits yields a value whose masklen is 128: + * set_masklen does not lose or alter the address, and masklen reports the new + * prefix exactly. + */ +PG_PROP(host_in_network) +{ + pg_prop_conn *conn; + char *err = NULL; + int len; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_inet6(tc, &i, &len); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT masklen(set_masklen(%s, 128)) = 128", i.data); + + require_true(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The abbreviated ("::"-compressed) rendering parses back to the same inet. + * abbrev() returns text; casting it to inet must reproduce the value. + */ +PG_PROP(abbrev_consistent) +{ + pg_prop_conn *conn; + char *err = NULL; + int len; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_inet6(tc, &i, &len); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s = (abbrev(%s))::inet", i.data, i.data); + + require_true(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Containment is reflexive: a value is both contained-or-equal in itself and + * contains-or-equal itself (I <<= I AND I >>= I). + */ +PG_PROP(contains_self) +{ + pg_prop_conn *conn; + char *err = NULL; + int len; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_inet6(tc, &i, &len); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s <<= %s AND %s >>= %s", + i.data, i.data, i.data, i.data); + + require_true(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The network base of an inet is <= the address itself under inet ordering: + * clearing host bits can only lower (or leave unchanged) the value. + */ +PG_PROP(network_le_host) +{ + pg_prop_conn *conn; + char *err = NULL; + int len; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_inet6(tc, &i, &len); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT network(%s) <= %s", i.data, i.data); + + require_true(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * An IPv6 value is never equal to an IPv4 value: inet equality partitions by + * family, so comparing against a fixed IPv4 address always yields false. + */ +PG_PROP(v6_distinct_from_v4) +{ + pg_prop_conn *conn; + char *err = NULL; + int len; + StringInfoData i, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&i); + draw_inet6(tc, &i, &len); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s <> inet '192.168.0.1'", i.data); + + require_true(tc, conn, q.data); + + pfree(i.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 92a42d713f97b..d082bfa17b1bb 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 38b899fdaaa817aa6f5369d61859dda18ceb7e41 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:12 -0400 Subject: [PATCH 123/338] prop: add macaddr property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_macaddr.c | 466 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 470 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_macaddr.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index c5ca740012c1e..c35118ed6d9b0 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -138,7 +138,8 @@ OBJS = \ prop_inetarith.o \ prop_networkops.o \ prop_cidrops.o \ - prop_inet6.o + prop_inet6.o \ + prop_macaddr.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 85b1a7b9e9169..4248c54e18fc5 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -136,6 +136,7 @@ prop_area_sources = files( 'prop_networkops.c', 'prop_cidrops.c', 'prop_inet6.c', + 'prop_macaddr.c', ) diff --git a/src/test/prop/prop_macaddr.c b/src/test/prop/prop_macaddr.c new file mode 100644 index 0000000000000..ecd9415b3e167 --- /dev/null +++ b/src/test/prop/prop_macaddr.c @@ -0,0 +1,466 @@ +/*------------------------------------------------------------------------- + * + * prop_macaddr.c + * Property-based tests for the macaddr and macaddr8 types. + * + * macaddr/macaddr8 parsing accepts several input formats (colon, dash, + * dotted, bare) and must canonicalize them identically; comparison is a + * byte-order total order. These check format tolerance, text round-trip, + * and ordering. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_macaddr.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "macaddr" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw 6 random bytes and render them as a 12-character lowercase hex string + * (no separators) into "hex", a caller-provided buffer of at least 13 bytes. + * Each byte is drawn with pg_prop_draw_int(0, 255) and printed "%02x", so the + * result is always fixed-width and lowercase -- the common value from which + * every input form (colon/dash/dotted/bare) is built, so equality across + * forms is the expected answer. + */ +static void +draw_mac6(pg_prop_tc *tc, char *hex) +{ + int i; + + for (i = 0; i < 6; i++) + { + int b = (int) pg_prop_draw_int(tc, 0, 255); + + snprintf(hex + i * 2, 3, "%02x", b); + } + hex[12] = '\0'; +} + +/* + * Draw 8 random bytes and render them as a 16-character lowercase hex string + * (no separators) into "hex", a caller-provided buffer of at least 17 bytes. + * This is the EUI-64 value a macaddr8 is built from. + */ +static void +draw_mac8(pg_prop_tc *tc, char *hex) +{ + int i; + + for (i = 0; i < 8; i++) + { + int b = (int) pg_prop_draw_int(tc, 0, 255); + + snprintf(hex + i * 2, 3, "%02x", b); + } + hex[16] = '\0'; +} + +/* + * Colon form of a 12-char hex string: xx:xx:xx:xx:xx:xx, appended to buf. + * Separators go between each pair of hex characters. + */ +static void +append_colon6(StringInfo buf, const char *hex) +{ + int i; + + for (i = 0; i < 6; i++) + { + if (i > 0) + appendStringInfoChar(buf, ':'); + appendBinaryStringInfo(buf, hex + i * 2, 2); + } +} + +/* + * Dash form of a 12-char hex string: xx-xx-xx-xx-xx-xx, appended to buf. + */ +static void +append_dash6(StringInfo buf, const char *hex) +{ + int i; + + for (i = 0; i < 6; i++) + { + if (i > 0) + appendStringInfoChar(buf, '-'); + appendBinaryStringInfo(buf, hex + i * 2, 2); + } +} + +/* + * Dotted (Cisco) form of a 12-char hex string: xxxx.xxxx.xxxx, appended to + * buf. Four hex characters per group, three groups separated by dots. + */ +static void +append_dotted6(StringInfo buf, const char *hex) +{ + int i; + + for (i = 0; i < 3; i++) + { + if (i > 0) + appendStringInfoChar(buf, '.'); + appendBinaryStringInfo(buf, hex + i * 4, 4); + } +} + +/* + * Colon form of a 16-char hex string (8 bytes, EUI-64): + * xx:xx:xx:xx:xx:xx:xx:xx, appended to buf. + */ +static void +append_colon8(StringInfo buf, const char *hex) +{ + int i; + + for (i = 0; i < 8; i++) + { + if (i > 0) + appendStringInfoChar(buf, ':'); + appendBinaryStringInfo(buf, hex + i * 2, 2); + } +} + +/* + * Dash form of a 16-char hex string (8 bytes, EUI-64): + * xx-xx-xx-xx-xx-xx-xx-xx, appended to buf. + */ +static void +append_dash8(StringInfo buf, const char *hex) +{ + int i; + + for (i = 0; i < 8; i++) + { + if (i > 0) + appendStringInfoChar(buf, '-'); + appendBinaryStringInfo(buf, hex + i * 2, 2); + } +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a validly-generated value parsed to NULL or errored, which + * is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Text round-trip: parsing a canonical colon-form macaddr and printing it + * yields a value that re-parses equal. (C::macaddr)::text::macaddr = + * C::macaddr. This exercises macaddr_in then macaddr_out then macaddr_in. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex[13]; + StringInfoData c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_mac6(tc, hex); + initStringInfo(&c); + append_colon6(&c, hex); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ('%s'::macaddr)::text::macaddr = '%s'::macaddr", + c.data, c.data); + + require_true(tc, conn, q.data); + + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Colon equals dash: the colon-separated and dash-separated input forms of + * the same six bytes denote the same macaddr, since macaddr_in canonicalizes + * both to the same value. + */ +PG_PROP(colon_equals_dash) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex[13]; + StringInfoData c, + d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_mac6(tc, hex); + initStringInfo(&c); + initStringInfo(&d); + append_colon6(&c, hex); + append_dash6(&d, hex); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ('%s'::macaddr) = ('%s'::macaddr)", + c.data, d.data); + + require_true(tc, conn, q.data); + + pfree(c.data); + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Colon equals dotted: the Cisco dotted form (xxxx.xxxx.xxxx) of the same six + * bytes denotes the same macaddr as the colon form. + */ +PG_PROP(colon_equals_dotted) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex[13]; + StringInfoData c, + dot, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_mac6(tc, hex); + initStringInfo(&c); + initStringInfo(&dot); + append_colon6(&c, hex); + append_dotted6(&dot, hex); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ('%s'::macaddr) = ('%s'::macaddr)", + c.data, dot.data); + + require_true(tc, conn, q.data); + + pfree(c.data); + pfree(dot.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Colon equals bare: the bare 12-hex form (no separators) of the same six + * bytes denotes the same macaddr as the colon form. + */ +PG_PROP(colon_equals_bare) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex[13]; + StringInfoData c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_mac6(tc, hex); + initStringInfo(&c); + append_colon6(&c, hex); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ('%s'::macaddr) = ('%s'::macaddr)", + c.data, hex); + + require_true(tc, conn, q.data); + + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Total order: for two independently drawn macaddrs exactly one of <, =, > + * holds, so the sum of those three booleans cast to int is 1. This checks + * macaddr comparison is a total order (trichotomy). + */ +PG_PROP(comparison_total_order) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex1[13], + hex2[13]; + StringInfoData c1, + c2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_mac6(tc, hex1); + draw_mac6(tc, hex2); + initStringInfo(&c1); + initStringInfo(&c2); + append_colon6(&c1, hex1); + append_colon6(&c2, hex2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (('%s'::macaddr < '%s'::macaddr)::int " + "+ ('%s'::macaddr = '%s'::macaddr)::int " + "+ ('%s'::macaddr > '%s'::macaddr)::int) = 1", + c1.data, c2.data, c1.data, c2.data, c1.data, c2.data); + + require_true(tc, conn, q.data); + + pfree(c1.data); + pfree(c2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Order matches the hex string: macaddr comparison must agree with the + * lexicographic comparison of the canonical lowercase 12-char hex (no + * separators). Both hex strings are lowercase fixed-width, so SQL text "<" is + * the right oracle, and the colon and bare forms are built from the same + * bytes. + */ +PG_PROP(order_matches_hex) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex1[13], + hex2[13]; + StringInfoData c1, + c2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_mac6(tc, hex1); + draw_mac6(tc, hex2); + initStringInfo(&c1); + initStringInfo(&c2); + append_colon6(&c1, hex1); + append_colon6(&c2, hex2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('%s'::macaddr < '%s'::macaddr) = ('%s' < '%s')", + c1.data, c2.data, hex1, hex2); + + require_true(tc, conn, q.data); + + pfree(c1.data); + pfree(c2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * macaddr8 text round-trip: parsing a canonical colon-form macaddr8 and + * printing it yields a value that re-parses equal. Exercises macaddr8_in, + * macaddr8_out, macaddr8_in. + */ +PG_PROP(macaddr8_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex[17]; + StringInfoData c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_mac8(tc, hex); + initStringInfo(&c); + append_colon8(&c, hex); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ('%s'::macaddr8)::text::macaddr8 = '%s'::macaddr8", + c.data, c.data); + + require_true(tc, conn, q.data); + + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * macaddr8 colon equals dash: the colon-separated and dash-separated input + * forms of the same eight bytes denote the same macaddr8, since macaddr8_in + * canonicalizes both to the same value. + */ +PG_PROP(macaddr8_colon_equals_dash) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex[17]; + StringInfoData c, + d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_mac8(tc, hex); + initStringInfo(&c); + initStringInfo(&d); + append_colon8(&c, hex); + append_dash8(&d, hex); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ('%s'::macaddr8) = ('%s'::macaddr8)", + c.data, d.data); + + require_true(tc, conn, q.data); + + pfree(c.data); + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index d082bfa17b1bb..c66788aceccc1 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From b63a7d079d8bd03b10e00be8c3c7721c6b46073a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:12 -0400 Subject: [PATCH 124/338] prop: add macaddr8 round-trip, trunc, and conversion property tests (macaddr8) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_macaddr8.c | 384 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 388 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_macaddr8.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index c35118ed6d9b0..4a5c71b0ada32 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -139,7 +139,8 @@ OBJS = \ prop_networkops.o \ prop_cidrops.o \ prop_inet6.o \ - prop_macaddr.o + prop_macaddr.o \ + prop_macaddr8.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 4248c54e18fc5..d26f5e66e1dfb 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -137,6 +137,7 @@ prop_area_sources = files( 'prop_cidrops.c', 'prop_inet6.c', 'prop_macaddr.c', + 'prop_macaddr8.c', ) diff --git a/src/test/prop/prop_macaddr8.c b/src/test/prop/prop_macaddr8.c new file mode 100644 index 0000000000000..c089380f040d0 --- /dev/null +++ b/src/test/prop/prop_macaddr8.c @@ -0,0 +1,384 @@ +/*------------------------------------------------------------------------- + * + * prop_macaddr8.c + * Property-based tests for macaddr8 (EUI-64) algebra and conversions. + * + * These cover the macaddr8-specific operations that prop_macaddr.c does not: + * trunc() (zeroing the trailing five bytes), macaddr8_set7bit() (idempotent + * OR of 0x02 into the first byte), the EUI-48 -> EUI-64 insertion of ff:fe + * when a 6-byte macaddr is cast to macaddr8, and that cast's reversibility. + * The oracle is each function's documented definition in mac8.c; a failure + * is a bug in trunc/set7bit/the conversion operators, not a rounding effect. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_macaddr8.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "macaddr8" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw 8 random bytes and render them as a 16-character lowercase hex string + * (no separators) into "hex", a caller-provided buffer of at least 17 bytes. + * Each byte is drawn with pg_prop_draw_int(0, 255) and printed "%02x", so the + * result is always fixed-width and lowercase. This is the EUI-64 value a + * macaddr8 is built from. + */ +static void +draw_mac8(pg_prop_tc *tc, char *hex) +{ + int i; + + for (i = 0; i < 8; i++) + { + int b = (int) pg_prop_draw_int(tc, 0, 255); + + snprintf(hex + i * 2, 3, "%02x", b); + } + hex[16] = '\0'; +} + +/* + * Draw 6 random bytes and render them as a 12-character lowercase hex string + * (no separators) into "hex", a caller-provided buffer of at least 13 bytes. + * This is the EUI-48 value a macaddr is built from. + */ +static void +draw_mac6(pg_prop_tc *tc, char *hex) +{ + int i; + + for (i = 0; i < 6; i++) + { + int b = (int) pg_prop_draw_int(tc, 0, 255); + + snprintf(hex + i * 2, 3, "%02x", b); + } + hex[12] = '\0'; +} + +/* + * Colon form of an n-byte hex string (2n hex characters): xx:xx:...:xx, + * appended to buf. Separators go between each pair of hex characters. + */ +static void +append_colon(StringInfo buf, const char *hex, int nbytes) +{ + int i; + + for (i = 0; i < nbytes; i++) + { + if (i > 0) + appendStringInfoChar(buf, ':'); + appendBinaryStringInfo(buf, hex + i * 2, 2); + } +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a validly-generated value parsed to NULL or errored, which + * is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * text_roundtrip: a drawn macaddr8 survives a text round trip. + * (M)::text::macaddr8 = M. Exercises macaddr8_in, macaddr8_out, macaddr8_in. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex[17]; + StringInfoData m, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_mac8(tc, hex); + initStringInfo(&m); + append_colon(&m, hex, 8); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('%s'::macaddr8)::text::macaddr8 = '%s'::macaddr8", + m.data, m.data); + + require_true(tc, conn, q.data); + + pfree(m.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * total_order: for two independently drawn macaddr8s exactly one of <, =, > + * holds, so the sum of those three booleans cast to int is 1 (trichotomy). + */ +PG_PROP(total_order) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex1[17], + hex2[17]; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_mac8(tc, hex1); + draw_mac8(tc, hex2); + initStringInfo(&a); + initStringInfo(&b); + append_colon(&a, hex1, 8); + append_colon(&b, hex2, 8); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (('%s'::macaddr8 < '%s'::macaddr8)::int " + "+ ('%s'::macaddr8 = '%s'::macaddr8)::int " + "+ ('%s'::macaddr8 > '%s'::macaddr8)::int) = 1", + a.data, b.data, a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * trunc_zeros_tail: trunc(M) keeps the first three bytes and zeroes the last + * five. Checked against the reconstructed value (first 3 bytes followed by + * five zero bytes), so this pins the exact manufacturer-prefix semantics of + * macaddr8_trunc rather than just the printed suffix. + */ +PG_PROP(trunc_zeros_tail) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex[17]; + StringInfoData m, + pfx, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_mac8(tc, hex); + initStringInfo(&m); + append_colon(&m, hex, 8); + + /* first three bytes in colon form, then five explicit zero bytes */ + initStringInfo(&pfx); + append_colon(&pfx, hex, 3); + appendStringInfoString(&pfx, ":00:00:00:00:00"); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT trunc('%s'::macaddr8) = '%s'::macaddr8", + m.data, pfx.data); + + require_true(tc, conn, q.data); + + pfree(m.data); + pfree(pfx.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * set7bit_idempotent: macaddr8_set7bit ORs 0x02 into the first byte, so + * applying it twice equals applying it once. + */ +PG_PROP(set7bit_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex[17]; + StringInfoData m, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_mac8(tc, hex); + initStringInfo(&m); + append_colon(&m, hex, 8); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT macaddr8_set7bit(macaddr8_set7bit('%s'::macaddr8)) " + "= macaddr8_set7bit('%s'::macaddr8)", + m.data, m.data); + + require_true(tc, conn, q.data); + + pfree(m.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * self_equal: a drawn macaddr8 equals itself (reflexivity of equality). + */ +PG_PROP(self_equal) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex[17]; + StringInfoData m, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_mac8(tc, hex); + initStringInfo(&m); + append_colon(&m, hex, 8); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT '%s'::macaddr8 = '%s'::macaddr8", + m.data, m.data); + + require_true(tc, conn, q.data); + + pfree(m.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * macaddr_to_macaddr8: casting a 6-byte macaddr to macaddr8 inserts ff:fe as + * the 4th and 5th bytes (the standard EUI-48 -> EUI-64 mapping), and that + * cast is reversible, so (MA::macaddr8)::macaddr = MA. The round trip pins + * both halves of the conversion at once. + */ +PG_PROP(macaddr_to_macaddr8) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex[13]; + StringInfoData ma, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_mac6(tc, hex); + initStringInfo(&ma); + append_colon(&ma, hex, 6); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT position(':ff:fe:' in ('%s'::macaddr8)::text) > 0 " + "AND (('%s'::macaddr)::macaddr8)::macaddr = '%s'::macaddr", + ma.data, ma.data, ma.data); + + require_true(tc, conn, q.data); + + pfree(ma.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * order_matches_bytes: for two macaddr8s that differ only in the first byte, + * the one with the smaller first byte compares less. Draw b1 < b1' and share + * the remaining seven bytes so the comparison is decided at byte one. + */ +PG_PROP(order_matches_bytes) +{ + pg_prop_conn *conn; + char *err = NULL; + char hex1[17], + hex2[17]; + int b1, + b1p, + t; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* draw two distinct first bytes with b1 < b1p */ + b1 = (int) pg_prop_draw_int(tc, 0, 254); + b1p = (int) pg_prop_draw_int(tc, b1 + 1, 255); + + /* build two 8-byte hex strings sharing the seven trailing bytes */ + snprintf(hex1, 3, "%02x", b1); + snprintf(hex2, 3, "%02x", b1p); + for (t = 1; t < 8; t++) + { + int byte = (int) pg_prop_draw_int(tc, 0, 255); + + snprintf(hex1 + t * 2, 3, "%02x", byte); + memcpy(hex2 + t * 2, hex1 + t * 2, 2); + } + hex1[16] = '\0'; + hex2[16] = '\0'; + + initStringInfo(&a); + append_colon(&a, hex1, 8); + + initStringInfo(&b); + append_colon(&b, hex2, 8); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT '%s'::macaddr8 < '%s'::macaddr8", + a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index c66788aceccc1..90ef2a6a5133f 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From f6d5ba4a3144e390e2c429142014169dde549bc6 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:12 -0400 Subject: [PATCH 125/338] prop: add tsquery property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_tsquery.c | 357 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 361 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_tsquery.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 4a5c71b0ada32..014da0613fb8d 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -140,7 +140,8 @@ OBJS = \ prop_cidrops.o \ prop_inet6.o \ prop_macaddr.o \ - prop_macaddr8.o + prop_macaddr8.o \ + prop_tsquery.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index d26f5e66e1dfb..e9b4eb9d23062 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -138,6 +138,7 @@ prop_area_sources = files( 'prop_inet6.c', 'prop_macaddr.c', 'prop_macaddr8.c', + 'prop_tsquery.c', ) diff --git a/src/test/prop/prop_tsquery.c b/src/test/prop/prop_tsquery.c new file mode 100644 index 0000000000000..10fda3e77a735 --- /dev/null +++ b/src/test/prop/prop_tsquery.c @@ -0,0 +1,357 @@ +/*------------------------------------------------------------------------- + * + * prop_tsquery.c + * Property-based tests for full-text search tsquery/tsvector. + * + * tsquery parsing/normalization and the @@ match operator have had fixes; + * these check tsquery text round-trip and the boolean semantics of @@ against + * generated tsvectors. + * + * Everything is built through the 'simple' text search configuration, so a + * lowercase ascii word maps to itself as a lexeme -- no stemming, no + * stopwords -- which makes the membership/boolean oracles below exact. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_tsquery.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "tsquery" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Tiny lexeme alphabet. Four single-letter words keep the generated docs and + * queries legible in a counterexample and, more importantly, make matches + * common: with only four possible words a random query term is likely to be + * present in a random document, so the boolean properties actually exercise + * both the true and false sides of @@. + */ +static const char *const lexemes[] = {"a", "b", "c", "d"}; +#define NLEX ((int) (sizeof(lexemes) / sizeof(lexemes[0]))) + +/* Draw one lexeme from the alphabet. */ +static const char * +draw_lexeme(pg_prop_tc *tc) +{ + return lexemes[pg_prop_draw_int(tc, 0, NLEX - 1)]; +} + +/* + * Draw a document: 0..6 space-separated lexemes, e.g. "a b a c". The raw + * words are emitted; the caller wraps them in to_tsvector('simple', ''). + */ +static void +draw_doc(pg_prop_tc *tc, StringInfo buf) +{ + int nwords = (int) pg_prop_draw_int(tc, 0, 6); + + for (int i = 0; i < nwords; i++) + { + if (i > 0) + appendStringInfoChar(buf, ' '); + appendStringInfoString(buf, draw_lexeme(tc)); + } +} + +/* + * Draw a boolean tsquery expression over the alphabet using & | ! and + * parentheses, e.g. "a & (b | !c)". Depth-limited and always parseable: + * parens are balanced here by construction and no operator is ever left + * dangling, so to_tsquery('simple', '') always succeeds. The result + * always contains at least one lexeme, so it never normalizes to the empty + * query (which would make @@ return NULL). + */ +static void +draw_query(pg_prop_tc *tc, StringInfo buf, int depth) +{ + int kind; + + /* + * At the depth cap only a leaf is allowed, guaranteeing termination and a + * non-empty query. + */ + if (depth >= 3) + kind = 0; + else + kind = (int) pg_prop_draw_int(tc, 0, 3); + + switch (kind) + { + case 0: /* leaf lexeme */ + appendStringInfoString(buf, draw_lexeme(tc)); + break; + case 1: /* negation: ! */ + appendStringInfoString(buf, "!"); + draw_query(tc, buf, depth + 1); + break; + case 2: /* conjunction: ( & ) */ + appendStringInfoChar(buf, '('); + draw_query(tc, buf, depth + 1); + appendStringInfoString(buf, " & "); + draw_query(tc, buf, depth + 1); + appendStringInfoChar(buf, ')'); + break; + default: /* disjunction: ( | ) */ + appendStringInfoChar(buf, '('); + draw_query(tc, buf, depth + 1); + appendStringInfoString(buf, " | "); + draw_query(tc, buf, depth + 1); + appendStringInfoChar(buf, ')'); + break; + } +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on valid input, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * tsvector text round-trip: a tsvector rendered to text and parsed back as a + * tsvector equals the original. Exercises tsvectorout/tsvectorin. + */ +PG_PROP(tsvector_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_doc(tc, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (v::text::tsvector = v) " + "FROM (SELECT to_tsvector('simple', '%s') AS v) s", + d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * tsquery text round-trip: to_tsquery normalizes the input, and rendering the + * normalized query to text and parsing it back is stable. Exercises + * tsqueryout/tsqueryin and the normalization in to_tsquery. + */ +PG_PROP(tsquery_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData qexpr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&qexpr); + draw_query(tc, &qexpr, 0); + + initStringInfo(&q); + + /* + * Compare the TEXT, not the tsquery = operator. tsquery equality is + * structural (it compares the serialized operator tree, including the + * grouping of & and |), and text output does not preserve that: a & (a & + * a) and (a & a) & a print identically but are unequal values. The sound + * round-trip is that parsing the printed form and reprinting reproduces + * the text. + */ + appendStringInfo(&q, + "SELECT (t::text::tsquery::text = t::text) " + "FROM (SELECT to_tsquery('simple', '%s') AS t) s", + qexpr.data); + + require_true(tc, conn, q.data); + + pfree(qexpr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A single-lexeme query matches a document iff that word is among the + * document's words. With the 'simple' config each lowercase ascii word is + * its own lexeme, so the oracle is independent: '' = ANY of the words of D + * split on spaces. Exercises the @@ operator on a one-term query. + */ +PG_PROP(single_lexeme_match_is_membership) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *l; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_doc(tc, &d); + l = draw_lexeme(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (to_tsvector('simple', '%s') @@ to_tsquery('simple', '%s')) " + "= ('%s' = ANY (string_to_array('%s', ' ')))", + d.data, l, l, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Conjunction: (D @@ (L1 & L2)) = ((D @@ L1) AND (D @@ L2)). The AND-query's + * match equals the AND of the two single-lexeme matches. + */ +PG_PROP(and_is_conjunction) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *l1, + *l2; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_doc(tc, &d); + l1 = draw_lexeme(tc); + l2 = draw_lexeme(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (v @@ to_tsquery('simple', '%s & %s')) " + "= ((v @@ to_tsquery('simple', '%s')) " + "AND (v @@ to_tsquery('simple', '%s'))) " + "FROM (SELECT to_tsvector('simple', '%s') AS v) s", + l1, l2, l1, l2, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Disjunction: (D @@ (L1 | L2)) = ((D @@ L1) OR (D @@ L2)). The OR-query's + * match equals the OR of the two single-lexeme matches. + */ +PG_PROP(or_is_disjunction) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *l1, + *l2; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_doc(tc, &d); + l1 = draw_lexeme(tc); + l2 = draw_lexeme(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (v @@ to_tsquery('simple', '%s | %s')) " + "= ((v @@ to_tsquery('simple', '%s')) " + "OR (v @@ to_tsquery('simple', '%s'))) " + "FROM (SELECT to_tsvector('simple', '%s') AS v) s", + l1, l2, l1, l2, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Negation: (D @@ !L) = NOT (D @@ L). The '!L' query (a valid tsquery + * meaning "not containing L") matches iff the positive query does not. + */ +PG_PROP(not_is_negation) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *l; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_doc(tc, &d); + l = draw_lexeme(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (v @@ to_tsquery('simple', '!%s')) " + "= NOT (v @@ to_tsquery('simple', '%s')) " + "FROM (SELECT to_tsvector('simple', '%s') AS v) s", + l, l, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 90ef2a6a5133f..40085996bdd9d 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From ee60aa84d29e6ed2847c1df2dd09d810ffd0922c Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:12 -0400 Subject: [PATCH 126/338] prop: add tsvector property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_tsvector.c | 306 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 310 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_tsvector.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 014da0613fb8d..2227069e85afc 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -141,7 +141,8 @@ OBJS = \ prop_inet6.o \ prop_macaddr.o \ prop_macaddr8.o \ - prop_tsquery.o + prop_tsquery.o \ + prop_tsvector.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index e9b4eb9d23062..a17c6520eee9e 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -139,6 +139,7 @@ prop_area_sources = files( 'prop_macaddr.c', 'prop_macaddr8.c', 'prop_tsquery.c', + 'prop_tsvector.c', ) diff --git a/src/test/prop/prop_tsvector.c b/src/test/prop/prop_tsvector.c new file mode 100644 index 0000000000000..e264b6df532e1 --- /dev/null +++ b/src/test/prop/prop_tsvector.c @@ -0,0 +1,306 @@ +/*------------------------------------------------------------------------- + * + * prop_tsvector.c + * Property-based tests for tsvector structure. + * + * tsvector concatenation, setweight, strip, and length (tsvector_op.c) have + * had fixes; these check structural invariants of tsvector independent of @@ + * matching (which the tsquery area covers). + * + * Everything is built through the 'simple' text search configuration, so a + * lowercase ascii word maps to itself as a lexeme -- no stemming, no + * stopwords -- which makes the distinct-lexeme oracles below exact. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_tsvector.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "tsvector" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Tiny lexeme alphabet. Four single-letter words keep the generated docs + * legible in a counterexample and make duplicate words common, so the + * distinct-count and union-bound properties below actually exercise dedup + * (D with repeats) and overlap (D1, D2 sharing words). + */ +static const char *const lexemes[] = {"a", "b", "c", "d"}; +#define NLEX ((int) (sizeof(lexemes) / sizeof(lexemes[0]))) + +/* Draw one lexeme from the alphabet. */ +static const char * +draw_lexeme(pg_prop_tc *tc) +{ + return lexemes[pg_prop_draw_int(tc, 0, NLEX - 1)]; +} + +/* + * Draw a document: 0..8 space-separated lowercase words, e.g. "a b a c". The + * raw words are emitted single-space-separated with no leading or trailing + * space; the caller wraps them in to_tsvector('simple', ''). + */ +static void +draw_doc(pg_prop_tc *tc, StringInfo buf) +{ + int nwords = (int) pg_prop_draw_int(tc, 0, 8); + + for (int i = 0; i < nwords; i++) + { + if (i > 0) + appendStringInfoChar(buf, ' '); + appendStringInfoString(buf, draw_lexeme(tc)); + } +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on valid input, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * length(tsvector) is the distinct-lexeme count. With the 'simple' config + * each lowercase ascii word maps 1:1 to a lexeme, so the length must equal the + * number of distinct words in D, counted independently by unnesting the raw + * document on spaces and DISTINCT-counting. draw_doc emits single-space + * tokens with no empty slots, but w <> '' is filtered defensively so a stray + * empty token (from any accidental double space) never inflates the oracle. + */ +PG_PROP(length_is_distinct_lexeme_count) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_doc(tc, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(to_tsvector('simple', '%s')) " + "= (SELECT count(DISTINCT w) " + "FROM unnest(string_to_array('%s', ' ')) w " + "WHERE w <> '')", + d.data, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Concatenation contains every lexeme of its left operand: for each word of + * D1, the vector (V1 || V2) still matches a single-lexeme query for that word. + * This checks that || drops no lexeme from the left side. bool_and over zero + * rows (empty D1) is NULL, so coalesce(..., true) treats the vacuous case as + * holding. + */ +PG_PROP(concat_contains_both) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d1, + d2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d1); + initStringInfo(&d2); + draw_doc(tc, &d1); + draw_doc(tc, &d2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(bool_and(" + "(to_tsvector('simple', '%s') || to_tsvector('simple', '%s')) " + "@@ to_tsquery('simple', w)), true) " + "FROM unnest(string_to_array('%s', ' ')) w " + "WHERE w <> ''", + d1.data, d2.data, d1.data); + + require_true(tc, conn, q.data); + + pfree(d1.data); + pfree(d2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * strip() removes positions and weights but keeps the lexeme set, so the + * stripped vector has the same length as the original. + */ +PG_PROP(strip_preserves_lexemes) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_doc(tc, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(strip(to_tsvector('simple', '%s'))) " + "= length(to_tsvector('simple', '%s'))", + d.data, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * setweight() changes the weight label on positions but not the lexeme set, + * so the length is unchanged after relabelling every position to weight 'A'. + */ +PG_PROP(setweight_preserves_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_doc(tc, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(setweight(to_tsvector('simple', '%s'), 'A')) " + "= length(to_tsvector('simple', '%s'))", + d.data, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * tsvector text round-trip: a tsvector rendered to text and parsed back as a + * tsvector equals the original. Exercises tsvectorout/tsvectorin on the + * structure this area builds (positions, dedup, ordering). + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_doc(tc, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_tsvector('simple', '%s')::text::tsvector " + "= to_tsvector('simple', '%s')", + d.data, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Concatenation is a union of lexeme sets: the distinct-lexeme count of + * (V1 || V2) is at least each operand's count (nothing is lost) and at most + * their sum (no lexeme is counted twice beyond the two sides). Equality to + * the max happens when one set contains the other; equality to the sum when + * they are disjoint. + */ +PG_PROP(concat_length_bounds) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d1, + d2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d1); + initStringInfo(&d2); + draw_doc(tc, &d1); + draw_doc(tc, &d2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(v1 || v2) >= greatest(length(v1), length(v2)) " + "AND length(v1 || v2) <= length(v1) + length(v2) " + "FROM (SELECT to_tsvector('simple', '%s') AS v1, " + "to_tsvector('simple', '%s') AS v2) s", + d1.data, d2.data); + + require_true(tc, conn, q.data); + + pfree(d1.data); + pfree(d2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 40085996bdd9d..30f52c89b8f0e 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 1fa4095f819e055e77f3d5748eb76901abe4a64c Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:12 -0400 Subject: [PATCH 127/338] prop: add tsvector concat, setweight, strip, and delete property tests (tsvector2) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_tsvector2.c | 364 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 368 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_tsvector2.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 2227069e85afc..fc43ae8f3a402 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -142,7 +142,8 @@ OBJS = \ prop_macaddr.o \ prop_macaddr8.o \ prop_tsquery.o \ - prop_tsvector.o + prop_tsvector.o \ + prop_tsvector2.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index a17c6520eee9e..5f6190b5a8b5f 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -140,6 +140,7 @@ prop_area_sources = files( 'prop_macaddr8.c', 'prop_tsquery.c', 'prop_tsvector.c', + 'prop_tsvector2.c', ) diff --git a/src/test/prop/prop_tsvector2.c b/src/test/prop/prop_tsvector2.c new file mode 100644 index 0000000000000..a2a4b00335cf2 --- /dev/null +++ b/src/test/prop/prop_tsvector2.c @@ -0,0 +1,364 @@ +/*------------------------------------------------------------------------- + * + * prop_tsvector2.c + * Property-based tests for tsvector @@ matching and editing semantics. + * + * Where prop_tsvector.c checks structural invariants (length, concat bounds, + * text round-trip), this area checks how tsvector operations interact with @@ + * matching: concatenation keeps the left operand's matches, strip() keeps + * lexeme matching while dropping positions, setweight() does not change + * matching, ts_delete() removes a lexeme's matches, and a document matches a + * query for any word it contains. A violation here is a bug in tsvector_op.c + * (concat/setweight/strip/ts_delete) or in @@ evaluation. + * + * Everything is built through the 'simple' text search configuration, so a + * lowercase ascii word maps to itself as a lexeme -- no stemming, no + * stopwords -- which makes "word present in the doc" exactly "lexeme in the + * tsvector" and keeps the oracles below exact. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_tsvector2.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "tsvector2" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Tiny lexeme alphabet. Four single-letter words keep generated docs legible + * in a counterexample and make duplicate words common, so concat dedup and + * ts_delete overlap are actually exercised. + */ +static const char *const lexemes[] = {"a", "b", "c", "d"}; +#define NLEX ((int) (sizeof(lexemes) / sizeof(lexemes[0]))) +#define MAXWORDS 8 + +/* + * Draw a non-empty document into buf (space-separated words) and record each + * drawn word in words[]. Returns the word count (1..MAXWORDS). The doc is + * never empty so at least one lexeme is always present, which every property + * below relies on when it picks a "present word" w = words[i]. + */ +static int +draw_doc(pg_prop_tc *tc, StringInfo buf, const char *words[]) +{ + int nwords = (int) pg_prop_draw_int(tc, 1, MAXWORDS); + + for (int i = 0; i < nwords; i++) + { + words[i] = lexemes[pg_prop_draw_int(tc, 0, NLEX - 1)]; + if (i > 0) + appendStringInfoChar(buf, ' '); + appendStringInfoString(buf, words[i]); + } + return nwords; +} + +/* Draw one of the words that is present in the just-built doc. */ +static const char * +draw_present_word(pg_prop_tc *tc, const char *words[], int nwords) +{ + return words[pg_prop_draw_int(tc, 0, nwords - 1)]; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on valid input, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Concatenation keeps the left operand's matches: for a word w present in V1, + * (V1 || V2) still matches to_tsquery('simple', w). Since w is drawn from the + * words of D1, V1 @@ w holds, and || must not drop it. + */ +PG_PROP(concat_contains_both) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *words1[MAXWORDS]; + const char *words2[MAXWORDS]; + const char *w; + int nwords; + StringInfoData d1, + d2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d1); + initStringInfo(&d2); + nwords = draw_doc(tc, &d1, words1); + (void) draw_doc(tc, &d2, words2); + w = draw_present_word(tc, words1, nwords); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (to_tsvector('simple', '%s') " + "|| to_tsvector('simple', '%s')) " + "@@ to_tsquery('simple', '%s')", + d1.data, d2.data, w); + + require_true(tc, conn, q.data); + + pfree(d1.data); + pfree(d2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * length() is non-negative and equals the distinct-lexeme count. With the + * 'simple' config each lowercase word maps 1:1 to a lexeme, so the length must + * match a DISTINCT count of the raw words (filtering any empty token + * defensively). + */ +PG_PROP(length_nonneg) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *words[MAXWORDS]; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + (void) draw_doc(tc, &d, words); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(to_tsvector('simple', '%s')) >= 0 " + "AND length(to_tsvector('simple', '%s')) " + "= (SELECT count(DISTINCT w) " + "FROM unnest(string_to_array('%s', ' ')) w " + "WHERE w <> '')", + d.data, d.data, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * strip() drops positions but keeps lexemes, so for a non-phrase query (a + * single lexeme) strip(V) @@ Q matches exactly when V @@ Q. The two sides are + * compared for a word w drawn from the doc. + */ +PG_PROP(strip_removes_positions) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *words[MAXWORDS]; + const char *w; + int nwords; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + nwords = draw_doc(tc, &d, words); + w = draw_present_word(tc, words, nwords); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (strip(to_tsvector('simple', '%s')) " + "@@ to_tsquery('simple', '%s')) " + "= (to_tsvector('simple', '%s') " + "@@ to_tsquery('simple', '%s'))", + d.data, w, d.data, w); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * setweight() relabels weights but does not change which lexemes match, so + * setweight(V, 'A') @@ Q matches exactly when V @@ Q for a word w drawn from + * the doc. + */ +PG_PROP(setweight_preserves_match) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *words[MAXWORDS]; + const char *w; + int nwords; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + nwords = draw_doc(tc, &d, words); + w = draw_present_word(tc, words, nwords); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (setweight(to_tsvector('simple', '%s'), 'A') " + "@@ to_tsquery('simple', '%s')) " + "= (to_tsvector('simple', '%s') " + "@@ to_tsquery('simple', '%s'))", + d.data, w, d.data, w); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ts_delete() removes a lexeme: after deleting 'a', the vector no longer + * matches to_tsquery('simple', 'a'). Holds for every doc (ts_delete of an + * absent lexeme is a no-op, and the absent lexeme still does not match). + */ +PG_PROP(delete_removes_lexeme) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *words[MAXWORDS]; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + (void) draw_doc(tc, &d, words); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT NOT (ts_delete(to_tsvector('simple', '%s'), 'a') " + "@@ to_tsquery('simple', 'a'))", + d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Concatenation length is bounded by the sum of operand lengths; dedup of + * shared lexemes can only reduce it. length(V1 || V2) <= length(V1) + + * length(V2). + */ +PG_PROP(concat_length_bound) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *words[MAXWORDS]; + StringInfoData d1, + d2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d1); + initStringInfo(&d2); + (void) draw_doc(tc, &d1, words); + (void) draw_doc(tc, &d2, words); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(v1 || v2) <= length(v1) + length(v2) " + "FROM (SELECT to_tsvector('simple', '%s') AS v1, " + "to_tsvector('simple', '%s') AS v2) s", + d1.data, d2.data); + + require_true(tc, conn, q.data); + + pfree(d1.data); + pfree(d2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Self-match: a document matches a single-lexeme query for any word it + * contains. w is drawn from the doc's own words, so V @@ to_tsquery(w) must + * hold. + */ +PG_PROP(self_match) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *words[MAXWORDS]; + const char *w; + int nwords; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + nwords = draw_doc(tc, &d, words); + w = draw_present_word(tc, words, nwords); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_tsvector('simple', '%s') " + "@@ to_tsquery('simple', '%s')", + d.data, w); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 30f52c89b8f0e..19a91a2be9baa 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From f062e5600f87d0c77739da52baa741f3fa5d6a1f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:12 -0400 Subject: [PATCH 128/338] prop: add tsquery boolean laws and matching property tests (tsquerylaw) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_tsquerylaw.c | 342 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 346 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_tsquerylaw.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index fc43ae8f3a402..ff802e2df7c90 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -143,7 +143,8 @@ OBJS = \ prop_macaddr8.o \ prop_tsquery.o \ prop_tsvector.o \ - prop_tsvector2.o + prop_tsvector2.o \ + prop_tsquerylaw.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 5f6190b5a8b5f..01337fec1f516 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -141,6 +141,7 @@ prop_area_sources = files( 'prop_tsquery.c', 'prop_tsvector.c', 'prop_tsvector2.c', + 'prop_tsquerylaw.c', ) diff --git a/src/test/prop/prop_tsquerylaw.c b/src/test/prop/prop_tsquerylaw.c new file mode 100644 index 0000000000000..6360a01ced51f --- /dev/null +++ b/src/test/prop/prop_tsquerylaw.c @@ -0,0 +1,342 @@ +/*------------------------------------------------------------------------- + * + * prop_tsquerylaw.c + * Property-based tests for the boolean algebra of tsquery under @@. + * + * These check that the tsquery boolean connectives obey the laws of boolean + * algebra when matched against a tsvector: & and | commute, an AND match + * implies the corresponding OR match, ! negates, and de Morgan relates the + * negation of a conjunction to the disjunction of the negations. The oracle + * is boolean algebra itself -- these equalities hold for every document and + * every pair of lexemes -- so any "f" is a bug in tsquery normalization or in + * the @@ executor, not an input artefact. Distinct from prop_tsquery (which + * decomposes @@ against SQL AND/OR), prop_tsvector/prop_tsrank. + * + * Everything is built through the 'simple' configuration, so each lowercase + * ascii word is its own lexeme (no stemming, no stopwords), which keeps the + * laws exact. + * + * Tier B: runs SQL against the cluster the TAP driver started; see pg_prop.h + * and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_tsquerylaw.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "tsquerylaw" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Tiny lexeme alphabet. Four single-letter words keep counterexamples + * legible and make matches common: with only four possible words a random + * query term is likely present in a random document, so the laws exercise + * both the true and false sides of @@. + */ +static const char *const lexemes[] = {"a", "b", "c", "d"}; +#define NLEX ((int) (sizeof(lexemes) / sizeof(lexemes[0]))) + +/* Draw one lexeme from the alphabet. */ +static const char * +draw_lexeme(pg_prop_tc *tc) +{ + return lexemes[pg_prop_draw_int(tc, 0, NLEX - 1)]; +} + +/* + * Draw a document: 0..6 space-separated lexemes, e.g. "a b a c". The raw + * words are emitted; the caller wraps them in to_tsvector('simple', ''). + */ +static void +draw_doc(pg_prop_tc *tc, StringInfo buf) +{ + int nwords = (int) pg_prop_draw_int(tc, 0, 6); + + for (int i = 0; i < nwords; i++) + { + if (i > 0) + appendStringInfoChar(buf, ' '); + appendStringInfoString(buf, draw_lexeme(tc)); + } +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on valid input, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Commutativity of &: V @@ (a & b) = V @@ (b & a) for a drawn document V and + * drawn lexemes a, b. + */ +PG_PROP(and_commutative_match) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *a, + *b; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_doc(tc, &d); + a = draw_lexeme(tc); + b = draw_lexeme(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (v @@ to_tsquery('simple', '%s & %s')) " + "= (v @@ to_tsquery('simple', '%s & %s')) " + "FROM (SELECT to_tsvector('simple', '%s') AS v) s", + a, b, b, a, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Commutativity of |: V @@ (a | b) = V @@ (b | a). + */ +PG_PROP(or_commutative_match) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *a, + *b; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_doc(tc, &d); + a = draw_lexeme(tc); + b = draw_lexeme(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (v @@ to_tsquery('simple', '%s | %s')) " + "= (v @@ to_tsquery('simple', '%s | %s')) " + "FROM (SELECT to_tsvector('simple', '%s') AS v) s", + a, b, b, a, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * AND implies OR: whenever V @@ (a & b), also V @@ (a | b). A conjunction + * match is strictly stronger than the matching disjunction, so the material + * implication (NOT and) OR (or) must hold. + */ +PG_PROP(and_implies_or) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *a, + *b; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_doc(tc, &d); + a = draw_lexeme(tc); + b = draw_lexeme(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (NOT (v @@ to_tsquery('simple', '%s & %s'))) " + "OR (v @@ to_tsquery('simple', '%s | %s')) " + "FROM (SELECT to_tsvector('simple', '%s') AS v) s", + a, b, a, b, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Negation: V @@ (!a) = NOT (V @@ a). The '!a' query matches iff a is absent + * from the document, which is exactly the negation of the positive match. + */ +PG_PROP(not_negates) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *a; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_doc(tc, &d); + a = draw_lexeme(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (v @@ to_tsquery('simple', '!%s')) " + "= NOT (v @@ to_tsquery('simple', '%s')) " + "FROM (SELECT to_tsvector('simple', '%s') AS v) s", + a, a, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * de Morgan: V @@ (!(a & b)) = V @@ ((!a) | (!b)). Negating a conjunction is + * the disjunction of the negations; both sides must match the same documents. + */ +PG_PROP(de_morgan_match) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *a, + *b; + StringInfoData d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_doc(tc, &d); + a = draw_lexeme(tc); + b = draw_lexeme(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (v @@ to_tsquery('simple', '!(%s & %s)')) " + "= (v @@ to_tsquery('simple', '(!%s) | (!%s)')) " + "FROM (SELECT to_tsvector('simple', '%s') AS v) s", + a, b, a, b, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Single-lexeme match, constant anchor: 'a b' @@ 'a' is true and 'a b' @@ 'c' + * is false. A fixed positive and negative case pins the @@ operator so the + * boolean laws above are not all vacuously satisfied by a @@ that always + * returns the same value. + */ +PG_PROP(single_lexeme_match) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT (to_tsvector('simple', 'a b') " + "@@ to_tsquery('simple', 'a')) " + "AND NOT (to_tsvector('simple', 'a b') " + "@@ to_tsquery('simple', 'c'))"); + + pg_prop_disconnect(conn); +} + +/* + * tsquery text round-trip: a normalized tsquery rendered to text and parsed + * back equals the original. Fixed 'a & b' here; the structural = holds + * because the printed form reparses to the same operator tree. + */ +PG_PROP(tsquery_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT (to_tsquery('simple', 'a & b')::text::tsquery " + "= to_tsquery('simple', 'a & b'))"); + + pg_prop_disconnect(conn); +} + +/* + * Empty document never positively matches: to_tsvector('simple','') @@ + * to_tsquery('simple','a') is false. An empty tsvector contains no lexemes, + * so a positive single-term query cannot match. + */ +PG_PROP(empty_doc_no_positive_match) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT NOT (to_tsvector('simple', '') " + "@@ to_tsquery('simple', 'a'))"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 19a91a2be9baa..e2736dea685af 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 79fe50eadb683a80e0b9b86af9f6654152a2e057 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:13 -0400 Subject: [PATCH 129/338] prop: add full-text ranking bounds and determinism property tests (tsrank) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_tsrank.c | 384 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 388 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_tsrank.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index ff802e2df7c90..807e19066eb69 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -144,7 +144,8 @@ OBJS = \ prop_tsquery.o \ prop_tsvector.o \ prop_tsvector2.o \ - prop_tsquerylaw.o + prop_tsquerylaw.o \ + prop_tsrank.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 01337fec1f516..70b8dbf5de777 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -142,6 +142,7 @@ prop_area_sources = files( 'prop_tsvector.c', 'prop_tsvector2.c', 'prop_tsquerylaw.c', + 'prop_tsrank.c', ) diff --git a/src/test/prop/prop_tsrank.c b/src/test/prop/prop_tsrank.c new file mode 100644 index 0000000000000..d9281a74578a0 --- /dev/null +++ b/src/test/prop/prop_tsrank.c @@ -0,0 +1,384 @@ +/*------------------------------------------------------------------------- + * + * prop_tsrank.c + * Property-based tests for full-text ranking (ts_rank / ts_rank_cd). + * + * ts_rank and ts_rank_cd score a tsvector against a tsquery; these check the + * score bounds, determinism, and the match-is-positive relationship, + * using the 'simple' text search configuration so that lowercase ascii words + * map to themselves as lexemes -- no stemming, no stopwords -- which makes the + * match oracle (@@) and the resulting rank sign predictable. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_tsrank.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "tsrank" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Tiny lexeme alphabet. Four single-letter words keep the generated docs and + * queries legible in a counterexample and make matches common, so that both + * the matching and ranking properties get exercised. + */ +static const char *const lexemes[] = {"a", "b", "c", "d"}; +#define NLEX ((int) (sizeof(lexemes) / sizeof(lexemes[0]))) + +/* Draw one lexeme from the alphabet. */ +static const char * +draw_lexeme(pg_prop_tc *tc) +{ + return lexemes[pg_prop_draw_int(tc, 0, NLEX - 1)]; +} + +/* + * Draw a document: 0..8 space-separated lexemes, e.g. "a b a c". The raw + * words are emitted; the caller wraps them in to_tsvector('simple', ''). + */ +static void +draw_doc(pg_prop_tc *tc, StringInfo buf) +{ + int nwords = (int) pg_prop_draw_int(tc, 0, 8); + + for (int i = 0; i < nwords; i++) + { + if (i > 0) + appendStringInfoChar(buf, ' '); + appendStringInfoString(buf, draw_lexeme(tc)); + } +} + +/* + * Draw a boolean tsquery expression over the alphabet using & | ! and + * parentheses, e.g. "a & (b | !c)". Depth-limited and always parseable: + * parens are balanced here by construction and no operator is ever left + * dangling, so to_tsquery('simple', '') always succeeds. The result + * always contains at least one lexeme, so it never normalizes to the empty + * query (which would make @@ and the rank functions return NULL / zero + * trivially). + */ +static void +draw_query(pg_prop_tc *tc, StringInfo buf, int depth, bool allow_neg) +{ + int kind; + int maxkind; + + /* + * At the depth cap only a leaf is allowed, guaranteeing termination and a + * non-empty query. + */ + maxkind = allow_neg ? 3 : 2; + if (depth >= 3) + kind = 0; + else + kind = (int) pg_prop_draw_int(tc, 0, maxkind); + + /* + * When negation is disallowed, remap the negation case (1) to a leaf so + * the query stays purely positive. + */ + if (!allow_neg && kind == 1) + kind = 0; + + switch (kind) + { + case 0: /* leaf lexeme */ + appendStringInfoString(buf, draw_lexeme(tc)); + break; + case 1: /* negation: ! */ + appendStringInfoString(buf, "!"); + draw_query(tc, buf, depth + 1, allow_neg); + break; + case 2: /* conjunction: ( & ) */ + appendStringInfoChar(buf, '('); + draw_query(tc, buf, depth + 1, allow_neg); + appendStringInfoString(buf, " & "); + draw_query(tc, buf, depth + 1, allow_neg); + appendStringInfoChar(buf, ')'); + break; + default: /* disjunction: ( | ) */ + appendStringInfoChar(buf, '('); + draw_query(tc, buf, depth + 1, allow_neg); + appendStringInfoString(buf, " | "); + draw_query(tc, buf, depth + 1, allow_neg); + appendStringInfoChar(buf, ')'); + break; + } +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on valid input, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * ts_rank is non-negative for any document and query. A rank is a relevance + * score and can never be negative, regardless of whether the query matches. + */ +PG_PROP(rank_nonnegative) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d, + qexpr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + initStringInfo(&qexpr); + draw_doc(tc, &d); + draw_query(tc, &qexpr, 0, true); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ts_rank(to_tsvector('simple', '%s'), " + "to_tsquery('simple', '%s')) >= 0", + d.data, qexpr.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(qexpr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ts_rank is deterministic. Because ts_rank scores by lexeme co-occurrence + * (not by the boolean @@ result), the clean invariant is that identical + * inputs produce an identical rank. + */ +PG_PROP(rank_deterministic) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d, + qexpr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + initStringInfo(&qexpr); + draw_doc(tc, &d); + draw_query(tc, &qexpr, 0, true); + + /* + * ts_rank scores by lexeme co-occurrence, not by the boolean @@ result, + * so a non-matching query can still have a positive rank (any query + * lexeme present in the document contributes). The sound invariant is + * determinism: the same document and query always yield the same rank. + */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ts_rank(to_tsvector('simple', '%s'), " + "to_tsquery('simple', '%s')) " + "= ts_rank(to_tsvector('simple', '%s'), " + "to_tsquery('simple', '%s'))", + d.data, qexpr.data, d.data, qexpr.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(qexpr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Match => positive rank. When the document matches a positive (negation- + * free) query, every query lexeme is present in the document, so there is at + * least one lexeme to score and ts_rank is strictly > 0. The CASE returns + * true in the no-match case, where this property makes no claim. + */ +PG_PROP(rank_positive_when_match) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d, + qexpr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + initStringInfo(&qexpr); + draw_doc(tc, &d); + draw_query(tc, &qexpr, 0, false); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN (to_tsvector('simple', '%s') " + "@@ to_tsquery('simple', '%s')) " + "THEN ts_rank(to_tsvector('simple', '%s'), " + "to_tsquery('simple', '%s')) > 0 " + "ELSE true END", + d.data, qexpr.data, d.data, qexpr.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(qexpr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ts_rank_cd (cover-density ranking) is non-negative for any document and + * query, for the same reason ts_rank is: a relevance score cannot be + * negative. + */ +PG_PROP(rank_cd_nonnegative) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d, + qexpr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + initStringInfo(&qexpr); + draw_doc(tc, &d); + draw_query(tc, &qexpr, 0, true); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ts_rank_cd(to_tsvector('simple', '%s'), " + "to_tsquery('simple', '%s')) >= 0", + d.data, qexpr.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(qexpr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * No match => zero cover-density rank. When the document does not match the + * query there is no cover to measure, so ts_rank_cd is exactly 0. The CASE + * returns true in the match case, where this property makes no claim. + */ +PG_PROP(rank_cd_zero_when_no_match) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d, + qexpr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + initStringInfo(&qexpr); + draw_doc(tc, &d); + draw_query(tc, &qexpr, 0, false); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN NOT (to_tsvector('simple', '%s') " + "@@ to_tsquery('simple', '%s')) " + "THEN ts_rank_cd(to_tsvector('simple', '%s'), " + "to_tsquery('simple', '%s')) = 0 " + "ELSE true END", + d.data, qexpr.data, d.data, qexpr.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(qexpr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Default-weights equivalence: the 2-arg ts_rank(vector, query) is defined to + * use the documented default weight array {0.1, 0.2, 0.4, 1.0} (for the D, C, + * B, A labels). So the 4-arg ts_rank with that explicit array must equal the + * 2-arg form for every document and query. This exercises the weighted + * accumulation path without relying on any subtle monotonicity between + * different weight sets. + */ +PG_PROP(rank_weights_monotone) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d, + qexpr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + initStringInfo(&qexpr); + draw_doc(tc, &d); + draw_query(tc, &qexpr, 0, true); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ts_rank(to_tsvector('simple', '%s'), " + "to_tsquery('simple', '%s')) " + "= ts_rank('{0.1,0.2,0.4,1.0}'::float4[], " + "to_tsvector('simple', '%s'), " + "to_tsquery('simple', '%s'))", + d.data, qexpr.data, d.data, qexpr.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(qexpr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index e2736dea685af..49421b6a55e51 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From e1f613325ee0b285c2661c46fce75c2fdff8683b Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:13 -0400 Subject: [PATCH 130/338] prop: add date property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_date.c | 407 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 411 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_date.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 807e19066eb69..b077566188651 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -145,7 +145,8 @@ OBJS = \ prop_tsvector.o \ prop_tsvector2.o \ prop_tsquerylaw.o \ - prop_tsrank.o + prop_tsrank.o \ + prop_date.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 70b8dbf5de777..54aca5eebbd38 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -143,6 +143,7 @@ prop_area_sources = files( 'prop_tsvector2.c', 'prop_tsquerylaw.c', 'prop_tsrank.c', + 'prop_date.c', ) diff --git a/src/test/prop/prop_date.c b/src/test/prop/prop_date.c new file mode 100644 index 0000000000000..49fa4255b6877 --- /dev/null +++ b/src/test/prop/prop_date.c @@ -0,0 +1,407 @@ +/*------------------------------------------------------------------------- + * + * prop_date.c + * Property-based tests for the date type (src/backend/utils/adt/date.c). + * + * Date arithmetic, parsing, and the +/- integer and date-minus-date + * operations have had overflow and rounding fixes; these check round-trips + * and arithmetic identities over generated dates. + * + * The oracle here is the type's own algebra and I/O: a date formatted to text + * and parsed back must equal the original; adding then subtracting an integer + * number of days is the identity; date - date yields the integer day count + * that reconstructs the second date; make_date and extract are mutual + * inverses; the comparator is a total order; and casting through timestamp + * (midnight) round-trips. Each invariant is computed as a boolean in SQL and + * the scalar result is required to be "t". + * + * Dates for the identity properties are drawn in a safe range (year + * 2000..2050, day <= 28) so neither invalid-date nor overflow errors arise + * from the literal itself; only large day offsets can legally overflow the + * date range, and those cases are discarded via is_overflow rather than + * reported. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_date.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "date" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * True when a SQL error is a legitimate out-of-range/overflow condition rather + * than a wrong-results bug. Adding a large integer number of days to a date + * can legally run off the ends of the supported date range; those cases are + * discarded, not reported. + */ +static bool +is_overflow(const char *errmsg) +{ + if (errmsg == NULL) + return false; + return strstr(errmsg, "out of range") != NULL || + strstr(errmsg, "overflow") != NULL; +} + +/* + * Draw a valid DATE literal into buf, e.g. DATE '2024-03-15'. + * + * The year is kept in 2000..2050 (well inside the supported range so no + * arithmetic driven by the literal itself overflows), the month in 1..12, and + * the day capped at 28 so every month is valid and no spurious invalid-date + * error can arise. + */ +static void +draw_date(pg_prop_tc *tc, StringInfo buf) +{ + int year; + int mon; + int day; + + year = (int) pg_prop_draw_int(tc, 2000, 2050); + mon = (int) pg_prop_draw_int(tc, 1, 12); + day = (int) pg_prop_draw_int(tc, 1, 28); + + appendStringInfo(buf, "DATE '%04d-%02d-%02d'", year, mon, day); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operation errored or produced NULL on valid inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * 1. text_roundtrip + * + * Formatting a date to text and parsing it back must yield the same date. + * This exercises the date output and input paths against each other. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_date(tc, &d); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)::text::date = %s", d.data, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. add_sub_days_inverse + * + * Adding an integer number of days and then subtracting the same count must + * return the original date: (D + k) - k = D. A large k can legally run off + * the date range, so an out-of-range error is discarded rather than reported. + */ +PG_PROP(add_sub_days_inverse) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d; + StringInfoData q; + int64 k; + char *result; + bool failed; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_date(tc, &d); + k = pg_prop_draw_int(tc, -100000, 100000); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s + %lld) - %lld = %s", + d.data, (long long) k, (long long) k, d.data); + + result = pg_prop_sql_scalar(conn, q.data, &failed, &err); + + if (failed) + { + if (is_overflow(err)) + { + /* arithmetic ran off the supported range: legal, skip the case */ + pg_prop_note(tc, "overflow (skipped): %s", err); + if (err) + pg_free(err); + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); + pg_prop_assume(tc, false); + return; /* unreachable: assume(false) longjmps */ + } + pg_prop_fail(tc, + "unexpected error on (D + k) - k = D\n" + " D: %s\n k: %lld\n error: %s", + d.data, (long long) k, err ? err : "(unknown)"); + } + + if (result == NULL || strcmp(result, "t") != 0) + pg_prop_fail(tc, + "(D + k) - k != D\n" + " D: %s\n k: %lld\n result: %s", + d.data, (long long) k, result ? result : "NULL"); + + pg_free(result); + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. diff_then_add + * + * date - date yields an int4 number of days; adding it back to the first date + * must reconstruct the second: D1 + (D2 - D1) = D2. Both dates are in the + * safe range so no overflow is expected. + */ +PG_PROP(diff_then_add) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d1; + StringInfoData d2; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d1); + initStringInfo(&d2); + draw_date(tc, &d1); + draw_date(tc, &d2); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s + (%s - %s) = %s", + d1.data, d2.data, d1.data, d2.data); + + require_true(tc, conn, q.data); + + pfree(d1.data); + pfree(d2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. monotonic_add + * + * Adding a strictly positive number of days must yield a strictly greater + * date: (D + k) > D for k > 0. A large k can legally overflow the date + * range, so an out-of-range error is discarded rather than reported. + */ +PG_PROP(monotonic_add) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d; + StringInfoData q; + int64 k; + char *result; + bool failed; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_date(tc, &d); + k = pg_prop_draw_int(tc, 1, 100000); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s + %lld) > %s", + d.data, (long long) k, d.data); + + result = pg_prop_sql_scalar(conn, q.data, &failed, &err); + + if (failed) + { + if (is_overflow(err)) + { + /* arithmetic ran off the supported range: legal, skip the case */ + pg_prop_note(tc, "overflow (skipped): %s", err); + if (err) + pg_free(err); + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); + pg_prop_assume(tc, false); + return; /* unreachable: assume(false) longjmps */ + } + pg_prop_fail(tc, + "unexpected error on (D + k) > D\n" + " D: %s\n k: %lld\n error: %s", + d.data, (long long) k, err ? err : "(unknown)"); + } + + if (result == NULL || strcmp(result, "t") != 0) + pg_prop_fail(tc, + "(D + k) is not strictly greater than D for k > 0\n" + " D: %s\n k: %lld\n result: %s", + d.data, (long long) k, result ? result : "NULL"); + + pg_free(result); + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. extract_reconstruct + * + * Reconstructing a date from its extracted year/month/day fields via + * make_date (the independent constructor) must yield the original date. This + * exercises extract and make_date as mutual inverses. + */ +PG_PROP(extract_reconstruct) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_date(tc, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT make_date(" + "extract(year from %s)::int, " + "extract(month from %s)::int, " + "extract(day from %s)::int) = %s", + d.data, d.data, d.data, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. comparison_total_order + * + * For any two dates D1, D2, exactly one of (D1 < D2), (D1 = D2), (D1 > D2) + * must hold. This checks that the date comparator is a proper total order. + */ +PG_PROP(comparison_total_order) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d1; + StringInfoData d2; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d1); + initStringInfo(&d2); + draw_date(tc, &d1); + draw_date(tc, &d2); + + /* exactly one of <, =, > must be true */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ( ((%s) < (%s))::int " + "+ ((%s) = (%s))::int " + "+ ((%s) > (%s))::int ) = 1", + d1.data, d2.data, + d1.data, d2.data, + d1.data, d2.data); + + require_true(tc, conn, q.data); + + pfree(d1.data); + pfree(d2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. timestamp_cast_roundtrip + * + * Casting a date to timestamp (midnight) and back must yield the same date: + * (D::timestamp)::date = D. This exercises the date<->timestamp conversion + * paths against each other. + */ +PG_PROP(timestamp_cast_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData d; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&d); + draw_date(tc, &d); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s::timestamp)::date = %s", d.data, d.data); + + require_true(tc, conn, q.data); + + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 49421b6a55e51..8edd0dd046f7c 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 3dea9f5ca7ae099bc2c98587a4e598e94c9a287e Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:13 -0400 Subject: [PATCH 131/338] prop: add date arithmetic and field extraction property tests (dateparts) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_dateparts.c | 286 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 290 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_dateparts.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index b077566188651..40b26559078f7 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -146,7 +146,8 @@ OBJS = \ prop_tsvector2.o \ prop_tsquerylaw.o \ prop_tsrank.o \ - prop_date.o + prop_date.o \ + prop_dateparts.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 54aca5eebbd38..70da2998afca9 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -144,6 +144,7 @@ prop_area_sources = files( 'prop_tsquerylaw.c', 'prop_tsrank.c', 'prop_date.c', + 'prop_dateparts.c', ) diff --git a/src/test/prop/prop_dateparts.c b/src/test/prop/prop_dateparts.c new file mode 100644 index 0000000000000..f51208167e58f --- /dev/null +++ b/src/test/prop/prop_dateparts.c @@ -0,0 +1,286 @@ +/*------------------------------------------------------------------------- + * + * prop_dateparts.c + * Property-based tests for date arithmetic and field extraction. + * + * Exercises date operations distinct from the "date" area: date +/- integer + * days, date - date yielding integer days, make_date(), extract() of + * year/month/day/dow/doy, the text round-trip, and the total order of date + * comparison. The oracle is each operation's defining law -- adding then + * subtracting the same offset is the identity, D1 + (D2 - D1) = D2, extracting + * a field returns the component that built the date, day-of-week and + * day-of-year lie in their fixed ranges, text output re-parses, and exactly + * one of <,=,> holds for any pair -- so any "f" result is a bug in date.c, + * not a tolerance. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_dateparts.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "dateparts" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a (year, month, day) triple into *y/*mo/*d, with y 2000..2030, + * mo 1..12 and d 1..28 (so every month is valid without leap/length cases). + */ +static void +draw_ymd(pg_prop_tc *tc, int *y, int *mo, int *d) +{ + *y = (int) pg_prop_draw_int(tc, 2000, 2030); + *mo = (int) pg_prop_draw_int(tc, 1, 12); + *d = (int) pg_prop_draw_int(tc, 1, 28); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operation errored or produced NULL on valid inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Add/subtract of an integer-day offset is the identity: + * (make_date(y,mo,d) + k) - k = make_date(y,mo,d), for k in -500..500. + */ +PG_PROP(date_plus_minus_int) +{ + pg_prop_conn *conn; + char *err = NULL; + int y, + mo, + d, + k; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_ymd(tc, &y, &mo, &d); + k = (int) pg_prop_draw_int(tc, -500, 500); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (make_date(%d, %d, %d) + (%d)) - (%d) " + "= make_date(%d, %d, %d)", + y, mo, d, k, k, y, mo, d); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Date difference round-trip: D2 - D1 is an integer number of days and + * D1 + (D2 - D1) = D2, for two independently drawn dates. + */ +PG_PROP(date_diff_days) +{ + pg_prop_conn *conn; + char *err = NULL; + int y1, + mo1, + d1, + y2, + mo2, + d2; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_ymd(tc, &y1, &mo1, &d1); + draw_ymd(tc, &y2, &mo2, &d2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT make_date(%d, %d, %d) " + "+ (make_date(%d, %d, %d) - make_date(%d, %d, %d)) " + "= make_date(%d, %d, %d)", + y1, mo1, d1, + y2, mo2, d2, y1, mo1, d1, + y2, mo2, d2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Field extraction returns the components that built the date: + * extract(year from D) = y, extract(month from D) = mo, extract(day from D) + * = d. + */ +PG_PROP(extract_year_month_day) +{ + pg_prop_conn *conn; + char *err = NULL; + int y, + mo, + d; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_ymd(tc, &y, &mo, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT extract(year from make_date(%d, %d, %d)) = %d " + "AND extract(month from make_date(%d, %d, %d)) = %d " + "AND extract(day from make_date(%d, %d, %d)) = %d", + y, mo, d, y, + y, mo, d, mo, + y, mo, d, d); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Range bounds: extract(dow from D) is in 0..6 and extract(doy from D) is in + * 1..366 for any valid date. + */ +PG_PROP(dow_range) +{ + pg_prop_conn *conn; + char *err = NULL; + int y, + mo, + d; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_ymd(tc, &y, &mo, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT extract(dow from make_date(%d, %d, %d)) " + "BETWEEN 0 AND 6 " + "AND extract(doy from make_date(%d, %d, %d)) " + "BETWEEN 1 AND 366", + y, mo, d, y, mo, d); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Text round-trip: (make_date(y,mo,d))::text::date = make_date(y,mo,d). The + * output text must re-parse to the same date. + */ +PG_PROP(make_date_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int y, + mo, + d; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_ymd(tc, &y, &mo, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (make_date(%d, %d, %d))::text::date " + "= make_date(%d, %d, %d)", + y, mo, d, y, mo, d); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Date comparison is a total order: for two drawn dates A and B exactly one of + * A < B, A = B, A > B holds. The three mutually exclusive outcomes must sum + * to exactly one true. + */ +PG_PROP(date_le_total_order) +{ + pg_prop_conn *conn; + char *err = NULL; + int y1, + mo1, + d1, + y2, + mo2, + d2; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_ymd(tc, &y1, &mo1, &d1); + draw_ymd(tc, &y2, &mo2, &d2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (CASE WHEN make_date(%d, %d, %d) " + "< make_date(%d, %d, %d) THEN 1 ELSE 0 END " + "+ CASE WHEN make_date(%d, %d, %d) " + "= make_date(%d, %d, %d) THEN 1 ELSE 0 END " + "+ CASE WHEN make_date(%d, %d, %d) " + "> make_date(%d, %d, %d) THEN 1 ELSE 0 END) = 1", + y1, mo1, d1, y2, mo2, d2, + y1, mo1, d1, y2, mo2, d2, + y1, mo1, d1, y2, mo2, d2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 8edd0dd046f7c..cf286e4bb870f 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 8ccce037cbab081655e42a93ad8d400ad5dc5786 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:13 -0400 Subject: [PATCH 132/338] prop: add to_char/to_date for dates property tests (datefmt) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_datefmt.c | 344 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 348 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_datefmt.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 40b26559078f7..c49d68489d830 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -147,7 +147,8 @@ OBJS = \ prop_tsquerylaw.o \ prop_tsrank.o \ prop_date.o \ - prop_dateparts.o + prop_dateparts.o \ + prop_datefmt.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 70da2998afca9..0ced329a8d5c5 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -145,6 +145,7 @@ prop_area_sources = files( 'prop_tsrank.c', 'prop_date.c', 'prop_dateparts.c', + 'prop_datefmt.c', ) diff --git a/src/test/prop/prop_datefmt.c b/src/test/prop/prop_datefmt.c new file mode 100644 index 0000000000000..b7e3a486735aa --- /dev/null +++ b/src/test/prop/prop_datefmt.c @@ -0,0 +1,344 @@ +/*------------------------------------------------------------------------- + * + * prop_datefmt.c + * Property-based tests for date formatting via to_char / to_date + * (src/backend/utils/adt/formatting.c). + * + * These exercise the date-specific to_char/to_date paths distinct from the + * generic number/roman masks (prop_tochar), the date-cast chain + * (prop_datecast) and the EXTRACT part machinery (prop_dateparts): format a + * date to text and parse it back (round-trips), check the individual + * YYYY/MM/DD/DDD/ID/Q fields against zero-padded oracles computed in SQL, and + * cross-check to_char against EXTRACT. A mismatch is a wrong-results bug in + * the field-formatting or field-parsing code, not a rounding artefact. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_datefmt.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "datefmt" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the formatting/parsing path errored or produced NULL on + * valid inputs, which is itself a bug -- so either is a failure, reported with + * the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Draw a valid (y, mo, d) with day capped at 28 so every month/year + * combination is valid and no invalid-date error can occur. + */ +static void +draw_ymd(pg_prop_tc *tc, int *y, int *mo, int *d) +{ + *y = (int) pg_prop_draw_int(tc, 2000, 2030); + *mo = (int) pg_prop_draw_int(tc, 1, 12); + *d = (int) pg_prop_draw_int(tc, 1, 28); +} + +/* + * 1. to_char_iso_roundtrip + * + * Formatting a date with the ISO YYYY-MM-DD mask and parsing it back must + * return the original date. This is a pure identity, sound for every drawn + * (y, mo, d). + */ +PG_PROP(to_char_iso_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int y; + int mo; + int d; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_ymd(tc, &y, &mo, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_date(to_char(make_date(%d,%d,%d), 'YYYY-MM-DD'), " + "'YYYY-MM-DD') = make_date(%d,%d,%d)", + y, mo, d, y, mo, d); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. to_char_components + * + * The YYYY/MM/DD fields produced by to_char must match the zero-padded + * decimal rendering of the input components (lpad as the oracle). Checks the + * per-field width/zero-pad behaviour of the date formatter. + */ +PG_PROP(to_char_components) +{ + pg_prop_conn *conn; + char *err = NULL; + int y; + int mo; + int d; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_ymd(tc, &y, &mo, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_char(make_date(%d,%d,%d), 'YYYY') = lpad(%d::text,4,'0') " + "AND to_char(make_date(%d,%d,%d), 'MM') = lpad(%d::text,2,'0') " + "AND to_char(make_date(%d,%d,%d), 'DD') = lpad(%d::text,2,'0')", + y, mo, d, y, + y, mo, d, mo, + y, mo, d, d); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. to_char_doy + * + * The day-of-year field (DDD) must fall in 1..366 for any valid date. A + * sound range check on the DDD computation. + */ +PG_PROP(to_char_doy) +{ + pg_prop_conn *conn; + char *err = NULL; + int y; + int mo; + int d; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_ymd(tc, &y, &mo, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_char(make_date(%d,%d,%d), 'DDD')::int BETWEEN 1 AND 366", + y, mo, d); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. to_date_ignores_format_extras + * + * A compact (no-separator) YYYYMMDD mask must also round-trip: to_char then + * to_date with the same unseparated mask returns the original date. + * Exercises the fixed-width field parsing of to_date without delimiters. + */ +PG_PROP(to_date_ignores_format_extras) +{ + pg_prop_conn *conn; + char *err = NULL; + int y; + int mo; + int d; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_ymd(tc, &y, &mo, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_date(to_char(make_date(%d,%d,%d), 'YYYYMMDD'), " + "'YYYYMMDD') = make_date(%d,%d,%d)", + y, mo, d, y, mo, d); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. to_char_day_of_week + * + * The ISO day-of-week field (ID) must fall in 1..7 for any valid date. A + * sound range check on the ISO-weekday computation. + */ +PG_PROP(to_char_day_of_week) +{ + pg_prop_conn *conn; + char *err = NULL; + int y; + int mo; + int d; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_ymd(tc, &y, &mo, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_char(make_date(%d,%d,%d), 'ID')::int BETWEEN 1 AND 7", + y, mo, d); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. extract_matches_to_char + * + * EXTRACT(year ...) and the YYYY field of to_char must agree. A differential + * oracle: two independent paths to the same year value must produce the same + * answer. + */ +PG_PROP(extract_matches_to_char) +{ + pg_prop_conn *conn; + char *err = NULL; + int y; + int mo; + int d; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_ymd(tc, &y, &mo, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT extract(year from make_date(%d,%d,%d)) " + "= to_char(make_date(%d,%d,%d),'YYYY')::int", + y, mo, d, y, mo, d); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. to_char_month_name_parse + * + * A mask that renders the month as an abbreviated name (Mon) must also + * round-trip: formatting then parsing with the same YYYY-Mon-DD mask returns + * the original date. Exercises the month-name formatting and name-parsing + * paths of to_date. + */ +PG_PROP(to_char_month_name_parse) +{ + pg_prop_conn *conn; + char *err = NULL; + int y; + int mo; + int d; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_ymd(tc, &y, &mo, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_date(to_char(make_date(%d,%d,%d), 'YYYY-Mon-DD'), " + "'YYYY-Mon-DD') = make_date(%d,%d,%d)", + y, mo, d, y, mo, d); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 8. to_char_quarter + * + * The quarter field (Q) must equal (mo-1)/3 + 1 for the drawn month. The + * oracle is computed from the month alone and is sound for every valid date. + */ +PG_PROP(to_char_quarter) +{ + pg_prop_conn *conn; + char *err = NULL; + int y; + int mo; + int d; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_ymd(tc, &y, &mo, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_char(make_date(%d,%d,%d), 'Q')::int = ((%d-1)/3 + 1)", + y, mo, d, mo); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index cf286e4bb870f..3b9cb2ec1a257 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 7c8812a8eed1e829d67b64098930e98b9289f0b9 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:13 -0400 Subject: [PATCH 133/338] prop: add date/timestamp/timestamptz cast relationships property tests (datecast) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_datecast.c | 348 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 352 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_datecast.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index c49d68489d830..22ba930f23a58 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -148,7 +148,8 @@ OBJS = \ prop_tsrank.o \ prop_date.o \ prop_dateparts.o \ - prop_datefmt.o + prop_datefmt.o \ + prop_datecast.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 0ced329a8d5c5..60059746dec77 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -146,6 +146,7 @@ prop_area_sources = files( 'prop_date.c', 'prop_dateparts.c', 'prop_datefmt.c', + 'prop_datecast.c', ) diff --git a/src/test/prop/prop_datecast.c b/src/test/prop/prop_datecast.c new file mode 100644 index 0000000000000..bd4fe5c435980 --- /dev/null +++ b/src/test/prop/prop_datecast.c @@ -0,0 +1,348 @@ +/*------------------------------------------------------------------------- + * + * prop_datecast.c + * Property-based tests for date/timestamp/timestamptz cast relationships. + * + * This file targets the cross-type cast algebra -- the conversions between + * date, timestamp (without time zone) and timestamptz -- as distinct from + * date arithmetic (prop_date), field extraction (prop_dateparts), time-of-day + * operations (prop_timeops) and truncation (prop_datetrunc). A date casts to + * timestamp as midnight of that day; a timestamp casts to date by dropping the + * time; and under a UTC session the timestamp<->timestamptz conversion via AT + * TIME ZONE 'UTC' is the identity. make_date / make_time / make_timestamp are + * the independent constructors used to build the operands. + * + * The oracle is the type's own cast algebra, evaluated in SQL as a boolean + * that must be "t": each identity holds for every drawn date/time, so an "f" + * (or a NULL / SQL error on valid inputs) is a cast bug, not a timezone + * artefact. Every property pins the session TimeZone to 'UTC' (via + * connect_tz) so the timestamp<->timestamptz conversions are deterministic + * and free of DST. + * + * Component ranges: year 2001..2030, month 1..12, day 1..28 (every month + * valid, no invalid-date error), hour 0..23, minute 0..59, second 0..59. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_datecast.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "datecast" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Drawn date/time components. Ranges per the spec: a day cap of 28 keeps + * every month valid so no invalid-date error can arise from the literal, and + * all fields stay well inside the supported range so no cast overflows. + */ +typedef struct dt_parts +{ + int y; + int mo; + int d; + int h; + int mi; + int s; +} dt_parts; + +static void +draw_parts(pg_prop_tc *tc, dt_parts *p) +{ + p->y = (int) pg_prop_draw_int(tc, 2001, 2030); + p->mo = (int) pg_prop_draw_int(tc, 1, 12); + p->d = (int) pg_prop_draw_int(tc, 1, 28); + p->h = (int) pg_prop_draw_int(tc, 0, 23); + p->mi = (int) pg_prop_draw_int(tc, 0, 59); + p->s = (int) pg_prop_draw_int(tc, 0, 59); +} + +/* + * Connect to the test cluster and pin the session TimeZone to 'UTC' so the + * timestamp<->timestamptz conversions (AT TIME ZONE 'UTC', ::timestamptz) are + * deterministic and free of DST regardless of the cluster default. Fails the + * case if either step fails. + */ +static pg_prop_conn * +connect_tz(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, "SET TIME ZONE 'UTC'", &err)) + pg_prop_fail(tc, "could not SET TIME ZONE 'UTC': %s", err ? err : ""); + + return conn; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a cast errored or produced NULL on valid inputs, which is + * itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * 1. date_to_timestamp + * + * Casting a date to timestamp yields midnight of that day: D::timestamp equals + * the timestamp built from the same y-mo-d string with a 00:00:00 time. Both + * sides are formed independently (make_date vs. a text literal) so a wrong + * time-of-day in the cast is caught. + */ +PG_PROP(date_to_timestamp) +{ + pg_prop_conn *conn = connect_tz(tc); + dt_parts p; + StringInfoData q; + + draw_parts(tc, &p); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (make_date(%d,%d,%d))::timestamp " + "= ('%04d-%02d-%02d 00:00:00')::timestamp", + p.y, p.mo, p.d, + p.y, p.mo, p.d); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. timestamp_to_date + * + * Casting a timestamp to date drops the time and returns the calendar day: + * (make_timestamp(y,mo,d,h,mi,s))::date = make_date(y,mo,d) for the drawn + * components. + */ +PG_PROP(timestamp_to_date) +{ + pg_prop_conn *conn = connect_tz(tc); + dt_parts p; + StringInfoData q; + + draw_parts(tc, &p); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (make_timestamp(%d,%d,%d,%d,%d,%d))::date " + "= make_date(%d,%d,%d)", + p.y, p.mo, p.d, p.h, p.mi, p.s, + p.y, p.mo, p.d); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. date_timestamp_roundtrip + * + * A date cast to timestamp (midnight) and back to date is the identity: + * ((D)::timestamp)::date = D. This exercises both cast directions against + * each other. + */ +PG_PROP(date_timestamp_roundtrip) +{ + pg_prop_conn *conn = connect_tz(tc); + dt_parts p; + StringInfoData q; + + draw_parts(tc, &p); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((make_date(%d,%d,%d))::timestamp)::date " + "= make_date(%d,%d,%d)", + p.y, p.mo, p.d, + p.y, p.mo, p.d); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. timestamp_to_timestamptz + * + * Under a UTC session, promoting a timestamp to timestamptz and then reading + * it back AT TIME ZONE 'UTC' is the identity: + * ((TS)::timestamptz) AT TIME ZONE 'UTC' = TS. + * The result of AT TIME ZONE 'UTC' on a timestamptz is a plain timestamp, so + * the comparison is timestamp-vs-timestamp. + */ +PG_PROP(timestamp_to_timestamptz) +{ + pg_prop_conn *conn = connect_tz(tc); + dt_parts p; + StringInfoData q; + + draw_parts(tc, &p); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((make_timestamp(%d,%d,%d,%d,%d,%d))::timestamptz) " + "AT TIME ZONE 'UTC' = make_timestamp(%d,%d,%d,%d,%d,%d)", + p.y, p.mo, p.d, p.h, p.mi, p.s, + p.y, p.mo, p.d, p.h, p.mi, p.s); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. timestamptz_to_timestamp + * + * The reverse direction, stated as a round-trip through timestamptz under UTC: + * taking a timestamp to timestamptz and back AT TIME ZONE 'UTC' recovers the + * original timestamp. (Equivalent to property 4 by UTC's symmetry; kept + * separate to assert the reverse conversion path explicitly as the spec + * requires.) + */ +PG_PROP(timestamptz_to_timestamp) +{ + pg_prop_conn *conn = connect_tz(tc); + dt_parts p; + StringInfoData q; + + draw_parts(tc, &p); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (((make_timestamp(%d,%d,%d,%d,%d,%d))::timestamptz) " + "AT TIME ZONE 'UTC') = make_timestamp(%d,%d,%d,%d,%d,%d)", + p.y, p.mo, p.d, p.h, p.mi, p.s, + p.y, p.mo, p.d, p.h, p.mi, p.s); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. date_plus_time_eq_timestamp + * + * A date plus a time-of-day equals the timestamp with the same components: + * make_date(y,mo,d) + make_time(h,mi,s) = make_timestamp(y,mo,d,h,mi,s). This + * ties the date+time composition operator to the direct timestamp constructor. + */ +PG_PROP(date_plus_time_eq_timestamp) +{ + pg_prop_conn *conn = connect_tz(tc); + dt_parts p; + StringInfoData q; + + draw_parts(tc, &p); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT make_date(%d,%d,%d) + make_time(%d,%d,%d) " + "= make_timestamp(%d,%d,%d,%d,%d,%d)", + p.y, p.mo, p.d, p.h, p.mi, p.s, + p.y, p.mo, p.d, p.h, p.mi, p.s); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. extract_consistency + * + * The year read from a date and from that date cast to timestamp must agree, + * and both must equal the drawn year: + * extract(year from D::timestamp) = extract(year from D) = y. + * The cast to timestamp must not perturb the calendar year. + */ +PG_PROP(extract_consistency) +{ + pg_prop_conn *conn = connect_tz(tc); + dt_parts p; + StringInfoData q; + + draw_parts(tc, &p); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT extract(year from (make_date(%d,%d,%d))::timestamp) " + "= extract(year from make_date(%d,%d,%d)) " + "AND extract(year from make_date(%d,%d,%d)) = %d", + p.y, p.mo, p.d, + p.y, p.mo, p.d, + p.y, p.mo, p.d, p.y); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 8. date_order_preserved + * + * The date->timestamp cast is strictly monotonic: for D1 < D2 built as + * D2 = D1 + k days with k > 0, (D1)::timestamp < (D2)::timestamp. k is kept + * small (1..60) so D2 stays within the drawn-day neighbourhood and no cast + * overflow arises; the day offset makes D1 < D2 by construction. + */ +PG_PROP(date_order_preserved) +{ + pg_prop_conn *conn = connect_tz(tc); + dt_parts p; + int k; + StringInfoData q; + + draw_parts(tc, &p); + k = (int) pg_prop_draw_int(tc, 1, 60); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (make_date(%d,%d,%d))::timestamp " + "< ((make_date(%d,%d,%d) + %d))::timestamp", + p.y, p.mo, p.d, + p.y, p.mo, p.d, k); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 3b9cb2ec1a257..aab0b3cdcea70 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From f91254f020a48b8df958c0a51e98b957eb8f399f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:13 -0400 Subject: [PATCH 134/338] prop: add date_trunc and date_part across units property tests (datetrunc) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_datetrunc.c | 335 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 339 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_datetrunc.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 22ba930f23a58..9d4a5d5fb2db6 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -149,7 +149,8 @@ OBJS = \ prop_date.o \ prop_dateparts.o \ prop_datefmt.o \ - prop_datecast.o + prop_datecast.o \ + prop_datetrunc.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 60059746dec77..b598569127e9f 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -147,6 +147,7 @@ prop_area_sources = files( 'prop_dateparts.c', 'prop_datefmt.c', 'prop_datecast.c', + 'prop_datetrunc.c', ) diff --git a/src/test/prop/prop_datetrunc.c b/src/test/prop/prop_datetrunc.c new file mode 100644 index 0000000000000..5892292915829 --- /dev/null +++ b/src/test/prop/prop_datetrunc.c @@ -0,0 +1,335 @@ +/*------------------------------------------------------------------------- + * + * prop_datetrunc.c + * Property-based tests for date_trunc()/date_part() on plain TIMESTAMP + * (src/backend/utils/adt/timestamp.c). + * + * Distinct from prop_date/prop_dateparts/prop_timeops/prop_tstzarith: this + * file targets the truncation and field-extraction algebra of date_trunc() + * and date_part() on TIMESTAMP WITHOUT TIME ZONE across the hour/day/month/ + * year units. The oracle is the operations' own definition: truncation can + * only move a timestamp earlier-or-equal, is idempotent, zeroes every field + * below the unit, and coarser units land earlier-or-equal than finer ones; + * date_part agrees with extract on the drawn field. Each is TRUE for every + * input, so an "f" result is a bug in the truncation/extraction code. + * + * Every property pins the session TimeZone to 'UTC' (via connect_tz) and uses + * plain TIMESTAMP literals (no time zone, hence no DST), so results are + * deterministic and free of timezone artefacts. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_datetrunc.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "datetrunc" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Connect to the test cluster and pin the session TimeZone to 'UTC'. Plain + * TIMESTAMP has no time zone, but date_trunc on larger units and extract are + * evaluated in session context; pinning UTC keeps every case deterministic + * and free of DST regardless of the cluster default. Fails the case on error. + */ +static pg_prop_conn * +connect_tz(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, "SET TIME ZONE 'UTC'", &err)) + pg_prop_fail(tc, "could not SET TIME ZONE 'UTC': %s", err ? err : ""); + + return conn; +} + +/* + * Build a random plain TIMESTAMP (no time zone) literal into buf, e.g. + * TIMESTAMP '2024-03-15 08:30:00' + * + * Year 2001..2030 keeps the literal well inside the supported range; day is + * capped at 28 so every month is valid. + */ +static void +draw_ts(pg_prop_tc *tc, StringInfo buf) +{ + int year; + int mon; + int day; + int hour; + int min; + int sec; + + year = (int) pg_prop_draw_int(tc, 2001, 2030); + mon = (int) pg_prop_draw_int(tc, 1, 12); + day = (int) pg_prop_draw_int(tc, 1, 28); + hour = (int) pg_prop_draw_int(tc, 0, 23); + min = (int) pg_prop_draw_int(tc, 0, 59); + sec = (int) pg_prop_draw_int(tc, 0, 59); + + appendStringInfo(buf, "TIMESTAMP '%04d-%02d-%02d %02d:%02d:%02d'", + year, mon, day, hour, min, sec); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operation errored or produced NULL on finite inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * 1. trunc_le_original: truncating to any unit can only move a timestamp + * earlier-or-equal to itself, for every unit {hour, day, month, year}. + */ +PG_PROP(trunc_le_original) +{ + pg_prop_conn *conn = connect_tz(tc); + StringInfoData ts, + q; + + initStringInfo(&ts); + draw_ts(tc, &ts); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT date_trunc('hour', %s) <= %s " + "AND date_trunc('day', %s) <= %s " + "AND date_trunc('month', %s) <= %s " + "AND date_trunc('year', %s) <= %s", + ts.data, ts.data, + ts.data, ts.data, + ts.data, ts.data, + ts.data, ts.data); + + require_true(tc, conn, q.data); + + pfree(ts.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. trunc_idempotent: truncating an already-truncated value changes nothing, + * for the hour/day/month units. + */ +PG_PROP(trunc_idempotent) +{ + pg_prop_conn *conn = connect_tz(tc); + StringInfoData ts, + q; + + initStringInfo(&ts); + draw_ts(tc, &ts); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT date_trunc('hour', date_trunc('hour', %s)) " + "= date_trunc('hour', %s) " + "AND date_trunc('day', date_trunc('day', %s)) " + "= date_trunc('day', %s) " + "AND date_trunc('month', date_trunc('month', %s)) " + "= date_trunc('month', %s)", + ts.data, ts.data, + ts.data, ts.data, + ts.data, ts.data); + + require_true(tc, conn, q.data); + + pfree(ts.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. trunc_hour_zeroes_subhour: truncating to the hour zeroes the minute and + * second fields. + */ +PG_PROP(trunc_hour_zeroes_subhour) +{ + pg_prop_conn *conn = connect_tz(tc); + StringInfoData ts, + q; + + initStringInfo(&ts); + draw_ts(tc, &ts); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT extract(minute from date_trunc('hour', %s)) = 0 " + "AND extract(second from date_trunc('hour', %s)) = 0", + ts.data, ts.data); + + require_true(tc, conn, q.data); + + pfree(ts.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. trunc_day_zeroes_time: truncating to the day leaves midnight, i.e. the + * time-of-day component is 00:00:00. + */ +PG_PROP(trunc_day_zeroes_time) +{ + pg_prop_conn *conn = connect_tz(tc); + StringInfoData ts, + q; + + initStringInfo(&ts); + draw_ts(tc, &ts); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT date_trunc('day', %s)::time = '00:00:00'::time", + ts.data); + + require_true(tc, conn, q.data); + + pfree(ts.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. trunc_month_first_day: truncating to the month lands on day 1. + */ +PG_PROP(trunc_month_first_day) +{ + pg_prop_conn *conn = connect_tz(tc); + StringInfoData ts, + q; + + initStringInfo(&ts); + draw_ts(tc, &ts); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT extract(day from date_trunc('month', %s)) = 1", + ts.data); + + require_true(tc, conn, q.data); + + pfree(ts.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. trunc_year_jan: truncating to the year lands on January 1st. + */ +PG_PROP(trunc_year_jan) +{ + pg_prop_conn *conn = connect_tz(tc); + StringInfoData ts, + q; + + initStringInfo(&ts); + draw_ts(tc, &ts); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT extract(month from date_trunc('year', %s)) = 1 " + "AND extract(day from date_trunc('year', %s)) = 1", + ts.data, ts.data); + + require_true(tc, conn, q.data); + + pfree(ts.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. trunc_coarser_le_finer: coarser truncation is earlier-or-equal than + * finer, in a single chain year <= month <= day <= hour <= original. + */ +PG_PROP(trunc_coarser_le_finer) +{ + pg_prop_conn *conn = connect_tz(tc); + StringInfoData ts, + q; + + initStringInfo(&ts); + draw_ts(tc, &ts); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT date_trunc('year', %s) <= date_trunc('month', %s) " + "AND date_trunc('month', %s) <= date_trunc('day', %s) " + "AND date_trunc('day', %s) <= date_trunc('hour', %s) " + "AND date_trunc('hour', %s) <= %s", + ts.data, ts.data, + ts.data, ts.data, + ts.data, ts.data, + ts.data, ts.data); + + require_true(tc, conn, q.data); + + pfree(ts.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 8. date_part_eq_extract: date_part() and extract() are the same operation + * spelled two ways; they must agree on the drawn hour. + */ +PG_PROP(date_part_eq_extract) +{ + pg_prop_conn *conn = connect_tz(tc); + StringInfoData ts, + q; + + initStringInfo(&ts); + draw_ts(tc, &ts); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT date_part('hour', %s) = extract(hour from %s)", + ts.data, ts.data); + + require_true(tc, conn, q.data); + + pfree(ts.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index aab0b3cdcea70..0c0298956e457 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 8d9c6372510a841adc9b157adcf71d4030ebb780 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:14 -0400 Subject: [PATCH 135/338] prop: add time and timestamp round-trips and arithmetic property tests (timeops) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_timeops.c | 394 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 398 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_timeops.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 9d4a5d5fb2db6..47c0f70143d75 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -150,7 +150,8 @@ OBJS = \ prop_dateparts.o \ prop_datefmt.o \ prop_datecast.o \ - prop_datetrunc.o + prop_datetrunc.o \ + prop_timeops.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index b598569127e9f..2e5e29db97490 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -148,6 +148,7 @@ prop_area_sources = files( 'prop_datefmt.c', 'prop_datecast.c', 'prop_datetrunc.c', + 'prop_timeops.c', ) diff --git a/src/test/prop/prop_timeops.c b/src/test/prop/prop_timeops.c new file mode 100644 index 0000000000000..d5c4391472846 --- /dev/null +++ b/src/test/prop/prop_timeops.c @@ -0,0 +1,394 @@ +/*------------------------------------------------------------------------- + * + * prop_timeops.c + * Property-based tests for time / timetz / timestamp (without time zone). + * + * The oracle here is the types' own algebra: text round-trips, field + * extraction, timestamp subtraction yielding an interval, date_trunc + * idempotence, and the cast to date. Each property is a SELECT of a boolean + * that must be "t" for every drawn input; a NULL or SQL error is a bug. We + * use plain TIMESTAMP (no tz) to avoid DST ambiguity, and restrict day to + * 1..28 so no month-length edge case can make a literal invalid. + * + * Distinct from the "date", "timestamptz", "interval" and "intervalparts" + * areas: this targets the time and timestamp-without-tz types specifically. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_timeops.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "timeops" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operation errored or produced NULL on a valid input, + * which is itself a bug -- so either is a failure, reported with the SQL. + * Copied from prop_numeric.c's require_true (Shape A, scalar boolean oracle). + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Draw a time-of-day and append a TIME literal "TIME '%02d:%02d:%02d'" to + * buf. The drawn h/m/s are returned via the out-params so a caller can also + * build an extract() oracle from the same values. + */ +static void +draw_time(pg_prop_tc *tc, StringInfo buf, int *h, int *m, int *s) +{ + *h = (int) pg_prop_draw_int(tc, 0, 23); + *m = (int) pg_prop_draw_int(tc, 0, 59); + *s = (int) pg_prop_draw_int(tc, 0, 59); + appendStringInfo(buf, "TIME '%02d:%02d:%02d'", *h, *m, *s); +} + +/* + * Draw a date-and-time and append a plain TIMESTAMP (no tz) literal + * "TIMESTAMP '%04d-%02d-%02d %02d:%02d:%02d'" to buf. Day is restricted to + * 1..28 so every (year, month, day) is a valid calendar date. Components are + * returned so a caller can build make_date()/extract() oracles. + */ +static void +draw_timestamp(pg_prop_tc *tc, StringInfo buf, + int *y, int *mo, int *d, int *h, int *m, int *s) +{ + *y = (int) pg_prop_draw_int(tc, 2000, 2030); + *mo = (int) pg_prop_draw_int(tc, 1, 12); + *d = (int) pg_prop_draw_int(tc, 1, 28); + *h = (int) pg_prop_draw_int(tc, 0, 23); + *m = (int) pg_prop_draw_int(tc, 0, 59); + *s = (int) pg_prop_draw_int(tc, 0, 59); + appendStringInfo(buf, "TIMESTAMP '%04d-%02d-%02d %02d:%02d:%02d'", + *y, *mo, *d, *h, *m, *s); +} + +/* + * 1. time_text_roundtrip: (T)::text::time = T for a drawn TIME. + */ +PG_PROP(time_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData t, + q; + int h, + m, + s; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&t); + draw_time(tc, &t, &h, &m, &s); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s)::text::time = %s)", t.data, t.data); + + require_true(tc, conn, q.data); + + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. timestamp_text_roundtrip: (TS)::text::timestamp = TS. + */ +PG_PROP(timestamp_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData ts, + q; + int y, + mo, + d, + h, + m, + s; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&ts); + draw_timestamp(tc, &ts, &y, &mo, &d, &h, &m, &s); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s)::text::timestamp = %s)", + ts.data, ts.data); + + require_true(tc, conn, q.data); + + pfree(ts.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. time_extract_fields: extract(hour from T)=h, extract(minute from T)=m, + * extract(second from T)=s for the drawn h, m, s. + */ +PG_PROP(time_extract_fields) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData t, + q; + int h, + m, + s; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&t); + draw_time(tc, &t, &h, &m, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT extract(hour from %s) = %d " + "AND extract(minute from %s) = %d " + "AND extract(second from %s) = %d", + t.data, h, t.data, m, t.data, s); + + require_true(tc, conn, q.data); + + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. timestamp_minus_timestamp_interval: TS2 - TS1 is an interval, and + * TS1 + (TS2 - TS1) = TS2 for two drawn timestamps. The pg_typeof check + * confirms the subtraction yields interval; the identity confirms the + * round-trip through that interval. + */ +PG_PROP(timestamp_minus_timestamp_interval) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData ts1, + ts2, + q; + int y, + mo, + d, + h, + m, + s; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&ts1); + initStringInfo(&ts2); + draw_timestamp(tc, &ts1, &y, &mo, &d, &h, &m, &s); + draw_timestamp(tc, &ts2, &y, &mo, &d, &h, &m, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT pg_typeof(%s - %s) = 'interval'::regtype " + "AND (%s + (%s - %s)) = %s", + ts2.data, ts1.data, + ts1.data, ts2.data, ts1.data, ts2.data); + + require_true(tc, conn, q.data); + + pfree(ts1.data); + pfree(ts2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. timestamp_plus_interval_roundtrip: + * TS + make_interval(days=>d) - make_interval(days=>d) = TS for d in -100..100. + * Adding then subtracting the same whole-day interval is an identity on a + * plain (no-tz) timestamp, where there is no DST discontinuity. + */ +PG_PROP(timestamp_plus_interval_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData ts, + q; + int y, + mo, + d, + h, + m, + s; + int days; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&ts); + draw_timestamp(tc, &ts, &y, &mo, &d, &h, &m, &s); + days = (int) pg_prop_draw_int(tc, -100, 100); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s + make_interval(days => (%d)) " + "- make_interval(days => (%d))) = %s", + ts.data, days, days, ts.data); + + require_true(tc, conn, q.data); + + pfree(ts.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. time_order_total: for two drawn times A, B exactly one of <, =, > holds, + * so the three comparisons' integer casts sum to 1. + */ +PG_PROP(time_order_total) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + int h, + m, + s; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_time(tc, &a, &h, &m, &s); + draw_time(tc, &b, &h, &m, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (((%s < %s)::int + (%s = %s)::int " + "+ (%s > %s)::int) = 1)", + a.data, b.data, a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. date_trunc_idempotent: + * date_trunc('hour', date_trunc('hour', TS)) = date_trunc('hour', TS). + * Truncating an already-truncated value is a no-op, so applying it twice + * equals applying it once. + */ +PG_PROP(date_trunc_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData ts, + q; + int y, + mo, + d, + h, + m, + s; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&ts); + draw_timestamp(tc, &ts, &y, &mo, &d, &h, &m, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT date_trunc('hour', date_trunc('hour', %s)) " + "= date_trunc('hour', %s)", + ts.data, ts.data); + + require_true(tc, conn, q.data); + + pfree(ts.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 8. timestamp_cast_to_date: (TS)::date equals the date built from the drawn + * year/month/day components: (TS)::date = make_date(y, mo, d). + */ +PG_PROP(timestamp_cast_to_date) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData ts, + q; + int y, + mo, + d, + h, + m, + s; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&ts); + draw_timestamp(tc, &ts, &y, &mo, &d, &h, &m, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s)::date = make_date(%d, %d, %d)", + ts.data, y, mo, d); + + require_true(tc, conn, q.data); + + pfree(ts.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 0c0298956e457..716554332021a 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From dddc3a828d11e352707930f68a828698dc749b8d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:14 -0400 Subject: [PATCH 136/338] prop: add time field extraction and arithmetic property tests (timeextract) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_timeextract.c | 375 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 379 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_timeextract.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 47c0f70143d75..1572ea724981f 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -151,7 +151,8 @@ OBJS = \ prop_datefmt.o \ prop_datecast.o \ prop_datetrunc.o \ - prop_timeops.o + prop_timeops.o \ + prop_timeextract.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 2e5e29db97490..8fbb6a43f66ed 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -149,6 +149,7 @@ prop_area_sources = files( 'prop_datecast.c', 'prop_datetrunc.c', 'prop_timeops.c', + 'prop_timeextract.c', ) diff --git a/src/test/prop/prop_timeextract.c b/src/test/prop/prop_timeextract.c new file mode 100644 index 0000000000000..50ef69eb8ba0a --- /dev/null +++ b/src/test/prop/prop_timeextract.c @@ -0,0 +1,375 @@ +/*------------------------------------------------------------------------- + * + * prop_timeextract.c + * Property-based tests for extract()/date_part() on time/timetz and the + * algebra of time arithmetic (time +/- interval, time - time). + * + * The oracle here is each identity's own definition: the hour/minute/second + * fields of a drawn TIME must equal the components we built it from, the epoch + * of a time is h*3600 + mi*60 + s, adding the zero interval is a no-op, a + * +1h/-1h round-trip returns the original (kept away from the 24h wrap), + * subtracting two times yields an interval that re-adds to the second, and the + * total order on time is trichotomous. make_time and the text cast must both + * reproduce the same TIME literal. Each property is a SELECT of a boolean + * that must be "t"; a NULL or SQL error on a valid time is itself a bug. + * + * Distinct from the "timeops", "dateparts" and "datetrunc" areas: this + * targets extract/date_part on the time types specifically, plus time-time + * and time-interval arithmetic (not date_trunc, not date fields, not + * timestamp subtraction). + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_timeextract.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "timeextract" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operation errored or produced NULL on a valid input, + * which is itself a bug -- so either is a failure, reported with the SQL. + * Copied from prop_numeric.c's require_true (Shape A, scalar boolean oracle). + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Draw a time-of-day h/m/s (whole-second components, each within its natural + * range) and append a zero-padded TIME literal "TIME '%02d:%02d:%02d'" to buf. + * The components are returned so a caller can also build the extract()/epoch + * oracle from the same values. + */ +static void +draw_time(pg_prop_tc *tc, StringInfo buf, int *h, int *mi, int *s) +{ + *h = (int) pg_prop_draw_int(tc, 0, 23); + *mi = (int) pg_prop_draw_int(tc, 0, 59); + *s = (int) pg_prop_draw_int(tc, 0, 59); + appendStringInfo(buf, "TIME '%02d:%02d:%02d'", *h, *mi, *s); +} + +/* + * 1. extract_fields: extract(hour from T) = h, extract(minute from T) = mi, + * and extract(second from T) = s, for a drawn whole-second TIME. extract + * returns numeric, so compare each against the integer we drew. + */ +PG_PROP(extract_fields) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData t, + q; + int h, + mi, + s; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&t); + draw_time(tc, &t, &h, &mi, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT extract(hour from %s) = %d " + "AND extract(minute from %s) = %d " + "AND extract(second from %s) = %d", + t.data, h, t.data, mi, t.data, s); + + require_true(tc, conn, q.data); + + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. epoch_seconds: extract(epoch from T) = h*3600 + mi*60 + s. For a + * time-of-day the epoch is the number of seconds since midnight. + */ +PG_PROP(epoch_seconds) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData t, + q; + int h, + mi, + s; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&t); + draw_time(tc, &t, &h, &mi, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT extract(epoch from %s) = %d", + t.data, h * 3600 + mi * 60 + s); + + require_true(tc, conn, q.data); + + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. time_plus_interval: T + interval '0' = T, and (for a time at least an + * hour before midnight so no 24h wrap occurs) T + interval '1 hour' - + * interval '1 hour' = T. We rebuild the literal with h capped at 22 so the + * +1 hour step cannot cross midnight and wrap. + */ +PG_PROP(time_plus_interval) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData t, + q; + int h, + mi, + s; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* keep h <= 22 so T + interval '1 hour' does not wrap past 24:00 */ + h = (int) pg_prop_draw_int(tc, 0, 22); + mi = (int) pg_prop_draw_int(tc, 0, 59); + s = (int) pg_prop_draw_int(tc, 0, 59); + + initStringInfo(&t); + appendStringInfo(&t, "TIME '%02d:%02d:%02d'", h, mi, s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s + interval '0') = %s " + "AND (%s + interval '1 hour' - interval '1 hour') = %s", + t.data, t.data, t.data, t.data); + + require_true(tc, conn, q.data); + + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. time_minus_time: for two drawn times T1, T2 the difference (T2 - T1) is + * an interval, and T1 + (T2 - T1) = T2. This is the additive-inverse + * identity for time subtraction and holds for any ordering of T1, T2. + */ +PG_PROP(time_minus_time) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData t1, + t2, + q; + int h1, + mi1, + s1, + h2, + mi2, + s2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&t1); + initStringInfo(&t2); + draw_time(tc, &t1, &h1, &mi1, &s1); + draw_time(tc, &t2, &h2, &mi2, &s2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT pg_typeof(%s - %s) = 'interval'::regtype " + "AND (%s + (%s - %s)) = %s", + t2.data, t1.data, + t1.data, t2.data, t1.data, t2.data); + + require_true(tc, conn, q.data); + + pfree(t1.data); + pfree(t2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. time_order: for two drawn times exactly one of T1 < T2, T1 = T2, T1 > T2 + * holds (trichotomy of the total order on time). + */ +PG_PROP(time_order) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData t1, + t2, + q; + int h1, + mi1, + s1, + h2, + mi2, + s2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&t1); + initStringInfo(&t2); + draw_time(tc, &t1, &h1, &mi1, &s1); + draw_time(tc, &t2, &h2, &mi2, &s2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (CASE WHEN %s < %s THEN 1 ELSE 0 END) " + "+ (CASE WHEN %s = %s THEN 1 ELSE 0 END) " + "+ (CASE WHEN %s > %s THEN 1 ELSE 0 END) = 1", + t1.data, t2.data, t1.data, t2.data, t1.data, t2.data); + + require_true(tc, conn, q.data); + + pfree(t1.data); + pfree(t2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. make_time_roundtrip: make_time(h, mi, s) = TIME '%02d:%02d:%02d' built + * from the same components. make_time takes a double for seconds; passing + * the whole-second integer we drew must reproduce the padded literal. + */ +PG_PROP(make_time_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData t, + q; + int h, + mi, + s; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&t); + draw_time(tc, &t, &h, &mi, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT make_time(%d, %d, %d) = %s", + h, mi, s, t.data); + + require_true(tc, conn, q.data); + + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. time_text_roundtrip: (T)::text::time = T for a drawn TIME. The text + * output of a time must parse back to the same value. + */ +PG_PROP(time_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData t, + q; + int h, + mi, + s; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&t); + draw_time(tc, &t, &h, &mi, &s); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s)::text::time = %s)", t.data, t.data); + + require_true(tc, conn, q.data); + + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 8. extract_from_make_time: extract(hour from make_time(h, mi, s)) = h. + * Composition of make_time and extract must recover the hour we supplied. + */ +PG_PROP(extract_from_make_time) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int h, + mi, + s; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + h = (int) pg_prop_draw_int(tc, 0, 23); + mi = (int) pg_prop_draw_int(tc, 0, 59); + s = (int) pg_prop_draw_int(tc, 0, 59); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT extract(hour from make_time(%d, %d, %d)) = %d", + h, mi, s, h); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 716554332021a..70c1501edc82b 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 2e15de64fc43746573e9c48687873be0fb46dace Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:14 -0400 Subject: [PATCH 137/338] prop: add timestamptz property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_timestamptz.c | 451 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 455 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_timestamptz.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 1572ea724981f..6917f1013b659 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -152,7 +152,8 @@ OBJS = \ prop_datecast.o \ prop_datetrunc.o \ prop_timeops.o \ - prop_timeextract.o + prop_timeextract.o \ + prop_timestamptz.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 8fbb6a43f66ed..bfb380213bf47 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -150,6 +150,7 @@ prop_area_sources = files( 'prop_datetrunc.c', 'prop_timeops.c', 'prop_timeextract.c', + 'prop_timestamptz.c', ) diff --git a/src/test/prop/prop_timestamptz.c b/src/test/prop/prop_timestamptz.c new file mode 100644 index 0000000000000..732676597c68b --- /dev/null +++ b/src/test/prop/prop_timestamptz.c @@ -0,0 +1,451 @@ +/*------------------------------------------------------------------------- + * + * prop_timestamptz.c + * Property-based tests for timestamp/timestamptz + * (src/backend/utils/adt/timestamp.c), focusing on timestamptz, + * date_trunc, date_bin, and extract. + * + * timestamp[tz] has had ~24 fixes (date_bin rounding d163fdbfea8, date_trunc + * on infinities, extract fields, overflow in +/- interval); these check + * round-trips, date_trunc/date_bin idempotence and bounds, and extract + * consistency. To keep timezone effects deterministic, every property sets + * the session TimeZone to 'UTC' first (see pg_prop_connect_utc), and every + * literal carries an explicit +00 offset so it is unambiguous UTC. + * + * The oracle here is the type's own algebra evaluated in SQL: text round-trip, + * date_trunc idempotence and its "never moves forward" bound, date_bin's + * bucket containment, extract(epoch) monotonicity, the add/subtract inverse + * for day/time-only intervals, and the comparator being a proper total order. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_timestamptz.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "timestamptz" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * True when a SQL error is a legitimate out-of-range/overflow condition rather + * than a wrong-results bug. Interval arithmetic and date_bin near the ends of + * the supported range can legally overflow; those cases are discarded, not + * reported. + */ +static bool +is_overflow(const char *errmsg) +{ + if (errmsg == NULL) + return false; + return strstr(errmsg, "out of range") != NULL || + strstr(errmsg, "overflow") != NULL; +} + +/* + * Connect to the test cluster and pin the session TimeZone to 'UTC' so that + * date_trunc, text I/O, and extract are deterministic regardless of the + * cluster's default. Fails the case if either step fails. + */ +static pg_prop_conn * +pg_prop_connect_utc(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, "SET TimeZone = 'UTC'", &err)) + pg_prop_fail(tc, "could not set TimeZone to UTC: %s", err ? err : ""); + + return conn; +} + +/* + * Build a random timestamptz literal into buf, e.g. + * TIMESTAMPTZ '2024-03-15 08:30:00+00' + * + * The range is kept well inside the supported timestamp range (year + * 2000..2050) so the literal itself never overflows and the identities under + * test are only driven by the operation. day is capped at 28 so every month + * is valid. The explicit +00 offset makes the literal unambiguous UTC. + */ +static void +draw_ts(pg_prop_tc *tc, StringInfo buf) +{ + int year; + int mon; + int day; + int hour; + int min; + int sec; + + year = (int) pg_prop_draw_int(tc, 2000, 2050); + mon = (int) pg_prop_draw_int(tc, 1, 12); + day = (int) pg_prop_draw_int(tc, 1, 28); + hour = (int) pg_prop_draw_int(tc, 0, 23); + min = (int) pg_prop_draw_int(tc, 0, 59); + sec = (int) pg_prop_draw_int(tc, 0, 59); + + appendStringInfo(buf, "TIMESTAMPTZ '%04d-%02d-%02d %02d:%02d:%02d+00'", + year, mon, day, hour, min, sec); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operation errored or produced NULL on finite inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * 1. text_roundtrip + * + * Formatting a timestamptz to text and parsing it back must yield the same + * value: (T)::text::timestamptz = T. This targets the text I/O round-trip. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + StringInfoData t; + StringInfoData q; + + conn = pg_prop_connect_utc(tc); + + initStringInfo(&t); + draw_ts(tc, &t); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)::text::timestamptz = %s", + t.data, t.data); + + require_true(tc, conn, q.data); + + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* Valid date_trunc fields, sampled by index. */ +static const char *const trunc_fields[] = { + "microseconds", "second", "minute", "hour", "day", "month", "year" +}; + +/* + * 2. date_trunc_idempotent + * + * date_trunc is idempotent: truncating to a unit and truncating again to the + * same unit yields the same value. date_trunc('', T) = + * date_trunc('', date_trunc('', T)). + */ +PG_PROP(date_trunc_idempotent) +{ + pg_prop_conn *conn; + StringInfoData t; + StringInfoData q; + const char *f; + + conn = pg_prop_connect_utc(tc); + + f = trunc_fields[pg_prop_draw_int(tc, 0, lengthof(trunc_fields) - 1)]; + + initStringInfo(&t); + draw_ts(tc, &t); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT date_trunc('%s', %s) " + "= date_trunc('%s', date_trunc('%s', %s))", + f, t.data, f, f, t.data); + + require_true(tc, conn, q.data); + + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. date_trunc_not_after + * + * Truncation rounds down toward the start of the unit, so it never moves the + * timestamp forward: date_trunc('', T) <= T. + */ +PG_PROP(date_trunc_not_after) +{ + pg_prop_conn *conn; + StringInfoData t; + StringInfoData q; + const char *f; + + conn = pg_prop_connect_utc(tc); + + f = trunc_fields[pg_prop_draw_int(tc, 0, lengthof(trunc_fields) - 1)]; + + initStringInfo(&t); + draw_ts(tc, &t); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT date_trunc('%s', %s) <= %s", + f, t.data, t.data); + + require_true(tc, conn, q.data); + + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. date_bin_in_bucket + * + * date_bin places T into a bucket whose start b satisfies b <= T < b + stride, + * for a positive stride. This directly targets the date_bin rounding fixes + * (d163fdbfea8). Overflow near the range ends is a legitimate error and the + * case is discarded. + * + * date_bin signature: date_bin(stride interval, source timestamptz, + * origin timestamptz). + */ +PG_PROP(date_bin_in_bucket) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData t; + StringInfoData o; + StringInfoData q; + char *result; + bool failed; + int stride; + + conn = pg_prop_connect_utc(tc); + + stride = (int) pg_prop_draw_int(tc, 1, 86400); + + initStringInfo(&t); + draw_ts(tc, &t); + + initStringInfo(&o); + draw_ts(tc, &o); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (b <= %s) AND (%s < b + ('%d seconds')::interval) " + "FROM (SELECT date_bin(('%d seconds')::interval, %s, %s) " + "AS b) s", + t.data, t.data, stride, + stride, t.data, o.data); + + result = pg_prop_sql_scalar(conn, q.data, &failed, &err); + + if (failed) + { + if (is_overflow(err)) + { + /* arithmetic ran off the supported range: legal, skip the case */ + pg_prop_note(tc, "overflow (skipped): %s", err); + if (err) + pg_free(err); + pfree(t.data); + pfree(o.data); + pfree(q.data); + pg_prop_disconnect(conn); + pg_prop_assume(tc, false); + return; /* unreachable: assume(false) longjmps */ + } + pg_prop_fail(tc, + "unexpected error on date_bin containment\n" + " T: %s\n O: %s\n stride: %d s\n error: %s", + t.data, o.data, stride, err ? err : "(unknown)"); + } + + if (result == NULL || strcmp(result, "t") != 0) + pg_prop_fail(tc, + "date_bin bucket does not contain T\n" + " T: %s\n O: %s\n stride: %d s\n result: %s", + t.data, o.data, stride, result ? result : "NULL"); + + pg_free(result); + pfree(t.data); + pfree(o.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. extract_epoch_monotonic + * + * The ordering of two timestamps matches the ordering of their epoch seconds: + * (T1 < T2) = (extract(epoch from T1) < extract(epoch from T2)). This checks + * that extract(epoch ...) is consistent with timestamptz comparison. + */ +PG_PROP(extract_epoch_monotonic) +{ + pg_prop_conn *conn; + StringInfoData t1; + StringInfoData t2; + StringInfoData q; + + conn = pg_prop_connect_utc(tc); + + initStringInfo(&t1); + initStringInfo(&t2); + draw_ts(tc, &t1); + draw_ts(tc, &t2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s < %s) " + "= (extract(epoch from %s) < extract(epoch from %s))", + t1.data, t2.data, t1.data, t2.data); + + require_true(tc, conn, q.data); + + pfree(t1.data); + pfree(t2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. plus_minus_interval_inverse + * + * For a day/time-only interval I (no months/years, so it is exactly + * invertible) and any timestamptz T, (T + I) - I must equal T. Months and + * years are deliberately excluded because they denote variable spans and are + * not invertible this way. Overflow near the range ends is a legitimate error + * and the case is discarded. + */ +PG_PROP(plus_minus_interval_inverse) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData t; + StringInfoData q; + char *result; + bool failed; + int days; + int hours; + int mins; + int secs; + + conn = pg_prop_connect_utc(tc); + + initStringInfo(&t); + draw_ts(tc, &t); + + /* day/time-only interval -- no months/years, so exactly invertible */ + days = (int) pg_prop_draw_int(tc, 0, 1000); + hours = (int) pg_prop_draw_int(tc, 0, 23); + mins = (int) pg_prop_draw_int(tc, 0, 59); + secs = (int) pg_prop_draw_int(tc, 0, 59); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s + '%d days %d hours %d mins %d secs'::interval " + "- '%d days %d hours %d mins %d secs'::interval) = %s", + t.data, days, hours, mins, secs, + days, hours, mins, secs, t.data); + + result = pg_prop_sql_scalar(conn, q.data, &failed, &err); + + if (failed) + { + if (is_overflow(err)) + { + /* arithmetic ran off the supported range: legal, skip the case */ + pg_prop_note(tc, "overflow (skipped): %s", err); + if (err) + pg_free(err); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); + pg_prop_assume(tc, false); + return; /* unreachable: assume(false) longjmps */ + } + pg_prop_fail(tc, + "unexpected error on (T + I - I) = T\n" + " T: %s\n I: %d days %d hours %d mins %d secs\n error: %s", + t.data, days, hours, mins, secs, err ? err : "(unknown)"); + } + + if (result == NULL || strcmp(result, "t") != 0) + pg_prop_fail(tc, + "(T + I - I) != T for day/time interval\n" + " T: %s\n I: %d days %d hours %d mins %d secs\n result: %s", + t.data, days, hours, mins, secs, + result ? result : "NULL"); + + pg_free(result); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. comparison_total_order + * + * For any two timestamps T1, T2, exactly one of (T1 < T2), (T1 = T2), + * (T1 > T2) must hold. This checks that the timestamptz comparator is a + * proper total order. + */ +PG_PROP(comparison_total_order) +{ + pg_prop_conn *conn; + StringInfoData t1; + StringInfoData t2; + StringInfoData q; + + conn = pg_prop_connect_utc(tc); + + initStringInfo(&t1); + initStringInfo(&t2); + draw_ts(tc, &t1); + draw_ts(tc, &t2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s < %s)::int + (%s = %s)::int " + "+ (%s > %s)::int) = 1", + t1.data, t2.data, t1.data, t2.data, t1.data, t2.data); + + require_true(tc, conn, q.data); + + pfree(t1.data); + pfree(t2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 70c1501edc82b..143d43e12b412 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 762481f6082a92fa2099ae42489c99f18c511fff Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:14 -0400 Subject: [PATCH 138/338] prop: add timestamptz arithmetic and timezone conversion property tests (tstzarith) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_tstzarith.c | 378 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 382 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_tstzarith.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 6917f1013b659..7983be0a9ebeb 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -153,7 +153,8 @@ OBJS = \ prop_datetrunc.o \ prop_timeops.o \ prop_timeextract.o \ - prop_timestamptz.o + prop_timestamptz.o \ + prop_tstzarith.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index bfb380213bf47..e675845a1b2dd 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -151,6 +151,7 @@ prop_area_sources = files( 'prop_timeops.c', 'prop_timeextract.c', 'prop_timestamptz.c', + 'prop_tstzarith.c', ) diff --git a/src/test/prop/prop_tstzarith.c b/src/test/prop/prop_tstzarith.c new file mode 100644 index 0000000000000..fba0093382da8 --- /dev/null +++ b/src/test/prop/prop_tstzarith.c @@ -0,0 +1,378 @@ +/*------------------------------------------------------------------------- + * + * prop_tstzarith.c + * Property-based tests for timestamptz arithmetic and timezone + * conversion (src/backend/utils/adt/timestamp.c). + * + * Distinct from prop_timestamptz.c (text/date_trunc/date_bin/extract): this + * file targets the arithmetic and timezone-conversion algebra -- AT TIME ZONE + * round-trips, interval add/subtract inverses, age() as a calendar interval, + * extract(epoch) differences, and the comparator as a total order. Every + * property pins the session TimeZone to 'UTC' (via connect_tz) and every + * literal carries an explicit +00 offset so results are deterministic and + * free of DST; a failure is therefore a bug in the arithmetic, not a timezone + * artefact. + * + * The oracle is the type's own algebra evaluated in SQL (round-trips, + * add/subtract inverses, a differential epoch identity, and a total-order + * check), so each identity holds for every drawn input with no tolerance + * (except extract(epoch) differences, compared within 1e-6 for float8). + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_tstzarith.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "tstzarith" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Connect to the test cluster and pin the session TimeZone to 'UTC' so AT TIME + * ZONE, age(), extract(epoch), and comparison are deterministic and free of + * DST regardless of the cluster default. Fails the case if either step fails. + */ +static pg_prop_conn * +connect_tz(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, "SET TIME ZONE 'UTC'", &err)) + pg_prop_fail(tc, "could not SET TIME ZONE 'UTC': %s", err ? err : ""); + + return conn; +} + +/* + * Build a random timestamptz literal into buf, e.g. + * TIMESTAMPTZ '2024-03-15 08:30:00+00' + * + * Year 2001..2030 keeps the literal and its arithmetic well inside the + * supported range; day is capped at 28 so every month is valid; the explicit + * +00 offset makes the literal unambiguous UTC. + */ +static void +draw_ts(pg_prop_tc *tc, StringInfo buf) +{ + int year; + int mon; + int day; + int hour; + int min; + int sec; + + year = (int) pg_prop_draw_int(tc, 2001, 2030); + mon = (int) pg_prop_draw_int(tc, 1, 12); + day = (int) pg_prop_draw_int(tc, 1, 28); + hour = (int) pg_prop_draw_int(tc, 0, 23); + min = (int) pg_prop_draw_int(tc, 0, 59); + sec = (int) pg_prop_draw_int(tc, 0, 59); + + appendStringInfo(buf, "TIMESTAMPTZ '%04d-%02d-%02d %02d:%02d:%02d+00'", + year, mon, day, hour, min, sec); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operation errored or produced NULL on finite inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * 1. attz_roundtrip + * + * Converting a timestamptz to a timestamp at UTC and back to timestamptz at + * UTC is the identity: (TS AT TIME ZONE 'UTC') AT TIME ZONE 'UTC' = TS. In + * UTC there is no DST, so this double conversion is lossless. + */ +PG_PROP(attz_roundtrip) +{ + pg_prop_conn *conn; + StringInfoData t; + StringInfoData q; + + conn = connect_tz(tc); + + initStringInfo(&t); + draw_ts(tc, &t); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s AT TIME ZONE 'UTC') AT TIME ZONE 'UTC') = %s", + t.data, t.data); + + require_true(tc, conn, q.data); + + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. plus_minus_interval + * + * Adding and then subtracting the same whole-hour interval is the identity: + * TS + make_interval(hours => h) - make_interval(hours => h) = TS, for h in + * -500..500. An hour interval is a fixed span (no months/years), so it is + * exactly invertible in UTC. + */ +PG_PROP(plus_minus_interval) +{ + pg_prop_conn *conn; + StringInfoData t; + StringInfoData q; + int h; + + conn = connect_tz(tc); + + initStringInfo(&t); + draw_ts(tc, &t); + + h = (int) pg_prop_draw_int(tc, -500, 500); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s + make_interval(hours => %d) " + "- make_interval(hours => %d)) = %s", + t.data, h, h, t.data); + + require_true(tc, conn, q.data); + + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. epoch_diff + * + * extract(epoch) distributes over subtraction of two timestamptz: + * extract(epoch from (TS2 - TS1)) = extract(epoch from TS2) + * - extract(epoch from TS1). + * Both sides are float8 seconds, compared within 1e-6 to absorb float + * rounding. + */ +PG_PROP(epoch_diff) +{ + pg_prop_conn *conn; + StringInfoData t1; + StringInfoData t2; + StringInfoData q; + + conn = connect_tz(tc); + + initStringInfo(&t1); + initStringInfo(&t2); + draw_ts(tc, &t1); + draw_ts(tc, &t2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(extract(epoch from (%s - %s)) " + "- (extract(epoch from %s) - extract(epoch from %s))) " + "< 1e-6", + t2.data, t1.data, t2.data, t1.data); + + require_true(tc, conn, q.data); + + pfree(t1.data); + pfree(t2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. age_plus_back + * + * age(TS2, TS1) is a calendar interval (variable-length months/days) that + * spans from TS1 to TS2. Adding it back recovers TS2: + * TS1 + age(TS2, TS1) = TS2. This holds exactly in UTC because there is no + * DST crossing to make the calendar interval ambiguous. + */ +PG_PROP(age_sign_matches_order) +{ + pg_prop_conn *conn; + StringInfoData t1; + StringInfoData t2; + StringInfoData q; + + conn = connect_tz(tc); + + initStringInfo(&t1); + initStringInfo(&t2); + draw_ts(tc, &t1); + draw_ts(tc, &t2); + + /* + * age(a, b) is not invertible by addition -- it returns a symbolic + * month/day/time interval, and a + age(b, a) need not land back on b when + * the time-of-day differs (the month carry does not undo cleanly). The + * sound invariant is the sign: age(a, b) is positive, zero, or negative + * exactly as a is after, equal to, or before b. + */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (age(%s, %s) > interval '0') = (%s > %s) " + "AND (age(%s, %s) = interval '0') = (%s = %s)", + t2.data, t1.data, t2.data, t1.data, + t2.data, t1.data, t2.data, t1.data); + + require_true(tc, conn, q.data); + + pfree(t1.data); + pfree(t2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. order_total + * + * For any two timestamptz A, B exactly one of (A < B), (A = B), (A > B) holds: + * the comparator is a proper total order. + */ +PG_PROP(order_total) +{ + pg_prop_conn *conn; + StringInfoData a; + StringInfoData b; + StringInfoData q; + + conn = connect_tz(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_ts(tc, &a); + draw_ts(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s < %s)::int + (%s = %s)::int " + "+ (%s > %s)::int) = 1", + a.data, b.data, a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. truncate_le + * + * date_trunc rounds down toward the start of the unit, so truncating to hour + * or day never moves the timestamp forward: + * date_trunc('hour', TS) <= TS AND date_trunc('day', TS) <= TS. + */ +PG_PROP(truncate_le) +{ + pg_prop_conn *conn; + StringInfoData t; + StringInfoData q; + + conn = connect_tz(tc); + + initStringInfo(&t); + draw_ts(tc, &t); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT date_trunc('hour', %s) <= %s " + "AND date_trunc('day', %s) <= %s", + t.data, t.data, t.data, t.data); + + require_true(tc, conn, q.data); + + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. epoch_monotone + * + * extract(epoch) is strictly increasing on a strictly increasing timestamptz. + * TS2 is built as TS1 plus a strictly positive interval, so TS1 < TS2 by + * construction; then extract(epoch from TS1) < extract(epoch from TS2) must + * hold. The positive interval is a fixed span (days/hours/minutes/seconds, + * at least one second) so TS2 strictly exceeds TS1 in UTC. + */ +PG_PROP(epoch_monotone) +{ + pg_prop_conn *conn; + StringInfoData t1; + StringInfoData q; + int days; + int hours; + int mins; + int secs; + + conn = connect_tz(tc); + + initStringInfo(&t1); + draw_ts(tc, &t1); + + /* + * Draw a strictly positive, fixed-span interval. secs starts at 1 so the + * interval is never zero even when days/hours/mins are all 0; the year is + * capped at 2030 and the span at well under a year, so TS2 stays inside + * the supported range and never overflows. + */ + days = (int) pg_prop_draw_int(tc, 0, 300); + hours = (int) pg_prop_draw_int(tc, 0, 23); + mins = (int) pg_prop_draw_int(tc, 0, 59); + secs = (int) pg_prop_draw_int(tc, 1, 59); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT extract(epoch from %s) " + "< extract(epoch from " + "(%s + '%d days %d hours %d mins %d secs'::interval))", + t1.data, t1.data, days, hours, mins, secs); + + require_true(tc, conn, q.data); + + pfree(t1.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 143d43e12b412..835034aac1361 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From f2bc1a047a85543032ca994e57976ec154e62d2d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:14 -0400 Subject: [PATCH 139/338] prop: add timezone conversion identities property tests (tzconv) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_tzconv.c | 343 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 347 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_tzconv.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 7983be0a9ebeb..c790d40b5ec5f 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -154,7 +154,8 @@ OBJS = \ prop_timeops.o \ prop_timeextract.o \ prop_timestamptz.o \ - prop_tstzarith.o + prop_tstzarith.o \ + prop_tzconv.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index e675845a1b2dd..35f69245bbeb6 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -152,6 +152,7 @@ prop_area_sources = files( 'prop_timeextract.c', 'prop_timestamptz.c', 'prop_tstzarith.c', + 'prop_tzconv.c', ) diff --git a/src/test/prop/prop_tzconv.c b/src/test/prop/prop_tzconv.c new file mode 100644 index 0000000000000..2b01d5674f483 --- /dev/null +++ b/src/test/prop/prop_tzconv.c @@ -0,0 +1,343 @@ +/*------------------------------------------------------------------------- + * + * prop_tzconv.c + * Property-based tests for timezone conversion identities. + * + * AT TIME ZONE, make_timestamptz, and the timestamptz<->timestamp conversion + * paths (src/backend/utils/adt/timestamp.c, datetime.c) implement the + * instant<->wall-clock mapping. These properties check the identities that + * must hold for fixed, DST-free zones: AT TIME ZONE is its own inverse at a + * fixed offset, converting to two fixed offsets differs by exactly the offset + * delta, the epoch of an instant is independent of display zone, text I/O + * round-trips under a UTC session, make_timestamptz recovers its fields, and + * conversion to a fixed zone preserves instant order. A false result is a + * bug in the conversion, not a DST artefact (every zone used here is a fixed + * offset or an Etc/GMT zone with no DST). + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_tzconv.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "tzconv" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Connect to the test cluster and pin the session TimeZone to 'UTC' so that + * text I/O, extract, and any implicit display conversion are deterministic. + * The properties that compare against a named zone always name it explicitly + * via AT TIME ZONE, so only the session default matters here. + */ +static pg_prop_conn * +connect_tz(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, "SET TIME ZONE 'UTC'", &err)) + pg_prop_fail(tc, "could not set TIME ZONE to UTC: %s", err ? err : ""); + + return conn; +} + +/* + * Build a random timestamptz literal into buf, e.g. + * TIMESTAMPTZ '2024-03-15 08:30:00+00' + * + * Shape A domain: year 2001..2030, month 1..12, day 1..28 (valid in every + * month), hour 0..23, minute 0..59, second 0..59. The explicit +00 offset + * makes the literal an unambiguous instant. The range is well inside the + * supported timestamp range so the literal and the small shifts below never + * overflow. + */ +static void +draw_ts(pg_prop_tc *tc, StringInfo buf) +{ + int year; + int mon; + int day; + int hour; + int min; + int sec; + + year = (int) pg_prop_draw_int(tc, 2001, 2030); + mon = (int) pg_prop_draw_int(tc, 1, 12); + day = (int) pg_prop_draw_int(tc, 1, 28); + hour = (int) pg_prop_draw_int(tc, 0, 23); + min = (int) pg_prop_draw_int(tc, 0, 59); + sec = (int) pg_prop_draw_int(tc, 0, 59); + + appendStringInfo(buf, "TIMESTAMPTZ '%04d-%02d-%02d %02d:%02d:%02d+00'", + year, mon, day, hour, min, sec); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the conversion errored or produced NULL on finite inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * 1. attz_inverse_fixed_offset + * + * AT TIME ZONE is its own inverse at a fixed offset. (TS AT TIME ZONE '+05') + * converts the instant TS to the local wall clock in zone '+05', yielding a + * plain timestamp; applying AT TIME ZONE '+05' to that timestamp reads it back + * as an instant in '+05', recovering TS. Zone '+05' is a fixed offset with no + * DST, so this round-trip is exact for every instant: + * ((TS AT TIME ZONE '+05') AT TIME ZONE '+05') = TS. + */ +PG_PROP(attz_inverse_fixed_offset) +{ + pg_prop_conn *conn; + StringInfoData ts; + StringInfoData q; + + conn = connect_tz(tc); + + initStringInfo(&ts); + draw_ts(tc, &ts); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s AT TIME ZONE '+05') AT TIME ZONE '+05') = %s", + ts.data, ts.data); + + require_true(tc, conn, q.data); + + pfree(ts.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. offset_shifts_by_hours + * + * AT TIME ZONE converts an instant to the local wall clock in that zone, + * producing a plain timestamp. The wall clocks for two fixed offsets differ + * by exactly the offset delta, so (TS AT TIME ZONE '+05') and + * (TS AT TIME ZONE 'UTC') differ by 5 hours. POSIX offset sign for a bare + * '+05' is counter-intuitive (it can mean west of UTC), so we assert only the + * magnitude, which is sound regardless of sign: + * abs(extract(epoch from ((TS AT '+05') - (TS AT 'UTC')))) = 5*3600. + */ +PG_PROP(offset_shifts_by_hours) +{ + pg_prop_conn *conn; + StringInfoData ts; + StringInfoData q; + + conn = connect_tz(tc); + + initStringInfo(&ts); + draw_ts(tc, &ts); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(extract(epoch from " + "((%s AT TIME ZONE '+05') - (%s AT TIME ZONE 'UTC')))) " + "= 5*3600", + ts.data, ts.data); + + require_true(tc, conn, q.data); + + pfree(ts.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. epoch_invariant_under_zone + * + * The epoch of an instant does not depend on the display zone. Converting an + * instant TS to the wall clock in 'UTC' (a plain timestamp) and reinterpreting + * that timestamp as an instant in the UTC session recovers the same instant, + * so its epoch is unchanged: + * extract(epoch from (TS AT TIME ZONE 'UTC')::timestamptz) + * = extract(epoch from TS) + * holds because the session TimeZone is pinned to UTC. + */ +PG_PROP(epoch_invariant_under_zone) +{ + pg_prop_conn *conn; + StringInfoData ts; + StringInfoData q; + + conn = connect_tz(tc); + + initStringInfo(&ts); + draw_ts(tc, &ts); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT extract(epoch from " + "(%s AT TIME ZONE 'UTC')::timestamptz) " + "= extract(epoch from %s)", + ts.data, ts.data); + + require_true(tc, conn, q.data); + + pfree(ts.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. roundtrip_text_utc + * + * Under a UTC session, formatting a timestamptz to text and parsing it back + * yields the same instant: (TS)::text::timestamptz = TS. This targets the + * text I/O round-trip with a pinned display zone. + */ +PG_PROP(roundtrip_text_utc) +{ + pg_prop_conn *conn; + StringInfoData ts; + StringInfoData q; + + conn = connect_tz(tc); + + initStringInfo(&ts); + draw_ts(tc, &ts); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)::text::timestamptz = %s", + ts.data, ts.data); + + require_true(tc, conn, q.data); + + pfree(ts.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. make_timestamptz_roundtrip + * + * make_timestamptz(y, mo, d, h, mi, s, 'UTC') builds a valid instant from its + * fields in UTC; reading the year field back in UTC must recover y: + * extract(year from make_timestamptz(...) AT TIME ZONE 'UTC') = y. + * The field domain (year 2001..2030, day 1..28, etc.) is always a valid + * calendar instant, so no error is expected. + */ +PG_PROP(make_timestamptz_roundtrip) +{ + pg_prop_conn *conn; + StringInfoData q; + int year; + int mon; + int day; + int hour; + int min; + int sec; + + conn = connect_tz(tc); + + year = (int) pg_prop_draw_int(tc, 2001, 2030); + mon = (int) pg_prop_draw_int(tc, 1, 12); + day = (int) pg_prop_draw_int(tc, 1, 28); + hour = (int) pg_prop_draw_int(tc, 0, 23); + min = (int) pg_prop_draw_int(tc, 0, 59); + sec = (int) pg_prop_draw_int(tc, 0, 59); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT extract(year from " + "make_timestamptz(%d, %d, %d, %d, %d, %d, 'UTC') " + "AT TIME ZONE 'UTC') = %d", + year, mon, day, hour, min, sec, year); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. order_preserved_under_conversion + * + * Conversion to a fixed-offset zone is monotone: for TS1 < TS2, + * (TS1 AT TIME ZONE '+05') < (TS2 AT TIME ZONE '+05'). TS2 is built as TS1 + * plus a strictly positive day/time interval (so TS1 < TS2 by construction, + * and the span is exactly invertible with no month/year ambiguity). Zone + * '+05' is a fixed offset, so applying the same offset to both instants shifts + * both wall clocks equally and preserves order. + */ +PG_PROP(order_preserved_under_conversion) +{ + pg_prop_conn *conn; + StringInfoData ts1; + StringInfoData ts2; + StringInfoData q; + int days; + int hours; + int mins; + int secs; + + conn = connect_tz(tc); + + initStringInfo(&ts1); + draw_ts(tc, &ts1); + + /* strictly positive day/time interval: at least one second */ + days = (int) pg_prop_draw_int(tc, 0, 1000); + hours = (int) pg_prop_draw_int(tc, 0, 23); + mins = (int) pg_prop_draw_int(tc, 0, 59); + secs = (int) pg_prop_draw_int(tc, 1, 59); + + /* TS2 = TS1 + positive interval, well inside the supported range */ + initStringInfo(&ts2); + appendStringInfo(&ts2, + "(%s + '%d days %d hours %d mins %d secs'::interval)", + ts1.data, days, hours, mins, secs); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s AT TIME ZONE '+05') < (%s AT TIME ZONE '+05')", + ts1.data, ts2.data); + + require_true(tc, conn, q.data); + + pfree(ts1.data); + pfree(ts2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 835034aac1361..efd8b1db5203f 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 0d1e1778e078d610e9e4c236a65a8d2228a8987b Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:14 -0400 Subject: [PATCH 140/338] prop: add interval property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_interval.c | 418 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 422 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_interval.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index c790d40b5ec5f..a0ed117e889e9 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -155,7 +155,8 @@ OBJS = \ prop_timeextract.o \ prop_timestamptz.o \ prop_tstzarith.o \ - prop_tzconv.o + prop_tzconv.o \ + prop_interval.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 35f69245bbeb6..2ab908be14cd5 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -153,6 +153,7 @@ prop_area_sources = files( 'prop_timestamptz.c', 'prop_tstzarith.c', 'prop_tzconv.c', + 'prop_interval.c', ) diff --git a/src/test/prop/prop_interval.c b/src/test/prop/prop_interval.c new file mode 100644 index 0000000000000..b729cde2d1c71 --- /dev/null +++ b/src/test/prop/prop_interval.c @@ -0,0 +1,418 @@ +/*------------------------------------------------------------------------- + * + * prop_interval.c + * Property-based tests for interval and timestamp arithmetic + * (src/backend/utils/adt/timestamp.c, datetime.c). + * + * Target #8 in the design. The interval/timestamp types have taken ~19 fixes + * since 2023: overflow in timestamp + interval (4019285c064), date_bin + * rounding (d163fdbfea8), interval field extraction (6be39d77a70), and the + * infinite-interval rework (519fc1bd9e9). That is a dense bug surface because + * intervals carry three independent fields (months, days, microseconds) whose + * mixed-sign and overflow interactions are easy to get subtly wrong. + * + * The oracle here is round-trip and algebraic identity, evaluated entirely in + * SQL: an interval formatted to text and parsed back must equal the original; + * a day/time interval added then subtracted must return the original + * timestamp; the comparator must be a proper total order (including + * infinities); and justify_hours must preserve the represented value. Each + * invariant is computed as a boolean in the query and the scalar result is + * required to be "t". + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_interval.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "interval" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * True when a SQL error is a legitimate out-of-range/overflow condition rather + * than a wrong-results bug. Random field counts added to a timestamp near the + * ends of its supported range can legally overflow; those cases are discarded, + * not reported. + */ +static bool +is_overflow(const char *errmsg) +{ + if (errmsg == NULL) + return false; + return strstr(errmsg, "out of range") != NULL || + strstr(errmsg, "overflow") != NULL; +} + +/* + * Build a random interval literal into buf, e.g. + * '1 years 2 mons -3 days 4 hours 5 mins 6.5 secs'::interval + * + * Field counts are kept small so counterexamples stay legible and so the + * oracle itself does not drown in uninteresting overflow. The whole interval + * is randomly negated. About 1 in 10 draws instead emits an infinite interval + * to exercise the infinite-interval paths (519fc1bd9e9). + * + * When time_only is true, no year/month fields are drawn, so the interval is + * exactly invertible under timestamp addition (months and years are not, by + * design, because they denote variable spans). + */ +static void +draw_interval(pg_prop_tc *tc, StringInfo buf, bool allow_infinite, bool time_only) +{ + int years; + int mons; + int days; + int hours; + int mins; + int secs; + bool frac; + bool negate; + + if (allow_infinite && pg_prop_draw_int(tc, 0, 9) == 0) + { + appendStringInfoString(buf, + pg_prop_draw_bool(tc) + ? "'infinity'::interval" + : "'-infinity'::interval"); + return; + } + + years = time_only ? 0 : (int) pg_prop_draw_int(tc, -5, 5); + mons = time_only ? 0 : (int) pg_prop_draw_int(tc, -15, 15); + days = (int) pg_prop_draw_int(tc, -40, 40); + hours = (int) pg_prop_draw_int(tc, -50, 50); + mins = (int) pg_prop_draw_int(tc, -120, 120); + secs = (int) pg_prop_draw_int(tc, -120, 120); + frac = pg_prop_draw_bool(tc); + negate = pg_prop_draw_bool(tc); + + appendStringInfoChar(buf, '\''); + appendStringInfo(buf, "%d years %d mons %d days %d hours %d mins ", + years, mons, days, hours, mins); + if (frac) + appendStringInfo(buf, "%d.5 secs", secs); + else + appendStringInfo(buf, "%d secs", secs); + appendStringInfoString(buf, "'::interval"); + + if (negate) + { + /* Prepend unary minus; wrap so operator precedence is unambiguous. */ + char *inner = pstrdup(buf->data); + + resetStringInfo(buf); + appendStringInfo(buf, "(- (%s))", inner); + pfree(inner); + } +} + +/* + * Build a random timestamp literal into buf, e.g. + * '2024-03-15 08:30:00'::timestamp + * + * The range is kept well inside the supported timestamp range (year + * 1900..2100) so that only interval arithmetic, not the literal itself, drives + * any overflow. day is capped at 28 so every month is valid. About 1 in 10 + * draws instead emits an infinite timestamp. + */ +static void +draw_timestamp(pg_prop_tc *tc, StringInfo buf, bool allow_infinite) +{ + int year; + int mon; + int day; + int hour; + int min; + int sec; + + if (allow_infinite && pg_prop_draw_int(tc, 0, 9) == 0) + { + appendStringInfoString(buf, + pg_prop_draw_bool(tc) + ? "'infinity'::timestamp" + : "'-infinity'::timestamp"); + return; + } + + year = (int) pg_prop_draw_int(tc, 1900, 2100); + mon = (int) pg_prop_draw_int(tc, 1, 12); + day = (int) pg_prop_draw_int(tc, 1, 28); + hour = (int) pg_prop_draw_int(tc, 0, 23); + min = (int) pg_prop_draw_int(tc, 0, 59); + sec = (int) pg_prop_draw_int(tc, 0, 59); + + appendStringInfo(buf, "'%04d-%02d-%02d %02d:%02d:%02d'::timestamp", + year, mon, day, hour, min, sec); +} + +/* + * 1. interval_text_roundtrip + * + * Formatting a (finite) interval to text and parsing it back must yield the + * same interval, under every IntervalStyle. This targets the IntervalStyle + * round-trip and mixed-sign formatting: the format and parse paths must agree. + */ +PG_PROP(interval_text_roundtrip) +{ + static const char *const styles[] = { + "postgres", "postgres_verbose", "sql_standard", "iso_8601" + }; + pg_prop_conn *conn; + char *err = NULL; + StringInfoData e; + StringInfoData q; + const char *style; + char *setsql; + char *result; + bool failed; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* a non-infinite interval -- infinities are gated out for this one */ + initStringInfo(&e); + draw_interval(tc, &e, false, false); + + /* exercise all four IntervalStyle output formats */ + style = styles[pg_prop_draw_int(tc, 0, lengthof(styles) - 1)]; + pg_prop_note(tc, "IntervalStyle = %s", style); + + setsql = psprintf("SET IntervalStyle = '%s'", style); + if (!pg_prop_sql_exec(conn, setsql, &err)) + pg_prop_fail(tc, "could not set IntervalStyle '%s': %s", + style, err ? err : ""); + pfree(setsql); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s) = (%s)::text::interval", e.data, e.data); + + result = pg_prop_sql_scalar(conn, q.data, &failed, &err); + + if (failed) + { + /* + * A valid interval that cannot parse its own formatted output is a + * round-trip bug, not a legitimate overflow. + */ + pg_prop_fail(tc, + "interval failed to round-trip through text " + "(IntervalStyle %s)\n interval: %s\n error: %s", + style, e.data, err ? err : "(unknown)"); + } + + if (result == NULL || strcmp(result, "t") != 0) + pg_prop_fail(tc, + "interval != (interval)::text::interval " + "(IntervalStyle %s)\n interval: %s\n result: %s", + style, e.data, result ? result : "NULL"); + + pg_free(result); + pfree(e.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. timestamp_plus_minus_interval + * + * For a day/time-only interval E (no month/year components) and any timestamp + * T, (T + E - E) must equal T. Months and years are deliberately excluded + * because they denote variable spans and are not invertible this way. + * + * Targets 4019285c064 (overflow in timestamp + interval) and the add/subtract + * inverse. Timestamp and interval are both finite here; an overflow near the + * range ends is a legitimate error and the case is discarded. + */ +PG_PROP(timestamp_plus_minus_interval) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData t; + StringInfoData e; + StringInfoData q; + char *result; + bool failed; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&t); + draw_timestamp(tc, &t, false); /* non-infinite */ + + initStringInfo(&e); + draw_interval(tc, &e, false, true); /* non-infinite, day/time only */ + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s + %s - %s) = %s", + t.data, e.data, e.data, t.data); + + result = pg_prop_sql_scalar(conn, q.data, &failed, &err); + + if (failed) + { + if (is_overflow(err)) + { + /* arithmetic ran off the supported range: legal, skip the case */ + pg_prop_note(tc, "overflow (skipped): %s", err); + if (err) + pg_free(err); + pfree(t.data); + pfree(e.data); + pfree(q.data); + pg_prop_disconnect(conn); + pg_prop_assume(tc, false); + return; /* unreachable: assume(false) longjmps */ + } + pg_prop_fail(tc, + "unexpected error on (T + E - E) = T\n" + " T: %s\n E: %s\n error: %s", + t.data, e.data, err ? err : "(unknown)"); + } + + if (result == NULL || strcmp(result, "t") != 0) + pg_prop_fail(tc, + "(T + E - E) != T for day/time interval\n" + " T: %s\n E: %s\n result: %s", + t.data, e.data, result ? result : "NULL"); + + pg_free(result); + pfree(t.data); + pfree(e.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. interval_comparison_total_order + * + * For any two intervals E1, E2 (infinities included), exactly one of + * (E1 < E2), (E1 = E2), (E1 > E2) must hold. This checks that the interval + * comparator is a proper total order, including across the infinite endpoints + * introduced by 519fc1bd9e9. + */ +PG_PROP(interval_comparison_total_order) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData e1; + StringInfoData e2; + StringInfoData q; + char *result; + bool failed; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&e1); + initStringInfo(&e2); + draw_interval(tc, &e1, true, false); /* infinities allowed */ + draw_interval(tc, &e2, true, false); + + /* exactly one of <, =, > must be true */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ( ((%s) < (%s))::int " + "+ ((%s) = (%s))::int " + "+ ((%s) > (%s))::int ) = 1", + e1.data, e2.data, + e1.data, e2.data, + e1.data, e2.data); + + result = pg_prop_sql_scalar(conn, q.data, &failed, &err); + + if (failed) + pg_prop_fail(tc, + "error evaluating interval total order\n" + " E1: %s\n E2: %s\n error: %s", + e1.data, e2.data, err ? err : "(unknown)"); + + if (result == NULL || strcmp(result, "t") != 0) + pg_prop_fail(tc, + "interval comparator is not a total order\n" + " E1: %s\n E2: %s\n (exactly one of <,=,> expected) result: %s", + e1.data, e2.data, result ? result : "NULL"); + + pg_free(result); + pfree(e1.data); + pfree(e2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. justify_interval_preserves_value + * + * justify_hours only redistributes the microsecond field into days without + * changing the value an interval represents. For a day/time-only interval E + * and a fixed reference timestamp R, (R + E) must equal (R + justify_hours(E)). + * + * E is gated to day/time-only (no months/years) and justify_hours is used + * rather than justify_interval, because justify_interval's day<->month + * redistribution genuinely changes results across month boundaries and is not + * a value-preserving identity under timestamp addition. + */ +PG_PROP(justify_interval_preserves_value) +{ + static const char *const ref = "'2000-06-15 12:00:00'::timestamp"; + pg_prop_conn *conn; + char *err = NULL; + StringInfoData e; + StringInfoData q; + char *result; + bool failed; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&e); + draw_interval(tc, &e, false, true); /* non-infinite, day/time only */ + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s + (%s)) = (%s + justify_hours(%s))", + ref, e.data, ref, e.data); + + result = pg_prop_sql_scalar(conn, q.data, &failed, &err); + + if (failed) + { + if (is_overflow(err)) + { + pg_prop_note(tc, "overflow (skipped): %s", err); + if (err) + pg_free(err); + pfree(e.data); + pfree(q.data); + pg_prop_disconnect(conn); + pg_prop_assume(tc, false); + return; /* unreachable: assume(false) longjmps */ + } + pg_prop_fail(tc, + "unexpected error on justify_hours identity\n" + " E: %s\n error: %s", + e.data, err ? err : "(unknown)"); + } + + if (result == NULL || strcmp(result, "t") != 0) + pg_prop_fail(tc, + "justify_hours changed the represented value\n" + " E: %s\n R + E != R + justify_hours(E); result: %s", + e.data, result ? result : "NULL"); + + pg_free(result); + pfree(e.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index efd8b1db5203f..a5af9e97b6832 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From db72ac669ee56f0f64841d36195ee09e0a5a5aae Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:14 -0400 Subject: [PATCH 141/338] prop: add interval field extraction and justification property tests (intervalparts) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_intervalparts.c | 343 +++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 347 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_intervalparts.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index a0ed117e889e9..433e968251203 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -156,7 +156,8 @@ OBJS = \ prop_timestamptz.o \ prop_tstzarith.o \ prop_tzconv.o \ - prop_interval.o + prop_interval.o \ + prop_intervalparts.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 2ab908be14cd5..0ce55df2eff24 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -154,6 +154,7 @@ prop_area_sources = files( 'prop_tstzarith.c', 'prop_tzconv.c', 'prop_interval.c', + 'prop_intervalparts.c', ) diff --git a/src/test/prop/prop_intervalparts.c b/src/test/prop/prop_intervalparts.c new file mode 100644 index 0000000000000..bf199afb28754 --- /dev/null +++ b/src/test/prop/prop_intervalparts.c @@ -0,0 +1,343 @@ +/*------------------------------------------------------------------------- + * + * prop_intervalparts.c + * Property-based tests for interval field extraction and justification + * (extract/date_part on intervals, justify_hours/justify_days, + * make_interval), in src/backend/utils/adt/timestamp.c. + * + * Interval field extraction and make_interval have precise relationships: the + * fields passed to make_interval are stored verbatim and extract() returns + * them back (within non-carrying ranges); justify_hours and justify_days + * normalize carries (24h -> 1 day, 30d -> 1 month) without changing the value + * an interval represents; and extract(epoch) of a time-only interval is the + * exact field-weighted sum of seconds. These properties check them. + * + * This is a distinct area from the existing "interval" tests (text round-trip, + * add/subtract inverse, total order, justify_hours value-preservation): here + * the subject is field extraction and the make_interval constructor. + * + * The oracle is the exact arithmetic of each field: make_interval stores the + * named fields, extract() reports them, justify redistributes them within + * fixed group sizes, and a day is exactly 86400 seconds for an interval's + * epoch. Each invariant is computed as a boolean in SQL and required "t". + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_intervalparts.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "intervalparts" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the expression errored or produced NULL on finite inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * 1. make_interval_extract_roundtrip + * + * For an interval built from a SINGLE field within a range that does not carry + * into a coarser field, extract() of that field returns the exact input. A + * field count is drawn per non-carrying range (year, month, day, hour, minute, + * second) and checked individually. Days do not auto-carry to months (that + * only happens under justify), so an arbitrary small day count is safe; hours + * stay in -23..23, minutes/seconds in -59..59. + */ +PG_PROP(make_interval_extract_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int y; + int mo; + int d; + int h; + int mi; + int s; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + y = (int) pg_prop_draw_int(tc, -5, 5); + mo = (int) pg_prop_draw_int(tc, -11, 11); + d = (int) pg_prop_draw_int(tc, -40, 40); + h = (int) pg_prop_draw_int(tc, -23, 23); + mi = (int) pg_prop_draw_int(tc, -59, 59); + s = (int) pg_prop_draw_int(tc, -59, 59); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT extract(year from make_interval(years => %d)) = %d " + "AND extract(month from make_interval(months => %d)) = %d " + "AND extract(day from make_interval(days => %d)) = %d " + "AND extract(hour from make_interval(hours => %d)) = %d " + "AND extract(minute from make_interval(mins => %d)) = %d " + "AND extract(second from make_interval(secs => %d)) = %d", + y, y, mo, mo, d, d, h, h, mi, mi, s, s); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. justify_hours_normalizes_hours + * + * justify_hours moves whole 24-hour groups into the day field, so after + * justify_hours the hour field is strictly within (-24, 24). + */ +PG_PROP(justify_hours_normalizes_hours) +{ + pg_prop_conn *conn; + char *err = NULL; + int h; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + h = (int) pg_prop_draw_int(tc, -200, 200); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(extract(hour from " + "justify_hours(make_interval(hours => %d)))) < 24", + h); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. justify_days_normalizes_days + * + * justify_days moves whole 30-day groups into the month field, so after + * justify_days the day field is strictly within (-30, 30). + */ +PG_PROP(justify_days_normalizes_days) +{ + pg_prop_conn *conn; + char *err = NULL; + int d; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + d = (int) pg_prop_draw_int(tc, -400, 400); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(extract(day from " + "justify_days(make_interval(days => %d)))) < 30", + d); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. justify_preserves_total_when_added + * + * justify_interval only redistributes the fields of an interval, so adding it + * to a fixed timestamp yields the same instant as adding its justified form. + * justify_hours redistributes 24-hour groups into days but never carries days + * into months, so for a plain TIMESTAMP (no DST, where 1 day == 24 hours + * exactly) adding the interval before or after justify_hours gives the same + * instant. (justify_interval is avoided here: its 30-day -> 1-month carry + * would legitimately move the result, since 30 days and 1 month are different + * calendar spans.) + */ +PG_PROP(justify_preserves_total_when_added) +{ + static const char *const ref = "TIMESTAMP '2000-06-15 00:00:00'"; + pg_prop_conn *conn; + char *err = NULL; + int d; + int h; + int m; + int s; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + d = (int) pg_prop_draw_int(tc, -40, 40); + h = (int) pg_prop_draw_int(tc, -50, 50); + m = (int) pg_prop_draw_int(tc, -120, 120); + s = (int) pg_prop_draw_int(tc, -120, 120); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s + make_interval(days => %d, hours => %d, " + "mins => %d, secs => %d)) " + "= (%s + justify_hours(make_interval(days => %d, " + "hours => %d, mins => %d, secs => %d)))", + ref, d, h, m, s, + ref, d, h, m, s); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. epoch_additive_for_time_fields + * + * For a time-only interval (days/hours/mins/secs, no months/years) the epoch + * is exactly the field-weighted sum of seconds, with a day counted as 86400 + * seconds (intervals, unlike timestamps across DST, treat a day as a fixed + * 86400s). + */ +PG_PROP(epoch_additive_for_time_fields) +{ + pg_prop_conn *conn; + char *err = NULL; + int d; + int h; + int m; + int s; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + d = (int) pg_prop_draw_int(tc, -40, 40); + h = (int) pg_prop_draw_int(tc, -50, 50); + m = (int) pg_prop_draw_int(tc, -120, 120); + s = (int) pg_prop_draw_int(tc, -120, 120); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT extract(epoch from make_interval(days => %d, " + "hours => %d, mins => %d, secs => %d)) " + "= (%d*86400 + %d*3600 + %d*60 + %d)::numeric", + d, h, m, s, + d, h, m, s); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. make_interval_additive + * + * Interval addition is componentwise, so make_interval(months => a) + + * make_interval(months => b) equals make_interval(months => a + b). + */ +PG_PROP(make_interval_additive) +{ + pg_prop_conn *conn; + char *err = NULL; + int a; + int b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (int) pg_prop_draw_int(tc, -15, 15); + b = (int) pg_prop_draw_int(tc, -15, 15); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT make_interval(months => %d) " + "+ make_interval(months => %d) " + "= make_interval(months => %d)", + a, b, a + b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. negate_make_interval + * + * Negating an interval negates every field, so -make_interval(y, mo, d, h) + * equals make_interval with all those fields negated. + */ +PG_PROP(negate_make_interval) +{ + pg_prop_conn *conn; + char *err = NULL; + int y; + int mo; + int d; + int h; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + y = (int) pg_prop_draw_int(tc, -5, 5); + mo = (int) pg_prop_draw_int(tc, -15, 15); + d = (int) pg_prop_draw_int(tc, -40, 40); + h = (int) pg_prop_draw_int(tc, -50, 50); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (- make_interval(years => %d, months => %d, " + "days => %d, hours => %d)) " + "= make_interval(years => %d, months => %d, " + "days => %d, hours => %d)", + y, mo, d, h, + -y, -mo, -d, -h); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index a5af9e97b6832..950cd371b4141 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From b0a9a0cf28cf45af7d26b1423d1bc97747e18024 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:14 -0400 Subject: [PATCH 142/338] prop: add interval comparison and arithmetic property tests (intervalcmp) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_intervalcmp.c | 371 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 375 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_intervalcmp.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 433e968251203..5aa3ef9db01b0 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -157,7 +157,8 @@ OBJS = \ prop_tstzarith.o \ prop_tzconv.o \ prop_interval.o \ - prop_intervalparts.o + prop_intervalparts.o \ + prop_intervalcmp.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 0ce55df2eff24..45ef563d6402e 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -155,6 +155,7 @@ prop_area_sources = files( 'prop_tzconv.c', 'prop_interval.c', 'prop_intervalparts.c', + 'prop_intervalcmp.c', ) diff --git a/src/test/prop/prop_intervalcmp.c b/src/test/prop/prop_intervalcmp.c new file mode 100644 index 0000000000000..e1c0c766c722d --- /dev/null +++ b/src/test/prop/prop_intervalcmp.c @@ -0,0 +1,371 @@ +/*------------------------------------------------------------------------- + * + * prop_intervalcmp.c + * Property-based tests for interval comparison, ordering, and arithmetic. + * + * This area targets the interval type's algebra distinct from parsing: the + * +/- operators, scalar multiply and divide, negation, and the total order + * (interval_cmp / <,=,>) plus how it interacts with timestamp addition. The + * oracle is the type's own algebra: with day/hour/minute fields only (no + * month ambiguity), interval add/subtract/negate are exact and scalar * and / + * by small integers stay exact, so these identities must hold with no slack. + * Any "f" result is a bug in interval arithmetic or comparison, not rounding. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_intervalcmp.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "intervalcmp" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the arithmetic errored or produced NULL on finite inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Draw an interval field count: days -100..100, hours -50..50, mins -120..120. + * Each returns a plain int the caller substitutes into make_interval(). The + * ranges span zero and the boundaries the engine favours (min/max/0/+-1). + */ +static int +draw_days(pg_prop_tc *tc) +{ + return (int) pg_prop_draw_int(tc, -100, 100); +} + +static int +draw_hours(pg_prop_tc *tc) +{ + return (int) pg_prop_draw_int(tc, -50, 50); +} + +static int +draw_mins(pg_prop_tc *tc) +{ + return (int) pg_prop_draw_int(tc, -120, 120); +} + +/* + * add_commutative: make_interval(days => a) + make_interval(days => b) equals + * make_interval(days => b) + make_interval(days => a). Day-only intervals so + * addition carries no month/day/time ambiguity; interval + interval must + * commute exactly. + */ +PG_PROP(add_commutative) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = draw_days(tc); + b = draw_days(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (make_interval(days => (%d)) " + "+ make_interval(days => (%d))) " + "= (make_interval(days => (%d)) " + "+ make_interval(days => (%d)))", + a, b, b, a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * negate_inverse: I + (-I) = interval '0' for I = make_interval(days, hours). + * Negation then addition of a value and its inverse must collapse exactly to + * the zero interval. + */ +PG_PROP(negate_inverse) +{ + pg_prop_conn *conn; + char *err = NULL; + int d, + h; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + d = draw_days(tc); + h = draw_hours(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (make_interval(days => (%d), hours => (%d)) " + "+ (- make_interval(days => (%d), hours => (%d)))) " + "= interval '0'", + d, h, d, h); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * scalar_mul_distributes: 2 * I = I + I. Multiplying an interval by the + * scalar 2 must equal adding it to itself; a carry bug in interval_mul would + * break this. + */ +PG_PROP(scalar_mul_distributes) +{ + pg_prop_conn *conn; + char *err = NULL; + int d, + h, + m; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + d = draw_days(tc); + h = draw_hours(tc); + m = draw_mins(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (2 * make_interval(days => (%d), hours => (%d), " + "mins => (%d))) " + "= (make_interval(days => (%d), hours => (%d), " + "mins => (%d)) " + "+ make_interval(days => (%d), hours => (%d), " + "mins => (%d)))", + d, h, m, d, h, m, d, h, m); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * div_mul_roundtrip: (I * 4) / 4 = I for I built from days and hours only. + * Days and hours keep every intermediate integral, so multiplying by 4 then + * dividing by 4 round-trips exactly with no fractional carry across fields. + */ +PG_PROP(div_mul_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int d, + h; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + d = draw_days(tc); + h = draw_hours(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((make_interval(days => (%d), hours => (%d)) " + "* 4) / 4) " + "= make_interval(days => (%d), hours => (%d))", + d, h, d, h); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * total_order: for two all-days intervals A and B, exactly one of A < B, + * A = B, A > B holds. Day-only intervals make the comparison well-defined + * (no month/day/time field crossing), so the three relations must partition. + */ +PG_PROP(total_order) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = draw_days(tc); + b = draw_days(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (CASE WHEN A < B THEN 1 ELSE 0 END) " + "+ (CASE WHEN A = B THEN 1 ELSE 0 END) " + "+ (CASE WHEN A > B THEN 1 ELSE 0 END) = 1 " + "FROM (SELECT make_interval(days => (%d)) AS A, " + "make_interval(days => (%d)) AS B) s", + a, b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * add_to_timestamp_order: for day-only intervals A, B with A.days < B.days + * (so A < B as intervals), ts + A < ts + B at a fixed timestamp. Monotonicity + * of timestamp + interval must respect the interval order. A and B are built + * so A's day count is strictly less than B's. + */ +PG_PROP(add_to_timestamp_order) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = draw_days(tc); + b = draw_days(tc); + + /* force a strictly less than b */ + if (a > b) + { + int t = a; + + a = b; + b = t; + } + if (a == b) + b = b + 1; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (timestamp '2000-06-15 12:00:00' " + "+ make_interval(days => (%d))) " + "< (timestamp '2000-06-15 12:00:00' " + "+ make_interval(days => (%d)))", + a, b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * zero_identity: I + interval '0' = I. The zero interval is the additive + * identity, so adding it must return the operand unchanged. + */ +PG_PROP(zero_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + int d, + h, + m; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + d = draw_days(tc); + h = draw_hours(tc); + m = draw_mins(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (make_interval(days => (%d), hours => (%d), " + "mins => (%d)) + interval '0') " + "= make_interval(days => (%d), hours => (%d), " + "mins => (%d))", + d, h, m, d, h, m); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * mul_zero: I * 0 = interval '0'. Scaling any interval by the scalar 0 must + * annihilate every field to the zero interval. + */ +PG_PROP(mul_zero) +{ + pg_prop_conn *conn; + char *err = NULL; + int d, + h, + m; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + d = draw_days(tc); + h = draw_hours(tc); + m = draw_mins(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (make_interval(days => (%d), hours => (%d), " + "mins => (%d)) * 0) = interval '0'", + d, h, m); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 950cd371b4141..1d290a879b57e 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 23d86c27eb6752ed110cb15f4ca73074518b4865 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:15 -0400 Subject: [PATCH 143/338] prop: add interval text and ISO 8601 I/O round-trips property tests (intervalio) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_intervalio.c | 312 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 316 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_intervalio.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 5aa3ef9db01b0..88829fc85e5f4 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -158,7 +158,8 @@ OBJS = \ prop_tzconv.o \ prop_interval.o \ prop_intervalparts.o \ - prop_intervalcmp.o + prop_intervalcmp.o \ + prop_intervalio.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 45ef563d6402e..dff29de04387e 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -156,6 +156,7 @@ prop_area_sources = files( 'prop_interval.c', 'prop_intervalparts.c', 'prop_intervalcmp.c', + 'prop_intervalio.c', ) diff --git a/src/test/prop/prop_intervalio.c b/src/test/prop/prop_intervalio.c new file mode 100644 index 0000000000000..4ba81325d8274 --- /dev/null +++ b/src/test/prop/prop_intervalio.c @@ -0,0 +1,312 @@ +/*------------------------------------------------------------------------- + * + * prop_intervalio.c + * Property-based tests for interval text I/O (ISO 8601 and verbose). + * + * The oracle is round-trip identity through interval's textual + * representation: an interval rendered to text and parsed back must equal + * itself, and an ISO 8601 literal ('P1DT2H') must denote the same value + * make_interval() builds from the same fields. All intervals here are + * month-free (days/hours/mins/secs only), so epoch extraction is exact + * (a day is 86400s) and reconstruction from fields is lossless. A broken + * round-trip is a bug in interval_in/interval_out or the ISO/verbose + * parsing, distinct from interval arithmetic, parts, and comparison. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_intervalio.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "intervalio" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means interval I/O errored or produced NULL on valid inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Draw a month-free make_interval(...) expression into buf from small field + * counts: days, hours, mins, secs. Months are deliberately excluded so epoch + * is exact and field reconstruction is lossless. Returns the drawn fields + * through the out-params so callers can also build an epoch expectation. + */ +static void +draw_make_interval(pg_prop_tc *tc, StringInfo buf, + int *d, int *h, int *m, int *s) +{ + *d = (int) pg_prop_draw_int(tc, -5, 5); + *h = (int) pg_prop_draw_int(tc, -24, 24); + *m = (int) pg_prop_draw_int(tc, -59, 59); + *s = (int) pg_prop_draw_int(tc, -59, 59); + + appendStringInfo(buf, + "make_interval(days=>(%d), hours=>(%d), mins=>(%d), secs=>(%d))", + *d, *h, *m, *s); +} + +/* + * Property 1: text round-trip. Rendering an interval to text and parsing it + * back must yield the original interval. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData iv, + q; + int d, + h, + m, + s; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&iv); + draw_make_interval(tc, &iv, &d, &h, &m, &s); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s)::text::interval = %s)", + iv.data, iv.data); + + require_true(tc, conn, q.data); + + pfree(iv.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 2: ISO 8601 round-trip. An ISO interval literal must denote the + * same value make_interval() builds from the same days/hours fields. + */ +PG_PROP(iso8601_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int d, + h; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + d = (int) pg_prop_draw_int(tc, 0, 9); + h = (int) pg_prop_draw_int(tc, 0, 23); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('P%dDT%dH'::interval = make_interval(days=>(%d), hours=>(%d)))", + d, h, d, h); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 3: justify_hours output is stable. Rendering justify_hours(I) to + * text and parsing it back must equal justify_hours(I). + */ +PG_PROP(justify_text_stable) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData iv, + q; + int d, + h, + m, + s; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&iv); + draw_make_interval(tc, &iv, &d, &h, &m, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (justify_hours(%s)::text::interval = justify_hours(%s))", + iv.data, iv.data); + + require_true(tc, conn, q.data); + + pfree(iv.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 4: seconds literal round-trip. interval ' seconds' must equal + * make_interval(secs=>) for n in -3600..3600. + */ +PG_PROP(seconds_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, -3600, 3600); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (interval '%d seconds' = make_interval(secs=>(%d)))", + n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 5: negative interval round-trip. With a strictly negative hours + * field, rendering to text and parsing back must yield the original interval. + */ +PG_PROP(negative_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData iv, + q; + int h; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + h = (int) pg_prop_draw_int(tc, -48, -1); + + initStringInfo(&iv); + appendStringInfo(&iv, "make_interval(hours=>(%d))", h); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s)::text::interval = %s)", + iv.data, iv.data); + + require_true(tc, conn, q.data); + + pfree(iv.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 6: epoch is exact for month-free intervals. extract(epoch from I) + * must equal d*86400 + h*3600 + m*60 + s for the drawn fields (an interval day + * is exactly 86400s; no months means no variable-length month to make this + * approximate). + */ +PG_PROP(extract_epoch_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData iv, + q; + int d, + h, + m, + s; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&iv); + draw_make_interval(tc, &iv, &d, &h, &m, &s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (extract(epoch from %s) = (%d::bigint*86400 + %d::bigint*3600 + %d::bigint*60 + %d::bigint))", + iv.data, d, h, m, s); + + require_true(tc, conn, q.data); + + pfree(iv.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 7: the zero interval. interval '0' must equal make_interval() and + * must equal the difference of an interval with itself. + */ +PG_PROP(zero_interval) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT (interval '0' = make_interval() " + "AND interval '0' = (make_interval(hours=>1) - make_interval(hours=>1)))"); + + pg_prop_disconnect(conn); +} + +/* + * Property 8: ISO zero output parses to the zero interval. 'P0D'::interval + * must equal interval '0'. + */ +PG_PROP(iso_output_parse) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, "SELECT ('P0D'::interval = interval '0')"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 1d290a879b57e..332679b5bb435 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From e46923281092d53807b1b9cedc04d1950dcd78d2 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:15 -0400 Subject: [PATCH 144/338] prop: add tochar property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_tochar.c | 321 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 325 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_tochar.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 88829fc85e5f4..3e81fc6d4660c 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -159,7 +159,8 @@ OBJS = \ prop_interval.o \ prop_intervalparts.o \ prop_intervalcmp.o \ - prop_intervalio.o + prop_intervalio.o \ + prop_tochar.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index dff29de04387e..4b09b40f24c6c 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -157,6 +157,7 @@ prop_area_sources = files( 'prop_intervalparts.c', 'prop_intervalcmp.c', 'prop_intervalio.c', + 'prop_tochar.c', ) diff --git a/src/test/prop/prop_tochar.c b/src/test/prop/prop_tochar.c new file mode 100644 index 0000000000000..571d5235a6e0b --- /dev/null +++ b/src/test/prop/prop_tochar.c @@ -0,0 +1,321 @@ +/*------------------------------------------------------------------------- + * + * prop_tochar.c + * Property-based tests for to_char / to_number / to_date formatting + * (src/backend/utils/adt/formatting.c). + * + * to_char/to_number/to_date in formatting.c have had ~18 fixes (roman + * numerals, overflow, rounding, field parsing); these check round-trips + * (format then parse back) and digit/width invariants. + * + * The oracle here is round-trip and width/shape: a value formatted to text + * and parsed back must equal the original, and a fixed-width mask must yield + * a result of the mask's width. Each invariant is computed as a boolean in + * the query and the scalar result is required to be "t". + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_tochar.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "tochar" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * True when a SQL error is a legitimate out-of-range/overflow/unsupported + * condition rather than a wrong-results bug. The bounded input ranges below + * are chosen so this should not fire, but it is kept so that any genuine + * overflow (e.g. a too-wide to_number) is skipped rather than misreported. + */ +static bool +is_overflow(const char *errmsg) +{ + if (errmsg == NULL) + return false; + return strstr(errmsg, "out of range") != NULL || + strstr(errmsg, "overflow") != NULL || + strstr(errmsg, "cannot") != NULL; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the formatting path errored or produced NULL on valid + * inputs, which is itself a bug -- so either is a failure, reported with the + * SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Like require_true, but a SQL error that is a legitimate out-of-range or + * overflow condition discards the case rather than failing it. Used by the + * to_number round-trip, where a pathological drawn value could in principle + * exceed the parse range; the bounded generators make this rare, but routing + * through here keeps a genuine overflow from being misreported as a bug. + */ +static void +require_true_or_skip(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + bool overflow = is_overflow(err); + + if (err) + pg_free(err); + if (overflow) + pg_prop_assume(tc, false); /* legitimate overflow: skip */ + pg_prop_fail(tc, "SQL error for: %s", sql); + } + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); +} + +/* + * 1. number_roundtrip_via_to_number + * + * Format an integer with a fixed-width digit mask in fill mode (FM, no + * padding) then parse it back: the value must survive the round-trip. FM + * suppresses the leading spaces that would otherwise confuse to_number. The + * range is bounded well inside the mask width so no overflow can occur. + */ +PG_PROP(number_roundtrip_via_to_number) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, -999999999, 999999999); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_number(to_char(%lld, 'FM999999999999'), " + "'999999999999')::bigint = %lld", + (long long) n, (long long) n); + + require_true_or_skip(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. roman_numeral_in_range_roundtrips + * + * Roman numerals are defined for 1..3999. There is no reliable RN parser to + * round-trip against, so this is a sound shape check instead: to_char(n, + * 'FMRN') must be composed only of roman digits (I V X L C D M) and have a + * length between 1 and 15. This guards the roman overflow/format fixes + * without needing an RN parser. + */ +PG_PROP(roman_numeral_in_range_roundtrips) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, 1, 3999); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_char(%lld, 'FMRN') ~ '^[IVXLCDM]+$' " + "AND length(to_char(%lld, 'FMRN')) BETWEEN 1 AND 15", + (long long) n, (long long) n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. date_roundtrip_via_to_date + * + * Formatting a date to text and parsing it back must return the same date. + * The day is capped at 28 so every month/year combination is valid and no + * invalid-date error can occur. + */ +PG_PROP(date_roundtrip_via_to_date) +{ + pg_prop_conn *conn; + char *err = NULL; + int year; + int mon; + int day; + char iso[16]; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + year = (int) pg_prop_draw_int(tc, 1900, 2100); + mon = (int) pg_prop_draw_int(tc, 1, 12); + day = (int) pg_prop_draw_int(tc, 1, 28); + + snprintf(iso, sizeof(iso), "%04d-%02d-%02d", year, mon, day); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_date(to_char(DATE '%s', 'YYYY-MM-DD'), " + "'YYYY-MM-DD') = DATE '%s'", + iso, iso); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. to_char_number_digit_count + * + * The number of characters in to_char(n, 'FM999999999999') must equal the + * number of digits in n's decimal representation. n is drawn in 1..999999999 + * (0 is excluded because the "9" mask has no significant digit to show for a + * leading zero; its width behaviour is checked separately by the 0-mask + * property). + */ +PG_PROP(to_char_number_digit_count) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, 1, 999999999); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(to_char(%lld, 'FM999999999999')) " + "= length(%lld::text)", + (long long) n, (long long) n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. leading_zero_mask_fixed_width + * + * The "0" mask forces leading zeros; FM trims padding spaces but not those + * zeros, so to_char(n, 'FM0000') always yields exactly 4 characters for any + * n in 0..9999. Exercises the zero-padding path. + */ +PG_PROP(leading_zero_mask_fixed_width) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, 0, 9999); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(to_char(%lld, 'FM0000')) = 4", + (long long) n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. time_roundtrip + * + * Formatting a parsed time back to text must yield the original. h/m/s are + * drawn in valid ranges so no invalid-time error can occur; the round-trip + * goes through to_timestamp with an HH24:MI:SS mask. + */ +PG_PROP(time_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int h; + int m; + int s; + char hms[16]; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + h = (int) pg_prop_draw_int(tc, 0, 23); + m = (int) pg_prop_draw_int(tc, 0, 59); + s = (int) pg_prop_draw_int(tc, 0, 59); + + snprintf(hms, sizeof(hms), "%02d:%02d:%02d", h, m, s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_char(to_timestamp('%s', 'HH24:MI:SS'), " + "'HH24:MI:SS') = '%s'", + hms, hms); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 332679b5bb435..a138b2048648b 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 2933ae67711e3333ff17a2f0c7beb24079cde043 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:15 -0400 Subject: [PATCH 145/338] prop: add to_char/to_number/to_date formatting round-trips property tests (toformat) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_toformat.c | 338 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 342 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_toformat.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 3e81fc6d4660c..0e075e56efe2b 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -160,7 +160,8 @@ OBJS = \ prop_intervalparts.o \ prop_intervalcmp.o \ prop_intervalio.o \ - prop_tochar.o + prop_tochar.o \ + prop_toformat.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 4b09b40f24c6c..a368b167c0451 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -158,6 +158,7 @@ prop_area_sources = files( 'prop_intervalcmp.c', 'prop_intervalio.c', 'prop_tochar.c', + 'prop_toformat.c', ) diff --git a/src/test/prop/prop_toformat.c b/src/test/prop/prop_toformat.c new file mode 100644 index 0000000000000..fec577f7c94b1 --- /dev/null +++ b/src/test/prop/prop_toformat.c @@ -0,0 +1,338 @@ +/*------------------------------------------------------------------------- + * + * prop_toformat.c + * Property-based tests for to_char / to_number / to_date / to_timestamp + * formatting round-trips (src/backend/utils/adt/formatting.c). + * + * These check that a value formatted to text with a numeric/date mask and + * parsed back equals the original, and that fixed-width and sign masks shape + * the output the way the mask spells it. A failure means a numeric digit + * mask, a zero-pad width, a date/time field, or a sign was emitted or parsed + * wrongly -- a formatting.c bug, not a rounding artefact. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_toformat.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "toformat" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * True when a SQL error is a legitimate out-of-range/overflow/unsupported + * condition rather than a wrong-results bug. The bounded input ranges below + * are chosen so this should not fire, but it is kept so that any genuine + * overflow (e.g. a to_number that cannot represent the formatted width) is + * skipped rather than misreported. + */ +static bool +is_overflow(const char *errmsg) +{ + if (errmsg == NULL) + return false; + return strstr(errmsg, "out of range") != NULL || + strstr(errmsg, "overflow") != NULL || + strstr(errmsg, "cannot") != NULL; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A legitimate + * out-of-range/overflow error discards the case (the mask or value could not + * represent the number); any other error, a NULL, or an "f" is a failure. + * Copied from the require_true_or_skip style in prop_tochar.c / prop_float.c. + */ +static void +require_true_or_skip(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + bool overflow = is_overflow(err); + + if (err) + pg_free(err); + if (overflow) + pg_prop_assume(tc, false); /* legitimate overflow: skip */ + pg_prop_fail(tc, "SQL error for: %s", sql); + } + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); +} + +/* + * 1. to_number_to_char_roundtrip + * + * For n in 0..999999, parsing back the FM-formatted value recovers n: + * to_number(to_char(n, 'FM999999'), '999999') = n. FM strips the padding + * that would otherwise leave leading spaces for to_number to choke on. + */ +PG_PROP(to_number_to_char_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, 0, 999999); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_number(to_char((%lld), 'FM999999'), " + "'999999') = (%lld)", + (long long) n, (long long) n); + + require_true_or_skip(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. to_char_pads_width + * + * A '0' mask zero-pads to a fixed width, so for n in 0..999999 the formatted + * text is always exactly 6 characters: length(to_char(n, '000000')) = 6. + */ +PG_PROP(to_char_pads_width) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, 0, 999999); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(to_char((%lld), 'FM000000')) = 6", + (long long) n); + + require_true_or_skip(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. to_date_roundtrip + * + * A zero-padded ISO date parsed by to_date and reformatted with the same mask + * returns the identical string, for y 2000..2030, mo 1..12, d 1..28 (always a + * valid day). Components are zero-padded in C to match YYYY-MM-DD exactly. + */ +PG_PROP(to_date_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 y, + mo, + d; + char date[16]; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + y = pg_prop_draw_int(tc, 2000, 2030); + mo = pg_prop_draw_int(tc, 1, 12); + d = pg_prop_draw_int(tc, 1, 28); + + snprintf(date, sizeof(date), "%04d-%02d-%02d", + (int) y, (int) mo, (int) d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_char(to_date('%s', 'YYYY-MM-DD'), " + "'YYYY-MM-DD') = '%s'", + date, date); + + require_true_or_skip(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. to_timestamp_roundtrip + * + * A zero-padded timestamp parsed by to_timestamp and reformatted with the + * same mask returns the identical string. Day capped at 28, hour 0..23, + * minute/second 0..59, so every drawn value is valid and no DST/overflow + * ambiguity arises. + */ +PG_PROP(to_timestamp_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 y, + mo, + d, + h, + mi, + s; + char ts[32]; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + y = pg_prop_draw_int(tc, 2000, 2030); + mo = pg_prop_draw_int(tc, 1, 12); + d = pg_prop_draw_int(tc, 1, 28); + h = pg_prop_draw_int(tc, 0, 23); + mi = pg_prop_draw_int(tc, 0, 59); + s = pg_prop_draw_int(tc, 0, 59); + + snprintf(ts, sizeof(ts), "%04d-%02d-%02d %02d:%02d:%02d", + (int) y, (int) mo, (int) d, (int) h, (int) mi, (int) s); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_char(to_timestamp('%s', " + "'YYYY-MM-DD HH24:MI:SS'), " + "'YYYY-MM-DD HH24:MI:SS') = '%s'", + ts, ts); + + require_true_or_skip(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. to_char_sign + * + * The S format forces an explicit sign; the FMS variant then formats a signed + * integer that to_number parses back exactly, for n in -999999..999999: + * to_number(to_char(n, 'FMS999999'), 'S999999') = n. This exercises the sign + * emit/parse path for both signs across zero. + */ +PG_PROP(to_char_sign) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, -999999, 999999); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_number(to_char((%lld), 'FMS999999'), " + "'S999999') = (%lld)", + (long long) n, (long long) n); + + require_true_or_skip(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. to_char_numeric_scale + * + * A three-decimal mask formats a numeric rounded to three places, and parsing + * that back equals round(x, 3): to_number(to_char(x, 'FM999999.000'), + * '999999.000') = round(x, 3). x is drawn with up to six integer digits so + * the mask always has room; FM strips the leading padding before to_number. + */ +PG_PROP(to_char_numeric_scale) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 ipart, + fpart; + bool neg; + char x[32]; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* up to 6 integer digits (fits '999999') and 4 fractional digits */ + ipart = pg_prop_draw_int(tc, 0, 999999); + fpart = pg_prop_draw_int(tc, 0, 9999); + neg = pg_prop_draw_bool(tc); + + snprintf(x, sizeof(x), "%s%d.%04d", + neg ? "-" : "", (int) ipart, (int) fpart); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_number(to_char((%s)::numeric, " + "'FM999999.000'), '999999.000') = " + "round((%s)::numeric, 3)", + x, x); + + require_true_or_skip(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 7. to_date_month_name + * + * Formatting a parsed date with 'MM' yields the zero-padded month number, for + * mo 1..12: to_char(to_date('2020--15', ...), 'MM') = lpad(mo::text, 2, + * '0'). Isolates the month field through the parse/format round-trip against + * an independent oracle built in SQL. + */ +PG_PROP(to_date_month_name) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 mo; + char date[16]; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + mo = pg_prop_draw_int(tc, 1, 12); + + snprintf(date, sizeof(date), "2020-%02d-15", (int) mo); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT to_char(to_date('%s', 'YYYY-MM-DD'), 'MM') = " + "lpad((%lld)::text, 2, '0')", + date, (long long) mo); + + require_true_or_skip(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index a138b2048648b..f292d9d39fb5a 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From e934aefe2dca6ebd7f250a0b06b2b7c57e9b94c2 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:15 -0400 Subject: [PATCH 146/338] prop: add miscellaneous scalar function contracts property tests (scalarfn) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_scalarfn.c | 281 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 285 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_scalarfn.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 0e075e56efe2b..e2bd965af80da 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -161,7 +161,8 @@ OBJS = \ prop_intervalcmp.o \ prop_intervalio.o \ prop_tochar.o \ - prop_toformat.o + prop_toformat.o \ + prop_scalarfn.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index a368b167c0451..4b880254dbe84 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -159,6 +159,7 @@ prop_area_sources = files( 'prop_intervalio.c', 'prop_tochar.c', 'prop_toformat.c', + 'prop_scalarfn.c', ) diff --git a/src/test/prop/prop_scalarfn.c b/src/test/prop/prop_scalarfn.c new file mode 100644 index 0000000000000..0ecd18e5166fc --- /dev/null +++ b/src/test/prop/prop_scalarfn.c @@ -0,0 +1,281 @@ +/*------------------------------------------------------------------------- + * + * prop_scalarfn.c + * Property-based tests for miscellaneous scalar functions. + * + * A grab-bag of scalar functions with clean algebraic contracts not covered + * by the type-specific areas: num_nulls/num_nonnulls, mod/div sign rules, the + * array form of width_bucket, the degrees/radians inverse, bit_count, and the + * gcd*lcm identity. Each property is a boolean the backend must answer "t". + * + * Tier B: runs SQL against the cluster the TAP driver started; see pg_prop.h + * and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_scalarfn.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "scalarfn" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* Run a boolean SELECT and require it to return "t". */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* Emit an int literal, parenthesizing negatives so a cast binds correctly. */ +static void +append_int(StringInfo buf, int n) +{ + if (n < 0) + appendStringInfo(buf, "(%d)", n); + else + appendStringInfo(buf, "%d", n); +} + +/* An int argument that is NULL ~25% of the time. */ +static void +append_int_or_null(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 3) == 0) + appendStringInfoString(buf, "NULL::int"); + else + append_int(buf, (int) pg_prop_draw_int(tc, -20, 20)); +} + +/* num_nulls(a,b,c) + num_nonnulls(a,b,c) = 3. */ +PG_PROP(num_nulls_nonnulls) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + { + StringInfoData args; + + /* same three args to both functions so the identity holds */ + initStringInfo(&args); + append_int_or_null(tc, &args); + appendStringInfoString(&args, ", "); + append_int_or_null(tc, &args); + appendStringInfoString(&args, ", "); + append_int_or_null(tc, &args); + + appendStringInfo(&q, + "SELECT num_nulls(%s) + num_nonnulls(%s) = 3 " + "AND num_nulls(1,2,3) = 0 " + "AND num_nonnulls(1,2,3) = 3 " + "AND num_nulls(NULL::int, 1, NULL::int, 2) = 2", + args.data, args.data); + pfree(args.data); + } + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* mod(a,b) has the sign of a (or is zero): sign(mod(a,b)) IN (0, sign(a)). */ +PG_PROP(mod_sign) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (int) pg_prop_draw_int(tc, -1000, 1000); + b = (int) pg_prop_draw_int(tc, 1, 1000); + if (pg_prop_draw_bool(tc)) + b = -b; + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT sign(mod("); + append_int(&q, a); + appendStringInfoString(&q, ", "); + append_int(&q, b); + appendStringInfoString(&q, ")) IN (0, sign("); + append_int(&q, a); + appendStringInfoString(&q, "))"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* div(a,b) = trunc(a::numeric / b) for b <> 0. */ +PG_PROP(div_func) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (int) pg_prop_draw_int(tc, -1000, 1000); + b = (int) pg_prop_draw_int(tc, 1, 1000); + if (pg_prop_draw_bool(tc)) + b = -b; + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT div("); + append_int(&q, a); + appendStringInfoString(&q, "::numeric, "); + append_int(&q, b); + appendStringInfoString(&q, ") = trunc("); + append_int(&q, a); + appendStringInfoString(&q, "::numeric / "); + append_int(&q, b); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* width_bucket(array form) is monotone: v1 <= v2 => bucket(v1) <= bucket(v2). */ +PG_PROP(width_bucket_array_monotone) +{ + pg_prop_conn *conn; + char *err = NULL; + int v1, + v2, + t; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + v1 = (int) pg_prop_draw_int(tc, -10, 40); + v2 = (int) pg_prop_draw_int(tc, -10, 40); + if (v1 > v2) + { + t = v1; + v1 = v2; + v2 = t; + } + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT width_bucket("); + append_int(&q, v1); + appendStringInfoString(&q, ", ARRAY[0,10,20,30]) <= width_bucket("); + append_int(&q, v2); + appendStringInfoString(&q, ", ARRAY[0,10,20,30])"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* radians(degrees(x)) = x within tolerance. */ +PG_PROP(degrees_radians_inverse) +{ + pg_prop_conn *conn; + char *err = NULL; + int whole, + frac; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + whole = (int) pg_prop_draw_int(tc, -5, 5); + frac = (int) pg_prop_draw_int(tc, 0, 9); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(radians(degrees('%d.%d'::float8)) " + "- '%d.%d'::float8) < 1e-9", + whole, frac, whole, frac); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* bit_count of 0xff is 8; of a zero byte is 0. */ +PG_PROP(bit_count_bytea) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT bit_count(decode('ff', 'hex')) = 8 " + "AND bit_count(decode('00', 'hex')) = 0 " + "AND bit_count(decode('ffff', 'hex')) = 16"); + + pg_prop_disconnect(conn); +} + +/* gcd(a,b) * lcm(a,b) = a*b for positive a, b. */ +PG_PROP(gcd_lcm_product) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = (int) pg_prop_draw_int(tc, 1, 100); + b = (int) pg_prop_draw_int(tc, 1, 100); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT gcd(%d, %d) * lcm(%d, %d) = %d * %d", + a, b, a, b, a, b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index f292d9d39fb5a..087bf5c8401c6 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 5997b1c8b0495dcea3289ec4c220a328850ba71b Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:15 -0400 Subject: [PATCH 147/338] prop: add hashfn property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_hashfn.c | 333 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 337 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_hashfn.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index e2bd965af80da..ca19c2b64c3a2 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -162,7 +162,8 @@ OBJS = \ prop_intervalio.o \ prop_tochar.o \ prop_toformat.o \ - prop_scalarfn.o + prop_scalarfn.o \ + prop_hashfn.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 4b880254dbe84..bcf3eaf85e18a 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -160,6 +160,7 @@ prop_area_sources = files( 'prop_tochar.c', 'prop_toformat.c', 'prop_scalarfn.c', + 'prop_hashfn.c', ) diff --git a/src/test/prop/prop_hashfn.c b/src/test/prop/prop_hashfn.c new file mode 100644 index 0000000000000..e03dcacc2a3ca --- /dev/null +++ b/src/test/prop/prop_hashfn.c @@ -0,0 +1,333 @@ +/*------------------------------------------------------------------------- + * + * prop_hashfn.c + * Property-based tests for the type hash functions. + * + * The per-type hash functions (hashtext, hashint4, hashint8, hash_numeric, + * and the seeded hashtextextended) must be deterministic and consistent with + * equality: equal values must hash equally. That contract is what hash joins + * and hash indexes rely on -- a value hashed on one side of a join, or on + * insert into a hash index, must land in the same bucket as an equal value + * hashed later, or lookups silently miss. These properties check the two + * halves of that contract via the hash_* SQL-exposed functions: determinism + * (the same input always hashes the same) and equal-values-hash-equally + * (independently constructed equal values, and numerically-equal numerics of + * different scale, hash identically). + * + * The oracle is the hash function's own stability and SQL equality: equality + * is computed by the backend, and the hashes must agree with it with no + * tolerance. A deterministic self-comparison catches uninitialized-byte and + * non-stable-state bugs; an equal-inputs comparison catches a hash that + * disagrees with equality. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_hashfn.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "hashfn" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw 0..16 characters of lowercase text from "a".."z" into buf as a bare + * string (no quoting). Letters only, so the literal never needs more than + * single-quote doubling (which it never triggers), keeping the generated SQL + * trivially well-formed. + */ +static void +draw_text(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abcdefghijklmnopqrstuvwxyz"; + int n = (int) pg_prop_draw_int(tc, 0, 16); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 25)]); +} + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabet never contains a quote, but do it defensively + * so the generated SQL is always well-formed. Modelled on prop_like.c. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the hash errored or produced NULL on a finite input, + * which is itself a bug -- so either is a failure, reported with the SQL. + * Modelled on prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Append v as a SQL int8 literal. INT64_MIN cannot be written as a plain + * negative literal (the lexer reads the magnitude 9223372036854775808 first, + * which overflows int8 before the minus is applied), so emit it as an + * expression that stays in range. + */ +static void +append_int8_literal(StringInfo buf, int64 v) +{ + if (v == INT64_MIN) + appendStringInfoString(buf, "(-9223372036854775807 - 1)"); + else + appendStringInfo(buf, "%lld", (long long) v); +} + +/* + * hashtext is deterministic: hashing the same string twice gives the same + * value. Trivial-looking, but a non-stable hash (uninitialized padding, a + * stray global) breaks it, and that would corrupt every hash lookup. + */ +PG_PROP(hashtext_deterministic) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT hashtext("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = hashtext("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Equal inputs hash equally. S and the server-side reconstruction + * substr(S,1,k) || substr(S,k+1) are the same string (the split-and-rejoin + * is the identity for any k), so their hashes must match. The split point k + * is clamped with least(k, length(S)) so the reconstruction equals S exactly + * for any drawn k, and the substrings are taken server-side so no C-side + * splitting can disagree with the backend's. This exercises the headline + * equal-values-hash-equally contract on text. + */ +PG_PROP(hashtext_equal_inputs_equal_hashes) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + int k; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + /* split point; clamped server-side via least(k, length(S)) */ + k = (int) pg_prop_draw_int(tc, 0, 20); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT hashtext("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = hashtext(substr("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", 1, least(%d, length(", k); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "))) || substr("); + append_sql_literal(&q, s.data); + appendStringInfo(&q, ", least(%d, length(", k); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) + 1))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * hashint4 is deterministic across the full INT32 range: the same int4 always + * hashes the same. The widest bounds are passed so the engine favours the + * INT32 extremes, where a sign/width bug would show. + */ +PG_PROP(hashint4_deterministic) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT hashint4(%lld) = hashint4(%lld)", + (long long) n, (long long) n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * hashint8 is deterministic across the full INT64 range: the same int8 always + * hashes the same. hashint4 and hashint8 are distinct functions and are not + * expected to agree on common values, so we check only int8 self-stability, + * over the full int64 domain so the engine favours the INT64 extremes. + */ +PG_PROP(hashint8_consistent_with_int4_on_small) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT hashint8("); + append_int8_literal(&q, n); + appendStringInfoString(&q, "::int8) = hashint8("); + append_int8_literal(&q, n); + appendStringInfoString(&q, "::int8)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The seeded extended hash is deterministic per seed: hashtextextended(S, k) + * is stable for a fixed seed k. Two different seeds (0 and 42) are each + * checked for self-stability -- a seed that leaked into stale state would + * break one or the other. We do not assert anything about cross-seed + * relations, only per-seed determinism. + */ +PG_PROP(hashtextextended_seed0_matches_or_differs) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT hashtextextended("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 0) = hashtextextended("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 0) AND hashtextextended("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 42) = hashtextextended("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", 42)"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Numerically-equal numerics hash equally regardless of scale. 1.0 and 1.00 + * are equal as numerics but stored with different scale, and n and n + 0.0 + * likewise; the numeric hash must respect numeric equality (a known subtlety, + * since a naive byte hash of the on-disk representation would differ by + * scale). hash_numeric is the SQL-callable per-type hash for numeric; if the + * numeric hash ignored scale-normalisation this property would fail. + */ +PG_PROP(equal_numeric_values_hash_equal) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, -1000, 1000); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT hash_numeric(1.0::numeric) = hash_numeric(1.00::numeric) " + "AND hash_numeric((%lld)::numeric) " + "= hash_numeric((%lld::numeric + 0.0))", + (long long) n, (long long) n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 087bf5c8401c6..45c8ce90c4e74 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From ede974a2b6e7559fe186cbdd8600f7454ea90424 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:15 -0400 Subject: [PATCH 148/338] prop: add hash-function determinism and consistency property tests (hashfns2) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_hashfns2.c | 365 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 369 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_hashfns2.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index ca19c2b64c3a2..625ba1037db52 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -163,7 +163,8 @@ OBJS = \ prop_tochar.o \ prop_toformat.o \ prop_scalarfn.o \ - prop_hashfn.o + prop_hashfn.o \ + prop_hashfns2.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index bcf3eaf85e18a..c70799def9c6e 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -161,6 +161,7 @@ prop_area_sources = files( 'prop_toformat.c', 'prop_scalarfn.c', 'prop_hashfn.c', + 'prop_hashfns2.c', ) diff --git a/src/test/prop/prop_hashfns2.c b/src/test/prop/prop_hashfns2.c new file mode 100644 index 0000000000000..a81649d1a3c29 --- /dev/null +++ b/src/test/prop/prop_hashfns2.c @@ -0,0 +1,365 @@ +/*------------------------------------------------------------------------- + * + * prop_hashfns2.c + * Property-based tests for hash/digest determinism and consistency. + * + * The text hash (hashtext) and the digests (md5, sha256) are pure functions: + * the same input must always produce the same output, and equal values must + * produce equal results. That contract is what hash joins, hash indexes and + * every content-addressed use of a digest rely on; a non-stable hash + * (uninitialized padding, stray global state) silently corrupts lookups. + * These properties check determinism (same input, same output), equality + * consistency (equal values hash equally), and the fixed output widths of the + * digests (md5 is 32 hex chars, sha256 is 64), each via a scalar boolean + * oracle. This area is distinct from "hashfn": it covers hashtext plus the + * md5/sha256 digests rather than the per-type hash_* functions. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_hashfns2.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "hashfns2" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw 0..16 characters from the quote-free single-byte alphabet "abc" into + * buf as a bare string (no quoting). Letters only -- all ASCII, never a + * single quote -- so the literal quoting is trivial and the only variable is + * the function under test. + */ +static void +draw_text(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abc"; + int n = (int) pg_prop_draw_int(tc, 0, 16); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 2)]); +} + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. The [abc] alphabet never contains a quote, but do it + * defensively so the generated SQL is always well-formed. Modelled on + * prop_hashfn.c. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the hash/digest errored or produced NULL on a finite + * input, which is itself a bug -- so either is a failure, reported with the + * SQL. Modelled on prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * hashint_deterministic: hashtext applied to the text rendering of a drawn + * int4 is stable. hashint4 is not relied on here (hashtext is the exposed, + * portable surface); rendering the integer to text and hashing it twice must + * agree, which is the determinism half of the hash contract on integer-shaped + * input over the full INT32 range. + */ +PG_PROP(hashint_deterministic) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT hashtext((%lld)::int4::text) " + "= hashtext((%lld)::int4::text)", + (long long) n, (long long) n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * hashtext_equal_inputs: hashing the same string twice gives the same value. + * The headline determinism property on text -- a non-stable hash would break + * this and corrupt every hash lookup. + */ +PG_PROP(hashtext_equal_inputs) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT hashtext("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = hashtext("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * hash_equal_values_equal_hash: equal values hash equally. S and S || '' are + * the same string (concatenating the empty string is the identity), built two + * different ways, so their hashes must match. This exercises the + * equal-values-hash-equally contract where the equality is proven by + * construction rather than asserted by the test. + */ +PG_PROP(hash_equal_values_equal_hash) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT hashtext("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = hashtext("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " || '')"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * hash_int_consistency: the same integer rendered to text the same way hashes + * the same. (n)::int4::text produced twice is byte-identical, so hashtext of + * the two renderings must agree -- determinism checked through the cast path + * rather than a literal. + */ +PG_PROP(hash_int_consistency) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT hashtext((%lld)::int4::text) " + "= hashtext(((%lld)::int4)::text)", + (long long) n, (long long) n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * hash_distributes_nothing: hashtext is a function, so repeated application to + * the same input always agrees. We deliberately assert ONLY determinism: + * two different strings MAY collide, so nothing is claimed about distinctness. + * Keeping it to self-equality keeps the oracle sound for every input. + */ +PG_PROP(hash_distributes_nothing) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT hashtext("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = hashtext("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * md5_deterministic_and_hex: md5(S) is stable and always 32 lowercase hex + * characters. The determinism and the fixed output shape are both bugs if + * broken, so both are checked in one oracle: md5(S) = md5(S) AND the result + * matches ^[0-9a-f]{32}$. + */ +PG_PROP(md5_deterministic_and_hex) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT md5("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = md5("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") AND md5("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") ~ '^[0-9a-f]{32}$'"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * sha256_length: sha256 produces 32 bytes, so its hex encoding is always 64 + * characters, independent of the input length. A width bug in the digest or + * its encoding would change this count. + */ +PG_PROP(sha256_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT length(encode(sha256("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "::bytea), 'hex')) = 64"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * hashtext_stable_across_calls: hashtext is stable across separate + * evaluations. Computing it in two independent subqueries and comparing the + * results catches any per-call state that leaks between invocations -- the + * result must be identical regardless of how many times it is evaluated. + */ +PG_PROP(hashtext_stable_across_calls) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_text(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT (SELECT hashtext("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) = (SELECT hashtext("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 45c8ce90c4e74..079a725b16354 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 0cb936b9e1194c73fb7a057830793ef79c58027f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:16 -0400 Subject: [PATCH 149/338] prop: add pglsn property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_pglsn.c | 364 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 368 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_pglsn.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 625ba1037db52..5ae5b6ca3c60f 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -164,7 +164,8 @@ OBJS = \ prop_toformat.o \ prop_scalarfn.o \ prop_hashfn.o \ - prop_hashfns2.o + prop_hashfns2.o \ + prop_pglsn.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index c70799def9c6e..128999583c8d6 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -162,6 +162,7 @@ prop_area_sources = files( 'prop_scalarfn.c', 'prop_hashfn.c', 'prop_hashfns2.c', + 'prop_pglsn.c', ) diff --git a/src/test/prop/prop_pglsn.c b/src/test/prop/prop_pglsn.c new file mode 100644 index 0000000000000..5290c1f78145f --- /dev/null +++ b/src/test/prop/prop_pglsn.c @@ -0,0 +1,364 @@ +/*------------------------------------------------------------------------- + * + * prop_pglsn.c + * Property-based tests for the pg_lsn datatype + * (src/backend/utils/adt/pg_lsn.c). + * + * pg_lsn is a 64-bit WAL position printed as X/Y hex; its text I/O, + * difference (lsn - lsn -> numeric bytes), and +/- numeric arithmetic have + * had fixes. These check text round-trip, the difference/addition inverse, + * and ordering. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_pglsn.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "pglsn" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * True when a SQL error is a legitimate out-of-range/overflow condition + * rather than a wrong-results bug. pg_lsn + numeric and pg_lsn - numeric + * run the result through numeric_pg_lsn, which raises "... out of range" when + * the byte count carries the position outside the 64-bit LSN space; the NaN + * guards raise "cannot add/subtract NaN". Those are legal, so discard them. + */ +static bool +is_overflow(const char *errmsg) +{ + if (errmsg == NULL) + return false; + return strstr(errmsg, "out of range") != NULL || + strstr(errmsg, "cannot") != NULL; +} + +/* + * Draw a 64-bit LSN as a pg_lsn literal into buf, e.g. "'1A2B/00C3'::pg_lsn". + * + * The value is drawn as two independent 32-bit halves and emitted upper-hex + * "high/low", so the whole 64-bit space is reachable, including the boundary + * values 0/0 and FFFFFFFF/FFFFFFFF. The raw 64-bit value (hi<<32 | lo) is + * returned via *raw for use as a C-side oracle when wanted. + */ +static void +draw_lsn(pg_prop_tc *tc, StringInfo buf, uint64 *raw) +{ + uint32 hi = (uint32) pg_prop_draw_int(tc, 0, 4294967295); + uint32 lo = (uint32) pg_prop_draw_int(tc, 0, 4294967295); + + appendStringInfo(buf, "'%X/%X'::pg_lsn", hi, lo); + + if (raw) + *raw = ((uint64) hi << 32) | (uint64) lo; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operator errored or produced NULL on valid inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * 1. text_roundtrip + * + * The text output of a pg_lsn, parsed back, must equal the original. This + * exercises pg_lsn_out / pg_lsn_in (pg_parse_lsn): L = (L)::text::pg_lsn for + * every drawn position, including the two extremes. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData l; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&l); + draw_lsn(tc, &l, NULL); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)::text::pg_lsn = (%s)", l.data, l.data); + + require_true(tc, conn, q.data); + + pfree(l.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. diff_then_add + * + * The difference L2 - L1 is a numeric byte count; adding it back to L1 must + * yield L2: L1 + (L2 - L1) = L2. (pg_lsn - pg_lsn -> numeric; pg_lsn + + * numeric -> pg_lsn.) When L2 < L1 the difference is negative, and the + * add can carry L1 below 0 -- that underflow is a legitimate out-of-range + * error, so the case is skipped rather than reported. + */ +PG_PROP(diff_then_add) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData l1; + StringInfoData l2; + StringInfoData q; + char *result; + bool failed; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&l1); + initStringInfo(&l2); + draw_lsn(tc, &l1, NULL); + draw_lsn(tc, &l2, NULL); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s + ((%s) - (%s))) = (%s)", + l1.data, l2.data, l1.data, l2.data); + + result = pg_prop_sql_scalar(conn, q.data, &failed, &err); + + if (failed) + { + if (is_overflow(err)) + { + /* add carried off the 64-bit range: legal, skip the case */ + pg_prop_note(tc, "overflow (skipped): %s", err); + if (err) + pg_free(err); + pfree(l1.data); + pfree(l2.data); + pfree(q.data); + pg_prop_disconnect(conn); + pg_prop_assume(tc, false); + return; /* unreachable: assume(false) longjmps */ + } + pg_prop_fail(tc, + "unexpected error on L1 + (L2 - L1) = L2\n" + " L1: %s\n L2: %s\n error: %s", + l1.data, l2.data, err ? err : "(unknown)"); + } + + if (result == NULL || strcmp(result, "t") != 0) + pg_prop_fail(tc, + "L1 + (L2 - L1) != L2\n" + " L1: %s\n L2: %s\n result: %s", + l1.data, l2.data, result ? result : "NULL"); + + pg_free(result); + pfree(l1.data); + pfree(l2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. add_sub_inverse + * + * Adding a byte count and then subtracting it must return the LSN: + * (L + d) - d = L. d is kept modest (0..1e6) so most positions survive, but + * a position within d bytes of the top of the range overflows on the add -- + * a legitimate out-of-range error, so that case is skipped. + */ +PG_PROP(add_sub_inverse) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData l; + StringInfoData q; + int64 d; + char *result; + bool failed; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&l); + draw_lsn(tc, &l, NULL); + d = pg_prop_draw_int(tc, 0, 1000000); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s + %lld::numeric) - %lld::numeric) = (%s)", + l.data, (long long) d, (long long) d, l.data); + + result = pg_prop_sql_scalar(conn, q.data, &failed, &err); + + if (failed) + { + if (is_overflow(err)) + { + /* add carried past FFFFFFFF/FFFFFFFF: legal, skip the case */ + pg_prop_note(tc, "overflow (skipped): %s", err); + if (err) + pg_free(err); + pfree(l.data); + pfree(q.data); + pg_prop_disconnect(conn); + pg_prop_assume(tc, false); + return; /* unreachable: assume(false) longjmps */ + } + pg_prop_fail(tc, + "unexpected error on (L + d) - d = L\n" + " L: %s\n d: %lld\n error: %s", + l.data, (long long) d, err ? err : "(unknown)"); + } + + if (result == NULL || strcmp(result, "t") != 0) + pg_prop_fail(tc, + "(L + d) - d != L\n" + " L: %s\n d: %lld\n result: %s", + l.data, (long long) d, result ? result : "NULL"); + + pg_free(result); + pfree(l.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. comparison_total_order + * + * For any two positions L1, L2 exactly one of (L1 < L2), (L1 = L2), + * (L1 > L2) must hold: pg_lsn is a total order. + */ +PG_PROP(comparison_total_order) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData l1; + StringInfoData l2; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&l1); + initStringInfo(&l2); + draw_lsn(tc, &l1, NULL); + draw_lsn(tc, &l2, NULL); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ( ((%s) < (%s))::int " + "+ ((%s) = (%s))::int " + "+ ((%s) > (%s))::int ) = 1", + l1.data, l2.data, + l1.data, l2.data, + l1.data, l2.data); + + require_true(tc, conn, q.data); + + pfree(l1.data); + pfree(l2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. order_matches_difference_sign + * + * The comparison order must agree with the sign of the numeric difference: + * L1 < L2 iff (L2 - L1) > 0, and L1 = L2 iff (L2 - L1) = 0. + */ +PG_PROP(order_matches_difference_sign) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData l1; + StringInfoData l2; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&l1); + initStringInfo(&l2); + draw_lsn(tc, &l1, NULL); + draw_lsn(tc, &l2, NULL); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s) < (%s)) = (((%s) - (%s)) > 0) " + "AND ((%s) = (%s)) = (((%s) - (%s)) = 0)", + l1.data, l2.data, l2.data, l1.data, + l1.data, l2.data, l2.data, l1.data); + + require_true(tc, conn, q.data); + + pfree(l1.data); + pfree(l2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. add_zero_identity + * + * Adding zero bytes is the identity: L + 0 = L for every drawn position. + */ +PG_PROP(add_zero_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData l; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&l); + draw_lsn(tc, &l, NULL); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s + 0::numeric) = (%s)", l.data, l.data); + + require_true(tc, conn, q.data); + + pfree(l.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 079a725b16354..1cc629ed59672 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 4d71d5d19458a443982aaa161cd95cd97975f87d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:16 -0400 Subject: [PATCH 150/338] prop: add width_bucket property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_width_bucket.c | 368 ++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 372 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_width_bucket.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 5ae5b6ca3c60f..fec2a1b2d1e7e 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -165,7 +165,8 @@ OBJS = \ prop_scalarfn.o \ prop_hashfn.o \ prop_hashfns2.o \ - prop_pglsn.o + prop_pglsn.o \ + prop_width_bucket.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 128999583c8d6..bc7b0a5001896 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -163,6 +163,7 @@ prop_area_sources = files( 'prop_hashfn.c', 'prop_hashfns2.c', 'prop_pglsn.c', + 'prop_width_bucket.c', ) diff --git a/src/test/prop/prop_width_bucket.c b/src/test/prop/prop_width_bucket.c new file mode 100644 index 0000000000000..5f6fa2e3a1451 --- /dev/null +++ b/src/test/prop/prop_width_bucket.c @@ -0,0 +1,368 @@ +/*------------------------------------------------------------------------- + * + * prop_width_bucket.c + * Property-based tests for the width_bucket() function. + * + * width_bucket(operand, low, high, count) assigns operand to one of count + * equal-width buckets, returning 0 for below-low and count+1 for at-or-above- + * high; recent fixes touched its boundary and NaN handling. These check the + * documented bucket semantics against an independent computation. + * + * The oracle is the documented definition itself, recomputed independently in + * SQL: buckets are 1-based, the low bound is inclusive (bucket 1), the high + * bound is exclusive (bucket count+1), anything strictly below low is 0, and + * for an in-range operand the bucket is floor((op-low)/(high-low)*count)+1. + * All arithmetic is done in numeric so the floor and division are exact and + * no float rounding can mask or manufacture a discrepancy. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_width_bucket.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "width_bucket" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means width_bucket errored or produced NULL on finite inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Draw the three shared inputs for a width_bucket case: an integer operand in + * [-50, 150], a pair of bounds ordered so low < high (bumping high when they + * collide), and a count in [1, 20]. Integers keep the numeric arithmetic + * exact and let the manual-formula oracle be checked without tolerance. + */ +static void +draw_inputs(pg_prop_tc *tc, int *operand, int *low, int *high, int *count) +{ + int a; + int b; + + *operand = (int) pg_prop_draw_int(tc, -50, 150); + + a = (int) pg_prop_draw_int(tc, -50, 150); + b = (int) pg_prop_draw_int(tc, -50, 150); + if (a > b) + { + int t = a; + + a = b; + b = t; + } + if (a == b) + b = b + 1; + *low = a; + *high = b; + + *count = (int) pg_prop_draw_int(tc, 1, 20); +} + +/* + * width_bucket always returns a value in [0, count+1]: a below-low operand is + * clamped to 0, an at-or-above-high operand to count+1, everything else lands + * in a numbered bucket. + */ +PG_PROP(result_in_range_0_to_count_plus_1) +{ + pg_prop_conn *conn; + char *err = NULL; + int operand, + low, + high, + count; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_inputs(tc, &operand, &low, &high, &count); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT width_bucket(%d::numeric, %d::numeric, " + "%d::numeric, %d) BETWEEN 0 AND %d + 1", + operand, low, high, count, count); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * An operand strictly below low gets bucket 0. low-1 < low, so it must fall + * in the below-range overflow bucket. + */ +PG_PROP(below_low_is_zero) +{ + pg_prop_conn *conn; + char *err = NULL; + int operand, + low, + high, + count; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_inputs(tc, &operand, &low, &high, &count); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT width_bucket((%d - 1)::numeric, %d::numeric, " + "%d::numeric, %d) = 0", + low, low, high, count); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * An operand at or above high gets bucket count+1. The high bound is + * exclusive, so operand = high already falls in the overflow bucket, as does + * high+1. + */ +PG_PROP(at_or_above_high_is_count_plus_1) +{ + pg_prop_conn *conn; + char *err = NULL; + int operand, + low, + high, + count; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_inputs(tc, &operand, &low, &high, &count); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT width_bucket(%d::numeric, %d::numeric, " + "%d::numeric, %d) = %d + 1 " + "AND width_bucket((%d + 1)::numeric, %d::numeric, " + "%d::numeric, %d) = %d + 1", + high, low, high, count, count, + high, low, high, count, count); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * An operand exactly at low gets bucket 1: buckets are 1-based and the low + * bound is inclusive. + */ +PG_PROP(at_low_is_bucket_1) +{ + pg_prop_conn *conn; + char *err = NULL; + int operand, + low, + high, + count; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_inputs(tc, &operand, &low, &high, &count); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT width_bucket(%d::numeric, %d::numeric, " + "%d::numeric, %d) = 1", + low, low, high, count); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * width_bucket is monotonic non-decreasing in the operand: for op1 <= op2, + * bucket(op1) <= bucket(op2). Two operands are drawn and ordered in C so the + * precondition holds by construction. + */ +PG_PROP(monotonic_nondecreasing) +{ + pg_prop_conn *conn; + char *err = NULL; + int operand, + low, + high, + count; + int op1, + op2; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_inputs(tc, &operand, &low, &high, &count); + + op1 = (int) pg_prop_draw_int(tc, -50, 150); + op2 = (int) pg_prop_draw_int(tc, -50, 150); + if (op1 > op2) + { + int t = op1; + + op1 = op2; + op2 = t; + } + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT width_bucket(%d::numeric, %d::numeric, " + "%d::numeric, %d) " + "<= width_bucket(%d::numeric, %d::numeric, " + "%d::numeric, %d)", + op1, low, high, count, + op2, low, high, count); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * For an in-range operand (low <= op < high) the bucket equals the manual + * definition floor((op-low)/(high-low)*count)+1. The operand is drawn in + * [low, high-1] so it is strictly in range, and the whole formula is computed + * in numeric so the floor is exact and independent of width_bucket's own + * arithmetic. + */ +PG_PROP(matches_manual_formula) +{ + pg_prop_conn *conn; + char *err = NULL; + int operand, + low, + high, + count; + int op; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_inputs(tc, &operand, &low, &high, &count); + + /* draw an operand strictly in range: low <= op <= high - 1 */ + op = (int) pg_prop_draw_int(tc, low, high - 1); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT width_bucket(%d::numeric, %d::numeric, " + "%d::numeric, %d) " + "= floor((%d::numeric - %d) * %d / (%d - %d)) + 1", + op, low, high, count, + op, low, count, high, low); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * width_bucket also supports descending bounds (high < low). For an operand + * strictly inside the open interval (low, high), the ascending and the + * bound-swapped descending results mirror each other and sum to count+1: + * width_bucket(op, low, high, count) + width_bucket(op, high, low, count) + * = count + 1. The operand is drawn in [low+1, high-1], which requires a span + * of at least 2; draw_inputs can give span 1, so bounds are redrawn here with + * a guaranteed span >= 2. + */ +PG_PROP(reverse_bounds_in_range) +{ + pg_prop_conn *conn; + char *err = NULL; + int operand, + low, + high, + count; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + draw_inputs(tc, &operand, &low, &high, &count); + if (high - low < 2) + high = low + 2; + + /* + * width_bucket also accepts descending bounds (high < low). The result + * stays in [0, count+1] for any operand, and an operand at or above the + * descending "low" (= the larger value) maps to bucket 0 while one at or + * below the descending "high" (= the smaller value) maps to count+1. + */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT width_bucket(%d::numeric, %d::numeric, " + "%d::numeric, %d) BETWEEN 0 AND %d + 1 " + "AND width_bucket((%d + 1)::numeric, %d::numeric, " + "%d::numeric, %d) = 0 " + "AND width_bucket((%d - 1)::numeric, %d::numeric, " + "%d::numeric, %d) = %d + 1", + operand, high, low, count, count, + high, high, low, count, + low, high, low, count, count); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 1cc629ed59672..e33ab9732d248 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 5be26874cb39c08b6c02ead3918b11066ef3c810 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:16 -0400 Subject: [PATCH 151/338] prop: add genseries property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_genseries.c | 290 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 294 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_genseries.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index fec2a1b2d1e7e..5586dc7e275c6 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -166,7 +166,8 @@ OBJS = \ prop_hashfn.o \ prop_hashfns2.o \ prop_pglsn.o \ - prop_width_bucket.o + prop_width_bucket.o \ + prop_genseries.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index bc7b0a5001896..a41aa300e7e97 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -164,6 +164,7 @@ prop_area_sources = files( 'prop_hashfns2.c', 'prop_pglsn.c', 'prop_width_bucket.c', + 'prop_genseries.c', ) diff --git a/src/test/prop/prop_genseries.c b/src/test/prop/prop_genseries.c new file mode 100644 index 0000000000000..a0f6e7f497b92 --- /dev/null +++ b/src/test/prop/prop_genseries.c @@ -0,0 +1,290 @@ +/*------------------------------------------------------------------------- + * + * prop_genseries.c + * Property-based tests for the generate_series set-returning function. + * + * generate_series has exact count/first/last/step invariants; these pin them + * down across positive, negative, and degenerate steps. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_genseries.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "genseries" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means generate_series errored or produced NULL on valid inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Draw a non-zero step in [-20, 20] \ {0}: draw a magnitude in 1..20 and a + * random sign. generate_series errors on step 0, so this never yields 0. + */ +static int64 +draw_step(pg_prop_tc *tc) +{ + int64 mag = pg_prop_draw_int(tc, 1, 20); + bool neg = pg_prop_draw_bool(tc); + + return neg ? -mag : mag; +} + +/* + * The number of elements of generate_series(a, b, s) is exactly: + * if (s > 0 and a > b) or (s < 0 and a < b) then 0 + * else floor(abs(b - a) / abs(s)) + 1 + * This is the core count invariant. Compute both sides in SQL and compare. + */ +PG_PROP(count_matches_formula) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a, + b, + s; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = pg_prop_draw_int(tc, -1000, 1000); + b = pg_prop_draw_int(tc, -1000, 1000); + s = draw_step(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM generate_series(%lld, %lld, %lld)) " + "= CASE WHEN (%lld > 0 AND %lld > %lld) OR (%lld < 0 AND %lld < %lld) THEN 0 " + "ELSE floor(abs(%lld - %lld) / abs(%lld))::int + 1 END", + (long long) a, (long long) b, (long long) s, + (long long) s, (long long) a, (long long) b, + (long long) s, (long long) a, (long long) b, + (long long) b, (long long) a, (long long) s); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The first emitted value (in series order) is always the start a, when the + * series is non-empty. coalesce(..., true) makes an empty series pass + * vacuously (the LIMIT 1 subselect is NULL, so the equality is NULL). + */ +PG_PROP(first_is_start_when_nonempty) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a, + b, + s; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = pg_prop_draw_int(tc, -1000, 1000); + b = pg_prop_draw_int(tc, -1000, 1000); + s = draw_step(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce((SELECT g FROM generate_series(%lld, %lld, %lld) g LIMIT 1) = %lld, true)", + (long long) a, (long long) b, (long long) s, + (long long) a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Every consecutive pair of elements differs by exactly the step s. WITH + * ORDINALITY gives a deterministic position to lag over. bool_and over the + * differences, coalesce'd to true so a 0- or 1-element series (no consecutive + * pair) passes vacuously. + */ +PG_PROP(step_between_consecutive) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a, + b, + s; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = pg_prop_draw_int(tc, -1000, 1000); + b = pg_prop_draw_int(tc, -1000, 1000); + s = draw_step(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(bool_and(d = %lld), true) FROM (" + "SELECT g - lag(g) OVER (ORDER BY ord) AS d " + "FROM generate_series(%lld, %lld, %lld) WITH ORDINALITY AS t(g, ord)" + ") s WHERE d IS NOT NULL", + (long long) s, + (long long) a, (long long) b, (long long) s); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Every element lies within [least(a, b), greatest(a, b)]. bool_and over the + * BETWEEN test, coalesce'd to true so an empty series passes vacuously. + */ +PG_PROP(all_within_bounds) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a, + b, + s; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = pg_prop_draw_int(tc, -1000, 1000); + b = pg_prop_draw_int(tc, -1000, 1000); + s = draw_step(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(bool_and(g BETWEEN least(%lld, %lld) AND greatest(%lld, %lld)), true) " + "FROM generate_series(%lld, %lld, %lld) g", + (long long) a, (long long) b, (long long) a, (long long) b, + (long long) a, (long long) b, (long long) s); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * generate_series(a, b, s) and generate_series(b, a, -s) walk the same values + * in opposite directions and so have equal sorted arrays -- but ONLY when + * (b - a) is an exact multiple of s, otherwise the two walks land on different + * grids and the last term differs. Use step +/-1, where every integer in the + * span is hit, so the reversal is always exact. IS NOT DISTINCT FROM also + * passes the empty case (both NULL). + */ +PG_PROP(reverse_step_reverses_series) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a, + b, + s; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = pg_prop_draw_int(tc, -1000, 1000); + b = pg_prop_draw_int(tc, -1000, 1000); + s = pg_prop_draw_bool(tc) ? 1 : -1; /* unit step: span is always a multiple */ + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT array_agg(g ORDER BY g) FROM generate_series(%lld, %lld, %lld) g) " + "IS NOT DISTINCT FROM " + "(SELECT array_agg(g ORDER BY g) FROM generate_series(%lld, %lld, -(%lld)) g)", + (long long) a, (long long) b, (long long) s, + (long long) b, (long long) a, (long long) s); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * For step 1 and a <= b, the count is exactly b - a + 1. a <= b is enforced + * by swapping the drawn endpoints in C before building the SQL. + */ +PG_PROP(unit_step_count_is_span_plus_one) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a, + b, + tmp; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = pg_prop_draw_int(tc, -1000, 1000); + b = pg_prop_draw_int(tc, -1000, 1000); + if (a > b) + { + tmp = a; + a = b; + b = tmp; + } + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM generate_series(%lld, %lld, 1)) = (%lld - %lld + 1)", + (long long) a, (long long) b, + (long long) b, (long long) a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index e33ab9732d248..0c715d9f9d192 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 395c1a6f95289dad71eec4770eb9a1821fbfbd49 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:16 -0400 Subject: [PATCH 152/338] prop: add generate_series over timestamp/numeric and subscripts property tests (genseries2) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_genseries2.c | 293 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 297 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_genseries2.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 5586dc7e275c6..a5be79339b0f5 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -167,7 +167,8 @@ OBJS = \ prop_hashfns2.o \ prop_pglsn.o \ prop_width_bucket.o \ - prop_genseries.o + prop_genseries.o \ + prop_genseries2.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index a41aa300e7e97..6b7d172c915f7 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -165,6 +165,7 @@ prop_area_sources = files( 'prop_pglsn.c', 'prop_width_bucket.c', 'prop_genseries.c', + 'prop_genseries2.c', ) diff --git a/src/test/prop/prop_genseries2.c b/src/test/prop/prop_genseries2.c new file mode 100644 index 0000000000000..b3776c9a08c85 --- /dev/null +++ b/src/test/prop/prop_genseries2.c @@ -0,0 +1,293 @@ +/*------------------------------------------------------------------------- + * + * prop_genseries2.c + * Property-based tests for generate_series over timestamps/numerics and + * for generate_subscripts. + * + * These pin exact count/first/last/sum invariants of the timestamp and + * numeric generate_series overloads, and the subscript range emitted by + * generate_subscripts, which the integer-only prop_genseries.c does not + * cover. A wrong count, bound, or sum on valid input is a bug. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_genseries2.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "genseries2" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on valid inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * A daily timestamp series from 2020-01-01 to 2020-01-01 + d days, stepping + * one day, emits exactly d + 1 rows for d in 0..30. + */ +PG_PROP(ts_series_count) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 d; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + d = pg_prop_draw_int(tc, 0, 30); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM generate_series(" + "TIMESTAMP '2020-01-01', " + "TIMESTAMP '2020-01-01' + make_interval(days => %lld), " + "interval '1 day')) = (%lld + 1)", + (long long) d, (long long) d); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A numeric series from 0 to n stepping 0.5 emits floor(n / 0.5) + 1 = 2n + 1 + * rows for n in 0..10. + */ +PG_PROP(numeric_series_count) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, 0, 10); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM generate_series(" + "0::numeric, %lld::numeric, 0.5)) = (2 * %lld + 1)", + (long long) n, (long long) n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * For an integer series with a <= b and default step 1, the minimum value is + * the start a and the maximum is the end b. Endpoints are drawn in -10..10 + * and swapped in C so a <= b. + */ +PG_PROP(series_first_last) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a, + b, + tmp; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = pg_prop_draw_int(tc, -10, 10); + b = pg_prop_draw_int(tc, -10, 10); + if (a > b) + { + tmp = a; + a = b; + b = tmp; + } + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT min(g) FROM generate_series((%lld), (%lld)) g) = (%lld) " + "AND (SELECT max(g) FROM generate_series((%lld), (%lld)) g) = (%lld)", + (long long) a, (long long) b, (long long) a, + (long long) a, (long long) b, (long long) b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * generate_subscripts(arr, 1) over an int array of m elements (m in 0..6) + * yields exactly the subscripts 1..m: m rows, and for m >= 1 the min is 1 and + * the max is m. The empty case (m = 0) is guarded: count = 0, and min is + * NULL (CASE yields NULL), matched with IS NOT DISTINCT FROM. + */ +PG_PROP(subscripts_match_bounds) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 m, + i; + StringInfoData arr; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + m = pg_prop_draw_int(tc, 0, 6); + + /* build an int4 array literal ARRAY[...]::int[] with m elements */ + initStringInfo(&arr); + appendStringInfoString(&arr, "ARRAY["); + for (i = 0; i < m; i++) + appendStringInfo(&arr, "%s%lld", i ? ", " : "", (long long) i); + appendStringInfoString(&arr, "]::int[]"); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM generate_subscripts(%s, 1)) = (%lld) " + "AND (SELECT min(s) FROM generate_subscripts(%s, 1) s) " + "IS NOT DISTINCT FROM (CASE WHEN %lld > 0 THEN 1 ELSE NULL END) " + "AND (SELECT max(s) FROM generate_subscripts(%s, 1) s) " + "IS NOT DISTINCT FROM (CASE WHEN %lld > 0 THEN %lld ELSE NULL END)", + arr.data, (long long) m, + arr.data, (long long) m, + arr.data, (long long) m, (long long) m); + + require_true(tc, conn, q.data); + + pfree(arr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A descending integer series generate_series(b, a, -1) with a <= b emits + * exactly b - a + 1 rows. Endpoints are drawn in -10..10 and swapped in C so + * a <= b. + */ +PG_PROP(series_step_negative) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a, + b, + tmp; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = pg_prop_draw_int(tc, -10, 10); + b = pg_prop_draw_int(tc, -10, 10); + if (a > b) + { + tmp = a; + a = b; + b = tmp; + } + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM generate_series((%lld), (%lld), (-1))) " + "= ((%lld) - (%lld) + 1)", + (long long) b, (long long) a, + (long long) b, (long long) a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * When the step points away from the end -- generate_series(0, 5, -1) -- the + * series is empty, so its count is 0. A fixed-input invariant, no draw. + */ +PG_PROP(series_empty_when_step_wrong) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM generate_series(0, 5, (-1))) = 0"); + + pg_prop_disconnect(conn); +} + +/* + * The sum of generate_series(1, n) equals the triangular number + * n * (n + 1) / 2 for n in 0..20. For n = 0 the series is empty, so sum() is + * NULL; coalesce to 0 matches the formula's 0. + */ +PG_PROP(series_sum) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = pg_prop_draw_int(tc, 0, 20); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce((SELECT sum(g) FROM generate_series(1, %lld) g), 0) " + "= (%lld * (%lld + 1) / 2)", + (long long) n, (long long) n, (long long) n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 0c715d9f9d192..3a04a6fd8cf56 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 78df5567edfb81e8ebbfe7741e02c2828d79baea Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:16 -0400 Subject: [PATCH 153/338] prop: add cross-type cast chains and text round-trips property tests (castchain) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_castchain.c | 340 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 344 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_castchain.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index a5be79339b0f5..82a7d380304d0 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -168,7 +168,8 @@ OBJS = \ prop_pglsn.o \ prop_width_bucket.o \ prop_genseries.o \ - prop_genseries2.o + prop_genseries2.o \ + prop_castchain.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 6b7d172c915f7..99250b56fbcaa 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -166,6 +166,7 @@ prop_area_sources = files( 'prop_width_bucket.c', 'prop_genseries.c', 'prop_genseries2.c', + 'prop_castchain.c', ) diff --git a/src/test/prop/prop_castchain.c b/src/test/prop/prop_castchain.c new file mode 100644 index 0000000000000..19afd5a851ecb --- /dev/null +++ b/src/test/prop/prop_castchain.c @@ -0,0 +1,340 @@ +/*------------------------------------------------------------------------- + * + * prop_castchain.c + * Property-based tests for cross-type cast chains and text round-trips. + * + * Casts and text I/O between the common built-in types form an algebra that + * must round-trip: a value cast out to another type and back, or serialized + * to text and parsed again, has to come back unchanged. These properties + * check that identity for bool<->int, int<->text, numeric<->text, the + * varchar/char/text family, oid<->int, and bytea<->text(hex), covering type + * I/O and the implicit/explicit cast paths that a formula-based oracle would + * not. Any "f" result is a cast or I/O bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_castchain.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "castchain" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a cast errored or produced NULL on a value it should + * accept, which is itself a bug -- so either is a failure, reported with the + * SQL. (Same shape as prop_numeric.c's require_true.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Draw an int4 as text, parenthesizing a negative so a leading '-' does not + * misparse when substituted into a larger expression (bare -N::int4 parses as + * -(N::int4), which overflows at INT_MIN). Kept inside int4 range. + */ +static void +draw_int4_lit(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + if (n < 0) + appendStringInfo(buf, "(%d)", n); + else + appendStringInfo(buf, "%d", n); +} + +/* + * Draw a short string over the alphabet [abc], length 0..maxlen, with no + * trailing space -- so a char(n) blank-pad/rtrim round-trip is sound. The + * literal is emitted single-quoted and ready to substitute. + */ +static void +draw_abc_lit(pg_prop_tc *tc, StringInfo buf, int maxlen) +{ + int len = (int) pg_prop_draw_int(tc, 0, maxlen); + + appendStringInfoChar(buf, '\''); + for (int i = 0; i < len; i++) + appendStringInfoChar(buf, (char) ('a' + pg_prop_draw_int(tc, 0, 2))); + appendStringInfoChar(buf, '\''); +} + +/* + * bool<->int round-trip. The constants pin the mapping (true->1, false->0 and + * back), and for a drawn bool b, b::int::bool = b. + */ +PG_PROP(bool_int_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + const char *b; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + b = pg_prop_draw_bool(tc) ? "true" : "false"; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT true::int = 1 AND false::int = 0 " + "AND 1::bool = true AND 0::bool = false " + "AND (%s::int::bool) = %s", + b, b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * int4 text round-trip: n::text::int4 = n for a drawn int4 n across the full + * range (including INT_MIN via the parenthesized negative). + */ +PG_PROP(int_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + draw_int4_lit(tc, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s::int4)::text::int4) = (%s::int4)", + n.data, n.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * numeric text round-trip: X built as "." has + * (X::numeric)::text::numeric = X::numeric. numeric text I/O is exact, so + * this holds with no tolerance. + */ +PG_PROP(numeric_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData x, + q; + int nint = (int) pg_prop_draw_int(tc, 1, 12); + int nfrac = (int) pg_prop_draw_int(tc, 1, 8); + bool neg = pg_prop_draw_bool(tc); + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* Build "(.)" with at least one digit on each side. */ + initStringInfo(&x); + appendStringInfoChar(&x, '('); + if (neg) + appendStringInfoChar(&x, '-'); + for (int i = 0; i < nint; i++) + appendStringInfoChar(&x, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + appendStringInfoChar(&x, '.'); + for (int i = 0; i < nfrac; i++) + appendStringInfoChar(&x, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + appendStringInfoChar(&x, ')'); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s::numeric)::text::numeric) = (%s::numeric)", + x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * varchar<->text round-trip: for S over [abc] length 0..10, + * S::varchar::text = S and S::varchar(20)::text = S. (len <= 10 < 20, so the + * length-qualified varchar never truncates.) + */ +PG_PROP(varchar_text_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc_lit(tc, &s, 10); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s::varchar::text) = %s " + "AND (%s::varchar(20)::text) = %s", + s.data, s.data, s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * char(n) blank-pad / rtrim round-trip: for S over [abc] with no trailing + * space, rtrim(S::char(20)) = S. char(n) right-pads with blanks to width n; + * rtrim strips them back. Since S has no trailing space, nothing of S is + * lost, so the padding is exactly recovered. (len <= 10 < 20, no truncation.) + */ +PG_PROP(text_bpchar_trailing_space) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc_lit(tc, &s, 10); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT rtrim(%s::char(20)) = %s", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * oid<->int round-trip: for a drawn n in 0..1000000, n::oid::int8 = n. oid is + * an unsigned 32-bit type, so keeping n in a small non-negative range avoids + * the modular wrap at 2^32 and the int8 compare is exact. + */ +PG_PROP(oid_int_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int n = (int) pg_prop_draw_int(tc, 0, 1000000); + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%d::oid::int8) = %d::int8", n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * bytea<->text(hex) round-trip: for S over [abc], + * decode(encode(S::bytea, 'hex'), 'hex') = S::bytea. encode/decode are exact + * inverses on bytea, so hex serialization and parsing must recover the bytes. + */ +PG_PROP(bytea_hex_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_abc_lit(tc, &s, 10); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT decode(encode(%s::bytea, 'hex'), 'hex') " + "= %s::bytea", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * int / numeric text consistency: for a drawn int4 n, n::numeric::text = + * n::text. An integer promoted to numeric renders with no decimal point, so + * both paths produce the identical digit string. + */ +PG_PROP(int_numeric_text_consistent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + draw_int4_lit(tc, &n); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s::int4)::numeric::text) = ((%s::int4)::text)", + n.data, n.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 3a04a6fd8cf56..7e416b2163054 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 27a09b3e5bba45be336e332b377ac1ae4b6f513d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:16 -0400 Subject: [PATCH 154/338] prop: add || operator across text/array/jsonb/bytea property tests (concatop) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_concatop.c | 370 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 374 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_concatop.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 82a7d380304d0..59ffcfa364bd8 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -169,7 +169,8 @@ OBJS = \ prop_width_bucket.o \ prop_genseries.o \ prop_genseries2.o \ - prop_castchain.o + prop_castchain.o \ + prop_concatop.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 99250b56fbcaa..6edf35e29b678 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -167,6 +167,7 @@ prop_area_sources = files( 'prop_genseries.c', 'prop_genseries2.c', 'prop_castchain.c', + 'prop_concatop.c', ) diff --git a/src/test/prop/prop_concatop.c b/src/test/prop/prop_concatop.c new file mode 100644 index 0000000000000..67d074484823a --- /dev/null +++ b/src/test/prop/prop_concatop.c @@ -0,0 +1,370 @@ +/*------------------------------------------------------------------------- + * + * prop_concatop.c + * Property-based tests for the || concatenation operator. + * + * The || operator is overloaded across text, array, jsonb and bytea, each + * with its own implementation but a shared algebra: concatenation is + * associative and length-additive (text/array/bytea), has an empty identity + * element, and for jsonb objects is right-biased on key conflicts. The + * oracle is that algebra asserted directly in SQL; any "f" is a bug in one of + * the concat implementations, not a rounding or representation artefact. + * + * Distinct from textfns/arrayops/jsonbops: those exercise per-type function + * families; here the single cross-type operator and its associativity are + * the target. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_concatop.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "concatop" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operator errored or produced NULL on finite inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Draw a single-quoted text literal over the alphabet a-c, length 0..8. No + * embedded quotes are possible from this alphabet, so no doubling is needed. + */ +static void +draw_text(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 8); + int i; + + appendStringInfoChar(buf, '\''); + for (i = 0; i < n; i++) + appendStringInfoChar(buf, (char) ('a' + pg_prop_draw_int(tc, 0, 2))); + appendStringInfoChar(buf, '\''); +} + +/* + * Draw an int4 array literal ARRAY[...]::int[] with 0..6 elements in -5..5. + */ +static void +draw_int_array(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 6); + int i; + + appendStringInfoString(buf, "ARRAY["); + for (i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoChar(buf, ','); + appendStringInfo(buf, "(%d)", (int) pg_prop_draw_int(tc, -5, 5)); + } + appendStringInfoString(buf, "]::int[]"); +} + +/* + * Draw a bytea literal decode('','hex') with 0..8 bytes. + */ +static void +draw_bytea(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 8); + int i; + + appendStringInfoString(buf, "decode('"); + for (i = 0; i < n; i++) + appendStringInfo(buf, "%02x", (int) pg_prop_draw_int(tc, 0, 255)); + appendStringInfoString(buf, "','hex')"); +} + +/* + * text || is associative: (a || b) || c = a || (b || c). + */ +PG_PROP(text_concat_associative) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + initStringInfo(&c); + draw_text(tc, &a); + draw_text(tc, &b); + draw_text(tc, &c); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s || %s) || %s) = (%s || (%s || %s))", + a.data, b.data, c.data, a.data, b.data, c.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * text || is length-additive: length(a || b) = length(a) + length(b). + */ +PG_PROP(text_concat_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_text(tc, &a); + draw_text(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(%s || %s) = length(%s) + length(%s)", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array || is associative: (a || b) || c = a || (b || c). + */ +PG_PROP(array_concat_associative) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + initStringInfo(&c); + draw_int_array(tc, &a); + draw_int_array(tc, &b); + draw_int_array(tc, &c); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s || %s) || %s) = (%s || (%s || %s))", + a.data, b.data, c.data, a.data, b.data, c.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array || is cardinality-additive: + * cardinality(a || b) = cardinality(a) + cardinality(b). + */ +PG_PROP(array_concat_cardinality) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_int_array(tc, &a); + draw_int_array(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cardinality(%s || %s) " + "= cardinality(%s) + cardinality(%s)", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * jsonb || on objects is right-biased on key conflicts: the right operand's + * value wins. This is a fixed-shape oracle (no drawn inputs needed) that + * pins the documented semantics. + */ +PG_PROP(jsonb_concat_right_wins) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT ((jsonb_build_object('k', 1) " + "|| jsonb_build_object('k', 2)) -> 'k') = '2'::jsonb"); + + pg_prop_disconnect(conn); +} + +/* + * bytea || is octet-length-additive: + * octet_length(a || b) = octet_length(a) + octet_length(b). + */ +PG_PROP(bytea_concat_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + draw_bytea(tc, &a); + draw_bytea(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT octet_length(%s || %s) " + "= octet_length(%s) + octet_length(%s)", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * text || has the empty string as a two-sided identity: + * a || '' = a AND '' || a = a. + */ +PG_PROP(text_concat_empty_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_text(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s || '') = %s AND ('' || %s) = %s", + a.data, a.data, a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * array || has the empty array as a two-sided identity: + * a || ARRAY[]::int[] = a AND ARRAY[]::int[] || a = a. + */ +PG_PROP(array_concat_empty_identity) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + draw_int_array(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s || ARRAY[]::int[]) = %s " + "AND (ARRAY[]::int[] || %s) = %s", + a.data, a.data, a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 7e416b2163054..68bd1b8503197 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From dcbf8569f8383bebba565ddb4b61990fd1cbf73e Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:16 -0400 Subject: [PATCH 155/338] prop: add type coercion and operator resolution property tests (coerce) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_coerce.c | 402 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 406 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_coerce.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 59ffcfa364bd8..c4c766ba429b7 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -170,7 +170,8 @@ OBJS = \ prop_genseries.o \ prop_genseries2.o \ prop_castchain.o \ - prop_concatop.o + prop_concatop.o \ + prop_coerce.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 6edf35e29b678..53ac7faed7a80 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -168,6 +168,7 @@ prop_area_sources = files( 'prop_genseries2.c', 'prop_castchain.c', 'prop_concatop.c', + 'prop_coerce.c', ) diff --git a/src/test/prop/prop_coerce.c b/src/test/prop/prop_coerce.c new file mode 100644 index 0000000000000..802b7d8adaeb3 --- /dev/null +++ b/src/test/prop/prop_coerce.c @@ -0,0 +1,402 @@ +/*------------------------------------------------------------------------- + * + * prop_coerce.c + * Property-based tests for implicit/explicit coercion and operator + * resolution. + * + * These properties pin down the type-promotion rules the parser and executor + * apply when operands of different types meet: int + numeric promotes the int + * to numeric, int < float8 promotes the int to float8, || coerces int to + * text, and UNION/CASE/COALESCE unify branch types to a common type. The + * oracle is the explicit-cast form of the same expression: an implicit + * coercion must produce exactly the value an explicit cast would, so any "f" + * (or a differing UNION result set) is a resolution bug, not a value bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_coerce.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "coerce" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a modest int4 literal, parenthesized so a leading minus never + * misparses when substituted into a larger expression (and so a bare -N is + * never fed to ::int4 where it would reassociate). Range is deliberately + * small so counterexamples stay legible; coercion resolution is independent + * of magnitude, so there is nothing extra to find at the int4 extremes here. + */ +static void +draw_int(pg_prop_tc *tc, StringInfo buf) +{ + appendStringInfo(buf, "(%d)", (int) pg_prop_draw_int(tc, -1000, 1000)); +} + +/* + * Draw a decimal numeric literal as ".", parenthesized, cast to + * numeric. Modest digit counts; a fractional part is always present so the + * numeric genuinely differs from an int and the promotion is exercised. + */ +static void +draw_numeric(pg_prop_tc *tc, StringInfo buf) +{ + int nint = (int) pg_prop_draw_int(tc, 1, 6); + int nfrac = (int) pg_prop_draw_int(tc, 1, 6); + bool neg = pg_prop_draw_bool(tc); + + appendStringInfoString(buf, "("); + if (neg) + appendStringInfoChar(buf, '-'); + for (int i = 0; i < nint; i++) + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + appendStringInfoChar(buf, '.'); + for (int i = 0; i < nfrac; i++) + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + appendStringInfoString(buf, ")::numeric"); +} + +/* + * Draw a float8 literal as ".", parenthesized, cast to float8. + */ +static void +draw_float8(pg_prop_tc *tc, StringInfo buf) +{ + int nint = (int) pg_prop_draw_int(tc, 1, 6); + int nfrac = (int) pg_prop_draw_int(tc, 1, 6); + bool neg = pg_prop_draw_bool(tc); + + appendStringInfoString(buf, "("); + if (neg) + appendStringInfoChar(buf, '-'); + for (int i = 0; i < nint; i++) + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + appendStringInfoChar(buf, '.'); + for (int i = 0; i < nfrac; i++) + appendStringInfoChar(buf, (char) ('0' + pg_prop_draw_int(tc, 0, 9))); + appendStringInfoString(buf, ")::float8"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means coercion resolution failed or produced an unexpected + * value, which is the bug these properties exist to catch. (Copied in style + * from prop_numeric.c's require_true.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Differential result-set oracle: two queries must return byte-identical + * row-major strings. Both queries MUST carry a total ORDER BY. (Copied from + * the skeleton in area_spec.md.) + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * int + numeric: the int operand promotes to numeric, so (n + x) must equal + * (n::numeric + x) exactly. If promotion instead went the other way (numeric + * truncated to int) the sum would differ. + */ +PG_PROP(int_plus_numeric_is_numeric) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + initStringInfo(&x); + draw_int(tc, &n); + draw_numeric(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s + %s) = (%s::numeric + %s)", + n.data, x.data, n.data, x.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * int division truncates toward zero, whereas numeric division is exact: so + * (n / m) with int operands must equal trunc(n::numeric / m) for m <> 0. + * This checks that the integer / operator was resolved (not promoted to a + * numeric / that would keep the fraction). The CASE guards m = 0. + */ +PG_PROP(int_div_vs_numeric_div) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + m, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + initStringInfo(&m); + draw_int(tc, &n); + draw_int(tc, &m); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN %s = 0 THEN true " + "ELSE (%s / %s)::numeric = trunc(%s::numeric / %s::numeric) END", + m.data, + n.data, m.data, n.data, m.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(m.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * int < float8: the int promotes to float8 for the comparison, so (n < f) + * must equal (n::float8 < f). A wrong resolution (truncating f to int) would + * flip the comparison near integer boundaries. + */ +PG_PROP(mixed_compare) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + f, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + initStringInfo(&f); + draw_int(tc, &n); + draw_float8(tc, &f); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s < %s) = (%s::float8 < %s)", + n.data, f.data, n.data, f.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(f.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * UNION type unification (differential, shape B): a UNION ALL of an int and a + * numeric unifies the column to numeric. The result set, ordered, must match + * the same two values built directly as numeric. A total ORDER BY x makes + * the comparison a sound multiset check; the two drawn values are distinct + * (one has a fractional part) so there are no ties to disambiguate -- but the + * explicit-numeric side is produced the same way, so even equal values would + * compare identically. + */ +PG_PROP(union_unifies_int_numeric) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + x, + q1, + q2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + initStringInfo(&x); + draw_int(tc, &n); + draw_numeric(tc, &x); + + initStringInfo(&q1); + initStringInfo(&q2); + appendStringInfo(&q1, + "SELECT x FROM (SELECT %s AS x UNION ALL SELECT %s) s ORDER BY x", + n.data, x.data); + appendStringInfo(&q2, + "SELECT x FROM (VALUES (%s::numeric),(%s)) v(x) ORDER BY x", + n.data, x.data); + + require_same_rows(tc, conn, q1.data, q2.data); + + pfree(n.data); + pfree(x.data); + pfree(q1.data); + pfree(q2.data); + pg_prop_disconnect(conn); +} + +/* + * CASE result unification: the branches (int, numeric) unify to numeric, so + * CASE WHEN true THEN n ELSE x END must equal n::numeric exactly. If the + * int branch were not promoted, the WHEN-true value would come back as int + * and still equal n::numeric -- but we also check the type path by comparing + * against the numeric cast, which only agrees when the common type is numeric. + */ +PG_PROP(case_result_unification) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + initStringInfo(&x); + draw_int(tc, &n); + draw_numeric(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (CASE WHEN true THEN %s ELSE %s END) = %s::numeric", + n.data, x.data, n.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * COALESCE type unification: COALESCE(int, numeric) unifies to numeric. Both + * branches are checked: a non-NULL int first argument comes back equal to + * n::numeric, and a NULL int first argument falls through to the numeric + * second argument equal to x. + */ +PG_PROP(coalesce_type_unification) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + x, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + initStringInfo(&x); + draw_int(tc, &n); + draw_numeric(tc, &x); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT COALESCE(%s, %s) = %s::numeric " + "AND COALESCE(NULL::int, %s) = %s", + n.data, x.data, n.data, x.data, x.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(x.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * || coerces int to text: 'x' || n must equal 'x' || n::text. The text + * concatenation operator takes text operands, so the int is implicitly cast; + * the oracle makes that cast explicit. Any difference means the int was + * rendered differently by the implicit path. + */ +PG_PROP(concat_coerces) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + draw_int(tc, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ('x' || %s) = ('x' || %s::text)", + n.data, n.data); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 68bd1b8503197..76d2cd8c42b0d 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 3f21fa598fd0991ea196261826c28171f862c495 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:17 -0400 Subject: [PATCH 156/338] prop: add COLLATE clause ordering determinism property tests (collate) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_collate.c | 329 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 333 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_collate.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index c4c766ba429b7..09d2e0d116a8a 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -171,7 +171,8 @@ OBJS = \ prop_genseries2.o \ prop_castchain.o \ prop_concatop.o \ - prop_coerce.o + prop_coerce.o \ + prop_collate.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 53ac7faed7a80..57de99cb0af42 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -169,6 +169,7 @@ prop_area_sources = files( 'prop_castchain.c', 'prop_concatop.c', 'prop_coerce.c', + 'prop_collate.c', ) diff --git a/src/test/prop/prop_collate.c b/src/test/prop/prop_collate.c new file mode 100644 index 0000000000000..34baf128e2250 --- /dev/null +++ b/src/test/prop/prop_collate.c @@ -0,0 +1,329 @@ +/*------------------------------------------------------------------------- + * + * prop_collate.c + * Property-based tests for COLLATE clauses and ordering determinism. + * + * The oracle is the agreement between a collation's comparison and a known + * reference order. The "C" collation is defined to be raw byte order, so a + * COLLATE "C" comparison must equal the bytea comparison of the UTF-8 bytes; + * COLLATE "C" must also be a total order (exactly one of <, =, > holds) and + * reflexive on equality. For pure-ASCII input, "C" and "ucs_basic" are both + * codepoint order and must agree. An ORDER BY under a fixed collation must be + * deterministic across identical runs. Any disagreement is a bug in collation + * handling or sort stability, not an artefact of the test. + * + * Only built-in collations guaranteed present everywhere are used ("C" and + * "ucs_basic"), and all input is drawn from the ASCII alphabet [a-cA-C] with + * strings of length 0..8, so every property is sound regardless of the host's + * ICU/libc locale support. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_collate.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "collate" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* Alphabet and length bounds for every drawn string. */ +#define COLLATE_ALPHABET "abcABC" +#define STR_MIN 0 +#define STR_MAX 8 + +/* + * Draw a string from [a-cA-C], length 0..8, and append it to buf as a + * single-quoted SQL text literal. The alphabet contains no single quote, so + * no escaping is needed; double any quote defensively in case the alphabet + * ever grows. + */ +static void +draw_str_literal(pg_prop_tc *tc, StringInfo buf) +{ + size_t len = 0; + char *s = pg_prop_draw_from(tc, COLLATE_ALPHABET, STR_MIN, STR_MAX, &len); + + appendStringInfoChar(buf, '\''); + for (size_t i = 0; i < len; i++) + { + if (s[i] == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, s[i]); + } + appendStringInfoChar(buf, '\''); + pg_free(s); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the collation comparison errored or produced NULL, which + * is itself a failure, reported with the SQL. (Copied in spirit from + * prop_numeric.c's require_true.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Differential result-set oracle: two queries must return identical rows in + * identical order. (Copied from the area_spec skeleton.) + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * C collation is byte order: for two strings S, T, + * (S COLLATE "C" < T COLLATE "C") = (convert_to(S,'UTF8') < convert_to(T,'UTF8')) + * where the right side is a bytea comparison. "C" is defined as raw byte + * (memcmp) order, so the text comparison under "C" must match the UTF-8 byte + * comparison exactly, for every pair. + */ +PG_PROP(c_collation_is_byte_order) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + t, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&t); + draw_str_literal(tc, &s); + draw_str_literal(tc, &t); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s COLLATE \"C\" < %s COLLATE \"C\") " + "= (convert_to(%s, 'UTF8') < convert_to(%s, 'UTF8'))", + s.data, t.data, s.data, t.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Total order: for two strings exactly one of <, =, > holds under COLLATE "C". + * The three indicators cast to int must sum to exactly 1. + */ +PG_PROP(collate_total_order) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + t, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&t); + draw_str_literal(tc, &s); + draw_str_literal(tc, &t); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s COLLATE \"C\" < %s COLLATE \"C\")::int " + "+ (%s COLLATE \"C\" = %s COLLATE \"C\")::int " + "+ (%s COLLATE \"C\" > %s COLLATE \"C\")::int) = 1", + s.data, t.data, s.data, t.data, s.data, t.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Reflexivity: a string equals itself under COLLATE "C". + */ +PG_PROP(collate_equal_reflexive) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_str_literal(tc, &s); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s COLLATE \"C\" = %s COLLATE \"C\"", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ORDER BY under a fixed collation is deterministic: the same query, run + * twice over the same VALUES list, must return identical rows in identical + * order. A small VALUES list is built from drawn strings; ORDER BY x COLLATE + * "C" is a stable, total order over distinct byte strings but may tie on + * duplicates, so a tie-break on the ordinal keeps the comparison a true + * total order and the oracle sound. + */ +PG_PROP(order_by_collate_deterministic) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData vals, + q; + int nrows; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + nrows = (int) pg_prop_draw_int(tc, 1, 12); + + initStringInfo(&vals); + for (int i = 0; i < nrows; i++) + { + if (i > 0) + appendStringInfoString(&vals, ", "); + appendStringInfoChar(&vals, '('); + draw_str_literal(tc, &vals); + appendStringInfo(&vals, ", %d)", i); + } + + /* + * Order primarily by the collated string; break ties on the ordinal so + * the ordering is total and the two runs are legitimately required to + * match. Select the ordinal too so a reordering is visible. + */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT x, n FROM (VALUES %s) v(x, n) " + "ORDER BY x COLLATE \"C\", n", + vals.data); + + require_same_rows(tc, conn, q.data, q.data); + + pfree(vals.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * For pure-ASCII [a-cA-C] input, "C" and "ucs_basic" are both codepoint + * order, so a strict-less comparison under one must equal the same comparison + * under the other: + * (S COLLATE "C" < T COLLATE "C") = (S COLLATE "ucs_basic" < T COLLATE "ucs_basic"). + */ +PG_PROP(ucs_basic_eq_c_for_ascii) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + t, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&t); + draw_str_literal(tc, &s); + draw_str_literal(tc, &t); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s COLLATE \"C\" < %s COLLATE \"C\") " + "= (%s COLLATE \"ucs_basic\" < %s COLLATE \"ucs_basic\")", + s.data, t.data, s.data, t.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Uppercase sorts before lowercase in byte order: under "C", 'A' < 'a'. + * This is a fixed fact of ASCII (0x41 < 0x61), so it takes no drawn input; + * it guards against a "C" collation that is not actually raw byte order. + */ +PG_PROP(upper_lower_collate_c) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, "SELECT 'A' COLLATE \"C\" < 'a' COLLATE \"C\""); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 76d2cd8c42b0d..642f67bca8c9a 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 233a759a8ff35237c1492138c56586d60181773a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:17 -0400 Subject: [PATCH 157/338] prop: add text comparison and ordering under C collation property tests (textord) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_textord.c | 395 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 399 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_textord.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 09d2e0d116a8a..4b16daa2ce5da 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -172,7 +172,8 @@ OBJS = \ prop_castchain.o \ prop_concatop.o \ prop_coerce.o \ - prop_collate.o + prop_collate.o \ + prop_textord.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 57de99cb0af42..597f82eb6d2cd 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -170,6 +170,7 @@ prop_area_sources = files( 'prop_concatop.c', 'prop_coerce.c', 'prop_collate.c', + 'prop_textord.c', ) diff --git a/src/test/prop/prop_textord.c b/src/test/prop/prop_textord.c new file mode 100644 index 0000000000000..65efec6d59f8d --- /dev/null +++ b/src/test/prop/prop_textord.c @@ -0,0 +1,395 @@ +/*------------------------------------------------------------------------- + * + * prop_textord.c + * Property-based tests for text comparison and ordering under COLLATE "C". + * + * The oracle is the algebra of a total order: under the byte-wise C collation, + * text comparison is a strict total order on strings (trichotomy, reflexivity, + * transitivity), the empty string is least, lexicographic order ignores length + * (shorter is not automatically smaller), and appending a byte never decreases + * a string's rank (strictly increases it). Every comparison forces COLLATE + * "C" so the result is deterministic and locale-independent; a broken identity + * here is a bug in C-collation text comparison, not a locale artefact. + * + * Distinct from the collate / textcase areas: those cover collation selection + * and case folding. This area asserts only the ordering laws of COLLATE "C". + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_textord.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "textord" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabet is [a-c] and never contains a quote, but do it + * defensively so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..8 letters from [a-c] into buf as a bare string. All single-byte, + * never a quote, so quoting is trivial and the byte-wise C ordering is exactly + * the ordering of these ASCII code points. + */ +static void +draw_letters(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 8); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, (char) ('a' + pg_prop_draw_int(tc, 0, 2))); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the comparison errored or produced NULL on well-formed + * [a-c] strings, which is itself a bug -- so either is a failure, reported + * with the SQL. Modelled on prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Trichotomy: for two strings S, T, exactly one of (<, =, >) holds under + * COLLATE "C". Counting the three mutually exclusive comparisons must sum to + * exactly 1 -- a total order never leaves two strings incomparable or + * simultaneously in two relations. + */ +PG_PROP(total_order_c) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + t, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&t); + draw_letters(tc, &s); + draw_letters(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " COLLATE \"C\" < "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, " COLLATE \"C\")::int + ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " COLLATE \"C\" = "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, " COLLATE \"C\")::int + ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " COLLATE \"C\" > "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, " COLLATE \"C\")::int = 1"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Prefix is <= its extension: S COLLATE "C" <= (S || 'a') COLLATE "C". Under + * C collation a string is never greater than itself with a byte appended -- + * either S is a proper prefix of (S || 'a') (so strictly less), or they are + * equal only if the appended byte produced an identical string, which it + * cannot since one is strictly longer; so <= always holds. + */ +PG_PROP(prefix_le) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " COLLATE \"C\" <= ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " || 'a') COLLATE \"C\""); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The empty string is least: '' COLLATE "C" <= S COLLATE "C" for any S. The + * empty string is a prefix of every string, so it is never greater than any + * string under C collation. + */ +PG_PROP(empty_least) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT '' COLLATE \"C\" <= "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " COLLATE \"C\""); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Appending a byte strictly increases: (S || 'x') COLLATE "C" > S COLLATE "C". + * S is a proper prefix of (S || 'x'), and under C collation a proper prefix is + * always strictly less than its extension, so the extension is strictly + * greater. ('x' is outside [a-c] but still a plain single-byte ASCII letter, + * which keeps the comparison byte-wise and quote-free.) + */ +PG_PROP(concat_monotone_prefix) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " || 'x') COLLATE \"C\" > "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " COLLATE \"C\""); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Ordering is lexicographic, not by length: 'b' COLLATE "C" > 'aa' COLLATE + * "C". A length-based order would rank the one-byte 'b' below the two-byte + * 'aa'; C collation compares byte-wise, where 'b' (0x62) exceeds 'a' (0x61) at + * the first byte, so 'b' is greater regardless of being shorter. Constant + * oracle -- no inputs drawn. + */ +PG_PROP(length_not_order) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT 'b' COLLATE \"C\" > 'aa' COLLATE \"C\""); + + pg_prop_disconnect(conn); +} + +/* + * Reflexivity: S COLLATE "C" = S, and S <= S, and S >= S. Every string equals + * itself and is both <= and >= itself under any total order. + */ +PG_PROP(reflexive) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + draw_letters(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " COLLATE \"C\" = "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " COLLATE \"C\" AND "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " COLLATE \"C\" <= "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " COLLATE \"C\" AND "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " COLLATE \"C\" >= "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " COLLATE \"C\""); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Transitivity: for S, T, U, NOT (S < T AND T < U) OR (S < U) -- i.e. S < T + * and T < U together imply S < U. A total order is transitive; a comparison + * bug that broke this would make sorts and index scans inconsistent. + */ +PG_PROP(transitivity) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + t, + u, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&t); + initStringInfo(&u); + draw_letters(tc, &s); + draw_letters(tc, &t); + draw_letters(tc, &u); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT NOT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " COLLATE \"C\" < "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, " COLLATE \"C\" AND "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, " COLLATE \"C\" < "); + append_sql_literal(&q, u.data); + appendStringInfoString(&q, " COLLATE \"C\") OR ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " COLLATE \"C\" < "); + append_sql_literal(&q, u.data); + appendStringInfoString(&q, " COLLATE \"C\")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(u.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * least/greatest consistency: least(S, T) COLLATE "C" <= greatest(S, T) + * COLLATE "C". least and greatest pick the min and max under the same C + * collation, so the chosen min can never exceed the chosen max. + */ +PG_PROP(min_max_consistency) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s, + t, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&s); + initStringInfo(&t); + draw_letters(tc, &s); + draw_letters(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT least("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " COLLATE \"C\", "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, " COLLATE \"C\") COLLATE \"C\" <= greatest("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " COLLATE \"C\", "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, " COLLATE \"C\") COLLATE \"C\""); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 642f67bca8c9a..522107888a15f 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From c3410566d9997add34b1f974f052a389e0e6a19d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:17 -0400 Subject: [PATCH 158/338] prop: add json vs jsonb normalization differences property tests (jsonvsjsonb) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonvsjsonb.c | 229 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 233 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonvsjsonb.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 4b16daa2ce5da..b47cc7be4fd5f 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -173,7 +173,8 @@ OBJS = \ prop_concatop.o \ prop_coerce.o \ prop_collate.o \ - prop_textord.o + prop_textord.o \ + prop_jsonvsjsonb.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 597f82eb6d2cd..d6b6376a92fd9 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -171,6 +171,7 @@ prop_area_sources = files( 'prop_coerce.c', 'prop_collate.c', 'prop_textord.c', + 'prop_jsonvsjsonb.c', ) diff --git a/src/test/prop/prop_jsonvsjsonb.c b/src/test/prop/prop_jsonvsjsonb.c new file mode 100644 index 0000000000000..323740aec08ef --- /dev/null +++ b/src/test/prop/prop_jsonvsjsonb.c @@ -0,0 +1,229 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonvsjsonb.c + * Property-based tests for json vs jsonb type-difference semantics. + * + * json is textual: it preserves input whitespace, key order and duplicate + * keys verbatim. jsonb is a decomposed binary form: it discards whitespace, + * sorts keys, keeps only the last of duplicate keys, and normalizes numbers. + * These oracles pin down each documented difference and the json->jsonb + * normalization bridge; a failure means the type's I/O or the cast changed a + * guaranteed semantic (e.g. jsonb stopped deduping, or json stopped + * preserving order), which is a bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonvsjsonb.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonvsjsonb" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error is itself a failure, reported with the SQL that produced it. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * json preserves input key order as text, while jsonb sorts keys. So the + * json text round-trips byte-for-byte with the input order, and the jsonb of + * the same input equals the jsonb of the key-sorted spelling. + */ +PG_PROP(json_preserves_order) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT '{\"b\":1,\"a\":2}'::json::text = '{\"b\":1,\"a\":2}' " + "AND '{\"b\":1,\"a\":2}'::jsonb = '{\"a\":2,\"b\":1}'::jsonb"); + + pg_prop_disconnect(conn); +} + +/* + * jsonb keeps only the last of duplicate keys; a json carrying duplicates + * cast to jsonb collapses to that same last-wins object. + */ +PG_PROP(json_keeps_duplicates_jsonb_dedups) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT '{\"a\":1,\"a\":2}'::jsonb = '{\"a\":2}'::jsonb " + "AND '{\"a\":1,\"a\":2}'::json::jsonb = '{\"a\":2}'::jsonb"); + + pg_prop_disconnect(conn); +} + +/* + * Casting a json object to jsonb normalizes it to the same value jsonb would + * build directly. For any drawn integer n, '{"x": n}'::json::jsonb equals + * jsonb_build_object('x', n). + */ +PG_PROP(json_to_jsonb_normalizes) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, INT32_MIN, INT32_MAX); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT '{\"x\": %d}'::json::jsonb " + "= jsonb_build_object('x', (%d)::int4)", + n, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * jsonb numbers are compared by numeric value, so differently spelled but + * numerically equal literals are equal: 1.0 = 1.00. + */ +PG_PROP(same_value_different_text) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT '1.0'::jsonb = '1.00'::jsonb"); + + pg_prop_disconnect(conn); +} + +/* + * Array element order is significant and preserved in both json and jsonb, so + * indexing a jsonb array yields the element at that input position. + */ +PG_PROP(json_array_order_preserved) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT ('[3,1,2]'::jsonb -> 0) = '3'::jsonb"); + + pg_prop_disconnect(conn); +} + +/* + * jsonb ignores insignificant whitespace: a spaced-out object parses to the + * same value as its compact spelling. + */ +PG_PROP(jsonb_whitespace_insensitive) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT '{ \"a\" : 1 }'::jsonb = '{\"a\":1}'::jsonb"); + + pg_prop_disconnect(conn); +} + +/* + * A built object serialized to json text round-trips to the same jsonb value: + * O::json::text::jsonb = O::jsonb. The json textual form carries all the + * information jsonb needs, so the detour through text is lossless at the jsonb + * level. + */ +PG_PROP(json_valid_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "WITH o AS (SELECT json_build_object(" + "'a', 1, 'b', '[2,3]'::json, 'c', 'x') AS j) " + "SELECT (j::text)::jsonb = j::jsonb FROM o"); + + pg_prop_disconnect(conn); +} + +/* + * row_to_json and to_jsonb agree on composite values once normalized through + * jsonb: row_to_json(ROW(1,2))::jsonb = to_jsonb(ROW(1,2)), both {"f1":1,"f2":2}. + */ +PG_PROP(row_to_json_vs_jsonb) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT row_to_json(ROW(1,2))::jsonb = to_jsonb(ROW(1,2))"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 522107888a15f..68f55aca15d20 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 8f8d14acd29e4a3bf267d65e1b604c34cdde8632 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:17 -0400 Subject: [PATCH 159/338] prop: add geopoint property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_geopoint.c | 342 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 346 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_geopoint.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index b47cc7be4fd5f..f4e52be7ea7e4 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -174,7 +174,8 @@ OBJS = \ prop_coerce.o \ prop_collate.o \ prop_textord.o \ - prop_jsonvsjsonb.o + prop_jsonvsjsonb.o \ + prop_geopoint.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index d6b6376a92fd9..69384eb2972e1 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -172,6 +172,7 @@ prop_area_sources = files( 'prop_collate.c', 'prop_textord.c', 'prop_jsonvsjsonb.c', + 'prop_geopoint.c', ) diff --git a/src/test/prop/prop_geopoint.c b/src/test/prop/prop_geopoint.c new file mode 100644 index 0000000000000..d98630b61cf9f --- /dev/null +++ b/src/test/prop/prop_geopoint.c @@ -0,0 +1,342 @@ +/*------------------------------------------------------------------------- + * + * prop_geopoint.c + * Property-based tests for geometric point/line/box operations. + * + * Target: src/backend/utils/adt/geo_ops.c. The geometric types (point, box, + * line segment) carry exact invariants (box containment, point-in-box) and + * metric invariants (distance symmetry, non-negativity, Euclidean distance), + * which these properties check with float tolerances where arithmetic is + * inexact and with exact equality where it is not. + * + * Coordinates are drawn as small integers (-100..100) and printed into point + * and box literals so that areas and containment tests are exact and the + * coordinate-range oracle (BETWEEN) matches the server's <@ exactly. + * Distances still involve sqrt, so those are compared within a tolerance. + * + * The oracle here is an independent formula for each operation: distance is + * checked against the Euclidean formula sqrt(dx^2+dy^2); box area against + * |x2-x1|*|y2-y1|; point-in-box against an explicit coordinate-range test. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_geopoint.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "geopoint" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw one small integer coordinate in [-100, 100]. Small integers keep the + * literals exact: box area and containment are then exact and the BETWEEN + * oracle matches the server's inclusive <@ on boundaries. + */ +static int +draw_coord(pg_prop_tc *tc) +{ + return (int) pg_prop_draw_int(tc, -100, 100); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the geometric operation errored or produced NULL on + * finite integer-coordinate inputs, which is itself a bug -- so either is a + * failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Point distance is a non-negative, symmetric metric: (P <-> Q) >= 0 and + * (P <-> Q) = (Q <-> P). The reverse ordering recomputes the same distance, + * so any asymmetry in the distance code surfaces here. + */ +PG_PROP(distance_nonneg_symmetric) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int px, + py, + qx, + qy; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + px = draw_coord(tc); + py = draw_coord(tc); + qx = draw_coord(tc); + qy = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('(%d,%d)'::point <-> '(%d,%d)'::point) >= 0 " + "AND ('(%d,%d)'::point <-> '(%d,%d)'::point) " + "= ('(%d,%d)'::point <-> '(%d,%d)'::point)", + px, py, qx, qy, + px, py, qx, qy, + qx, qy, px, py); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Distance of a point to itself is zero: (P <-> P) = 0. Integer coordinates + * make this exact, so no tolerance is needed. + */ +PG_PROP(distance_zero_iff_equal) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int px, + py; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + px = draw_coord(tc); + py = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('(%d,%d)'::point <-> '(%d,%d)'::point) = 0", + px, py, px, py); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Point distance equals the Euclidean distance sqrt(dx^2 + dy^2). The + * independent oracle is the Pythagorean formula computed in float8; sqrt makes + * this inexact, so compare within a tolerance. Coordinates are cast to + * float8 so ^ is float power rather than integer exponentiation. + */ +PG_PROP(distance_matches_pythagoras) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int px, + py, + qx, + qy; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + px = draw_coord(tc); + py = draw_coord(tc); + qx = draw_coord(tc); + qy = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(('(%d,%d)'::point <-> '(%d,%d)'::point) " + "- sqrt((%d - %d)::float8 ^ 2 + (%d - %d)::float8 ^ 2)) < 1e-6", + px, py, qx, qy, + px, qx, py, qy); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A box contains both of its defining corner points: B @> corner1 and + * B @> corner2. @> is box-contains-point; integer coordinates make the + * inclusive boundary test exact. + */ +PG_PROP(box_contains_its_corners) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int x1, + y1, + x2, + y2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x1 = draw_coord(tc); + y1 = draw_coord(tc); + x2 = draw_coord(tc); + y2 = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('(%d,%d),(%d,%d)'::box @> '(%d,%d)'::point) " + "AND ('(%d,%d),(%d,%d)'::box @> '(%d,%d)'::point)", + x1, y1, x2, y2, x1, y1, + x1, y1, x2, y2, x2, y2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A box contains its own center point. @@ applied to a box yields its center + * point, which must lie inside (or on the boundary of) the box. + */ +PG_PROP(box_contains_center) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int x1, + y1, + x2, + y2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x1 = draw_coord(tc); + y1 = draw_coord(tc); + x2 = draw_coord(tc); + y2 = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('(%d,%d),(%d,%d)'::box " + "@> (@@ '(%d,%d),(%d,%d)'::box))", + x1, y1, x2, y2, x1, y1, x2, y2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Box area is non-negative and equals |x2-x1| * |y2-y1|. The independent + * oracle is the width-times-height product in float8; area() uses the same + * exact integer coordinates, so the two agree within a tiny tolerance. + */ +PG_PROP(box_area_nonneg_and_matches) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int x1, + y1, + x2, + y2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x1 = draw_coord(tc); + y1 = draw_coord(tc); + x2 = draw_coord(tc); + y2 = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT area('(%d,%d),(%d,%d)'::box) >= 0 " + "AND abs(area('(%d,%d),(%d,%d)'::box) " + "- (abs(%d - %d) * abs(%d - %d))::float8) < 1e-6", + x1, y1, x2, y2, + x1, y1, x2, y2, + x2, x1, y2, y1); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Point-in-box agrees with the explicit coordinate-range test: (P <@ B) holds + * exactly when P.x lies in the box's x-range and P.y in its y-range. The + * independent oracle is the BETWEEN test over least/greatest of the corner + * coordinates; integer coordinates make BETWEEN exact, and <@ is inclusive on + * the boundary, matching BETWEEN. + */ +PG_PROP(point_in_box_consistent) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int px, + py, + x1, + y1, + x2, + y2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + px = draw_coord(tc); + py = draw_coord(tc); + x1 = draw_coord(tc); + y1 = draw_coord(tc); + x2 = draw_coord(tc); + y2 = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('(%d,%d)'::point <@ '(%d,%d),(%d,%d)'::box) " + "= (%d BETWEEN least(%d,%d) AND greatest(%d,%d) " + "AND %d BETWEEN least(%d,%d) AND greatest(%d,%d))", + px, py, x1, y1, x2, y2, + px, x1, x2, x1, x2, + py, y1, y2, y1, y2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 68f55aca15d20..da0e3db4535ca 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 42e1fc7d2cf886941387a4b310a1a9b68205155a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:17 -0400 Subject: [PATCH 160/338] prop: add line-segment and circle geometry property tests (geoseg) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_geoseg.c | 343 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 347 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_geoseg.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index f4e52be7ea7e4..0030dc9f94875 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -175,7 +175,8 @@ OBJS = \ prop_collate.o \ prop_textord.o \ prop_jsonvsjsonb.o \ - prop_geopoint.o + prop_geopoint.o \ + prop_geoseg.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 69384eb2972e1..c3c8436e2598d 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -173,6 +173,7 @@ prop_area_sources = files( 'prop_textord.c', 'prop_jsonvsjsonb.c', 'prop_geopoint.c', + 'prop_geoseg.c', ) diff --git a/src/test/prop/prop_geoseg.c b/src/test/prop/prop_geoseg.c new file mode 100644 index 0000000000000..250944152b9f2 --- /dev/null +++ b/src/test/prop/prop_geoseg.c @@ -0,0 +1,343 @@ +/*------------------------------------------------------------------------- + * + * prop_geoseg.c + * Property-based tests for geometric line-segment, line and circle ops. + * + * Target: src/backend/utils/adt/geo_ops.c, the lseg/line/circle operations + * distinct from the point/box area covered by prop_geopoint.c. Line segments + * and circles carry length/radius/containment invariants: a segment's length + * is the distance between its endpoints and its endpoints lie on it; a circle + * contains its own center, its area is pi*r^2, its stored radius round-trips, + * and a point lies in it exactly when its distance to the center is within the + * radius. These properties check those invariants with float tolerances where + * sqrt/pi make arithmetic inexact, and with exact equality where the stored + * integer radius is exact. + * + * Coordinates and radii are drawn as small integers and printed into lseg, + * point and circle literals so that lengths and areas are near-exact (only the + * sqrt/pi rounding needs tolerance) and the integer radius round-trips + * exactly. + * + * The oracle here is an independent formula for each operation: segment length + * against the point-distance operator; circle area against pi*r^2; point-in- + * circle against the explicit distance-to-center test. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_geoseg.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "geoseg" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw one small integer coordinate in [-100, 100]. Small integers keep the + * literals exact: segment lengths and circle areas are then off from the + * independent formula only by sqrt/pi rounding, and the integer radius is + * stored exactly. + */ +static int +draw_coord(pg_prop_tc *tc) +{ + return (int) pg_prop_draw_int(tc, -100, 100); +} + +/* + * Draw one strictly-positive integer radius in [1, 50]. r > 0 keeps the + * circle non-degenerate so it has an interior that contains its center. + */ +static int +draw_radius(pg_prop_tc *tc) +{ + return (int) pg_prop_draw_int(tc, 1, 50); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the geometric operation errored or produced NULL on finite + * integer-coordinate inputs, which is itself a bug -- so either is a failure, + * reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * A segment's length equals the distance between its endpoints: the independent + * oracle is the point-distance operator over the two endpoints. Both compute a + * Euclidean distance via sqrt, so compare within a tolerance. + */ +PG_PROP(lseg_length_matches_endpoint_distance) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int x1, + y1, + x2, + y2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x1 = draw_coord(tc); + y1 = draw_coord(tc); + x2 = draw_coord(tc); + y2 = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(length('[(%d,%d),(%d,%d)]'::lseg) " + "- ('(%d,%d)'::point <-> '(%d,%d)'::point)) < 1e-6", + x1, y1, x2, y2, + x1, y1, x2, y2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A segment's length is non-negative: length([P,Q]) >= 0. A length is a + * magnitude, so it can never be negative regardless of endpoint ordering. + */ +PG_PROP(lseg_length_nonneg) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int x1, + y1, + x2, + y2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x1 = draw_coord(tc); + y1 = draw_coord(tc); + x2 = draw_coord(tc); + y2 = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length('[(%d,%d),(%d,%d)]'::lseg) >= 0", + x1, y1, x2, y2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Both defining endpoints lie on the segment: P <@ [P,Q] and Q <@ [P,Q]. For + * point/lseg, <@ means "point is on segment"; the endpoints are trivially on + * it, and integer coordinates make the on-segment test exact. + */ +PG_PROP(lseg_endpoints_on_segment) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int x1, + y1, + x2, + y2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x1 = draw_coord(tc); + y1 = draw_coord(tc); + x2 = draw_coord(tc); + y2 = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('(%d,%d)'::point <@ '[(%d,%d),(%d,%d)]'::lseg) " + "AND ('(%d,%d)'::point <@ '[(%d,%d),(%d,%d)]'::lseg)", + x1, y1, x1, y1, x2, y2, + x2, y2, x1, y1, x2, y2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A circle contains its own center: C @> center. @> is circle-contains-point; + * for a positive radius the center is strictly interior, so containment must + * hold. + */ +PG_PROP(circle_center_inside) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int cx, + cy, + r; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + cx = draw_coord(tc); + cy = draw_coord(tc); + r = draw_radius(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT '<(%d,%d),%d>'::circle @> '(%d,%d)'::point", + cx, cy, r, cx, cy); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A circle's area equals pi * r^2: the independent oracle is pi()*r^2 computed + * in float8. area() and the formula both involve pi, so compare within a + * tolerance. The radius is cast to float8 so ^ is float power. + */ +PG_PROP(circle_area_matches_formula) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int cx, + cy, + r; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + cx = draw_coord(tc); + cy = draw_coord(tc); + r = draw_radius(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(area('<(%d,%d),%d>'::circle) " + "- (pi() * %d::float8 ^ 2)) < 1e-6", + cx, cy, r, r); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A circle's stored radius round-trips: radius('<(cx,cy),r>') = r. The radius + * is an integer and stored exactly, so this holds with no tolerance. + */ +PG_PROP(circle_radius_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int cx, + cy, + r; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + cx = draw_coord(tc); + cy = draw_coord(tc); + r = draw_radius(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT radius('<(%d,%d),%d>'::circle) = %d", + cx, cy, r, r); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A point lies in a circle exactly when its distance to the center is within + * the radius: (P <@ C) = (dist(center, P) <= r). The independent oracle is the + * point-distance operator against the center; a tiny epsilon on the right-hand + * side handles the boundary case (distance exactly r) consistently with the + * server's inclusive containment. + */ +PG_PROP(point_in_circle_iff_distance_le_radius) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int px, + py, + cx, + cy, + r; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + px = draw_coord(tc); + py = draw_coord(tc); + cx = draw_coord(tc); + cy = draw_coord(tc); + r = draw_radius(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('(%d,%d)'::point <@ '<(%d,%d),%d>'::circle) " + "= (('(%d,%d)'::point <-> '(%d,%d)'::point) " + "<= %d::float8 + 1e-9)", + px, py, cx, cy, r, + cx, cy, px, py, r); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index da0e3db4535ca..812e96040b8a0 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 1f92e77badfe3b0e8d81be3a4faa871bc71a9d8d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:17 -0400 Subject: [PATCH 161/338] prop: add geometric box and polygon operation property tests (boxops) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_boxops.c | 412 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 416 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_boxops.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 0030dc9f94875..dee97120d3e5c 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -176,7 +176,8 @@ OBJS = \ prop_textord.o \ prop_jsonvsjsonb.o \ prop_geopoint.o \ - prop_geoseg.o + prop_geoseg.o \ + prop_boxops.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index c3c8436e2598d..557877b86816d 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -174,6 +174,7 @@ prop_area_sources = files( 'prop_jsonvsjsonb.c', 'prop_geopoint.c', 'prop_geoseg.c', + 'prop_boxops.c', ) diff --git a/src/test/prop/prop_boxops.c b/src/test/prop/prop_boxops.c new file mode 100644 index 0000000000000..6847a119dc386 --- /dev/null +++ b/src/test/prop/prop_boxops.c @@ -0,0 +1,412 @@ +/*------------------------------------------------------------------------- + * + * prop_boxops.c + * Property-based tests for geometric box and polygon operations. + * + * Target: src/backend/utils/adt/geo_ops.c, the box/polygon operations distinct + * from the point/lseg/circle areas covered by prop_geopoint.c and + * prop_geoseg.c. A box carries area/center/containment invariants: its area is + * non-negative and equals the product of its side lengths; it contains its own + * center and corners; the intersection of two overlapping boxes is contained in + * each; the bounding box of two boxes covers both; and a polygon built from a + * box encloses the same area. width()/height() report the side lengths. + * + * The oracle here is an independent formula or a containment relation for each + * operation: box area against abs(x2-x1)*abs(y2-y1); self-containment of center + * and corners; intersection and bound_box containment; polygon(box) area + * against box area. Integer corners keep areas and side lengths exact, so only + * the area comparisons need a tolerance. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_boxops.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "boxops" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw one small integer coordinate in [-50, 50]. Small integers keep the + * literals exact: box areas and side lengths are then integer-valued and the + * area comparisons are off from the independent formula only by any float + * rounding inside area()/polygon(). + */ +static int +draw_coord(pg_prop_tc *tc) +{ + return (int) pg_prop_draw_int(tc, -50, 50); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the geometric operation errored or produced NULL on finite + * integer-coordinate inputs, which is itself a bug -- so either is a failure, + * reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * A box's area is non-negative: area(B) >= 0. An area is a magnitude, so it + * can never be negative regardless of corner ordering. (The box type + * normalises corners on input, but the invariant holds either way.) + */ +PG_PROP(area_nonneg) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int x1, + y1, + x2, + y2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x1 = draw_coord(tc); + y1 = draw_coord(tc); + x2 = draw_coord(tc); + y2 = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT area('((%d,%d),(%d,%d))'::box) >= 0", + x1, y1, x2, y2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A box's area equals the product of its side lengths: area(B) = + * abs(x2-x1)*abs(y2-y1). The box type normalises corners on input, so the + * identity is independent of which corner was given first; integer corners make + * both sides exact, so the only slack is any float rounding in area(). + */ +PG_PROP(area_formula) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int x1, + y1, + x2, + y2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x1 = draw_coord(tc); + y1 = draw_coord(tc); + x2 = draw_coord(tc); + y2 = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(area('((%d,%d),(%d,%d))'::box) " + "- (abs(%d - (%d)) * abs(%d - (%d)))::float8) < 1e-6", + x1, y1, x2, y2, + x2, x1, y2, y1); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A box contains its own center: B @> center(B). @> is box-contains-point and + * the center lies within the box's (inclusive) bounds, so containment must + * hold for any box. + */ +PG_PROP(contains_center) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int x1, + y1, + x2, + y2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x1 = draw_coord(tc); + y1 = draw_coord(tc); + x2 = draw_coord(tc); + y2 = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT '((%d,%d),(%d,%d))'::box " + "@> center('((%d,%d),(%d,%d))'::box)", + x1, y1, x2, y2, + x1, y1, x2, y2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A box contains its own defining corners: B @> point(x1,y1) AND + * B @> point(x2,y2). The corners lie on the box's boundary, and box + * containment is inclusive, so both must be contained. + */ +PG_PROP(contains_corners) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int x1, + y1, + x2, + y2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x1 = draw_coord(tc); + y1 = draw_coord(tc); + x2 = draw_coord(tc); + y2 = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('((%d,%d),(%d,%d))'::box @> point(%d, %d)) " + "AND ('((%d,%d),(%d,%d))'::box @> point(%d, %d))", + x1, y1, x2, y2, x1, y1, + x1, y1, x2, y2, x2, y2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * When two boxes overlap, their intersection is contained in each: when + * B1 && B2, (B1 # B2) <@ B1 AND (B1 # B2) <@ B2. # is box intersection and <@ + * is box-contained-in-box. The intersection is only defined when the boxes + * overlap, so the property is guarded by CASE WHEN B1 && B2 (the non-overlapping + * case is vacuously true). + */ +PG_PROP(intersection_contained) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int ax1, + ay1, + ax2, + ay2, + bx1, + by1, + bx2, + by2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + ax1 = draw_coord(tc); + ay1 = draw_coord(tc); + ax2 = draw_coord(tc); + ay2 = draw_coord(tc); + bx1 = draw_coord(tc); + by1 = draw_coord(tc); + bx2 = draw_coord(tc); + by2 = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN " + "'((%d,%d),(%d,%d))'::box && '((%d,%d),(%d,%d))'::box " + "THEN ('((%d,%d),(%d,%d))'::box # '((%d,%d),(%d,%d))'::box) " + "<@ '((%d,%d),(%d,%d))'::box " + "AND ('((%d,%d),(%d,%d))'::box # '((%d,%d),(%d,%d))'::box) " + "<@ '((%d,%d),(%d,%d))'::box " + "ELSE true END", + ax1, ay1, ax2, ay2, bx1, by1, bx2, by2, + ax1, ay1, ax2, ay2, bx1, by1, bx2, by2, + ax1, ay1, ax2, ay2, + ax1, ay1, ax2, ay2, bx1, by1, bx2, by2, + bx1, by1, bx2, by2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The bounding box of two boxes covers both: bound_box(B1, B2) @> B1 AND @> B2. + * bound_box is the smallest box covering both operands, so each operand is + * contained in it; @> here is box-contains-box. + */ +PG_PROP(bound_box_contains) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int ax1, + ay1, + ax2, + ay2, + bx1, + by1, + bx2, + by2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + ax1 = draw_coord(tc); + ay1 = draw_coord(tc); + ax2 = draw_coord(tc); + ay2 = draw_coord(tc); + bx1 = draw_coord(tc); + by1 = draw_coord(tc); + bx2 = draw_coord(tc); + by2 = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (bound_box('((%d,%d),(%d,%d))'::box, " + "'((%d,%d),(%d,%d))'::box) @> '((%d,%d),(%d,%d))'::box) " + "AND (bound_box('((%d,%d),(%d,%d))'::box, " + "'((%d,%d),(%d,%d))'::box) @> '((%d,%d),(%d,%d))'::box)", + ax1, ay1, ax2, ay2, bx1, by1, bx2, by2, + ax1, ay1, ax2, ay2, + ax1, ay1, ax2, ay2, bx1, by1, bx2, by2, + bx1, by1, bx2, by2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A polygon built from a box encloses the same area: area(polygon(B)) = + * area(B). polygon(box) produces the four-corner rectangle, so the two area + * computations must agree; integer corners make both exact up to any float + * rounding, so compare within a tolerance. + */ +PG_PROP(box_to_polygon_contains_center) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int x1, + y1, + x2, + y2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x1 = draw_coord(tc); + y1 = draw_coord(tc); + x2 = draw_coord(tc); + y2 = draw_coord(tc); + + /* + * area(polygon) is not defined, but the polygon derived from a box must + * still cover the box's centre point: polygon(box) @> center(box). + */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT polygon('((%d,%d),(%d,%d))'::box) " + "@> center('((%d,%d),(%d,%d))'::box)", + x1, y1, x2, y2, + x1, y1, x2, y2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A box's width and height equal its side lengths: width(B) = abs(x2-x1) AND + * height(B) = abs(y2-y1). The box type normalises corners on input, so each + * side length is independent of input order; integer corners make both sides + * exact, so compare within a tolerance to absorb any float rounding. + */ +PG_PROP(width_height) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int x1, + y1, + x2, + y2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + x1 = draw_coord(tc); + y1 = draw_coord(tc); + x2 = draw_coord(tc); + y2 = draw_coord(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(width('((%d,%d),(%d,%d))'::box) " + "- abs(%d - (%d))::float8) < 1e-6 " + "AND abs(height('((%d,%d),(%d,%d))'::box) " + "- abs(%d - (%d))::float8) < 1e-6", + x1, y1, x2, y2, x2, x1, + x1, y1, x2, y2, y2, y1); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 812e96040b8a0..b97e524190c26 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 1b74a50afc46188356487958a5b509263e875dac Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:17 -0400 Subject: [PATCH 162/338] prop: add XML construction and escaping property tests (xmlfmt) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_xmlfmt.c | 388 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 392 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_xmlfmt.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index dee97120d3e5c..ada2d62f6b93a 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -177,7 +177,8 @@ OBJS = \ prop_jsonvsjsonb.o \ prop_geopoint.o \ prop_geoseg.o \ - prop_boxops.o + prop_boxops.o \ + prop_xmlfmt.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 557877b86816d..e2382a2211475 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -175,6 +175,7 @@ prop_area_sources = files( 'prop_geopoint.c', 'prop_geoseg.c', 'prop_boxops.c', + 'prop_xmlfmt.c', ) diff --git a/src/test/prop/prop_xmlfmt.c b/src/test/prop/prop_xmlfmt.c new file mode 100644 index 0000000000000..7fea65e1de58e --- /dev/null +++ b/src/test/prop/prop_xmlfmt.c @@ -0,0 +1,388 @@ +/*------------------------------------------------------------------------- + * + * prop_xmlfmt.c + * Property-based tests for XML construction/predicate functions. + * + * The XML constructors (xmlelement, xmlforest, xmlconcat, xmlagg) and the + * xml predicates (IS DOCUMENT, xmlexists) in src/backend/utils/adt/xml.c + * escape content and build well-formed fragments; these check element + * wrapping, concatenation, escaping of the XML-special characters < > &, and + * the IS DOCUMENT predicate, keeping the source ASCII. + * + * The oracle is textual: xmlelement wraps content in matching tags and + * escapes markup; xmlconcat and xmlagg concatenate fragments in order; and + * IS DOCUMENT distinguishes a single-rooted value from a multi-root fragment. + * Each property compares the server's answer to an independently constructed + * expected string (built with SQL string concatenation so no raw '<' sits + * next to an interpolated identifier) and requires a boolean "t". + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_xmlfmt.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "xmlfmt" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our content alphabet never contains a single quote, but do + * it defensively so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + const char *p; + + appendStringInfoChar(buf, '\''); + for (p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw a valid XML element name of 1..6 characters from [a-z] into buf as a + * bare identifier (no quoting). Letters only, so the result is always a + * valid NCName and can be used as a bare identifier in xmlelement(name N,...). + */ +static void +draw_name(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abcdefghijklmnopqrstuvwxyz"; + int n = (int) pg_prop_draw_int(tc, 1, 6); + int i; + + for (i = 0; i < n; i++) + appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 25)]); +} + +/* + * Draw 0..10 characters of content into buf as a bare string (no quoting), + * from a pool that INCLUDES the XML-special characters < > & plus letters, a + * space, and a double quote -- exactly the bytes xmlelement must escape in + * element content. The single quote is deliberately excluded so the SQL + * literal quoting stays trivial. + */ +static void +draw_content(pg_prop_tc *tc, StringInfo buf) +{ + static const char pool[] = "abcdefghij<>& \""; + int n = (int) pg_prop_draw_int(tc, 0, 10); + int i; + + for (i = 0; i < n; i++) + appendStringInfoChar(buf, pool[pg_prop_draw_int(tc, 0, (int) sizeof(pool) - 2)]); +} + +/* + * Draw 0..10 characters of content from letters only (no < > & " space) into + * buf as a bare string. Used by properties whose expected text is built by + * literal substitution, where any special character would need escaping. + */ +static void +draw_content_letters(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abcdefghij"; + int n = (int) pg_prop_draw_int(tc, 0, 10); + int i; + + for (i = 0; i < n; i++) + appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 9)]); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the construction errored or produced NULL on well-formed + * ASCII inputs, which is itself a bug -- so either is a failure, reported with + * the SQL. Modelled on prop_numeric.c's require_true. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * xmlelement wraps its content in a matching start/end tag. For a letter name + * N and letter-only content C (no markup to escape), the rendered text is + * exactly 'C'. The expected string is assembled with SQL string + * concatenation ('<' || N || '>' || C || '') so no raw '<' sits + * adjacent to the interpolated identifier in the C source. + */ +PG_PROP(element_wraps_name) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + initStringInfo(&c); + draw_name(tc, &n); + draw_content_letters(tc, &c); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT xmlelement(name %s, ", n.data); + append_sql_literal(&q, c.data); + appendStringInfoString(&q, ")::text = '<' || "); + append_sql_literal(&q, n.data); + appendStringInfoString(&q, " || '>' || "); + append_sql_literal(&q, c.data); + appendStringInfoString(&q, " || ''"); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * xmlelement escapes markup in its content. No matter how many '<' characters + * the content C contains, each content '<' is escaped to '<', so the only + * raw '<' bytes left in the rendered output are the two tag delimiters (the + * '<' of the start tag and the '<' of the end tag). The raw-'<'-count-is-2 + * oracle (output length minus length with all '<' removed equals 2) soundly + * detects escaping without an XML parser. Content is drawn from the pool that + * includes < > & so this exercises the escaping path. + */ +PG_PROP(element_escapes_specials) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + c, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + initStringInfo(&c); + draw_name(tc, &n); + draw_content(tc, &c); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (length(xmlelement(name %s, ", n.data); + append_sql_literal(&q, c.data); + appendStringInfo(&q, ")::text) - length(replace(xmlelement(name %s, ", n.data); + append_sql_literal(&q, c.data); + appendStringInfoString(&q, ")::text, '<', ''))) = 2"); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * xmlconcat is textual concatenation of its fragments in order. Two simple + * elements E1, E2 concatenated render as E1's text followed by E2's text. + * Letter-only content keeps each fragment's text free of escaping subtleties, + * so the comparison is a clean textual identity. + */ +PG_PROP(xmlconcat_length_additive_ish) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n1, + n2, + c1, + c2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n1); + initStringInfo(&n2); + initStringInfo(&c1); + initStringInfo(&c2); + draw_name(tc, &n1); + draw_name(tc, &n2); + draw_content_letters(tc, &c1); + draw_content_letters(tc, &c2); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT xmlconcat(xmlelement(name %s, ", n1.data); + append_sql_literal(&q, c1.data); + appendStringInfo(&q, "), xmlelement(name %s, ", n2.data); + append_sql_literal(&q, c2.data); + appendStringInfo(&q, "))::text = xmlelement(name %s, ", n1.data); + append_sql_literal(&q, c1.data); + appendStringInfo(&q, ")::text || xmlelement(name %s, ", n2.data); + append_sql_literal(&q, c2.data); + appendStringInfoString(&q, ")::text"); + + require_true(tc, conn, q.data); + + pfree(n1.data); + pfree(n2.data); + pfree(c1.data); + pfree(c2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * IS DOCUMENT is true for a single-rooted well-formed XML value and false for + * a multi-root fragment. A lone xmlelement has exactly one root (a document), + * while the concatenation of two elements has two roots (a content fragment, + * not a document). + */ +PG_PROP(is_document_single_root) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n1, + n2, + c1, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n1); + initStringInfo(&n2); + initStringInfo(&c1); + draw_name(tc, &n1); + draw_name(tc, &n2); + draw_content_letters(tc, &c1); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (xmlelement(name %s, ", n1.data); + append_sql_literal(&q, c1.data); + appendStringInfo(&q, ") IS DOCUMENT) AND NOT (xmlconcat(xmlelement(name %s, 'x'), " + "xmlelement(name %s, 'y')) IS DOCUMENT)", + n1.data, n2.data); + + require_true(tc, conn, q.data); + + pfree(n1.data); + pfree(n2.data); + pfree(c1.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * xmlelement with empty content yields an empty element. PostgreSQL emits the + * self-closing form '', but accept the equivalent '' too so the + * oracle does not over-specify the serialization. Both acceptable forms are + * built with SQL string concatenation. + */ +PG_PROP(xmlelement_empty_content) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n); + draw_name(tc, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT xmlelement(name %s, '')::text IN ('<' || ", n.data); + append_sql_literal(&q, n.data); + appendStringInfoString(&q, " || '/>', '<' || "); + append_sql_literal(&q, n.data); + appendStringInfoString(&q, " || '>')"); + + require_true(tc, conn, q.data); + + pfree(n.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * xmlagg aggregates XML fragments by concatenation in order. Aggregating two + * element rows (ordered) must equal xmlconcat of the same two elements in that + * order. A VALUES-driven subquery supplies the two rows so no table is + * needed. + */ +PG_PROP(xmlagg_concatenates_rows) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData n1, + n2, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&n1); + initStringInfo(&n2); + draw_name(tc, &n1); + draw_name(tc, &n2); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT xmlagg(e ORDER BY ord) FROM " + "(VALUES (xmlelement(name %s, 'x'), 1), " + "(xmlelement(name %s, 'y'), 2)) t(e, ord))::text = " + "xmlconcat(xmlelement(name %s, 'x'), " + "xmlelement(name %s, 'y'))::text", + n1.data, n2.data, n1.data, n2.data); + + require_true(tc, conn, q.data); + + pfree(n1.data); + pfree(n2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index b97e524190c26..4e677febd67c8 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 939ca76b4116f9903de6ede747058ac5e79bcefd Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:18 -0400 Subject: [PATCH 163/338] prop: add aggregate property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_aggregate.c | 255 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 259 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_aggregate.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index ada2d62f6b93a..b97cbc7c60f4a 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -178,7 +178,8 @@ OBJS = \ prop_geopoint.o \ prop_geoseg.o \ prop_boxops.o \ - prop_xmlfmt.o + prop_xmlfmt.o \ + prop_aggregate.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index e2382a2211475..178be3ba57f46 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -176,6 +176,7 @@ prop_area_sources = files( 'prop_geoseg.c', 'prop_boxops.c', 'prop_xmlfmt.c', + 'prop_aggregate.c', ) diff --git a/src/test/prop/prop_aggregate.c b/src/test/prop/prop_aggregate.c new file mode 100644 index 0000000000000..8e0f04dfc701c --- /dev/null +++ b/src/test/prop/prop_aggregate.c @@ -0,0 +1,255 @@ +/*------------------------------------------------------------------------- + * + * prop_aggregate.c + * Property-based tests for the standard SQL aggregate functions. + * + * The standard aggregates sum(), avg(), min(), max() and count() must satisfy + * exact relationships that hold for any column of values: the sum equals the + * count times the average, the minimum is no greater than the average which + * is no greater than the maximum, count(*) counts every row while count(col) + * and sum()/avg()/min()/max() skip NULLs, and min()/max() are values actually + * present in the column. These properties check those relationships over a + * randomly generated integer column. + * + * The oracles are computed in SQL, not C, so there is no risk of C-side + * overflow and the type's own exact numeric arithmetic (avg(int) and sum(int) + * return numeric/bigint) is the reference. Comparisons use IS NOT DISTINCT + * FROM so that the empty / all-NULL table -- where every aggregate but + * count() is NULL -- is handled cleanly rather than yielding a NULL result. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_aggregate.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "aggregate" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the aggregate errored or produced an unexpected NULL, + * which is itself a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build a one-column integer table agg_t with a drawn number of rows (0..30). + * Each value is a small integer in [-100, 100] -- small enough that no + * aggregate over at most 30 of them can overflow -- or SQL NULL (~1 in 6), so + * NULL-skipping is exercised. The oracles are all derived in SQL from this + * table, so nothing is returned; the counterexample shows the drawn values + * through the generated INSERTs. DROP TABLE IF EXISTS runs first; the harness + * sets client_min_messages = warning so the "does not exist" NOTICE is + * suppressed. INSERTs are batched into one statement for speed. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows = (int) pg_prop_draw_int(tc, 0, 30); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS agg_t; " + "CREATE TABLE agg_t (v int); "); + + for (int r = 0; r < nrows; r++) + { + if (pg_prop_draw_int(tc, 0, 5) == 0) + appendStringInfoString(&ddl, "INSERT INTO agg_t VALUES (NULL); "); + else + appendStringInfo(&ddl, "INSERT INTO agg_t VALUES (%d); ", + (int) pg_prop_draw_int(tc, -100, 100)); + } + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + + pfree(ddl.data); +} + +/* + * avg(v) = sum(v) / count(v). avg returns numeric; the exact identity divides + * sum by the same count avg used, so it holds to the last digit (unlike + * sum = count*avg, which rounds). Empty / all-NULL: avg IS NULL. + */ +/* + * avg(v) is sum(v)/count(v) by definition. avg returns a finite-scale + * numeric, so sum = count*avg does NOT hold exactly (the multiply reintroduces + * rounding); the sound exact identity is avg = sum::numeric / count when there + * is at least one non-NULL value, and avg IS NULL otherwise. + */ +PG_PROP(avg_equals_sum_over_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN count(v) > 0 " + "THEN avg(v) = sum(v)::numeric / count(v) " + "ELSE avg(v) IS NULL END FROM agg_t"); + + pg_prop_sql_exec(conn, "DROP TABLE agg_t", &err); + pg_prop_disconnect(conn); +} + +/* + * min <= avg <= max. Holds whenever there is at least one non-NULL row; when + * there is none every aggregate is NULL and the comparison is NULL, so + * coalesce(..., true) accepts the empty / all-NULL case. + */ +PG_PROP(min_le_avg_le_max) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(min(v) <= avg(v) AND avg(v) <= max(v), true) " + "FROM agg_t"); + + pg_prop_sql_exec(conn, "DROP TABLE agg_t", &err); + pg_prop_disconnect(conn); +} + +/* + * count(*) counts every row; count(v) counts only non-NULL rows. So + * count(*) >= count(v), and the difference is exactly the number of NULL rows. + */ +PG_PROP(count_star_vs_count_col) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(*) >= count(v) AND (count(*) - count(v)) = " + "count(*) FILTER (WHERE v IS NULL) FROM agg_t"); + + pg_prop_sql_exec(conn, "DROP TABLE agg_t", &err); + pg_prop_disconnect(conn); +} + +/* + * min() and max() return values that actually occur in the column. Only + * meaningful when there is at least one non-NULL value; on an empty or + * all-NULL table min/max are NULL (and a NULL IN (empty set) is false, not + * NULL), so gate the test on count(v) > 0. + */ +PG_PROP(min_max_in_table) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN count(v) > 0 " + "THEN (min(v) IN (SELECT v FROM agg_t)) AND " + "(max(v) IN (SELECT v FROM agg_t)) " + "ELSE true END FROM agg_t"); + + pg_prop_sql_exec(conn, "DROP TABLE agg_t", &err); + pg_prop_disconnect(conn); +} + +/* + * sum() skips NULLs: the sum over the whole table equals the sum over the + * NULL-filtered rows. IS NOT DISTINCT FROM makes the empty / all-NULL case + * (both sums NULL) pass. + */ +PG_PROP(sum_null_skipping) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT sum(v) IS NOT DISTINCT FROM " + "(SELECT sum(v) FROM agg_t WHERE v IS NOT NULL) FROM agg_t"); + + pg_prop_sql_exec(conn, "DROP TABLE agg_t", &err); + pg_prop_disconnect(conn); +} + +/* + * count(DISTINCT v) <= count(v): distinct non-NULL values cannot outnumber + * all non-NULL values. Both are 0 on an empty / all-NULL table, so 0 <= 0. + */ +PG_PROP(count_distinct_le_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(DISTINCT v) <= count(v) FROM agg_t"); + + pg_prop_sql_exec(conn, "DROP TABLE agg_t", &err); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 4e677febd67c8..5b6aa9bec298e 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 2ebf5642ab75a52a992c49c5387ca011bd13a58f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:18 -0400 Subject: [PATCH 164/338] prop: add hash vs sorted aggregate equivalence property tests (aggstrategies) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_aggstrategies.c | 262 +++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 266 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_aggstrategies.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index b97cbc7c60f4a..f7cee1ae77b7d 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -179,7 +179,8 @@ OBJS = \ prop_geoseg.o \ prop_boxops.o \ prop_xmlfmt.o \ - prop_aggregate.o + prop_aggregate.o \ + prop_aggstrategies.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 178be3ba57f46..4f51e54a015b5 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -177,6 +177,7 @@ prop_area_sources = files( 'prop_boxops.c', 'prop_xmlfmt.c', 'prop_aggregate.c', + 'prop_aggstrategies.c', ) diff --git a/src/test/prop/prop_aggstrategies.c b/src/test/prop/prop_aggstrategies.c new file mode 100644 index 0000000000000..3aa7a7ee8f9c7 --- /dev/null +++ b/src/test/prop/prop_aggstrategies.c @@ -0,0 +1,262 @@ +/*------------------------------------------------------------------------- + * + * prop_aggstrategies.c + * Property-based tests for aggregate-strategy independence. + * + * An aggregation's result must not depend on which strategy the executor + * picks: hash aggregation (enable_hashagg = on) and grouped/sorted + * aggregation (enable_hashagg = off) must return identical rows. The oracle + * is differential -- the same GROUP BY query run under each strategy -- and + * any disagreement is a wrong-results bug in nodeAgg, not a performance + * difference, because the two paths are supposed to compute the same groups. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_aggstrategies.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "aggstrategies" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small integer domains keep the generated table legible in a counterexample + * and keep each case fast while still exercising the grouping key boundaries: + * g in -3..3 with ~15% NULL (so NULL grouping is tested on both paths), v + * small and signed (so FILTER (WHERE v>0)/(v<0) and ordered array_agg have + * both-sign content, ties, and NULLs). + */ +#define G_MIN (-3) +#define G_MAX 3 +#define V_MIN (-5) +#define V_MAX 5 + +/* Draw g in [G_MIN, G_MAX], NULL ~15% of the time. */ +static void +draw_g(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, G_MIN, G_MAX)); +} + +/* Draw v in [V_MIN, V_MAX], NULL ~15% of the time. */ +static void +draw_v(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, V_MIN, V_MAX)); +} + +/* + * (Re)build ag_t fresh with 0..30 random rows. id is a serial-ish PRIMARY + * KEY assigned here so the data is reproducible from the drawn choices. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS ag_t; " + "CREATE TABLE ag_t (id int PRIMARY KEY, g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 30); + for (int r = 0; r < nrows; r++) + { + StringInfoData gval; + StringInfoData vval; + + initStringInfo(&gval); + initStringInfo(&vval); + draw_g(tc, &gval); + draw_v(tc, &vval); + appendStringInfo(&ddl, "INSERT INTO ag_t VALUES (%d, %s, %s); ", + r, gval.data, vval.data); + pfree(gval.data); + pfree(vval.data); + } + appendStringInfoString(&ddl, "ANALYZE ag_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n SQL: %s", + err ? err : "", ddl.data); + pfree(ddl.data); +} + +/* + * Differential oracle: run the same aggregate query twice, once forcing hash + * aggregation (enable_hashagg = on) and once forcing grouped/sorted + * aggregation (enable_hashagg = off). Both must return identical rows. Each + * query must already carry a TOTAL ORDER BY on the grouping key(s) so the + * row-major string compares as an ordered result; NULL groups are coalesced + * to a sentinel in ORDER BY to give that a total order. RESET ALL after. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, const char *query) +{ + char *err = NULL; + bool f1 = false, + f2 = false; + char *r1, + *r2; + + if (!pg_prop_sql_exec(conn, "SET enable_hashagg = on", &err)) + pg_prop_fail(tc, "SET enable_hashagg=on failed: %s", err ? err : ""); + r1 = pg_prop_sql_rows(conn, query, &f1, &err); + if (f1) + pg_prop_fail(tc, "hashagg query failed: %s\n SQL: %s", + err ? err : "", query); + + if (!pg_prop_sql_exec(conn, "SET enable_hashagg = off", &err)) + pg_prop_fail(tc, "SET enable_hashagg=off failed: %s", err ? err : ""); + r2 = pg_prop_sql_rows(conn, query, &f2, &err); + if (f2) + pg_prop_fail(tc, "groupagg query failed: %s\n SQL: %s", + err ? err : "", query); + + pg_prop_sql_exec(conn, "RESET ALL", &err); + + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, + "aggregate result depends on strategy\n" + " SQL: %s\n" + " hashagg:\n%s\n" + " groupagg:\n%s", + query, r1, r2); + + pg_free(r1); + pg_free(r2); + if (err) + pg_free(err); +} + +/* + * Property 1: basic grouped aggregates (count/sum/avg/min/max) over g. + * avg is cast to a fixed scale so the two strategies' textual rendering is + * identical regardless of accumulation order. + */ +PG_PROP(basic_group) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + require_same_rows(tc, conn, + "SELECT g, count(*), sum(v), avg(v)::numeric(20,4), " + "min(v), max(v) " + "FROM ag_t GROUP BY g ORDER BY g NULLS FIRST"); + pg_prop_disconnect(conn); +} + +/* Property 2: count(DISTINCT v) per group. */ +PG_PROP(count_distinct_group) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + require_same_rows(tc, conn, + "SELECT g, count(DISTINCT v) " + "FROM ag_t GROUP BY g ORDER BY g NULLS FIRST"); + pg_prop_disconnect(conn); +} + +/* + * Property 3: grouping on two keys, one of them a boolean expression that + * can itself be NULL (v is NULL -> v>0 is NULL). Total order covers both. + */ +PG_PROP(group_two_keys) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + require_same_rows(tc, conn, + "SELECT g, (v>0) AS pos, count(*) " + "FROM ag_t GROUP BY g, (v>0) " + "ORDER BY g NULLS FIRST, pos NULLS FIRST"); + pg_prop_disconnect(conn); +} + +/* Property 4: FILTER-ed aggregates per group. */ +PG_PROP(filtered_agg) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + require_same_rows(tc, conn, + "SELECT g, count(*) FILTER (WHERE v>0), " + "sum(v) FILTER (WHERE v<0) " + "FROM ag_t GROUP BY g ORDER BY g NULLS FIRST"); + pg_prop_disconnect(conn); +} + +/* Property 5: HAVING on the aggregate count. */ +PG_PROP(having_clause) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + require_same_rows(tc, conn, + "SELECT g, count(*) FROM ag_t GROUP BY g " + "HAVING count(*) > 1 ORDER BY g NULLS FIRST"); + pg_prop_disconnect(conn); +} + +/* + * Property 6: ordered array_agg. The ORDER BY inside array_agg makes the + * array content deterministic, so both strategies must produce the same + * arrays per group. + */ +PG_PROP(array_agg_ordered) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + require_same_rows(tc, conn, + "SELECT g, array_agg(v ORDER BY v) " + "FROM ag_t GROUP BY g ORDER BY g NULLS FIRST"); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 5b6aa9bec298e..de88de68d7fc1 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From fb2d2a8447de28f64cf38a9bb661b0863f833d05 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:18 -0400 Subject: [PATCH 165/338] prop: add ordered-aggregate determinism and sorting property tests (aggorder) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_aggorder.c | 239 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 243 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_aggorder.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index f7cee1ae77b7d..09af9ac434cb8 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -180,7 +180,8 @@ OBJS = \ prop_boxops.o \ prop_xmlfmt.o \ prop_aggregate.o \ - prop_aggstrategies.o + prop_aggstrategies.o \ + prop_aggorder.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 4f51e54a015b5..c1527d5d59ef8 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -178,6 +178,7 @@ prop_area_sources = files( 'prop_xmlfmt.c', 'prop_aggregate.c', 'prop_aggstrategies.c', + 'prop_aggorder.c', ) diff --git a/src/test/prop/prop_aggorder.c b/src/test/prop/prop_aggorder.c new file mode 100644 index 0000000000000..ff598676d3716 --- /dev/null +++ b/src/test/prop/prop_aggorder.c @@ -0,0 +1,239 @@ +/*------------------------------------------------------------------------- + * + * prop_aggorder.c + * Property-based tests for ordered aggregates and aggregate ORDER BY. + * + * These check the ordering guarantees of aggregates that take an in-aggregate + * ORDER BY or a WITHIN GROUP sort: array_agg/string_agg/json_agg emit their + * elements in the sorted order requested, DESC is the reverse of ASC, + * percentile_cont over a sorted group is monotone, and a redundant ORDER BY + * in count() does not change the count. A failure means the executor's + * tuplesort-backed aggregate transition fed the final function values in the + * wrong order (or dropped/duplicated them), which is a wrong-results bug. + * + * The oracle is the type's own ordering algebra over a fixed, NULL-free input + * table (v is never NULL so every ordering is total and exact); each property + * asserts a scalar boolean that must be true for every generated table. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_aggorder.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "aggorder" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Build ao_t(id int PRIMARY KEY, g int, v int) with 0..25 rows, g in 0..2 and + * v in -5..5, never NULL so every ORDER BY over v is a total, exact order. + * Rebuilt fresh in each property so cases do not leak state. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS ao_t; " + "CREATE TABLE ao_t(id int PRIMARY KEY, g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 25); + for (int i = 0; i < nrows; i++) + { + int g = (int) pg_prop_draw_int(tc, 0, 2); + int v = (int) pg_prop_draw_int(tc, -5, 5); + + appendStringInfo(&ddl, "INSERT INTO ao_t VALUES (%d, %d, (%d)); ", + i, g, v); + } + appendStringInfoString(&ddl, "ANALYZE ao_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n SQL: %s", + err ? err : "", ddl.data); + + pfree(ddl.data); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error is itself a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * array_agg(v ORDER BY v) is non-decreasing: every adjacent pair a[i] <= + * a[i+1]. coalesce over generate_subscripts makes the empty/one-element array + * pass (bool_and over no rows is NULL -> true). + */ +PG_PROP(array_agg_sorted) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce((SELECT bool_and(a[i] <= a[i+1]) " + "FROM generate_subscripts(a, 1) i " + "WHERE i < cardinality(a)), true) " + "FROM (SELECT array_agg(v ORDER BY v) a FROM ao_t) s"); + + pg_prop_disconnect(conn); +} + +/* + * string_agg(v::text, ',' ORDER BY v, id) is deterministic: computed twice it + * must be identical. (v, id) is a total order, so the output is fixed. IS + * NOT DISTINCT FROM handles the empty case (both sides NULL -> true). + */ +PG_PROP(string_agg_order_deterministic) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT string_agg(v::text, ',' ORDER BY v, id) FROM ao_t) " + "IS NOT DISTINCT FROM " + "(SELECT string_agg(v::text, ',' ORDER BY v, id) FROM ao_t)"); + + pg_prop_disconnect(conn); +} + +/* + * array_agg(v ORDER BY v DESC) is the element-wise reverse of + * array_agg(v ORDER BY v): same cardinality and a[i] = b[card - i + 1] for + * every i. coalesce makes the empty case pass. + */ +PG_PROP(array_agg_desc_is_reverse_asc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT cardinality(a) IS NOT DISTINCT FROM cardinality(b) " + "AND coalesce((SELECT bool_and(a[i] = b[cardinality(b) - i + 1]) " + "FROM generate_subscripts(a, 1) i), true) " + "FROM (SELECT array_agg(v ORDER BY v DESC) a, " + "array_agg(v ORDER BY v) b FROM ao_t) s"); + + pg_prop_disconnect(conn); +} + +/* + * The first element of json_agg(v ORDER BY v) equals min(v): the sorted JSON + * array leads with the smallest value. Guarded on non-empty (count > 0) so + * the empty case trivially passes. + */ +PG_PROP(json_agg_order_first) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN count(*) = 0 THEN true " + "ELSE (json_agg(v ORDER BY v)::jsonb -> 0) = to_jsonb(min(v)) END " + "FROM ao_t"); + + pg_prop_disconnect(conn); +} + +/* + * percentile_cont(ARRAY[0, 0.5, 1]) WITHIN GROUP (ORDER BY v) returns a + * 3-element array (the min, median, max of a continuous interpolation) that + * must be non-decreasing. Guarded on non-empty; empty input yields NULL, + * which coalesce turns into a pass. + */ +PG_PROP(percentile_within_group_sorted) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN p IS NULL THEN true " + "ELSE cardinality(p) = 3 AND p[1] <= p[2] AND p[2] <= p[3] END " + "FROM (SELECT percentile_cont(ARRAY[0, 0.5, 1]) " + "WITHIN GROUP (ORDER BY v) p FROM ao_t) s"); + + pg_prop_disconnect(conn); +} + +/* + * A redundant ORDER BY inside count() does not change the result: + * count(v ORDER BY v) = count(v). Both count exactly one row even on empty + * input (count over 0 rows is 0, not NULL), so no guard is needed. + */ +PG_PROP(count_order_irrelevant) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(v ORDER BY v) = count(v) FROM ao_t"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index de88de68d7fc1..5aa24dad6ef94 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 0897cc16609ead5061c0fc3ac72f46754322c227 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:18 -0400 Subject: [PATCH 166/338] prop: add DISTINCT-in-aggregate relationships property tests (aggdistinct) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_aggdistinct.c | 323 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 327 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_aggdistinct.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 09af9ac434cb8..02292195892d5 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -181,7 +181,8 @@ OBJS = \ prop_xmlfmt.o \ prop_aggregate.o \ prop_aggstrategies.o \ - prop_aggorder.o + prop_aggorder.o \ + prop_aggdistinct.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index c1527d5d59ef8..e0cc3e8742ebc 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -179,6 +179,7 @@ prop_area_sources = files( 'prop_aggregate.c', 'prop_aggstrategies.c', 'prop_aggorder.c', + 'prop_aggdistinct.c', ) diff --git a/src/test/prop/prop_aggdistinct.c b/src/test/prop/prop_aggdistinct.c new file mode 100644 index 0000000000000..9af86197f2196 --- /dev/null +++ b/src/test/prop/prop_aggdistinct.c @@ -0,0 +1,323 @@ +/*------------------------------------------------------------------------- + * + * prop_aggdistinct.c + * Property-based tests for DISTINCT inside aggregates and combinations + * of DISTINCT aggregates. + * + * agg(DISTINCT v) deduplicates its input before aggregating, so count(DISTINCT + * v) is bounded by count(v), sum(DISTINCT v) by sum(v) for non-negative input, + * and count(DISTINCT v) must equal the row count of an explicit SELECT DISTINCT + * projection. These properties exercise the aggregate DISTINCT path (sort/hash + * of the per-aggregate inputs) against independent SQL oracles over a small + * table ad_t(id, g, v) whose tiny value domain guarantees heavy duplication, + * so distinct counts are strictly below raw counts and the dedup actually runs. + * + * This area is deliberately disjoint from "aggregate", "boolagg", "ordagg", + * "filteragg" and "aggorder": here the subject is specifically DISTINCT within + * aggregates and several DISTINCT aggregates coexisting in one query. + * + * The oracles are computed in SQL from the same table, so there is no C-side + * arithmetic to overflow; the small value range keeps every sum well within + * range. Empty-table and all-NULL cases are handled with coalesce()/nullif()/ + * CASE so those boundaries pass soundly (aggregates are NULL over no rows). + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_aggdistinct.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "aggdistinct" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the aggregate errored or produced an unexpected NULL, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build the table ad_t(id int PRIMARY KEY, g int, v int) and populate it with + * 0..30 rows. id is a dense surrogate key, the group label g is drawn in + * [0, 3] so there are only a few groups each with several rows, and v is drawn + * in [0, 4] so the value domain is tiny -- distinct values are far fewer than + * rows, which is exactly what makes the DISTINCT dedup observable (distinct + * count strictly below raw count). v is non-negative so sum(DISTINCT) <= + * sum() holds. The small range means no sum over at most 30 rows can + * overflow. Zero rows exercises the empty-aggregate path. ANALYZE updates + * statistics so the planner may choose either sort or hash aggregation paths. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS ad_t; " + "CREATE TABLE ad_t (id int PRIMARY KEY, " + "g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 30); + for (int r = 0; r < nrows; r++) + appendStringInfo(&ddl, "INSERT INTO ad_t VALUES (%d, %d, %d); ", + r, + (int) pg_prop_draw_int(tc, 0, 3), + (int) pg_prop_draw_int(tc, 0, 4)); + + appendStringInfoString(&ddl, "ANALYZE ad_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + + pfree(ddl.data); +} + +/* + * The dedup/non-null count chain: count(DISTINCT v) <= count(v) <= count(*). + * DISTINCT removes duplicates so cannot increase the count, and count(v) + * skips NULLs so cannot exceed count(*). Empty table: 0 <= 0 <= 0. + */ +PG_PROP(count_distinct_le_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(DISTINCT v) <= count(v) " + "AND count(v) <= count(*) FROM ad_t"); + + pg_prop_sql_exec(conn, "DROP TABLE ad_t", &err); + pg_prop_disconnect(conn); +} + +/* + * For non-negative v (drawn in 0..4), summing the distinct values cannot + * exceed summing every value, since dropping duplicate non-negative addends + * only removes non-negative contributions: sum(DISTINCT v) <= sum(v). + * coalesce(..., 0) makes the empty table (both NULL) pass as 0 <= 0. + */ +PG_PROP(sum_distinct_le_sum_when_nonneg) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(sum(DISTINCT v), 0) <= " + "coalesce(sum(v), 0) FROM ad_t"); + + pg_prop_sql_exec(conn, "DROP TABLE ad_t", &err); + pg_prop_disconnect(conn); +} + +/* + * count(DISTINCT v) must equal the number of distinct non-NULL values, + * computed independently as the row count of an explicit SELECT DISTINCT v + * (which, like count(DISTINCT), ignores NULL under count(*) over the distinct + * projection only when NULLs are excluded -- here count(DISTINCT) ignores NULL + * and SELECT DISTINCT yields a NULL row, so we count non-NULL rows). Both + * sides agree exactly. Empty table: both 0. + */ +PG_PROP(distinct_count_eq_subquery) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(DISTINCT v) = " + "(SELECT count(*) FROM (SELECT DISTINCT v FROM ad_t) s) " + "FROM ad_t"); + + pg_prop_sql_exec(conn, "DROP TABLE ad_t", &err); + pg_prop_disconnect(conn); +} + +/* + * avg(DISTINCT v) is by definition sum(DISTINCT v) / count(DISTINCT v), so it + * must equal the manual quotient of the two DISTINCT aggregates. IS NOT + * DISTINCT FROM makes the empty / all-NULL case (both sides NULL via nullif + * on a zero distinct count) pass soundly. The cast to numeric gives the same + * exact-decimal arithmetic avg uses internally for integer input. + */ +PG_PROP(avg_distinct_manual) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT avg(DISTINCT v) IS NOT DISTINCT FROM " + "sum(DISTINCT v)::numeric / nullif(count(DISTINCT v), 0) " + "FROM ad_t"); + + pg_prop_sql_exec(conn, "DROP TABLE ad_t", &err); + pg_prop_disconnect(conn); +} + +/* + * Two DISTINCT aggregates coexisting in one query must each still equal their + * own independent subquery count: count(DISTINCT v) and count(DISTINCT g) are + * computed together but must not interfere. This exercises the + * multiple-per-aggregate-DISTINCT sort/hash path. Empty table: all 0. + */ +PG_PROP(multiple_distinct_aggs) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(DISTINCT v) = " + "(SELECT count(*) FROM (SELECT DISTINCT v FROM ad_t) s) " + "AND count(DISTINCT g) = " + "(SELECT count(*) FROM (SELECT DISTINCT g FROM ad_t) t) " + "FROM ad_t"); + + pg_prop_sql_exec(conn, "DROP TABLE ad_t", &err); + pg_prop_disconnect(conn); +} + +/* + * Per group, DISTINCT cannot produce more values than there are rows: + * count(DISTINCT v) <= count(*) within each GROUP BY g bucket. bool_and over + * the per-group comparisons must be true, and coalesce(..., true) makes the + * empty table (no groups, bool_and NULL) pass. + */ +PG_PROP(grouped_distinct_sums) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(cd <= c), true) FROM " + "(SELECT count(DISTINCT v) cd, count(*) c " + "FROM ad_t GROUP BY g) s"); + + pg_prop_sql_exec(conn, "DROP TABLE ad_t", &err); + pg_prop_disconnect(conn); +} + +/* + * string_agg(DISTINCT v::text, ',') concatenates the distinct values, so the + * number of comma-separated pieces equals count(DISTINCT v). v::text never + * contains a comma (v is a small integer), so splitting on ',' recovers one + * piece per distinct value. When count(DISTINCT v) is 0 (empty or all-NULL) + * string_agg is NULL -> coalesce to '' -> array_length of a one-empty-element + * array is 1, which does not equal 0, so a CASE short-circuits that case to + * true; otherwise the piece count must equal the distinct count. + */ +PG_PROP(string_agg_distinct_dedup) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN count(DISTINCT v) = 0 THEN true " + "ELSE array_length(string_to_array(" + "coalesce(string_agg(DISTINCT v::text, ','), ''), ','), 1) " + "= count(DISTINCT v) END FROM ad_t"); + + pg_prop_sql_exec(conn, "DROP TABLE ad_t", &err); + pg_prop_disconnect(conn); +} + +/* + * DISTINCT does not change the result of bool_and: whether or not duplicate + * boolean inputs are collapsed, the AND over {true, false} is the same. So + * bool_and(DISTINCT (v > 0)) must equal bool_and(v > 0). coalesce(..., true) + * makes the empty table (both NULL) pass as true = true. + */ +PG_PROP(bool_and_distinct_same) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(DISTINCT (v > 0)), true) = " + "coalesce(bool_and(v > 0), true) FROM ad_t"); + + pg_prop_sql_exec(conn, "DROP TABLE ad_t", &err); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 5aa24dad6ef94..3df29eed02df8 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From ffe999869c4fab17aa46f1d504ec6612d9c8914f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:18 -0400 Subject: [PATCH 167/338] prop: add aggregate ALL/DISTINCT and GROUP BY consistency property tests (aggdedup) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_aggdedup.c | 311 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 315 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_aggdedup.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 02292195892d5..b2f0e71f50530 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -182,7 +182,8 @@ OBJS = \ prop_aggregate.o \ prop_aggstrategies.o \ prop_aggorder.o \ - prop_aggdistinct.o + prop_aggdistinct.o \ + prop_aggdedup.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index e0cc3e8742ebc..904666cc926e1 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -180,6 +180,7 @@ prop_area_sources = files( 'prop_aggstrategies.c', 'prop_aggorder.c', 'prop_aggdistinct.c', + 'prop_aggdedup.c', ) diff --git a/src/test/prop/prop_aggdedup.c b/src/test/prop/prop_aggdedup.c new file mode 100644 index 0000000000000..ac93e2e741b1f --- /dev/null +++ b/src/test/prop/prop_aggdedup.c @@ -0,0 +1,311 @@ +/*------------------------------------------------------------------------- + * + * prop_aggdedup.c + * Property-based tests for aggregate ALL/DISTINCT duplicate handling and + * GROUP-wide counting identities. + * + * ALL is the default duplicate-handling for aggregates, so agg(ALL x) must + * equal agg(x); DISTINCT within an aggregate must agree with an explicit + * SELECT DISTINCT, and must be insensitive to a redundant inner ORDER BY; + * per-group counts must sum to the whole-table count; and count(DISTINCT + * composite) is bounded by count(*). Each is checked against an independent + * SQL oracle over a tiny table ad2_t(id, g, v) whose small value domain forces + * heavy duplication, so any dedup/ALL mishandling shows up. + * + * This area is deliberately disjoint from "aggregate", "aggdistinct" and + * "grouphaving": here the subject is specifically the ALL-vs-DISTINCT default + * equivalence and the whole-vs-grouped counting identities, not DISTINCT + * bounds (aggdistinct) nor HAVING/grouping rewrites (grouphaving). + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_aggdedup.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "aggdedup" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Differential result-set oracle (shape B): q1 and q2 must return identical + * rows in identical order. Both callers supply a TOTAL ORDER BY so the + * row-major string compare behaves as a multiset compare. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Build ad2_t(id int PRIMARY KEY, g int, v int) with 0..30 rows. id is a + * dense surrogate key, g is drawn in [0, 3] (a few groups), and v is drawn in + * [0, 4] (tiny value domain, so distinct values are far fewer than rows and + * dedup is observable). v is non-negative and never NULL, so count(v) = + * count(*) and the ALL-vs-DISTINCT reasoning is clean. Zero rows exercises + * the empty-aggregate path. ANALYZE lets the planner pick sort or hash + * aggregation. Rebuilt fresh in each property so cases are independent. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS ad2_t; " + "CREATE TABLE ad2_t (id int PRIMARY KEY, " + "g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 30); + for (int r = 0; r < nrows; r++) + appendStringInfo(&ddl, "INSERT INTO ad2_t VALUES (%d, %d, %d); ", + r, + (int) pg_prop_draw_int(tc, 0, 3), + (int) pg_prop_draw_int(tc, 0, 4)); + + appendStringInfoString(&ddl, "ANALYZE ad2_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + + pfree(ddl.data); +} + +/* + * ALL is the default duplicate-handling, so count(ALL v) must equal count(v): + * writing ALL explicitly changes nothing. Both ignore NULL (v is non-NULL + * here anyway). Empty table: 0 = 0. + */ +PG_PROP(count_all_eq_count_star) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(ALL v) = count(v) FROM ad2_t"); + + pg_prop_sql_exec(conn, "DROP TABLE ad2_t", &err); + pg_prop_disconnect(conn); +} + +/* + * sum(ALL v) must equal sum(v): ALL is the default, so making it explicit does + * not change the sum. Both are NULL over an empty table, and NULL = NULL + * would be NULL -- but every input here has either both NULL (empty) or both + * equal, so IS NOT DISTINCT FROM covers the empty case soundly. + */ +PG_PROP(sum_all_eq_sum) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT sum(ALL v) IS NOT DISTINCT FROM sum(v) FROM ad2_t"); + + pg_prop_sql_exec(conn, "DROP TABLE ad2_t", &err); + pg_prop_disconnect(conn); +} + +/* + * Per-group count(DISTINCT v) equals the distinct count computed by a + * correlated subquery over the distinct grouping keys. IS NOT DISTINCT FROM + * joins the correlation NULL-safely to match GROUP BY semantics (g is + * non-NULL here, but the form stays sound if that changes). Both sides ORDER + * BY g, a total order over the distinct keys, so the row-major compare behaves + * as a multiset compare. Empty table: both return zero rows. + */ +PG_PROP(distinct_group_vs_subquery) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT g, count(DISTINCT v) FROM ad2_t " + "GROUP BY g ORDER BY g", + "SELECT g, (SELECT count(DISTINCT v) FROM ad2_t u " + "WHERE u.g IS NOT DISTINCT FROM o.g) " + "FROM (SELECT DISTINCT g FROM ad2_t) o ORDER BY g"); + + pg_prop_sql_exec(conn, "DROP TABLE ad2_t", &err); + pg_prop_disconnect(conn); +} + +/* + * count(DISTINCT v) is a set cardinality and so cannot depend on a redundant + * ORDER BY in a subquery that feeds it: ordering the input rows changes + * nothing about the distinct set. So count(DISTINCT v) over the table equals + * the same over an ORDER BY v subquery. Empty table: 0 = 0. + */ +PG_PROP(agg_distinct_order_irrelevant) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(DISTINCT v) FROM ad2_t) = " + "(SELECT count(DISTINCT v) FROM " + "(SELECT v FROM ad2_t ORDER BY v) s)"); + + pg_prop_sql_exec(conn, "DROP TABLE ad2_t", &err); + pg_prop_disconnect(conn); +} + +/* + * Several aggregates computed together in one query must be internally + * consistent: sum(v) equals sum(v) FILTER (WHERE true) (a trivially-true + * filter removes nothing), and count(*) equals count(v) plus count(*) FILTER + * (WHERE v IS NULL) (every row is either counted by count(v) or is a NULL-v + * row; v is non-NULL here so count(v) = count(*) and the FILTER count is 0). + * IS NOT DISTINCT FROM covers the empty-table sum (both NULL). + */ +PG_PROP(multiple_aggs_consistent) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (sum(v) IS NOT DISTINCT FROM " + "sum(v) FILTER (WHERE true)) " + "AND count(*) = count(v) + " + "count(*) FILTER (WHERE v IS NULL) FROM ad2_t"); + + pg_prop_sql_exec(conn, "DROP TABLE ad2_t", &err); + pg_prop_disconnect(conn); +} + +/* + * Partitioning the table by g and summing the per-group counts recovers the + * whole-table count: every row lands in exactly one group, so the group counts + * sum to count(*). coalesce(sum(cnt), 0) makes the empty table (no groups, + * sum NULL) pass as 0 = 0. + */ +PG_PROP(group_all_rows) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce((SELECT sum(cnt) FROM " + "(SELECT count(*) cnt FROM ad2_t GROUP BY g) s), 0) = " + "(SELECT count(*) FROM ad2_t)"); + + pg_prop_sql_exec(conn, "DROP TABLE ad2_t", &err); + pg_prop_disconnect(conn); +} + +/* + * The number of distinct composite values (g, v) cannot exceed the number of + * rows: DISTINCT only collapses duplicates. count(DISTINCT (g,v)) <= count(*). + * Empty table: 0 <= 0. + */ +PG_PROP(distinct_le_total) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(DISTINCT (g, v)) <= count(*) FROM ad2_t"); + + pg_prop_sql_exec(conn, "DROP TABLE ad2_t", &err); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 3df29eed02df8..5c196133ddb87 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From bd834ec1e89ac5be8c58d0d0a931585f54ba69b4 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:18 -0400 Subject: [PATCH 168/338] prop: add EVERY/SOME, FILTER, and grouped aggregates property tests (boolagg) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_boolagg.c | 281 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 285 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_boolagg.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index b2f0e71f50530..0659a769e2526 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -183,7 +183,8 @@ OBJS = \ prop_aggstrategies.o \ prop_aggorder.o \ prop_aggdistinct.o \ - prop_aggdedup.o + prop_aggdedup.o \ + prop_boolagg.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 904666cc926e1..af771acf45c59 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -181,6 +181,7 @@ prop_area_sources = files( 'prop_aggorder.c', 'prop_aggdistinct.c', 'prop_aggdedup.c', + 'prop_boolagg.c', ) diff --git a/src/test/prop/prop_boolagg.c b/src/test/prop/prop_boolagg.c new file mode 100644 index 0000000000000..6b28753552b72 --- /dev/null +++ b/src/test/prop/prop_boolagg.c @@ -0,0 +1,281 @@ +/*------------------------------------------------------------------------- + * + * prop_boolagg.c + * Property-based tests for the quantifier aggregates, FILTER clauses and + * DISTINCT/GROUP BY aggregate modifiers. + * + * EVERY() is the SQL-standard spelling of bool_and and SOME()/ANY() of + * bool_or; count(*) FILTER (WHERE cond) counts only matching rows; and + * aggregating over GROUP BY and then re-aggregating the per-group results must + * reconcile with the ungrouped total. These properties check the exact + * relationships those features promise over a randomly generated two-column + * table ba_t(g, v): the quantifier/count duality, that complementary FILTERs + * partition the rows, that count(DISTINCT g) counts the distinct groups, and + * that summing (or counting) per group adds back to the grand total. + * + * This area is deliberately disjoint from "aggregate" (sum/avg/min/max/count, + * count-distinct bound) and "ordagg" (bool_and/bool_or, bit_and/bit_or, + * percentile, mode): here the subject is EVERY/SOME, the FILTER clause, + * count(DISTINCT) against an explicit distinct projection, and the + * GROUP-BY-then-reaggregate identities. + * + * The oracles are computed in SQL from the same table, so there is no C-side + * arithmetic to overflow; the small value range keeps every sum well within + * range. Empty-table cases are handled with coalesce() so the empty table + * passes consistently with each oracle (aggregates are NULL over no rows). + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_boolagg.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "boolagg" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the aggregate errored or produced an unexpected NULL, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build the table ba_t(g int, v int) and populate it with 0..25 rows. The + * group label g is drawn in [0, 3] so there are only a few groups and most + * have several rows (so GROUP BY has something to reconcile), and v is drawn + * in [-20, 20] so there is a healthy mix of positive, zero and negative values + * for the quantifier and FILTER predicates to split on. The range is small + * enough that no sum over at most 25 rows can overflow. Zero rows exercises + * the empty-aggregate path. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS ba_t; " + "CREATE TABLE ba_t (g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 25); + for (int r = 0; r < nrows; r++) + appendStringInfo(&ddl, "INSERT INTO ba_t VALUES (%d, %d); ", + (int) pg_prop_draw_int(tc, 0, 3), + (int) pg_prop_draw_int(tc, -20, 20)); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + + pfree(ddl.data); +} + +/* + * EVERY(v > 0) -- the SQL-standard spelling of bool_and -- is true iff no row + * has v <= 0, which the count filter measures directly. Empty table: EVERY is + * NULL -> coalesce true, and count filter = 0 -> true, so the two sides agree. + */ +PG_PROP(every_equals_count_filter) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(every(v > 0), true) = " + "(count(*) FILTER (WHERE v <= 0) = 0) FROM ba_t"); + + pg_prop_sql_exec(conn, "DROP TABLE ba_t", &err); + pg_prop_disconnect(conn); +} + +/* + * Two complementary FILTERs partition the rows: count(*) FILTER (WHERE v > 0) + * plus count(*) FILTER (WHERE v <= 0) equals count(*). Every row satisfies + * exactly one of v > 0 / v <= 0, so the matched counts sum to the total. + * Empty table: 0 + 0 = 0. + */ +PG_PROP(count_filter_partitions) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (count(*) FILTER (WHERE v > 0) + " + "count(*) FILTER (WHERE v <= 0)) = count(*) FROM ba_t"); + + pg_prop_sql_exec(conn, "DROP TABLE ba_t", &err); + pg_prop_disconnect(conn); +} + +/* + * count(DISTINCT g) equals the number of distinct groups, computed + * independently as the row count of an explicit SELECT DISTINCT. The two must + * agree exactly. Empty table: both are 0. + */ +PG_PROP(count_distinct_matches_group_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(DISTINCT g) = " + "(SELECT count(*) FROM (SELECT DISTINCT g FROM ba_t) s) " + "FROM ba_t"); + + pg_prop_sql_exec(conn, "DROP TABLE ba_t", &err); + pg_prop_disconnect(conn); +} + +/* + * GROUP BY g, sum v per group, then sum the group sums: the result equals the + * grand total sum(v). coalesce(..., 0) makes the empty table (both sides + * NULL) pass as 0 = 0. + */ +PG_PROP(group_sums_add_to_total) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce((SELECT sum(gs) FROM " + "(SELECT sum(v) gs FROM ba_t GROUP BY g) s), 0) = " + "coalesce((SELECT sum(v) FROM ba_t), 0)"); + + pg_prop_sql_exec(conn, "DROP TABLE ba_t", &err); + pg_prop_disconnect(conn); +} + +/* + * GROUP BY g, count rows per group, then sum the group counts: the result + * equals the total row count count(*). The per-group count sum is NULL on an + * empty table (no groups), so coalesce(..., 0) matches count(*) = 0. + */ +PG_PROP(group_counts_add_to_total) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce((SELECT sum(gc) FROM " + "(SELECT count(*) gc FROM ba_t GROUP BY g) s), 0) = " + "(SELECT count(*) FROM ba_t)"); + + pg_prop_sql_exec(conn, "DROP TABLE ba_t", &err); + pg_prop_disconnect(conn); +} + +/* + * The FILTERed sum of the positive values is non-negative and no greater than + * the sum of absolute values: 0 <= sum(v) FILTER (WHERE v > 0) <= sum(abs(v)). + * coalesce(..., 0) makes the empty / all-filtered-out case pass as 0 <= 0 <= 0. + */ +PG_PROP(filter_sum_le_total_abs) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(sum(v) FILTER (WHERE v > 0), 0) >= 0 " + "AND coalesce(sum(v) FILTER (WHERE v > 0), 0) <= " + "coalesce(sum(abs(v)), 0) FROM ba_t"); + + pg_prop_sql_exec(conn, "DROP TABLE ba_t", &err); + pg_prop_disconnect(conn); +} + +/* + * De Morgan for the quantifier aggregates: NOT EVERY(p) = SOME(NOT p). Here + * p is v > 0, so NOT every(v > 0) must equal bool_or(NOT (v > 0)). Empty + * table: every and bool_or are both NULL, so coalesce(..., false) gives + * false = false, which holds. + */ +PG_PROP(every_some_duality) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(NOT every(v > 0), false) = " + "coalesce(bool_or(NOT (v > 0)), false) FROM ba_t"); + + pg_prop_sql_exec(conn, "DROP TABLE ba_t", &err); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 5c196133ddb87..60e74bf251f2b 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From dd5d6e006037bbb0f90db273303d822c915f0879 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:19 -0400 Subject: [PATCH 169/338] prop: add min/max/bool/bit aggregate identities property tests (minmaxagg) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_minmaxagg.c | 306 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 310 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_minmaxagg.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 0659a769e2526..d0e6b89bc4fbf 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -184,7 +184,8 @@ OBJS = \ prop_aggorder.o \ prop_aggdistinct.o \ prop_aggdedup.o \ - prop_boolagg.o + prop_boolagg.o \ + prop_minmaxagg.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index af771acf45c59..03dbd9abbf488 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -182,6 +182,7 @@ prop_area_sources = files( 'prop_aggdistinct.c', 'prop_aggdedup.c', 'prop_boolagg.c', + 'prop_minmaxagg.c', ) diff --git a/src/test/prop/prop_minmaxagg.c b/src/test/prop/prop_minmaxagg.c new file mode 100644 index 0000000000000..ac762deb7b121 --- /dev/null +++ b/src/test/prop/prop_minmaxagg.c @@ -0,0 +1,306 @@ +/*------------------------------------------------------------------------- + * + * prop_minmaxagg.c + * Property-based tests for the extremum and bitwise/boolean aggregates: + * min, max, bool_and, bool_or, bit_and, bit_or. + * + * These aggregates promise order and bit-lattice identities over whatever + * rows survive NULL skipping: min(v) <= max(v); min()/max() return values + * actually present (and ignore NULLs); bool_and/bool_or obey De Morgan; + * bit_and(v) is a submask of bit_or(v); and count(v) > 0 forces min/max + * non-NULL. A failure means an aggregate transition broke one of these, not + * a rounding artefact -- there is none here. + * + * This area is deliberately disjoint from "aggregate" (sum/avg/min/max/count + * numeric bounds), "boolagg" (EVERY/SOME, FILTER, GROUP BY reaggregation) and + * "ordagg" (percentile/mode/ordered-set): here the subject is the extremum + * and bit/bool lattice identities of min/max/bool_and/bool_or/bit_and/bit_or. + * + * The oracles are computed in SQL from the same table, so there is no C-side + * arithmetic to overflow. Empty / all-NULL cases are handled with coalesce(), + * CASE and IS NOT DISTINCT FROM so an aggregate NULL passes soundly. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_minmaxagg.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "minmaxagg" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the aggregate errored or produced an unexpected NULL, + * which is itself a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build the table mm_t(id int PRIMARY KEY, v int) with 0..30 rows. id is a + * dense sequence so the primary key is unique by construction; v is drawn in + * [-10, 10] -- small enough that no sum/product over at most 30 of them can + * overflow int, and signed so bit_and/bit_or exercise the sign bits -- or SQL + * NULL (~15%, 3 of 20), so NULL skipping is exercised. Zero rows exercises + * the empty-aggregate path. The oracles are all derived in SQL from this + * table; the counterexample shows the drawn values through the INSERTs. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows = (int) pg_prop_draw_int(tc, 0, 30); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS mm_t; " + "CREATE TABLE mm_t (id int PRIMARY KEY, v int); "); + + for (int r = 0; r < nrows; r++) + { + if (pg_prop_draw_int(tc, 0, 19) < 3) + appendStringInfo(&ddl, "INSERT INTO mm_t VALUES (%d, NULL); ", r); + else + appendStringInfo(&ddl, "INSERT INTO mm_t VALUES (%d, %d); ", + r, (int) pg_prop_draw_int(tc, -10, 10)); + } + + appendStringInfoString(&ddl, "ANALYZE mm_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + + pfree(ddl.data); +} + +/* + * min(v) <= max(v) over a non-empty column of values. On an empty / all-NULL + * table both are NULL, so coalesce the comparison to true (min <= max OR both + * NULL): the empty case passes soundly. + */ +PG_PROP(min_le_max) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(min(v) <= max(v), true) FROM mm_t"); + + pg_prop_sql_exec(conn, "DROP TABLE mm_t", &err); + pg_prop_disconnect(conn); +} + +/* + * min(v) is an actual value present in the column: either min is NULL (empty / + * all-NULL) or it occurs among the v values. coalesce(... IN ..., true) + * accepts the NULL-min case; a non-NULL min must be found in the table. + */ +PG_PROP(min_in_table) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN (SELECT min(v) FROM mm_t) IS NULL THEN true " + "ELSE ((SELECT min(v) FROM mm_t) IN (SELECT v FROM mm_t)) END"); + + pg_prop_sql_exec(conn, "DROP TABLE mm_t", &err); + pg_prop_disconnect(conn); +} + +/* + * max(v) dominates every non-NULL value: bool_and(v <= max(v)) over the + * non-NULL rows is true. Over no non-NULL rows bool_and is NULL, so + * coalesce(..., true) passes the empty / all-NULL case. + */ +PG_PROP(max_ge_all) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(v <= (SELECT max(v) FROM mm_t)), " + "true) FROM mm_t WHERE v IS NOT NULL"); + + pg_prop_sql_exec(conn, "DROP TABLE mm_t", &err); + pg_prop_disconnect(conn); +} + +/* + * De Morgan over the quantifier aggregates: bool_and(p) = NOT bool_or(NOT p), + * here with p = v > 0. IS NOT DISTINCT FROM makes the empty / all-NULL case + * (both sides NULL) pass; coalesce the whole comparison to true for the same + * reason that NULL IS NOT DISTINCT FROM NULL is already true. + */ +PG_PROP(bool_and_or_duality) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(" + "bool_and(v > 0) IS NOT DISTINCT FROM " + "NOT bool_or(NOT (v > 0)), true) FROM mm_t"); + + pg_prop_sql_exec(conn, "DROP TABLE mm_t", &err); + pg_prop_disconnect(conn); +} + +/* + * bit_and(v) is a submask of bit_or(v): a bit set in the AND of all values is + * necessarily set in the OR of all values, so (bit_and & bit_or) = bit_and. + * Both aggregates skip NULLs; over no non-NULL rows both are NULL, so + * coalesce the comparison to true for the empty / all-NULL case. + */ +PG_PROP(bit_and_submask_or) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce((bit_and(v) & bit_or(v)) = bit_and(v), true) " + "FROM mm_t WHERE v IS NOT NULL"); + + pg_prop_sql_exec(conn, "DROP TABLE mm_t", &err); + pg_prop_disconnect(conn); +} + +/* + * When count(v) > 0 there is at least one non-NULL value, so min(v) and + * max(v) are both non-NULL; when count(v) = 0 (empty / all-NULL) the CASE + * returns true, accepting that min/max are NULL. + */ +PG_PROP(count_min_max_nonnull) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN count(v) > 0 " + "THEN min(v) IS NOT NULL AND max(v) IS NOT NULL " + "ELSE true END FROM mm_t"); + + pg_prop_sql_exec(conn, "DROP TABLE mm_t", &err); + pg_prop_disconnect(conn); +} + +/* + * sum(v) is bracketed by count(v)*min(v) and count(v)*max(v) over the + * non-NULL rows: every value is in [min, max], so their sum is in + * [count*min, count*max]. Over no non-NULL rows sum/min/max are NULL and + * count is 0, so coalesce the comparison to true for the empty case. + */ +PG_PROP(sum_between_count_times_minmax) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(" + "count(v) * min(v) <= sum(v) " + "AND sum(v) <= count(v) * max(v), true) " + "FROM mm_t WHERE v IS NOT NULL"); + + pg_prop_sql_exec(conn, "DROP TABLE mm_t", &err); + pg_prop_disconnect(conn); +} + +/* + * min() ignores NULLs: min over all rows equals min over the NULL-filtered + * rows. IS NOT DISTINCT FROM makes the empty / all-NULL case (both min NULL) + * pass. + */ +PG_PROP(min_max_ignore_null) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT min(v) FROM mm_t) IS NOT DISTINCT FROM " + "(SELECT min(v) FROM mm_t WHERE v IS NOT NULL)"); + + pg_prop_sql_exec(conn, "DROP TABLE mm_t", &err); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 60e74bf251f2b..7c2c2c3ba1cc5 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From fbbb41a8f38e45768b86439a753105d65961f475 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:19 -0400 Subject: [PATCH 170/338] prop: add ordered-set and boolean/bit aggregates property tests (ordagg) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_ordagg.c | 279 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 283 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_ordagg.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index d0e6b89bc4fbf..027e558274287 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -185,7 +185,8 @@ OBJS = \ prop_aggdistinct.o \ prop_aggdedup.o \ prop_boolagg.o \ - prop_minmaxagg.o + prop_minmaxagg.o \ + prop_ordagg.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 03dbd9abbf488..88c2a991812c9 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -183,6 +183,7 @@ prop_area_sources = files( 'prop_aggdedup.c', 'prop_boolagg.c', 'prop_minmaxagg.c', + 'prop_ordagg.c', ) diff --git a/src/test/prop/prop_ordagg.c b/src/test/prop/prop_ordagg.c new file mode 100644 index 0000000000000..4ab26e1d928b6 --- /dev/null +++ b/src/test/prop/prop_ordagg.c @@ -0,0 +1,279 @@ +/*------------------------------------------------------------------------- + * + * prop_ordagg.c + * Property-based tests for ordered-set and boolean/bitwise aggregates. + * + * Ordered-set aggregates (percentile_cont, percentile_disc, mode) and the + * boolean/bitwise aggregates (bool_and, bool_or, bit_and, bit_or) have + * precise definitions; these check them over a generated integer column. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_ordagg.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "ordagg" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the aggregate errored or produced an unexpected NULL, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build the table oa_t(v int) and populate it with 0..25 rows of values in + * [-50, 50]. The narrow value range makes duplicates likely, so mode() has a + * meaningful most-frequent element and the discrete percentile has repeated + * elements to choose among. Zero rows exercise the empty-aggregate path. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS oa_t; " + "CREATE TABLE oa_t (v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 25); + for (int r = 0; r < nrows; r++) + appendStringInfo(&ddl, "INSERT INTO oa_t VALUES (%d); ", + (int) pg_prop_draw_int(tc, -50, 50)); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + + pfree(ddl.data); +} + +/* + * percentile_disc picks an actual element of the column. So the discrete + * median must be a value present in the column (empty table -> NULL IN (...) + * -> coalesce true). + */ +PG_PROP(percentile_disc_is_a_value) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN (SELECT count(*) FROM oa_t) = 0 THEN true ELSE " + "((SELECT percentile_disc(0.5) WITHIN GROUP (ORDER BY v) FROM oa_t) " + "IN (SELECT v FROM oa_t)) END"); + + pg_prop_disconnect(conn); +} + +/* + * percentile_cont interpolates between elements, so for any fraction p in + * [0,1] the result lies within [min(v), max(v)]. + */ +PG_PROP(percentile_cont_within_minmax) +{ + pg_prop_conn *conn; + char *err = NULL; + int pn; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + pn = (int) pg_prop_draw_int(tc, 0, 100); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(" + "percentile_cont(%d/100.0) WITHIN GROUP (ORDER BY v) " + "BETWEEN min(v) AND max(v), true) FROM oa_t", + pn); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * percentile_cont is non-decreasing in its fraction: for p1 <= p2 the + * interpolated value at p1 must not exceed that at p2. + */ +PG_PROP(percentile_cont_monotonic) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + b, + p1, + p2; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + a = (int) pg_prop_draw_int(tc, 0, 100); + b = (int) pg_prop_draw_int(tc, 0, 100); + p1 = (a <= b) ? a : b; + p2 = (a <= b) ? b : a; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(" + "(percentile_cont(%d/100.0) WITHIN GROUP (ORDER BY v)) " + "<= (percentile_cont(%d/100.0) WITHIN GROUP (ORDER BY v)), " + "true) FROM oa_t", + p1, p2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The extreme fractions pin to the extreme values: percentile_cont(0) = min(v) + * and percentile_cont(1) = max(v). + */ +PG_PROP(percentile_0_1_are_min_max) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(" + "(percentile_cont(0) WITHIN GROUP (ORDER BY v) = min(v)) " + "AND (percentile_cont(1) WITHIN GROUP (ORDER BY v) = max(v)), " + "true) FROM oa_t"); + + pg_prop_disconnect(conn); +} + +/* + * mode() returns a most-frequent value. Oracle: the count of rows equal to + * the mode equals the maximum per-value count. (Empty table -> mode is NULL, + * both subqueries are NULL -> coalesce true.) + */ +PG_PROP(mode_is_most_frequent) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN (SELECT count(*) FROM oa_t) = 0 THEN true ELSE " + "((SELECT count(*) FROM oa_t WHERE v = " + "(SELECT mode() WITHIN GROUP (ORDER BY v) FROM oa_t)) " + "= (SELECT max(c) FROM " + "(SELECT count(*) c FROM oa_t GROUP BY v) g)) END"); + + pg_prop_disconnect(conn); +} + +/* + * bool_and over a per-row predicate (v > 0) is true iff no row makes the + * predicate false; cross-check it against a direct count of falsifying rows. + * Empty table: bool_and is NULL -> coalesce true, and the count is 0 -> true, + * so the two sides agree. + */ +PG_PROP(bool_and_or_definition) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(v > 0), true) = " + "(coalesce((SELECT count(*) FROM oa_t WHERE NOT (v > 0)), 0) = 0) " + "FROM oa_t"); + + pg_prop_disconnect(conn); +} + +/* + * bit_or accumulates every bit any row has; bit_and keeps only bits all rows + * share. So bit_and(v) is a submask of bit_or(v): (bit_and & bit_or) = + * bit_and. (Empty table -> both NULL -> coalesce true.) + */ +PG_PROP(bit_and_or_bounds) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce((bit_and(v) & bit_or(v)) = bit_and(v), true) " + "FROM oa_t"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 7c2c2c3ba1cc5..735ba3545417e 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 64569657778c61ef164228685c384cc7c47ac28b Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:19 -0400 Subject: [PATCH 171/338] prop: add percentile_cont/disc and mode ordered-set aggregates property tests (percentilefn) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_percentilefn.c | 289 ++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 293 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_percentilefn.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 027e558274287..e3d785e5ee39a 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -186,7 +186,8 @@ OBJS = \ prop_aggdedup.o \ prop_boolagg.o \ prop_minmaxagg.o \ - prop_ordagg.o + prop_ordagg.o \ + prop_percentilefn.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 88c2a991812c9..31ea08a6e7899 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -184,6 +184,7 @@ prop_area_sources = files( 'prop_boolagg.c', 'prop_minmaxagg.c', 'prop_ordagg.c', + 'prop_percentilefn.c', ) diff --git a/src/test/prop/prop_percentilefn.c b/src/test/prop/prop_percentilefn.c new file mode 100644 index 0000000000000..9097c2e68649e --- /dev/null +++ b/src/test/prop/prop_percentilefn.c @@ -0,0 +1,289 @@ +/*------------------------------------------------------------------------- + * + * prop_percentilefn.c + * Property-based tests for the percentile and mode ordered-set aggregates. + * + * percentile_cont, percentile_disc and mode() are ordered-set aggregates with + * precise definitions: percentile_disc picks an actual column element, + * percentile_cont interpolates within [min, max] and is monotone in the + * fraction, and mode() returns a most-frequent value. Each oracle restates + * one of those definitions, so an "f" result is a bug in the aggregate. This + * area is kept distinct from ordagg/aggorder: it drills only the percentile + * and mode functions, including the ARRAY-argument percentile form. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_percentilefn.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "percentilefn" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the aggregate errored or produced an unexpected NULL, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build the table pf_t(id int PRIMARY KEY, v int) with 0..30 rows. Values in + * [-10, 10] with no NULLs: the narrow range makes duplicates likely, so + * mode() has a meaningful most-frequent element and percentile_disc has + * repeated elements to choose among. Zero rows exercise the empty-aggregate + * path (every property guards it). ANALYZE so the planner has stats. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS pf_t; " + "CREATE TABLE pf_t (id int PRIMARY KEY, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 30); + for (int r = 0; r < nrows; r++) + appendStringInfo(&ddl, "INSERT INTO pf_t VALUES (%d, %d); ", + r, (int) pg_prop_draw_int(tc, -10, 10)); + + appendStringInfoString(&ddl, "ANALYZE pf_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + + pfree(ddl.data); +} + +/* + * percentile_disc picks an actual element of the column, so the discrete + * median must be a value present in the column. (Empty table -> count 0 -> + * CASE true.) + */ +PG_PROP(percentile_disc_is_member) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN (SELECT count(*) FROM pf_t) = 0 THEN true ELSE " + "((SELECT percentile_disc(0.5) WITHIN GROUP (ORDER BY v) FROM pf_t) " + "IN (SELECT v FROM pf_t)) END"); + + pg_prop_disconnect(conn); +} + +/* + * percentile_cont interpolates between elements, so for any fraction p in + * [0,1] the result lies within [min(v), max(v)]. (Empty table -> count 0 -> + * CASE true.) + */ +PG_PROP(percentile_cont_in_range) +{ + pg_prop_conn *conn; + char *err = NULL; + int pn; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + pn = (int) pg_prop_draw_int(tc, 0, 100); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN (SELECT count(*) FROM pf_t) = 0 THEN true ELSE " + "((SELECT percentile_cont(%d/100.0) WITHIN GROUP (ORDER BY v) FROM pf_t) " + "BETWEEN (SELECT min(v) FROM pf_t) AND (SELECT max(v) FROM pf_t)) END", + pn); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The extreme fractions pin to the extreme values: percentile_cont(0) = min(v) + * and percentile_cont(1) = max(v). (Empty table -> count 0 -> CASE true.) + */ +PG_PROP(percentile_0_is_min) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN (SELECT count(*) FROM pf_t) = 0 THEN true ELSE " + "(((SELECT percentile_cont(0) WITHIN GROUP (ORDER BY v) FROM pf_t) " + "= (SELECT min(v) FROM pf_t)) " + "AND ((SELECT percentile_cont(1) WITHIN GROUP (ORDER BY v) FROM pf_t) " + "= (SELECT max(v) FROM pf_t))) END"); + + pg_prop_disconnect(conn); +} + +/* + * percentile_cont is non-decreasing in its fraction: for p1 <= p2 the + * interpolated value at p1 must not exceed that at p2. (Empty table -> + * count 0 -> CASE true.) + */ +PG_PROP(percentile_cont_monotone) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + b, + p1, + p2; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + a = (int) pg_prop_draw_int(tc, 0, 100); + b = (int) pg_prop_draw_int(tc, 0, 100); + p1 = (a <= b) ? a : b; + p2 = (a <= b) ? b : a; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN (SELECT count(*) FROM pf_t) = 0 THEN true ELSE " + "((SELECT percentile_cont(%d/100.0) WITHIN GROUP (ORDER BY v) FROM pf_t) " + "<= (SELECT percentile_cont(%d/100.0) WITHIN GROUP (ORDER BY v) FROM pf_t)) END", + p1, p2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The lowest discrete fraction pins to the smallest value: + * percentile_disc(0) = min(v). (Empty table -> count 0 -> CASE true.) + */ +PG_PROP(percentile_disc_0_is_min) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN (SELECT count(*) FROM pf_t) = 0 THEN true ELSE " + "((SELECT percentile_disc(0) WITHIN GROUP (ORDER BY v) FROM pf_t) " + "= (SELECT min(v) FROM pf_t)) END"); + + pg_prop_disconnect(conn); +} + +/* + * mode() returns a most-frequent value. Oracle: the count of rows equal to + * the mode equals the maximum per-value count. (Empty table -> count 0 -> + * CASE true.) + */ +PG_PROP(mode_is_frequent) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN (SELECT count(*) FROM pf_t) = 0 THEN true ELSE " + "((SELECT count(*) FROM pf_t WHERE v = " + "(SELECT mode() WITHIN GROUP (ORDER BY v) FROM pf_t)) " + "= (SELECT max(c) FROM " + "(SELECT count(*) c FROM pf_t GROUP BY v) g)) END"); + + pg_prop_disconnect(conn); +} + +/* + * The ARRAY form evaluates every fraction at once: percentile_cont(ARRAY + * [0,0.5,1]) returns a 3-element array that, since the fractions ascend, must + * be non-decreasing. (Empty table -> the array aggregate is NULL -> count 0 + * -> CASE true.) + */ +PG_PROP(percentile_array) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN (SELECT count(*) FROM pf_t) = 0 THEN true ELSE " + "(SELECT array_length(a, 1) = 3 AND a[1] <= a[2] AND a[2] <= a[3] " + "FROM (SELECT percentile_cont(ARRAY[0,0.5,1]) " + "WITHIN GROUP (ORDER BY v) AS a FROM pf_t) s) END"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 735ba3545417e..c0f96bcd9cf1c 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 18a94c02debe3a14a33017d4e1359e7b48cb38f4 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:19 -0400 Subject: [PATCH 172/338] prop: add aggregate FILTER vs CASE and DISTINCT-in-agg property tests (filteragg) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_filteragg.c | 322 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 326 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_filteragg.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index e3d785e5ee39a..a7b7de6378132 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -187,7 +187,8 @@ OBJS = \ prop_boolagg.o \ prop_minmaxagg.o \ prop_ordagg.o \ - prop_percentilefn.o + prop_percentilefn.o \ + prop_filteragg.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 31ea08a6e7899..9cc978351bf31 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -185,6 +185,7 @@ prop_area_sources = files( 'prop_minmaxagg.c', 'prop_ordagg.c', 'prop_percentilefn.c', + 'prop_filteragg.c', ) diff --git a/src/test/prop/prop_filteragg.c b/src/test/prop/prop_filteragg.c new file mode 100644 index 0000000000000..7844b8685476d --- /dev/null +++ b/src/test/prop/prop_filteragg.c @@ -0,0 +1,322 @@ +/*------------------------------------------------------------------------- + * + * prop_filteragg.c + * Property-based tests for aggregate FILTER clauses and DISTINCT-in-agg. + * + * An aggregate with a FILTER (WHERE p) clause must agg exactly the rows a + * CASE WHEN p THEN v END expression would feed it; the two spellings compile + * to different executor paths (Agg filter qual vs a CASE in the transfn + * input), so they are an independent oracle for each other. DISTINCT inside + * an aggregate deduplicates its input before the transfn runs; its result + * must stay consistent with the plain aggregate (count(DISTINCT) <= count()) + * and with a manual sum/count. A disagreement is a FILTER or DISTINCT-agg + * executor bug, not a rounding artefact. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_filteragg.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "filteragg" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the aggregate errored or produced an unexpected NULL, + * which is itself a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Compare two result sets rendered row-major; a mismatch is a failure. Both + * queries must carry a TOTAL ORDER BY so the string compare is a multiset + * compare (see area_spec.md). + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Build fa_t(id int PRIMARY KEY, g int, v int) with 0..30 rows. id is the + * row ordinal, g is a group in 0..3, and v is a small int in [-5, 5] or SQL + * NULL (~1 in 7, roughly 15%). Small domains keep collisions, duplicate v's + * (so DISTINCT does real work), and NULL/empty boundaries all exercised while + * keeping a counterexample legible. Rebuilt fresh in each property. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows = (int) pg_prop_draw_int(tc, 0, 30); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS fa_t; " + "CREATE TABLE fa_t (id int PRIMARY KEY, g int, v int); "); + + for (int r = 0; r < nrows; r++) + { + int g = (int) pg_prop_draw_int(tc, 0, 3); + + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfo(&ddl, "INSERT INTO fa_t VALUES (%d, %d, NULL); ", + r, g); + else + appendStringInfo(&ddl, "INSERT INTO fa_t VALUES (%d, %d, %d); ", + r, g, (int) pg_prop_draw_int(tc, -5, 5)); + } + + appendStringInfoString(&ddl, "ANALYZE fa_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + + pfree(ddl.data); +} + +/* + * FILTER equals CASE for sum: sum(v) FILTER (WHERE v>0) must equal + * sum(CASE WHEN v>0 THEN v END). Both aggregate exactly the rows where v>0. + * Over 0 rows (or no positive v) both sides are NULL; IS NOT DISTINCT FROM + * makes that pass. Wrapped in scalar subselects so a single row is returned + * even when fa_t is empty. + */ +PG_PROP(filter_eq_case) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT sum(v) FILTER (WHERE v>0) FROM fa_t) " + "IS NOT DISTINCT FROM " + "(SELECT sum(CASE WHEN v>0 THEN v END) FROM fa_t)"); + + pg_prop_disconnect(conn); +} + +/* + * FILTER equals CASE for count: count(*) FILTER (WHERE v>0) must equal + * count(CASE WHEN v>0 THEN 1 END). count(expr) skips NULLs, so the CASE's + * ELSE NULL arm drops exactly the rows the FILTER drops. count() is never + * NULL, so a plain = is sound even over 0 rows (both 0). + */ +PG_PROP(count_filter_eq_count_case) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FILTER (WHERE v>0) FROM fa_t) " + "= (SELECT count(CASE WHEN v>0 THEN 1 END) FROM fa_t)"); + + pg_prop_disconnect(conn); +} + +/* + * FILTER predicates partition the rows: the three mutually exclusive and + * exhaustive predicates v>0, v<=0, v IS NULL each count a disjoint subset, + * and together they must count every row -- so their sum equals count(*). + * All counts are non-NULL, so a plain = is sound even over 0 rows (0=0). + */ +PG_PROP(filter_partitions) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT " + "count(*) FILTER (WHERE v>0) " + "+ count(*) FILTER (WHERE v<=0) " + "+ count(*) FILTER (WHERE v IS NULL) " + "FROM fa_t) = (SELECT count(*) FROM fa_t)"); + + pg_prop_disconnect(conn); +} + +/* + * DISTINCT never increases a count: count(DISTINCT v) <= count(v). (The + * analogous sum(DISTINCT v) <= sum(v) is FALSE with negatives, so only the + * count form is asserted.) Both counts are non-NULL and 0 over an empty + * table, so 0<=0 holds. + */ +PG_PROP(distinct_count_le_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(DISTINCT v) FROM fa_t) " + "<= (SELECT count(v) FROM fa_t)"); + + pg_prop_disconnect(conn); +} + +/* + * avg(DISTINCT v) is, by definition, the sum of the distinct values over + * their count: avg(DISTINCT v) IS NOT DISTINCT FROM + * sum(DISTINCT v)::numeric / nullif(count(DISTINCT v), 0). nullif guards the + * empty / all-NULL case: both sides are then NULL and IS NOT DISTINCT FROM + * passes. The exact identity divides the same sum by the same count avg + * used, so it holds to the last digit. + */ +PG_PROP(avg_distinct_vs_manual) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT avg(DISTINCT v) FROM fa_t) " + "IS NOT DISTINCT FROM " + "(SELECT sum(DISTINCT v)::numeric " + "/ nullif(count(DISTINCT v), 0) FROM fa_t)"); + + pg_prop_disconnect(conn); +} + +/* + * Per-group FILTER sum equals a correlated-subquery sum (differential). The + * grouped aggregate sum(v) FILTER (WHERE v>0) per g must match, group for + * group, the sum of the same positive v's computed by a correlated subquery + * over the distinct groups. Both carry ORDER BY g (a total order over the + * distinct groups) and render NULL as "\\N", so an empty group's NULL on each + * side compares equal. A disagreement is a grouped-FILTER executor bug. + */ +PG_PROP(filter_grouped) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT g, sum(v) FILTER (WHERE v>0) " + "FROM fa_t GROUP BY g ORDER BY g", + "SELECT g, (SELECT sum(u.v) FROM fa_t u " + "WHERE u.g = t.g AND u.v > 0) " + "FROM (SELECT DISTINCT g FROM fa_t) t ORDER BY g"); + + pg_prop_disconnect(conn); +} + +/* + * string_agg(DISTINCT ...) emits no duplicate pieces: the number of + * comma-separated elements it produces equals count(DISTINCT v). Only + * asserted when there is at least one distinct value, because an empty + * string_agg is NULL -> coalesced to '' -> string_to_array('', ',') yields + * {''} (length 1) which would spuriously fail against count 0. The CASE + * short-circuits that boundary to true. v::text ORDER BY v::text makes the + * aggregate deterministic. + */ +PG_PROP(string_agg_distinct) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT CASE WHEN count(DISTINCT v) = 0 THEN true " + "ELSE cardinality(string_to_array(" + "coalesce(string_agg(DISTINCT v::text, ',' " + "ORDER BY v::text), ''), ',')) = count(DISTINCT v) END " + "FROM fa_t)"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index c0f96bcd9cf1c..55f351f367908 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 506db6da4ccc62b50a5f8fcf7fac596c06316ede Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:19 -0400 Subject: [PATCH 173/338] prop: add count/sum FILTER and boolean aggregation property tests (countfilter) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_countfilter.c | 260 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 264 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_countfilter.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index a7b7de6378132..c36ee9b52032c 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -188,7 +188,8 @@ OBJS = \ prop_minmaxagg.o \ prop_ordagg.o \ prop_percentilefn.o \ - prop_filteragg.o + prop_filteragg.o \ + prop_countfilter.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 9cc978351bf31..54cfae9b6ac42 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -186,6 +186,7 @@ prop_area_sources = files( 'prop_ordagg.c', 'prop_percentilefn.c', 'prop_filteragg.c', + 'prop_countfilter.c', ) diff --git a/src/test/prop/prop_countfilter.c b/src/test/prop/prop_countfilter.c new file mode 100644 index 0000000000000..02e5a34a57edc --- /dev/null +++ b/src/test/prop/prop_countfilter.c @@ -0,0 +1,260 @@ +/*------------------------------------------------------------------------- + * + * prop_countfilter.c + * Property-based tests for COUNT/SUM with FILTER and boolean-to-int + * aggregation. + * + * The oracle is the relationship between filtered aggregates and the + * equivalent boolean-predicate aggregates over the same rows: count(*) + * FILTER (WHERE p) must equal sum((p)::int) once NULL predicate rows are + * accounted for (a NULL predicate drops the row from the FILTER count, and + * (NULL)::int is NULL which sum skips -- so the two agree), count(*) must + * partition exactly into its FILTER complements, and bool_or/bool_and over a + * predicate must agree with the matching filtered counts. A failure here is + * a wrong-results bug in FILTER handling, boolean-to-int casting, or the + * boolean aggregates' NULL skipping. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_countfilter.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "countfilter" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the aggregate errored or produced NULL where it must not, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build cf_t(id int PRIMARY KEY, v int) with 0..30 rows: id is 0..n-1, v is + * drawn in -5..5 with ~15% NULL. Fresh each property so a counterexample is + * self-contained; ANALYZE so plan choice (hash vs sorted vs partial agg) is + * free to vary and the oracle holds regardless. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS cf_t; " + "CREATE TABLE cf_t (id int PRIMARY KEY, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 30); + for (int i = 0; i < nrows; i++) + { + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfo(&ddl, "INSERT INTO cf_t VALUES (%d, NULL); ", i); + else + appendStringInfo(&ddl, "INSERT INTO cf_t VALUES (%d, %d); ", + i, (int) pg_prop_draw_int(tc, -5, 5)); + } + appendStringInfoString(&ddl, "ANALYZE cf_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + pfree(ddl.data); +} + +/* + * count(*) FILTER (WHERE v > 0) = coalesce(sum((v > 0)::int), 0). + * + * A NULL v makes (v > 0) NULL, so the row is excluded from the FILTER count; + * (NULL)::int is NULL and sum skips it -- so both sides treat NULL rows + * identically. coalesce guards the all-NULL / empty case where sum is NULL + * but the FILTER count is 0. + */ +PG_PROP(count_filter_eq_sum_bool) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(sum((v > 0)::int), 0) " + "= count(*) FILTER (WHERE v > 0) FROM cf_t"); + + pg_prop_disconnect(conn); +} + +/* + * count(*) = count(v) + count(*) FILTER (WHERE v IS NULL). count(v) counts + * non-NULL v; the FILTER counts the NULL v; together they are every row. + */ +PG_PROP(count_star_eq_count_plus_nulls) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(*) " + "= count(v) + count(*) FILTER (WHERE v IS NULL) FROM cf_t"); + + pg_prop_disconnect(conn); +} + +/* + * coalesce(sum(v) FILTER (WHERE v > 0), 0) >= 0. Summing only positive values + * can never be negative; coalesce handles the no-positive-rows case (sum NULL). + */ +PG_PROP(sum_filter_positive_nonneg) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(sum(v) FILTER (WHERE v > 0), 0) >= 0 FROM cf_t"); + + pg_prop_disconnect(conn); +} + +/* + * The three FILTER complements partition every row exactly: + * count(*) FILTER (WHERE v>0) + count(*) FILTER (WHERE v<=0) + * + count(*) FILTER (WHERE v IS NULL) = count(*). + * Every row falls into exactly one of {v>0, v<=0, v IS NULL}. + */ +PG_PROP(filter_complement_partition) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(*) FILTER (WHERE v > 0) " + "+ count(*) FILTER (WHERE v <= 0) " + "+ count(*) FILTER (WHERE v IS NULL) " + "= count(*) FROM cf_t"); + + pg_prop_disconnect(conn); +} + +/* + * avg(v) FILTER (WHERE v > 0) lies in [1, 5] whenever any v > 0 exists: the + * values averaged are integers in 1..5, so their mean is in (0, 5], and being + * >= 1 (the smallest positive integer) and <= 5. The CASE guards the empty + * case (no positive rows): avg is NULL then, so return true. + */ +PG_PROP(avg_filter_in_range) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN count(*) FILTER (WHERE v > 0) = 0 THEN true " + "ELSE avg(v) FILTER (WHERE v > 0) BETWEEN 1 AND 5 END " + "FROM cf_t"); + + pg_prop_disconnect(conn); +} + +/* + * coalesce(bool_or(v > 0), false) = (count(*) FILTER (WHERE v > 0) > 0). + * bool_or over the predicate is true iff at least one row satisfies it, which + * is exactly the filtered count being positive. coalesce handles the empty / + * all-NULL case where bool_or is NULL and the count is 0 (false both sides). + */ +PG_PROP(bool_or_eq_count_gt_zero) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_or(v > 0), false) " + "= (count(*) FILTER (WHERE v > 0) > 0) FROM cf_t"); + + pg_prop_disconnect(conn); +} + +/* + * coalesce(bool_and(v > 0), true) = (count(*) FILTER (WHERE v <= 0) = 0). + * bool_and over (v > 0) skips NULL-v rows, so it is true iff no non-NULL row + * has v <= 0 -- i.e. the filtered count of v <= 0 is zero. coalesce handles + * the empty / all-NULL case (bool_and NULL, count 0: true both sides). + */ +PG_PROP(bool_and_eq_no_false) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(v > 0), true) " + "= (count(*) FILTER (WHERE v <= 0) = 0) FROM cf_t"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 55f351f367908..2befea8edf24f 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From de9fc3f9f73196e1c8a2c0e3041a4dc54a9fd3c8 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:19 -0400 Subject: [PATCH 174/338] prop: add string_agg and array_agg aggregation property tests (stringagg) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_stringagg.c | 313 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 317 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_stringagg.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index c36ee9b52032c..37c1e6191ad83 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -189,7 +189,8 @@ OBJS = \ prop_ordagg.o \ prop_percentilefn.o \ prop_filteragg.o \ - prop_countfilter.o + prop_countfilter.o \ + prop_stringagg.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 54cfae9b6ac42..96d9c5ebd6a2b 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -187,6 +187,7 @@ prop_area_sources = files( 'prop_percentilefn.c', 'prop_filteragg.c', 'prop_countfilter.c', + 'prop_stringagg.c', ) diff --git a/src/test/prop/prop_stringagg.c b/src/test/prop/prop_stringagg.c new file mode 100644 index 0000000000000..500e4e9802361 --- /dev/null +++ b/src/test/prop/prop_stringagg.c @@ -0,0 +1,313 @@ +/*------------------------------------------------------------------------- + * + * prop_stringagg.c + * Property-based tests for the string_agg and array_agg aggregates. + * + * string_agg and array_agg collect ordered input; these check the agg/unnest + * and agg/split inverses and length relationships over a generated table. + * + * The generated table sa_t(id int, v text) holds 0..20 rows with id = 1..n + * (unique) and v a short [a-z] string containing no comma, so the comma-join + * / string_to_array split round-trips exactly and every aggregate ordered BY + * id is deterministic. The oracle is each query's reference form: a split of + * the joined aggregate against the array aggregate, cardinality against + * count(*), string length against the sum of value lengths, an in-aggregate + * ORDER BY against an externally-ordered aggregate, string_agg's NULL-skipping + * against an explicit NOT NULL filter, and DISTINCT cardinality bounded by the + * total. IS NOT DISTINCT FROM / coalesce(...,0) make the empty table (all + * aggregates NULL / arrays NULL) pass. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_stringagg.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "stringagg" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the aggregate errored or produced NULL where the property + * expected a boolean, which is itself a bug -- so either is a failure, + * reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Append a short random [a-z] string literal (1..5 chars, no comma) to buf, + * single-quoted for use as a text value. + */ +static void +append_value_literal(pg_prop_tc *tc, StringInfo buf) +{ + int len = (int) pg_prop_draw_int(tc, 1, 5); + + appendStringInfoChar(buf, '\''); + for (int i = 0; i < len; i++) + appendStringInfoChar(buf, (char) ('a' + pg_prop_draw_int(tc, 0, 25))); + appendStringInfoChar(buf, '\''); +} + +/* + * Build (and first drop) the table sa_t(id int, v text) with n rows, id = + * 1..n assigned explicitly and v a short non-NULL [a-z] string. Fails the + * property if any DDL/DML errors. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn, int n) +{ + StringInfoData ddl; + char *err = NULL; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS sa_t; " + "CREATE TABLE sa_t (id int, v text); "); + + for (int i = 1; i <= n; i++) + { + appendStringInfo(&ddl, "INSERT INTO sa_t VALUES (%d, ", i); + append_value_literal(tc, &ddl); + appendStringInfoString(&ddl, "); "); + } + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s", err ? err : "(unknown)"); + + pfree(ddl.data); +} + +/* + * Build the table with some NULL v rows for the NULL-skipping property. id = + * 1..n; v is NULL ~30% of the time, otherwise a short [a-z] string. + */ +static void +build_table_with_nulls(pg_prop_tc *tc, pg_prop_conn *conn, int n) +{ + StringInfoData ddl; + char *err = NULL; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS sa_t; " + "CREATE TABLE sa_t (id int, v text); "); + + for (int i = 1; i <= n; i++) + { + appendStringInfo(&ddl, "INSERT INTO sa_t VALUES (%d, ", i); + if (pg_prop_draw_int(tc, 0, 2) == 0) + appendStringInfoString(&ddl, "NULL"); + else + append_value_literal(tc, &ddl); + appendStringInfoString(&ddl, "); "); + } + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s", err ? err : "(unknown)"); + + pfree(ddl.data); +} + +/* Drop the generated table. */ +static void +drop_table(pg_prop_conn *conn) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS sa_t", &err) && err) + pg_free(err); +} + +/* + * The comma-join aggregate split back on ',' recovers the ordered values. + * Because v contains no comma, string_to_array(string_agg(v, ',' ...), ',') + * must equal array_agg(v ...); both are NULL on the empty table, so compare + * with IS NOT DISTINCT FROM. + */ +PG_PROP(string_agg_split_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 0, 20); + build_table(tc, conn, n); + + require_true(tc, conn, + "SELECT string_to_array(string_agg(v, ',' ORDER BY id), ',') " + "IS NOT DISTINCT FROM array_agg(v ORDER BY id) " + "FROM sa_t"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * array_agg produces one element per row: cardinality(array_agg(v)) equals + * count(*). coalesce(...,0) handles the empty table, where the array is NULL. + */ +PG_PROP(array_agg_unnest_count) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 0, 20); + build_table(tc, conn, n); + + require_true(tc, conn, + "SELECT coalesce(cardinality(array_agg(v ORDER BY id)), 0) " + "= (SELECT count(*) FROM sa_t) FROM sa_t"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * The length of the empty-delimiter string_agg equals the sum of the value + * lengths: concatenation adds no separators, so length(string_agg(v, '')) + * equals sum(length(v)). coalesce(...,0) handles the empty table. + */ +PG_PROP(string_agg_length_sum) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 0, 20); + build_table(tc, conn, n); + + require_true(tc, conn, + "SELECT coalesce(length(string_agg(v, '' ORDER BY id)), 0) " + "= coalesce((SELECT sum(length(v)) FROM sa_t), 0) FROM sa_t"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * An in-aggregate ORDER BY matches an externally-ordered aggregate: + * array_agg(v ORDER BY id) equals array_agg(v2) over a subquery already + * ordered by id. Both NULL on the empty table, so IS NOT DISTINCT FROM. + */ +PG_PROP(array_agg_order_matches) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 0, 20); + build_table(tc, conn, n); + + require_true(tc, conn, + "SELECT array_agg(v ORDER BY id) IS NOT DISTINCT FROM " + "(SELECT array_agg(v2) FROM " + "(SELECT v AS v2 FROM sa_t ORDER BY id) s) " + "FROM sa_t"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * string_agg ignores NULL inputs: over a table that includes NULL v rows, the + * aggregate over all rows equals the aggregate restricted to the non-NULL + * rows. Both NULL when every row is NULL / the table is empty, so IS NOT + * DISTINCT FROM. + */ +PG_PROP(string_agg_skips_nulls) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 0, 20); + build_table_with_nulls(tc, conn, n); + + require_true(tc, conn, + "SELECT string_agg(v, ',' ORDER BY id) IS NOT DISTINCT FROM " + "(SELECT string_agg(v, ',' ORDER BY id) FROM sa_t " + "WHERE v IS NOT NULL) FROM sa_t"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * The distinct aggregate is no larger than the full one: + * cardinality(array_agg(DISTINCT v)) <= cardinality(array_agg(v)). + * coalesce(...,0) handles the empty table, where both arrays are NULL. + */ +PG_PROP(array_agg_distinct_le_total) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 0, 20); + build_table(tc, conn, n); + + require_true(tc, conn, + "SELECT coalesce(cardinality(array_agg(DISTINCT v)), 0) " + "<= coalesce(cardinality(array_agg(v)), 0) FROM sa_t"); + + drop_table(conn); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 2befea8edf24f..bd756faf7f770 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 337e3e36a9e3e79c9aec096700563eaab6360cdc Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:19 -0400 Subject: [PATCH 175/338] prop: add string_agg/array_to_string consistency property tests (strconcat) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_strconcat.c | 339 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 343 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_strconcat.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 37c1e6191ad83..cc263598b7f22 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -190,7 +190,8 @@ OBJS = \ prop_percentilefn.o \ prop_filteragg.o \ prop_countfilter.o \ - prop_stringagg.o + prop_stringagg.o \ + prop_strconcat.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 96d9c5ebd6a2b..0ee5018c143e1 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -188,6 +188,7 @@ prop_area_sources = files( 'prop_filteragg.c', 'prop_countfilter.c', 'prop_stringagg.c', + 'prop_strconcat.c', ) diff --git a/src/test/prop/prop_strconcat.c b/src/test/prop/prop_strconcat.c new file mode 100644 index 0000000000000..46ec03be1c35f --- /dev/null +++ b/src/test/prop/prop_strconcat.c @@ -0,0 +1,339 @@ +/*------------------------------------------------------------------------- + * + * prop_strconcat.c + * Property-based tests for string_agg / array_to_string / array_agg + * consistency. + * + * These check that the three string-concatenation aggregates agree: a + * comma-joined string_agg equals array_to_string(array_agg(...)), the + * empty-delimiter string_agg has one character per single-digit row, and the + * comma-joined string round-trips back through string_to_array to the ordered + * input rows. Each is a differential oracle (two independent ways must agree) + * or an exact identity, so any disagreement is a bug in one of the aggregates. + * + * The table sc_t(id int PRIMARY KEY, v int) holds 0..15 rows with id = 1..n + * (unique) and v a single decimal digit 0..9, so v::text is exactly one + * character and carries no comma; the comma-join / string_to_array split + * round-trips exactly and every aggregate ordered BY id is deterministic. + * coalesce(...,0) / IS NOT DISTINCT FROM make the empty table (all aggregates + * NULL) pass soundly. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_strconcat.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "strconcat" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the aggregate errored or produced NULL where the property + * expected a boolean, which is itself a bug -- so either is a failure, + * reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Differential result-set oracle: two queries must return identical rows in + * identical order. Both callers carry a total ORDER BY. (Copied from the + * area spec skeleton.) + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Build (and first drop) the table sc_t(id int PRIMARY KEY, v int) with n + * rows, id = 1..n assigned explicitly and v a single decimal digit 0..9 so + * that v::text is exactly one character and contains no comma. ANALYZE after + * load. Fails the property if any DDL/DML errors. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn, int n) +{ + StringInfoData ddl; + char *err = NULL; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS sc_t; " + "CREATE TABLE sc_t (id int PRIMARY KEY, v int); "); + + for (int i = 1; i <= n; i++) + appendStringInfo(&ddl, "INSERT INTO sc_t VALUES (%d, %d); ", + i, (int) pg_prop_draw_int(tc, 0, 9)); + + appendStringInfoString(&ddl, "ANALYZE sc_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s", err ? err : "(unknown)"); + + pfree(ddl.data); +} + +/* Drop the generated table. */ +static void +drop_table(pg_prop_conn *conn) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS sc_t", &err) && err) + pg_free(err); +} + +/* + * string_agg equals array_to_string(array_agg(...)): collecting ordered + * values and joining them with ',' must give the same string whether the join + * happens inside string_agg or afterward over array_agg. Both are NULL on the + * empty table, so compare with IS NOT DISTINCT FROM. + */ +PG_PROP(string_agg_eq_array_to_string) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 0, 15); + build_table(tc, conn, n); + + require_true(tc, conn, + "SELECT string_agg(v::text, ',' ORDER BY id) " + "IS NOT DISTINCT FROM " + "array_to_string(array_agg(v::text ORDER BY id), ',') " + "FROM sc_t"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * The empty-delimiter string_agg has exactly one character per row: each + * v::text is a single digit and '' adds no separators, so the length equals + * count(*). coalesce(...,0) handles the empty table, where the aggregate is + * NULL and count(*) is 0. + */ +PG_PROP(string_agg_length) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 0, 15); + build_table(tc, conn, n); + + require_true(tc, conn, + "SELECT coalesce(length(string_agg(v::text, '' ORDER BY id)), 0) " + "= count(*) FROM sc_t"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * The comma-joined string_agg round-trips back to rows: splitting it on ',' + * with ordinality recovers each v::text in id order. Both queries carry a + * total ORDER BY over the position (id in the reference, the split ordinality + * in the round-trip), so require_same_rows compares the pieces position by + * position -- a scrambled order would be caught, not masked. On the empty + * table both sides return zero rows (string_agg is NULL, unnest of NULL + * yields nothing), so the comparison is sound. + */ +PG_PROP(string_agg_pieces) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 0, 15); + build_table(tc, conn, n); + + require_same_rows(tc, conn, + "SELECT v::text AS x FROM sc_t ORDER BY id", + "SELECT q.x FROM " + "(SELECT u.x, u.ord FROM " + "(SELECT string_agg(v::text, ',' ORDER BY id) AS sa " + "FROM sc_t) s, " + "unnest(string_to_array(s.sa, ',')) " + "WITH ORDINALITY AS u(x, ord)) q ORDER BY q.ord"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * The comma-joined string_agg has exactly (count(*) - 1) commas when + * non-empty, and 0 when empty: each single-digit value contributes no comma, + * and the join inserts one delimiter between adjacent values. The comma count + * is length - length(string with commas stripped). coalesce(...,0) handles + * the empty table (string_agg NULL), and greatest(count(*)-1, 0) handles both + * the empty and single-row cases. + */ +PG_PROP(concat_ws_eq_string_agg) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 0, 15); + build_table(tc, conn, n); + + require_true(tc, conn, + "SELECT coalesce(" + "length(string_agg(v::text, ',')) " + "- length(replace(string_agg(v::text, ','), ',', '')), 0) " + "= greatest(count(*) - 1, 0) FROM sc_t"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * array_agg produces one element per row: cardinality(array_agg(v)) equals + * count(*). coalesce(...,0) handles the empty table, where the array is NULL. + */ +PG_PROP(array_agg_cardinality) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 0, 15); + build_table(tc, conn, n); + + require_true(tc, conn, + "SELECT coalesce(cardinality(array_agg(v ORDER BY id)), 0) " + "= count(*) FROM sc_t"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * An ordered string_agg is deterministic: computing it twice over the same + * table with the same ORDER BY must give the identical string. Both NULL on + * the empty table, so IS NOT DISTINCT FROM. + */ +PG_PROP(string_agg_order_deterministic) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 0, 15); + build_table(tc, conn, n); + + require_true(tc, conn, + "SELECT (SELECT string_agg(v::text, ',' ORDER BY id) FROM sc_t) " + "IS NOT DISTINCT FROM " + "(SELECT string_agg(v::text, ',' ORDER BY id) FROM sc_t)"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * string_agg over no rows is SQL NULL: an aggregate with no input produces + * NULL, so the WHERE false form must be NULL on every table. + */ +PG_PROP(empty_string_agg_null) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = (int) pg_prop_draw_int(tc, 0, 15); + build_table(tc, conn, n); + + require_true(tc, conn, + "SELECT (SELECT string_agg(v::text, ',') FROM sc_t WHERE false) " + "IS NULL"); + + drop_table(conn); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index bd756faf7f770..d43c834086031 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 5b4cf04ea83ace863add234c80bd3eccec5eb5d3 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:20 -0400 Subject: [PATCH 176/338] prop: add JSON aggregate array and object building property tests (jsonagg) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonagg.c | 289 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 293 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonagg.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index cc263598b7f22..b17da25441b9f 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -191,7 +191,8 @@ OBJS = \ prop_filteragg.o \ prop_countfilter.o \ prop_stringagg.o \ - prop_strconcat.o + prop_strconcat.o \ + prop_jsonagg.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 0ee5018c143e1..981860f1ce715 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -189,6 +189,7 @@ prop_area_sources = files( 'prop_countfilter.c', 'prop_stringagg.c', 'prop_strconcat.c', + 'prop_jsonagg.c', ) diff --git a/src/test/prop/prop_jsonagg.c b/src/test/prop/prop_jsonagg.c new file mode 100644 index 0000000000000..9d21b45e061f2 --- /dev/null +++ b/src/test/prop/prop_jsonagg.c @@ -0,0 +1,289 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonagg.c + * Property-based tests for the JSON aggregate functions. + * + * The JSON aggregates build composite JSON values from rows: json_agg and + * jsonb_agg collect their argument into an array, one element per row in + * aggregation order, while json_object_agg and jsonb_object_agg collect + * key/value pairs into an object, one member per row. These properties + * check the structural relationships that must hold between the aggregated + * result and the rows it was built from -- the array length (or object member + * count) equals the row count, a key lookup recovers the row that supplied it, + * and unnesting the aggregated array recovers the original multiset of values + * -- plus the agree-with-jsonb identity for the json (text) variants. All run + * over a small generated table ja_t(id int, k text, v int). + * + * The oracle is the table itself: count(*), to_jsonb() on a single row, and + * array_agg() of the raw column values are computed directly from the rows + * and compared against what the JSON aggregate produced, so the aggregate's + * answer is checked against the source data it summarised rather than against + * a second copy of itself. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonagg.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonagg" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the aggregate errored or produced NULL where the property + * expects a definite true/false, which is itself a bug -- so either is a + * failure, reported with the SQL. Mirrors require_true() in prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build the generated table ja_t(id int, k text, v int) with n rows, where n + * is drawn in [1, 15]. Row i (1-based) has id = i, a DISTINCT key + * 'k'||(i-1) (so 'k0'..'k(n-1)'), and a small integer v. Keys are distinct + * by construction, so jsonb_object_agg(k, v) has exactly one member per row + * with no dedup. n >= 1 guarantees 'k0' exists for the lookup property. + * + * Returns the row count n (always >= 1). pg_prop_fail()s on any DDL/DML + * error, since a failure to build the fixture is not something the property + * can meaningfully proceed past. + */ +static int +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int n; + + /* n >= 1 so key 'k0' always exists (jsonb_object_agg_lookup needs it). */ + n = (int) pg_prop_draw_int(tc, 1, 15); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS ja_t; " + "CREATE TABLE ja_t (id int, k text, v int); "); + + for (int i = 1; i <= n; i++) + { + int v = (int) pg_prop_draw_int(tc, -8, 8); + + /* id = i, k = 'k'||(i-1) distinct per row, v a small int */ + appendStringInfo(&ddl, + "INSERT INTO ja_t VALUES (%d, 'k%d', %d); ", + i, i - 1, v); + } + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + { + pg_prop_fail(tc, "failed to build ja_t: %s\n ddl: %s", + err ? err : "(no message)", ddl.data); + } + + pfree(ddl.data); + return n; +} + +/* Drop the generated table. Best-effort; ignores any error. */ +static void +drop_table(pg_prop_conn *conn) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS ja_t", &err)) + { + if (err) + pg_free(err); + } +} + +/* + * jsonb_agg(v) produces an array with exactly one element per row. Over no + * rows the aggregate is SQL NULL, so coalesce the array length to 0; the + * table always has >= 1 row here, but the coalesce keeps the invariant stated + * for the empty-table case too. + */ +PG_PROP(jsonb_agg_length_matches_rows) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(jsonb_array_length(jsonb_agg(v)), 0) " + "= (SELECT count(*) FROM ja_t) FROM ja_t"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * jsonb_object_agg(k, v) with DISTINCT keys has exactly one member per row: + * counting the members (jsonb_each over the object) equals the row count. + * Because the keys are distinct by construction there is no dedup, so no row + * can be collapsed away. + */ +PG_PROP(jsonb_object_agg_key_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(" + "(SELECT count(*) FROM jsonb_each(jsonb_object_agg(k, v))), 0) " + "= (SELECT count(*) FROM ja_t) FROM ja_t"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * A key lookup recovers the row that supplied it: for the key 'k0' (always + * present, since the table has >= 1 row), the aggregated object's 'k0' member + * equals that row's v rendered as jsonb. IS NOT DISTINCT FROM so the compare + * is well-defined even if the value were NULL. + */ +PG_PROP(jsonb_object_agg_lookup) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (jsonb_object_agg(k, v) -> 'k0') " + "IS NOT DISTINCT FROM " + "(SELECT to_jsonb(v) FROM ja_t WHERE k = 'k0') FROM ja_t"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * json_agg and jsonb_agg agree on value: the json (text) variant, cast to + * jsonb, equals the jsonb variant. json preserves textual form while jsonb + * normalises, but once json_agg's result is cast to jsonb the two must be + * identical. Both aggregate ORDER BY id so element order matches, and both + * coalesce to 'null'::jsonb over no rows. + */ +PG_PROP(json_agg_matches_jsonb_agg) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(json_agg(v ORDER BY id)::jsonb, 'null'::jsonb) " + "= coalesce(jsonb_agg(v ORDER BY id), 'null'::jsonb) FROM ja_t"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * Unnesting the aggregated array recovers the row values: the multiset of + * elements in jsonb_agg(v) equals the multiset of v values from the rows. + * Both sides are sorted into an int[] before comparing, so order does not + * matter and duplicates are preserved. Both coalesce to '{}'::int[] over no + * rows. + */ +PG_PROP(jsonb_agg_elements_match_rows) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(" + "(SELECT array_agg(e::int ORDER BY e::int) " + "FROM jsonb_array_elements(jsonb_agg(v)) e), '{}'::int[]) " + "= coalesce(" + "(SELECT array_agg(v ORDER BY v) FROM ja_t), '{}'::int[]) " + "FROM ja_t"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * jsonb_agg(v ORDER BY id) preserves row order: the first element of the + * aggregated array is the v of the smallest-id row. IS NOT DISTINCT FROM for + * a NULL-safe compare, and coalesce to true over no rows (where there is no + * first element to check). + */ +PG_PROP(jsonb_agg_order_preserved) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(" + "(jsonb_agg(v ORDER BY id) -> 0) IS NOT DISTINCT FROM " + "(SELECT to_jsonb(v) FROM ja_t ORDER BY id LIMIT 1), true) " + "FROM ja_t"); + + drop_table(conn); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index d43c834086031..c7f67ee496c30 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From a31dde1717d3b8a3b93bfce0a9f3ca23a6b18e3b Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:20 -0400 Subject: [PATCH 177/338] prop: add json/jsonb aggregate ordering and filtering property tests (jsonaggord) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonaggord.c | 315 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 319 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonaggord.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index b17da25441b9f..97bf0f1561e77 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -192,7 +192,8 @@ OBJS = \ prop_countfilter.o \ prop_stringagg.o \ prop_strconcat.o \ - prop_jsonagg.o + prop_jsonagg.o \ + prop_jsonaggord.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 981860f1ce715..3fd0e5b48ca1d 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -190,6 +190,7 @@ prop_area_sources = files( 'prop_stringagg.c', 'prop_strconcat.c', 'prop_jsonagg.c', + 'prop_jsonaggord.c', ) diff --git a/src/test/prop/prop_jsonaggord.c b/src/test/prop/prop_jsonaggord.c new file mode 100644 index 0000000000000..e3d689576d12d --- /dev/null +++ b/src/test/prop/prop_jsonaggord.c @@ -0,0 +1,315 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonaggord.c + * Property-based tests for jsonb aggregate ordering, filtering and keys. + * + * Exercises jsonb_agg()/jsonb_object_agg() behaviour distinct from the + * jsonagg/jsonbagg2 areas: the ORDER BY inside the aggregate (first element + * equals min), FILTER (WHERE ...) narrowing the aggregated set, DISTINCT + * de-duplication, element-sum against sum(v), and per-group length. The + * oracle is a count/sum/extremum identity against the plain aggregate -- so a + * mismatch is a bug in the jsonb aggregates, not a tolerance. Every property + * guards the empty / all-filtered-out case with coalesce so 0 rows passes + * soundly. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonaggord.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonaggord" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operation errored or produced NULL on valid inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build ja_t(id int PRIMARY KEY, g int, v int) with a drawn number of rows + * 0..20. id runs 0..nrows-1 (a dense distinct PK), g in 0..2 (so grouping has + * a few buckets), v in 0..9. Returns the row count so properties can guard + * the empty case. + */ +static int +build_ja_t(pg_prop_tc *tc, pg_prop_conn *conn) +{ + int nrows = (int) pg_prop_draw_int(tc, 0, 20); + int i; + char *err = NULL; + StringInfoData sql; + + initStringInfo(&sql); + appendStringInfoString(&sql, + "DROP TABLE IF EXISTS ja_t; " + "CREATE TABLE ja_t(id int PRIMARY KEY, g int, v int); "); + for (i = 0; i < nrows; i++) + { + int g = (int) pg_prop_draw_int(tc, 0, 2); + int v = (int) pg_prop_draw_int(tc, 0, 9); + + appendStringInfo(&sql, + "INSERT INTO ja_t VALUES (%d, %d, %d); ", + i, g, v); + } + appendStringInfoString(&sql, "ANALYZE ja_t;"); + + if (!pg_prop_sql_exec(conn, sql.data, &err)) + pg_prop_fail(tc, "build ja_t failed: %s\n SQL: %s", + err ? err : "", sql.data); + + pfree(sql.data); + return nrows; +} + +/* + * Ordered first element: jsonb_agg(v ORDER BY v) -> 0 equals to_jsonb(min(v)). + * The first element of a value-ordered array is the minimum. Guarded on + * nrows >= 1: over 0 rows the aggregate is NULL (-> 0 is NULL) and min(v) is + * NULL too, so there is nothing to compare; skip to a true constant. + */ +PG_PROP(jsonb_agg_ordered_first) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + nrows = build_ja_t(tc, conn); + + initStringInfo(&q); + if (nrows >= 1) + appendStringInfoString(&q, + "SELECT (jsonb_agg(v ORDER BY v) -> 0) " + "= to_jsonb(min(v)) FROM ja_t"); + else + appendStringInfoString(&q, "SELECT true"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Array length equals row count: jsonb_array_length(jsonb_agg(v)) = count(*). + * coalesce handles the empty table, where jsonb_agg is NULL: coalesce(NULL, 0) + * = count(*) = 0 holds soundly. + */ +PG_PROP(jsonb_agg_length) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + (void) build_ja_t(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT coalesce(jsonb_array_length(jsonb_agg(v)), 0) " + "= count(*) FROM ja_t"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Object_agg lookup by PK: jsonb_object_agg(id::text, v) looked up at the min + * id's text equals to_jsonb of that row's v. id is the PRIMARY KEY, so keys + * are distinct and the lookup is unambiguous. Guarded on nrows >= 1 so a + * minimum id exists; the empty case skips to a true constant. + */ +PG_PROP(jsonb_object_agg_lookup) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + nrows = build_ja_t(tc, conn); + + initStringInfo(&q); + if (nrows >= 1) + appendStringInfoString(&q, + "SELECT (jsonb_object_agg(id::text, v) " + "-> (SELECT min(id)::text FROM ja_t)) " + "= to_jsonb((SELECT v FROM ja_t " + "WHERE id = (SELECT min(id) FROM ja_t))) " + "FROM ja_t"); + else + appendStringInfoString(&q, "SELECT true"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * FILTER narrows the aggregate: the length of jsonb_agg(v) FILTER (WHERE v > 0) + * equals count(*) FILTER (WHERE v > 0). coalesce handles the all-filtered-out + * case: with no surviving row the aggregate is NULL, coalesce(NULL, 0) = 0 = + * the filtered count, so it passes soundly. + */ +PG_PROP(jsonb_agg_filter) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + (void) build_ja_t(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT coalesce(jsonb_array_length(" + "jsonb_agg(v) FILTER (WHERE v > 0)), 0) " + "= count(*) FILTER (WHERE v > 0) FROM ja_t"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * DISTINCT de-duplicates: jsonb_array_length(jsonb_agg(DISTINCT v)) equals + * count(DISTINCT v). coalesce handles the empty table (NULL -> 0 = 0). + */ +PG_PROP(jsonb_agg_distinct) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + (void) build_ja_t(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT coalesce(jsonb_array_length(" + "jsonb_agg(DISTINCT v)), 0) " + "= count(DISTINCT v) FROM ja_t"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Element sum equals sum(v): summing the integer elements of jsonb_agg(v) + * equals sum(v) over the table. Both sides coalesce to 0 on the empty table, + * where jsonb_agg is NULL (defaulted to '[]') and sum(v) is NULL. + */ +PG_PROP(jsonb_agg_elements_sum) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + (void) build_ja_t(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT (SELECT coalesce(sum(e::int), 0) " + "FROM jsonb_array_elements(" + "coalesce((SELECT jsonb_agg(v) FROM ja_t), " + "'[]'::jsonb)) e) " + "= (SELECT coalesce(sum(v), 0) FROM ja_t)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Per-group length: within every g bucket, jsonb_array_length(jsonb_agg(v)) + * equals that group's count(*). bool_and over the grouped subquery folds the + * per-group checks into one boolean; coalesce(..., true) makes the no-group + * (empty table) case pass soundly, since bool_and over 0 rows is NULL. + */ +PG_PROP(grouped_jsonb_agg) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + (void) build_ja_t(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT coalesce(bool_and(" + "jsonb_array_length(ja) = c), true) " + "FROM (SELECT jsonb_agg(v) ja, count(*) c " + "FROM ja_t GROUP BY g) s"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index c7f67ee496c30..d6ea7a0f0ae56 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 5111eafa890eb2c227dfd448160d0bfe6e68fd3a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:20 -0400 Subject: [PATCH 178/338] prop: add jsonb_object_agg over a table property tests (jsonobjectagg) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jsonobjectagg.c | 313 +++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 317 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jsonobjectagg.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 97bf0f1561e77..eb534a3b947d3 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -193,7 +193,8 @@ OBJS = \ prop_stringagg.o \ prop_strconcat.o \ prop_jsonagg.o \ - prop_jsonaggord.o + prop_jsonaggord.o \ + prop_jsonobjectagg.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 3fd0e5b48ca1d..9b84eaf716c25 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -191,6 +191,7 @@ prop_area_sources = files( 'prop_strconcat.c', 'prop_jsonagg.c', 'prop_jsonaggord.c', + 'prop_jsonobjectagg.c', ) diff --git a/src/test/prop/prop_jsonobjectagg.c b/src/test/prop/prop_jsonobjectagg.c new file mode 100644 index 0000000000000..2008c4d07ebe6 --- /dev/null +++ b/src/test/prop/prop_jsonobjectagg.c @@ -0,0 +1,313 @@ +/*------------------------------------------------------------------------- + * + * prop_jsonobjectagg.c + * Property-based tests for jsonb_object_agg and json_object_agg. + * + * The object aggregates collect key/value pairs from rows into a JSON object, + * one member per row. Over a table whose keys are DISTINCT by construction + * (k0..k(n-1)), there is no key collision, so every row contributes exactly + * one member. These properties check that structural fact against the source + * rows -- member count, key count, value sum, a key lookup, the empty-input + * NULL, a FILTER restriction -- plus the json(text)==jsonb agreement. Any + * mismatch means the aggregate summarised the rows wrongly, which is a bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jsonobjectagg.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jsonobjectagg" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the aggregate errored or produced NULL where the property + * expects a definite true/false, which is itself a bug -- so either is a + * failure, reported with the SQL. Mirrors require_true() in prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build jo_t(id int PRIMARY KEY, k text, v int) with n rows, n drawn in + * [0, 15]. Row i (1-based) has id = i, a DISTINCT key 'k'||(i-1) (so + * 'k0'..'k(n-1)'), and a small integer v. Keys are distinct by construction, + * so an object aggregate has exactly one member per row with no dedup. n may + * be 0 (empty table) to exercise the empty-input case; the properties coalesce + * for that. ANALYZE after load. This table name is distinct from the sibling + * jsonagg areas' ja_t so the areas never collide. + * + * Returns the row count n. pg_prop_fail()s on any DDL/DML error, since a + * failure to build the fixture is not something the property can proceed past. + */ +static int +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int n; + + n = (int) pg_prop_draw_int(tc, 0, 15); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS jo_t; " + "CREATE TABLE jo_t (id int PRIMARY KEY, k text, v int); "); + + for (int i = 1; i <= n; i++) + { + int v = (int) pg_prop_draw_int(tc, -8, 8); + + /* id = i, k = 'k'||(i-1) distinct per row, v a small int */ + appendStringInfo(&ddl, + "INSERT INTO jo_t VALUES (%d, 'k%d', %d); ", + i, i - 1, v); + } + + appendStringInfoString(&ddl, "ANALYZE jo_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + { + pg_prop_fail(tc, "failed to build jo_t: %s\n ddl: %s", + err ? err : "(no message)", ddl.data); + } + + pfree(ddl.data); + return n; +} + +/* Drop the generated table. Best-effort; ignores any error. */ +static void +drop_table(pg_prop_conn *conn) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS jo_t", &err)) + { + if (err) + pg_free(err); + } +} + +/* + * object_agg_member_count: jsonb_object_agg(k, v) with DISTINCT keys has + * exactly one member per row, so counting its members (jsonb_each over the + * object) equals the row count. Over no rows the aggregate is SQL NULL; + * coalesce it to '{}'::jsonb so the empty case counts 0 members and still + * holds. + */ +PG_PROP(object_agg_member_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM jsonb_each(" + "coalesce((SELECT jsonb_object_agg(k, v) FROM jo_t), " + "'{}'::jsonb))) " + "= (SELECT count(*) FROM jo_t)"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * object_agg_lookup: a key lookup recovers the row that supplied it. For key + * 'k0', the aggregated object's 'k0' member equals that row's v rendered as + * jsonb -- but only when 'k0' exists (the table may be empty). The CASE + * guards on existence: with no 'k0' row the property is vacuously true. + */ +PG_PROP(object_agg_lookup) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN EXISTS (SELECT 1 FROM jo_t WHERE k = 'k0') " + "THEN (SELECT jsonb_object_agg(k, v) FROM jo_t) -> 'k0' " + "IS NOT DISTINCT FROM " + "(SELECT to_jsonb(v) FROM jo_t WHERE k = 'k0') " + "ELSE true END"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * json_eq_jsonb: the json(text) variant cast to jsonb equals the jsonb + * variant -- the two aggregates build the same object, differing only in + * textual vs normalised form, which the cast erases. Over no rows both + * aggregates are SQL NULL; coalesce both sides so the empty case compares + * '{}'::jsonb = '{}'::jsonb and holds. + */ +PG_PROP(json_eq_jsonb) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce((SELECT json_object_agg(k, v) FROM jo_t)::jsonb, " + "'{}'::jsonb) " + "= coalesce((SELECT jsonb_object_agg(k, v) FROM jo_t), " + "'{}'::jsonb)"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * object_agg_keys_match: the number of keys in jsonb_object_agg(k, v) equals + * the number of distinct k values in the table. With DISTINCT keys by + * construction count(DISTINCT k) = count(*), so no key is dropped by the + * aggregate's last-wins dedup. coalesce to '{}'::jsonb over no rows (0 keys). + */ +PG_PROP(object_agg_keys_match) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM jsonb_object_keys(" + "coalesce((SELECT jsonb_object_agg(k, v) FROM jo_t), " + "'{}'::jsonb))) " + "= (SELECT count(DISTINCT k) FROM jo_t)"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * object_agg_values_sum: the sum of the values in jsonb_object_agg(k, v) + * equals sum(v) over the rows. Keys are distinct so every v survives as a + * member; summing the member values (jsonb_each_text, cast to int) recovers + * the row sum. coalesce both the object (empty -> '{}') and the sums (empty + * aggregate -> 0) so the empty case is 0 = 0. + */ +PG_PROP(object_agg_values_sum) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT coalesce(sum(value::int), 0) " + "FROM jsonb_each_text(" + "coalesce((SELECT jsonb_object_agg(k, v) FROM jo_t), " + "'{}'::jsonb))) " + "= (SELECT coalesce(sum(v), 0) FROM jo_t)"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * empty_object_agg_null: an object aggregate over zero rows is SQL NULL. A + * WHERE false scan feeds the aggregate no rows regardless of the generated + * table contents, so the result must be NULL. + */ +PG_PROP(empty_object_agg_null) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT jsonb_object_agg(k, v) FROM jo_t WHERE false) " + "IS NULL"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * object_agg_filter: jsonb_object_agg(k, v) FILTER (WHERE v > 0) builds an + * object only from the positive-v rows, so its member count equals the number + * of rows with v > 0. coalesce the filtered aggregate to '{}'::jsonb for the + * case where no row has v > 0 (the FILTER then yields SQL NULL). + */ +PG_PROP(object_agg_filter) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM jsonb_each(" + "coalesce((SELECT jsonb_object_agg(k, v) FILTER (WHERE v > 0) " + "FROM jo_t), '{}'::jsonb))) " + "= (SELECT count(*) FROM jo_t WHERE v > 0)"); + + drop_table(conn); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index d6ea7a0f0ae56..5526d0a9ed45d 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 22a820e12a177f3132d3dcb928d94f01260b9d6b Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:20 -0400 Subject: [PATCH 179/338] prop: add window property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_window.c | 266 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 270 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_window.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index eb534a3b947d3..6aebcdcb6d748 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -194,7 +194,8 @@ OBJS = \ prop_strconcat.o \ prop_jsonagg.o \ prop_jsonaggord.o \ - prop_jsonobjectagg.o + prop_jsonobjectagg.o \ + prop_window.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 9b84eaf716c25..2e208dce49629 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -192,6 +192,7 @@ prop_area_sources = files( 'prop_jsonagg.c', 'prop_jsonaggord.c', 'prop_jsonobjectagg.c', + 'prop_window.c', ) diff --git a/src/test/prop/prop_window.c b/src/test/prop/prop_window.c new file mode 100644 index 0000000000000..713ff3df0b263 --- /dev/null +++ b/src/test/prop/prop_window.c @@ -0,0 +1,266 @@ +/*------------------------------------------------------------------------- + * + * prop_window.c + * Property-based tests for SQL window functions. + * + * Window functions over an ordered partition must satisfy structural + * invariants: row_number() is a permutation of 1..n, rank() <= row_number() + * under the same ordering, dense_rank() is gapless (its maximum equals the + * number of distinct order-key values), lag/lead shift by their offset, and a + * running sum built with ROWS UNBOUNDED PRECEDING ends at the grand total. + * These properties check each invariant over a generated table. + * + * The oracle is the invariant itself, expressed in SQL and reduced to a + * single boolean: each per-row check is folded with bool_and(...) over the + * windowed subquery and wrapped in coalesce(..., true) so the empty table + * (n = 0) passes vacuously. IS NOT DISTINCT FROM is used wherever a value may + * be NULL (lag at the first row, the sum of an empty set). + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_window.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "window" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Value domain for the "v" column. Kept small so that running sums of up to + * 25 rows (|sum| <= 25 * 50 = 1250) cannot overflow int, and so a + * counterexample table stays legible. + */ +#define V_MIN (-50) +#define V_MAX 50 + +/* + * Build the test table agg_w(id int, v int) with a drawn number of rows in + * [0, 25]. id is assigned 1..n explicitly so that ORDER BY id is a strict + * total order (lag/lead are then fully deterministic); v is random in + * [V_MIN, V_MAX] and may tie, so any ordering that must be total uses + * ORDER BY v, id. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS agg_w; " + "CREATE TABLE agg_w (id int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 25); + for (int i = 1; i <= nrows; i++) + { + int v = (int) pg_prop_draw_int(tc, V_MIN, V_MAX); + + appendStringInfo(&ddl, "INSERT INTO agg_w VALUES (%d, %d); ", i, v); + } + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + + pfree(ddl.data); +} + +/* Drop the test table (best effort; ignore errors). */ +static void +drop_table(pg_prop_conn *conn) +{ + char *err = NULL; + + (void) pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS agg_w", &err); + if (err) + pg_free(err); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the window query errored or produced NULL where the + * invariant should have been decided, which is itself a failure, reported + * with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * row_number() OVER (ORDER BY v, id) over n rows yields exactly {1..n}: its + * maximum equals the row count and the number of distinct values equals the + * row count, so the generated numbers are a permutation of 1..n. + */ +PG_PROP(row_number_is_permutation) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce((max(rn) = count(*) " + "AND count(DISTINCT rn) = count(*)), true) " + "FROM (SELECT row_number() OVER (ORDER BY v, id) rn " + "FROM agg_w) s"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * For every row, rank() <= row_number() under the same ordering. rank() ties + * share the lower number while row_number() is strictly increasing, so rank() + * can only equal or trail row_number(). + */ +PG_PROP(rank_le_row_number) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(rk <= rn), true) " + "FROM (SELECT rank() OVER w rk, row_number() OVER w rn " + "FROM agg_w WINDOW w AS (ORDER BY v)) s"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * dense_rank() is gapless: its maximum over ORDER BY v equals the number of + * distinct v values. (On the empty table max() is NULL, so coalesce it to 0, + * which equals count(DISTINCT v) = 0.) + */ +PG_PROP(dense_rank_gapless) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce((SELECT max(dr) FROM " + "(SELECT dense_rank() OVER (ORDER BY v) dr FROM agg_w) s), 0) " + "= (SELECT count(DISTINCT v) FROM agg_w)"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * lag(v) OVER (ORDER BY id) at the row with id = k must equal the v of the row + * with id = k - 1. A self LEFT JOIN against the row at id - 1 is the oracle: + * at the first row there is no id - 1, b.v is NULL, and lag is NULL, so + * IS NOT DISTINCT FROM holds there too. + */ +PG_PROP(lag_lead_inverse) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(a.lagged IS NOT DISTINCT FROM b.v), true) " + "FROM (SELECT id, lag(v) OVER (ORDER BY id) lagged FROM agg_w) a " + "LEFT JOIN agg_w b ON b.id = a.id - 1"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * The running sum (sum(v) OVER ORDER BY id ROWS UNBOUNDED PRECEDING) at the + * last row equals the grand total sum(v). Both sides are NULL on the empty + * table, so IS NOT DISTINCT FROM decides the empty case cleanly. + */ +PG_PROP(running_sum_last_equals_total) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT rs FROM " + "(SELECT id, sum(v) OVER (ORDER BY id " + "ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) rs " + "FROM agg_w) s ORDER BY id DESC LIMIT 1) " + "IS NOT DISTINCT FROM (SELECT sum(v) FROM agg_w)"); + + drop_table(conn); + pg_prop_disconnect(conn); +} + +/* + * count(*) OVER () with no partition or frame reports the full row count n on + * every row. bool_and folds that over all rows; coalesce handles the empty + * table. + */ +PG_PROP(count_over_equals_n) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(c = (SELECT count(*) FROM agg_w)), true) " + "FROM (SELECT count(*) OVER () c FROM agg_w) s"); + + drop_table(conn); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 5526d0a9ed45d..ff4e1ebc63d05 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 9337b5ab753747cd9667a0bc69f28bbd55f903c6 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:20 -0400 Subject: [PATCH 180/338] prop: add window frame and running-aggregate property tests (windowframes) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_windowframes.c | 271 ++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 275 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_windowframes.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 6aebcdcb6d748..315bb2a936869 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -195,7 +195,8 @@ OBJS = \ prop_jsonagg.o \ prop_jsonaggord.o \ prop_jsonobjectagg.o \ - prop_window.o + prop_window.o \ + prop_windowframes.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 2e208dce49629..a818db5d7a507 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -193,6 +193,7 @@ prop_area_sources = files( 'prop_jsonaggord.c', 'prop_jsonobjectagg.c', 'prop_window.c', + 'prop_windowframes.c', ) diff --git a/src/test/prop/prop_windowframes.c b/src/test/prop/prop_windowframes.c new file mode 100644 index 0000000000000..59657467c3316 --- /dev/null +++ b/src/test/prop/prop_windowframes.c @@ -0,0 +1,271 @@ +/*------------------------------------------------------------------------- + * + * prop_windowframes.c + * Property-based tests for SQL window FRAME semantics. + * + * Distinct from the "window" area (which checks structural invariants of the + * window functions themselves -- row_number/rank/dense_rank/lag/lead), this + * area targets the window FRAME: the ROWS/RANGE/GROUPS clause that selects + * which peer rows an aggregate sees. A running sum built with ROWS UNBOUNDED + * PRECEDING must end at the partition total; a CURRENT-ROW count must equal + * the row's position; ROWS and RANGE must agree on a whole-partition frame; + * first_value/last_value/nth_value over a fully-bounded ordered frame must + * line up with min/max. A failure means the frame boundary logic (the + * WinSetMarkPosition / aggregate restart machinery) selected the wrong rows. + * + * The oracle is the invariant itself, expressed in SQL and reduced to a + * single boolean: each per-row check is folded with bool_and(...) over a + * windowed subquery and wrapped in coalesce(..., true) so an empty table + * passes vacuously. v is non-NULL so frame arithmetic is exact. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_windowframes.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "windowframes" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Value domain for v. Kept small (-5..5) and NEVER NULL so frame sums stay + * exact and legible; |sum| <= 25 * 5 = 125 cannot overflow int. g in 0..2 + * gives a few small partitions, each with several rows, so the frame logic + * runs over partitions that actually have interior/edge rows to distinguish. + */ +#define V_MIN (-5) +#define V_MAX 5 +#define G_MAX 2 + +/* + * Build wf_t(id int PRIMARY KEY, g int, v int) with a drawn number of rows in + * [0, 25]. id is 1..n so ORDER BY id is a strict total order within every + * partition (frames and running aggregates are then fully deterministic); g + * is drawn in 0..G_MAX; v is drawn in V_MIN..V_MAX and never NULL. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS wf_t; " + "CREATE TABLE wf_t(id int PRIMARY KEY, g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 25); + for (int i = 1; i <= nrows; i++) + { + int g = (int) pg_prop_draw_int(tc, 0, G_MAX); + int v = (int) pg_prop_draw_int(tc, V_MIN, V_MAX); + + appendStringInfo(&ddl, "INSERT INTO wf_t VALUES (%d, %d, %d); ", + i, g, v); + } + appendStringInfoString(&ddl, "ANALYZE wf_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + pfree(ddl.data); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the window query produced NULL/errored where an invariant + * demands true -- itself a bug -- so either is a failure, reported with SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * The last running sum (ROWS UNBOUNDED PRECEDING TO CURRENT ROW), taken at the + * final row of each partition, equals that partition's total sum. Picks the + * last row via row_number() DESC = 1; if the frame boundary logic stops short + * of CURRENT ROW the running sum would miss the final addend. + */ +PG_PROP(running_sum_last_equals_total) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(rs = tot), true) FROM (" + "SELECT g, id, " + "sum(v) OVER (PARTITION BY g ORDER BY id " + "ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) rs, " + "sum(v) OVER (PARTITION BY g) tot, " + "row_number() OVER (PARTITION BY g ORDER BY id DESC) rn " + "FROM wf_t) s WHERE rn = 1"); + + pg_prop_disconnect(conn); +} + +/* + * A ROWS UNBOUNDED PRECEDING TO CURRENT ROW count equals the row's 1-based + * position in its partition. A frame that over- or under-counts the rows up + * to and including the current row breaks this directly. + */ +PG_PROP(count_frame_equals_position) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(cnt = rn), true) FROM (" + "SELECT count(*) OVER (PARTITION BY g ORDER BY id " + "ROWS UNBOUNDED PRECEDING) cnt, " + "row_number() OVER (PARTITION BY g ORDER BY id) rn " + "FROM wf_t) s"); + + pg_prop_disconnect(conn); +} + +/* + * On a BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING frame the whole + * partition is in frame regardless of ROWS vs RANGE, so the two framings must + * produce the identical aggregate on every row. A difference means one + * framing mis-identifies the frame bounds for the "whole partition" case. + */ +PG_PROP(range_vs_rows_full_frame) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(a = b), true) FROM (" + "SELECT sum(v) OVER (PARTITION BY g " + "ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) a, " + "sum(v) OVER (PARTITION BY g " + "RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) b " + "FROM wf_t) s"); + + pg_prop_disconnect(conn); +} + +/* + * first_value(v) over a whole-partition frame ordered by v is the partition + * minimum of v. The frame is fully bounded (UNBOUNDED PRECEDING..FOLLOWING) + * and ordered ascending, so the first row in frame carries the smallest v; + * compared to the frame-independent min(v) over the partition. + */ +PG_PROP(first_value_is_min_ordered) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(fv = mn), true) FROM (" + "SELECT first_value(v) OVER (PARTITION BY g ORDER BY v " + "ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) fv, " + "min(v) OVER (PARTITION BY g) mn " + "FROM wf_t) s"); + + pg_prop_disconnect(conn); +} + +/* + * last_value(v) over a whole-partition frame ordered by v is the partition + * maximum of v. This is the canonical trap: last_value defaults to a frame + * ending at CURRENT ROW, so only an explicit UNBOUNDED FOLLOWING frame makes + * the last-in-frame row the global max. Compared to min/max's partition max. + */ +PG_PROP(last_value_full_frame_is_max_ordered) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(lv = mx), true) FROM (" + "SELECT last_value(v) OVER (PARTITION BY g ORDER BY v " + "ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) lv, " + "max(v) OVER (PARTITION BY g) mx " + "FROM wf_t) s"); + + pg_prop_disconnect(conn); +} + +/* + * nth_value(v, 1) and first_value(v) over the identical frame must agree on + * every row: both name the first row of the frame. A sound frame identity -- + * it holds for any frame, bounds, and ordering -- so it isolates whether the + * two functions resolve the same frame-start row, independent of the data. + */ +PG_PROP(nth_value_1_is_first_value) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(nv IS NOT DISTINCT FROM fv), true) FROM (" + "SELECT nth_value(v, 1) OVER (PARTITION BY g ORDER BY id " + "ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) nv, " + "first_value(v) OVER (PARTITION BY g ORDER BY id " + "ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) fv " + "FROM wf_t) s"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index ff4e1ebc63d05..cbd16cbcd92c3 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 7a00dbd65e1cb2dc6c501f3a398f5848145b9dc7 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:20 -0400 Subject: [PATCH 181/338] prop: add window ranking function relationships property tests (windowrank) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_windowrank.c | 292 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 296 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_windowrank.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 315bb2a936869..8561a3f481124 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -196,7 +196,8 @@ OBJS = \ prop_jsonaggord.o \ prop_jsonobjectagg.o \ prop_window.o \ - prop_windowframes.o + prop_windowframes.o \ + prop_windowrank.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index a818db5d7a507..f84b3bf3baaf7 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -194,6 +194,7 @@ prop_area_sources = files( 'prop_jsonobjectagg.c', 'prop_window.c', 'prop_windowframes.c', + 'prop_windowrank.c', ) diff --git a/src/test/prop/prop_windowrank.c b/src/test/prop/prop_windowrank.c new file mode 100644 index 0000000000000..e19e73b11a1f5 --- /dev/null +++ b/src/test/prop/prop_windowrank.c @@ -0,0 +1,292 @@ +/*------------------------------------------------------------------------- + * + * prop_windowrank.c + * Property-based tests for SQL window RANKING functions. + * + * Distinct from the "window" area (structural invariants of the window + * functions in general) and "windowframes" (the ROWS/RANGE/GROUPS frame), + * this area targets the ranking family and the relationships between its + * members: rank/dense_rank, percent_rank, cume_dist, ntile, and lag/lead. + * dense_rank must never exceed rank; percent_rank and cume_dist must stay in + * [0,1] with cume_dist >= percent_rank; ntile(4) must stay in [1,4]; the + * minimum rank in a partition is 1; max(dense_rank) equals the count of + * distinct order values; and lead(v) is non-NULL on every row but the + * partition last. A failure means a ranking peer-group or position + * computation selected the wrong value. + * + * The oracle is the invariant itself, expressed in SQL and reduced to a + * single boolean: each per-row/per-partition check is folded with + * bool_and(...) over a windowed subquery and wrapped in coalesce(..., true) + * so an empty table passes vacuously. v is non-NULL so the peer grouping and + * ranking are exact. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_windowrank.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "windowrank" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Value domain for v. Kept small (-5..5) and NEVER NULL so the ORDER BY peer + * grouping is exact (ties occur often, exercising rank vs dense_rank). g in + * 0..2 gives a few small partitions each with several rows, so the ranking + * logic runs over partitions with real interior/edge/tie structure. + */ +#define V_MIN (-5) +#define V_MAX 5 +#define G_MAX 2 + +/* + * Build wr_t(id int PRIMARY KEY, g int, v int) with a drawn number of rows in + * [0, 25]. id is 1..n so ORDER BY ..., id is a strict total order within + * every partition (ntile and lead are then fully deterministic); g is drawn + * in 0..G_MAX; v is drawn in V_MIN..V_MAX and never NULL. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS wr_t; " + "CREATE TABLE wr_t(id int PRIMARY KEY, g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 25); + for (int i = 1; i <= nrows; i++) + { + int g = (int) pg_prop_draw_int(tc, 0, G_MAX); + int v = (int) pg_prop_draw_int(tc, V_MIN, V_MAX); + + appendStringInfo(&ddl, "INSERT INTO wr_t VALUES (%d, %d, %d); ", + i, g, v); + } + appendStringInfoString(&ddl, "ANALYZE wr_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + pfree(ddl.data); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the window query produced NULL/errored where an invariant + * demands true -- itself a bug -- so either is a failure, reported with SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * dense_rank() never exceeds rank() over the same partition/order. Ties + * advance rank by the group size but dense_rank by one, so dense_rank <= rank + * must hold on every row; a peer-group miscount would push dense_rank past + * rank. + */ +PG_PROP(rank_le_dense_rank_scaled) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(dr <= rk), true) FROM (" + "SELECT rank() OVER (PARTITION BY g ORDER BY v) rk, " + "dense_rank() OVER (PARTITION BY g ORDER BY v) dr " + "FROM wr_t) s"); + + pg_prop_disconnect(conn); +} + +/* + * percent_rank() always lies in the closed unit interval [0, 1]. It is + * (rank-1)/(nrows-1), which is 0 on the first peer group and 1 on the last; a + * position/row-count error would push it outside [0,1]. + */ +PG_PROP(percent_rank_unit_interval) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(pr BETWEEN 0 AND 1), true) FROM (" + "SELECT percent_rank() OVER (PARTITION BY g ORDER BY v) pr " + "FROM wr_t) s"); + + pg_prop_disconnect(conn); +} + +/* + * cume_dist() lies in (0, 1] and is always >= percent_rank() over the same + * window. cume_dist counts rows up to and including the current peer group + * (so it reaches 1 at the top), while percent_rank counts rows strictly + * before it; a peer-boundary error would break the ordering between them or + * push cume_dist outside [0,1]. + */ +PG_PROP(cume_dist_unit_interval) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(cd BETWEEN 0 AND 1 AND cd >= pr), true) FROM (" + "SELECT cume_dist() OVER (PARTITION BY g ORDER BY v) cd, " + "percent_rank() OVER (PARTITION BY g ORDER BY v) pr " + "FROM wr_t) s"); + + pg_prop_disconnect(conn); +} + +/* + * ntile(4) over a strict total order (v, id) assigns every row a bucket in + * [1, 4]. A bucket-boundary arithmetic error (the classic remainder/division + * off-by-one in the ntile bucket sizing) would produce a 0 or a 5. + */ +PG_PROP(ntile_bounded) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(nt BETWEEN 1 AND 4), true) FROM (" + "SELECT ntile(4) OVER (PARTITION BY g ORDER BY v, id) nt " + "FROM wr_t) s"); + + pg_prop_disconnect(conn); +} + +/* + * The minimum rank() in every partition is 1. Regardless of ties, the first + * peer group always ranks 1; so the per-partition minimum of rank() must be + * exactly 1. A rank that failed to reset at a partition boundary would show a + * minimum other than 1. + */ +PG_PROP(rank_first_is_one) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(mr = 1), true) FROM (" + "SELECT g, min(rk) mr FROM (" + "SELECT g, rank() OVER (PARTITION BY g ORDER BY v) rk " + "FROM wr_t) a GROUP BY g) b"); + + pg_prop_disconnect(conn); +} + +/* + * The maximum dense_rank() in a partition equals the number of distinct order + * values (v) in that partition. dense_rank assigns consecutive integers + * 1..k to the k distinct peer groups, so its partition maximum is exactly k = + * count(distinct v). Computed on two independent paths (windowed dense_rank + * vs plain count(distinct)) and joined on g, this is a differential check: a + * peer-group miscount in dense_rank breaks the equality. + */ +PG_PROP(dense_rank_counts_distinct) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(mdr = dc), true) FROM (" + "SELECT g, max(dr) mdr FROM (" + "SELECT g, dense_rank() OVER (PARTITION BY g ORDER BY v) dr " + "FROM wr_t) a GROUP BY g) m " + "JOIN (SELECT g, count(DISTINCT v) dc FROM wr_t GROUP BY g) d " + "USING (g)"); + + pg_prop_disconnect(conn); +} + +/* + * lead(v) OVER (PARTITION BY g ORDER BY id) is non-NULL on exactly the rows + * that are not the partition last; v itself is never NULL, so the count of + * non-NULL lead values in each partition equals that partition's size minus + * one. A lead that reads the wrong offset, or fails to NULL the partition + * boundary, breaks this count. + */ +PG_PROP(lag_lead_shift) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(nn = sz - 1), true) FROM (" + "SELECT g, count(ld) nn, count(*) sz FROM (" + "SELECT g, lead(v) OVER (PARTITION BY g ORDER BY id) ld " + "FROM wr_t) a GROUP BY g) b"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index cbd16cbcd92c3..122257ef8f98c 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 2a3279ed58c037bd8991b39b9be1e302187abe28 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:21 -0400 Subject: [PATCH 182/338] prop: add window frame EXCLUDE and GROUPS mode property tests (windowexcl) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_windowexcl.c | 280 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 284 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_windowexcl.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 8561a3f481124..9c90ed1648ac8 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -197,7 +197,8 @@ OBJS = \ prop_jsonobjectagg.o \ prop_window.o \ prop_windowframes.o \ - prop_windowrank.o + prop_windowrank.o \ + prop_windowexcl.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index f84b3bf3baaf7..db1588ee6b368 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -195,6 +195,7 @@ prop_area_sources = files( 'prop_window.c', 'prop_windowframes.c', 'prop_windowrank.c', + 'prop_windowexcl.c', ) diff --git a/src/test/prop/prop_windowexcl.c b/src/test/prop/prop_windowexcl.c new file mode 100644 index 0000000000000..221c4015b496c --- /dev/null +++ b/src/test/prop/prop_windowexcl.c @@ -0,0 +1,280 @@ +/*------------------------------------------------------------------------- + * + * prop_windowexcl.c + * Property-based tests for window frame EXCLUDE clauses and GROUPS mode. + * + * Distinct from the window/windowframes/windowrank areas, this one targets + * the frame EXCLUSION options (EXCLUDE CURRENT ROW / GROUP / TIES / NO OTHERS) + * and the GROUPS framing mode, where a frame counts peer groups rather than + * rows or value ranges. EXCLUDE CURRENT ROW over a full partition must drop + * exactly this row's value; EXCLUDE NO OTHERS must match omitting the clause; + * a GROUPS running count must be monotone and end at the partition size; + * EXCLUDE GROUP must remove the whole peer group; EXCLUDE TIES can only remove + * rows versus NO OTHERS. A failure means the exclusion bookkeeping (the + * frameheadpos/frametailpos plus the peer-group exclusion in nodeWindowAgg) + * selected the wrong rows. + * + * The oracle is the invariant itself, expressed in SQL and reduced to a single + * boolean: each per-row check is folded with bool_and(...) over a windowed + * subquery and wrapped in coalesce(..., true) so an empty table passes + * vacuously. v is non-NULL and small (0..3) so peer groups actually form. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_windowexcl.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "windowexcl" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Value domain for v. Kept tiny (0..3) and NEVER NULL so many rows share a v + * within a partition and GROUPS peer groups actually form; g in 0..2 gives a + * few small partitions. v non-NULL keeps the EXCLUDE arithmetic exact. + */ +#define V_MAX 3 +#define G_MAX 2 + +/* + * Build we_t(id int PRIMARY KEY, g int, v int) with a drawn number of rows in + * [0, 25]. id is 1..n so ORDER BY id is a strict total order within every + * partition; g is drawn in 0..G_MAX; v is drawn in 0..V_MAX, never NULL. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS we_t; " + "CREATE TABLE we_t(id int PRIMARY KEY, g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 25); + for (int i = 1; i <= nrows; i++) + { + int g = (int) pg_prop_draw_int(tc, 0, G_MAX); + int v = (int) pg_prop_draw_int(tc, 0, V_MAX); + + appendStringInfo(&ddl, "INSERT INTO we_t VALUES (%d, %d, %d); ", + i, g, v); + } + appendStringInfoString(&ddl, "ANALYZE we_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + pfree(ddl.data); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the window query produced NULL/errored where an invariant + * demands true -- itself a bug -- so either is a failure, reported with SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * EXCLUDE CURRENT ROW over a whole-partition frame drops exactly this row's + * own value: sum(v) EXCLUDE CURRENT ROW = sum(v) over the full frame minus v. + * The whole-partition frame makes the excluded set exactly {current row}, so + * the two sums must differ by precisely v on every row. + */ +PG_PROP(exclude_current_row) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(sum_excl = sum_all - v), true) FROM (" + "SELECT v, " + "sum(v) OVER (PARTITION BY g " + "ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING " + "EXCLUDE CURRENT ROW) sum_excl, " + "sum(v) OVER (PARTITION BY g " + "ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) sum_all " + "FROM we_t) s"); + + pg_prop_disconnect(conn); +} + +/* + * The default EXCLUDE NO OTHERS excludes nothing, so a frame carrying it must + * produce the identical aggregate as the same frame with the clause omitted. + * A difference means the parser/executor treats the default option as doing + * something, which it must not. + */ +PG_PROP(exclude_no_others_default) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(a = b), true) FROM (" + "SELECT sum(v) OVER (PARTITION BY g ORDER BY id " + "ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) a, " + "sum(v) OVER (PARTITION BY g ORDER BY id " + "ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING " + "EXCLUDE NO OTHERS) b " + "FROM we_t) s"); + + pg_prop_disconnect(conn); +} + +/* + * A GROUPS running count (UNBOUNDED PRECEDING TO CURRENT ROW, ordered by v) + * counts whole peer groups up to and including this row's group, so taken at + * the last peer group of a partition it equals the partition size. The last + * group is where row_number() DESC within the partition is 1; its running + * count must be the full partition row count. + */ +PG_PROP(groups_mode_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(cnt = psize), true) FROM (" + "SELECT count(*) OVER (PARTITION BY g ORDER BY v " + "GROUPS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) cnt, " + "count(*) OVER (PARTITION BY g) psize, " + "row_number() OVER (PARTITION BY g ORDER BY v DESC) rn " + "FROM we_t) s WHERE rn = 1"); + + pg_prop_disconnect(conn); +} + +/* + * With GROUPS over the whole partition and EXCLUDE GROUP, the current row's + * entire peer group is removed, so the frame count equals partition size minus + * the size of this row's peer group. Peer-group size is computed independently + * as count(*) OVER (PARTITION BY g ORDER BY v RANGE CURRENT ROW), i.e. the set + * of rows that are peers of this one under the v ordering. + */ +PG_PROP(exclude_group) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(cnt = psize - peers), true) FROM (" + "SELECT count(*) OVER (PARTITION BY g ORDER BY v " + "GROUPS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING " + "EXCLUDE GROUP) cnt, " + "count(*) OVER (PARTITION BY g) psize, " + "count(*) OVER (PARTITION BY g ORDER BY v " + "RANGE BETWEEN CURRENT ROW AND CURRENT ROW) peers " + "FROM we_t) s"); + + pg_prop_disconnect(conn); +} + +/* + * EXCLUDE TIES removes the current row's peers (but keeps the current row), so + * on any frame its count can only be less than or equal to the same frame with + * EXCLUDE NO OTHERS -- excluding rows never adds any. A strict inequality in + * the other direction would mean exclusion invented rows. + */ +PG_PROP(exclude_ties_le) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(cnt_ties <= cnt_none), true) FROM (" + "SELECT count(*) OVER (PARTITION BY g ORDER BY v " + "GROUPS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING " + "EXCLUDE TIES) cnt_ties, " + "count(*) OVER (PARTITION BY g ORDER BY v " + "GROUPS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING " + "EXCLUDE NO OTHERS) cnt_none " + "FROM we_t) s"); + + pg_prop_disconnect(conn); +} + +/* + * A ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING frame is the whole + * partition, so sum(v) over it equals the frame-independent partition total + * sum(v) OVER (PARTITION BY g) on every row. A difference means the full-frame + * bounds mis-identify the partition. + */ +PG_PROP(full_frame_sum_is_partition_total) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(a = b), true) FROM (" + "SELECT sum(v) OVER (PARTITION BY g " + "ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) a, " + "sum(v) OVER (PARTITION BY g) b " + "FROM we_t) s"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 122257ef8f98c..42ac966acb4c2 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 1cf981768eee7a869f192b3d4e04adc3e22e6d6c Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:21 -0400 Subject: [PATCH 183/338] prop: add window aggregate vs GROUP BY equivalence property tests (windowagg) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_windowagg.c | 295 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 299 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_windowagg.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 9c90ed1648ac8..7ddac381c56ea 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -198,7 +198,8 @@ OBJS = \ prop_window.o \ prop_windowframes.o \ prop_windowrank.o \ - prop_windowexcl.o + prop_windowexcl.o \ + prop_windowagg.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index db1588ee6b368..68fe78d11ccb8 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -196,6 +196,7 @@ prop_area_sources = files( 'prop_windowframes.c', 'prop_windowrank.c', 'prop_windowexcl.c', + 'prop_windowagg.c', ) diff --git a/src/test/prop/prop_windowagg.c b/src/test/prop/prop_windowagg.c new file mode 100644 index 0000000000000..d84fa0a8f5d1d --- /dev/null +++ b/src/test/prop/prop_windowagg.c @@ -0,0 +1,295 @@ +/*------------------------------------------------------------------------- + * + * prop_windowagg.c + * Property-based tests for window aggregates vs GROUP BY aggregates. + * + * A window aggregate with PARTITION BY g must compute, on every row of a + * group, the same value a plain GROUP BY of that aggregate computes for the + * group. The two paths run entirely different executor nodes (WindowAgg vs + * grouped Agg/HashAgg), so a divergence is a wrong-results bug in one of + * them. This area is deliberately distinct from the window/windowframes/ + * windowrank/windowexcl areas: it tests only the aggregate-equals-group + * equivalence, not frame mechanics or ranking. + * + * Tier B: runs SQL against the cluster the TAP driver started; see pg_prop.h + * and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_windowagg.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "windowagg" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error is itself a failure (the oracle expression is written to be + * total: coalesce()/bool_and over an empty set yields "t"). + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Compare two result sets for exact row-major equality. Both queries MUST + * carry a total ORDER BY so the string compare is a sound multiset compare. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Build wa_t(id int PRIMARY KEY, g int, v int) with 0..25 rows, g in 0..3, + * v in -5..5 and never NULL. v is kept non-NULL so sum/avg/max/min are + * exact and the window-vs-group equivalence has no NULL-handling asymmetry. + * ANALYZE so the planner can pick its real plan for each path. Rebuilt fresh + * in every property. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS wa_t; " + "CREATE TABLE wa_t(id int PRIMARY KEY, g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 25); + for (int i = 0; i < nrows; i++) + appendStringInfo(&ddl, + "INSERT INTO wa_t VALUES (%d, %d, %d); ", + i, + (int) pg_prop_draw_int(tc, 0, 3), + (int) pg_prop_draw_int(tc, -5, 5)); + + appendStringInfoString(&ddl, "ANALYZE wa_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build failed: %s", err ? err : ""); + pfree(ddl.data); +} + +/* + * 1. sum(v) OVER (PARTITION BY g) equals the group's GROUP BY sum, on every + * row. Join the per-row window result to the grouped sum on g and require + * they agree everywhere; coalesce makes the empty table pass. + */ +PG_PROP(window_sum_eq_group_sum) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(ws = gs), true) FROM " + "(SELECT g, sum(v) OVER (PARTITION BY g) ws FROM wa_t) w " + "JOIN (SELECT g, sum(v) gs FROM wa_t GROUP BY g) a " + "USING (g)"); + + pg_prop_disconnect(conn); +} + +/* + * 2. count(*) OVER (PARTITION BY g) equals the group's GROUP BY count, on + * every row. + */ +PG_PROP(window_count_eq_group_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(wc = gc), true) FROM " + "(SELECT g, count(*) OVER (PARTITION BY g) wc FROM wa_t) w " + "JOIN (SELECT g, count(*) gc FROM wa_t GROUP BY g) a " + "USING (g)"); + + pg_prop_disconnect(conn); +} + +/* + * 3. avg(v) OVER (PARTITION BY g) equals the group's GROUP BY avg. avg over + * integers yields numeric and both paths average the identical multiset, + * so the result is exact; IS NOT DISTINCT FROM compares them total (no NULL + * special-casing needed, but it keeps the oracle sound at any boundary). + */ +PG_PROP(window_avg_eq_group_avg) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(wa IS NOT DISTINCT FROM ga), true) FROM " + "(SELECT g, avg(v) OVER (PARTITION BY g) wa FROM wa_t) w " + "JOIN (SELECT g, avg(v) ga FROM wa_t GROUP BY g) a " + "USING (g)"); + + pg_prop_disconnect(conn); +} + +/* + * 4. max(v) and min(v) OVER (PARTITION BY g) equal the group's GROUP BY max + * and min, on every row. + */ +PG_PROP(window_max_min) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(wmax = gmax AND wmin = gmin), true) FROM " + "(SELECT g, max(v) OVER (PARTITION BY g) wmax, " + "min(v) OVER (PARTITION BY g) wmin FROM wa_t) w " + "JOIN (SELECT g, max(v) gmax, min(v) gmin FROM wa_t GROUP BY g) a " + "USING (g)"); + + pg_prop_disconnect(conn); +} + +/* + * 5. sum(v) OVER () with no partition equals the grand total sum on every + * row. coalesce guards the empty table (both the per-row set and the + * scalar subquery are then NULL/empty). + */ +PG_PROP(unpartitioned_window_eq_total) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(ws = (SELECT sum(v) FROM wa_t)), true) " + "FROM (SELECT sum(v) OVER () ws FROM wa_t) w"); + + pg_prop_disconnect(conn); +} + +/* + * 6. A running row count within a group never exceeds the group's total row + * count: count(*) OVER (PARTITION BY g ORDER BY id ROWS UNBOUNDED PRECEDING) + * <= count(*) OVER (PARTITION BY g), on every row. (A running sum would + * not be monotone because v can be negative, so count is used instead.) + */ +PG_PROP(running_count_le_total) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(rc <= tc), true) FROM " + "(SELECT count(*) OVER (PARTITION BY g ORDER BY id " + "ROWS UNBOUNDED PRECEDING) rc, " + "count(*) OVER (PARTITION BY g) tc FROM wa_t) w"); + + pg_prop_disconnect(conn); +} + +/* + * 7. The distinct (g, window-sum) pairs equal the grouped (g, sum) rows. A + * window sum partitioned by g produces one value per group repeated across + * its rows, so DISTINCT collapses it to exactly the GROUP BY result. Both + * sides carry a total ORDER BY g (g is unique after the grouping/distinct), + * so this is a sound differential oracle. + */ +PG_PROP(distinct_in_window_vs_group) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT DISTINCT g, sum(v) OVER (PARTITION BY g) " + "FROM wa_t ORDER BY g", + "SELECT g, sum(v) FROM wa_t GROUP BY g ORDER BY g"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 42ac966acb4c2..11328354fd446 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 8edd081301da3f4796e83d79be02f155af46e335 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:21 -0400 Subject: [PATCH 184/338] prop: add window navigation lag/lead/first/last/nth_value property tests (windownav) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_windownav.c | 301 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 305 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_windownav.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 7ddac381c56ea..ff7c28e2565aa 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -199,7 +199,8 @@ OBJS = \ prop_windowframes.o \ prop_windowrank.o \ prop_windowexcl.o \ - prop_windowagg.o + prop_windowagg.o \ + prop_windownav.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 68fe78d11ccb8..7475aa78d08f0 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -197,6 +197,7 @@ prop_area_sources = files( 'prop_windowrank.c', 'prop_windowexcl.c', 'prop_windowagg.c', + 'prop_windownav.c', ) diff --git a/src/test/prop/prop_windownav.c b/src/test/prop/prop_windownav.c new file mode 100644 index 0000000000000..6776f487f2b3e --- /dev/null +++ b/src/test/prop/prop_windownav.c @@ -0,0 +1,301 @@ +/*------------------------------------------------------------------------- + * + * prop_windownav.c + * Property-based tests for SQL window NAVIGATION functions. + * + * Distinct from "window" (general structural invariants), "windowframes" + * (ROWS/RANGE/GROUPS frame), "windowrank" (ranking family), "windowexcl" + * (frame EXCLUDE) and "windowagg" (aggregates as window functions), this + * area targets the navigation family: lag/lead and the positional readers + * first_value/last_value/nth_value, plus row_number as the position oracle. + * lag's default fires on the first row of a partition; lead/lag have exactly + * (size-1) non-NULL results per partition; first_value/last_value over a full + * frame are constant per partition and equal the min-id/max-id row's v; + * nth_value(,1) equals first_value; and row_number is a dense 1..size bijection. + * A failure means a navigation/position computation read the wrong row. + * + * The oracle is the invariant itself, expressed in SQL and reduced to a + * single boolean: each per-row/per-partition check is folded with bool_and() + * over a windowed subquery and wrapped in coalesce(..., true) so an empty + * table passes vacuously. v is non-NULL so lead/lag NULLs mark only partition + * edges and positional reads are exact. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_windownav.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "windownav" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Value domain for v. Kept small (0..9) and NEVER NULL so lead/lag NULLs mark + * only the partition edges and the positional reads (first/last/nth_value) are + * exact. g in 0..2 gives a few small partitions each with several rows, so the + * navigation logic runs over partitions with real interior/edge structure. + */ +#define V_MIN 0 +#define V_MAX 9 +#define G_MAX 2 + +/* + * Build wn_t(id int PRIMARY KEY, g int, v int) with a drawn number of rows in + * [0, 25]. id is 1..n so ORDER BY id is a strict total order within every + * partition (lead/lag and the full-frame positional reads are then fully + * deterministic); g is drawn in 0..G_MAX; v is drawn in V_MIN..V_MAX and never + * NULL. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS wn_t; " + "CREATE TABLE wn_t(id int PRIMARY KEY, g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 25); + for (int i = 1; i <= nrows; i++) + { + int g = (int) pg_prop_draw_int(tc, 0, G_MAX); + int v = (int) pg_prop_draw_int(tc, V_MIN, V_MAX); + + appendStringInfo(&ddl, "INSERT INTO wn_t VALUES (%d, %d, %d); ", + i, g, v); + } + appendStringInfoString(&ddl, "ANALYZE wn_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + pfree(ddl.data); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the window query produced NULL/errored where an invariant + * demands true -- itself a bug -- so either is a failure, reported with SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * lag(v, 1, -1) OVER (PARTITION BY g ORDER BY id) returns the default (-1) on + * the first row of each partition and the previous row's v elsewhere. So on + * every row whose row_number() is 1, the lagged value must equal exactly the + * default -1. A default-handling or offset error on the partition's leading + * edge would make the first row's lag something other than -1. + */ +PG_PROP(lag_default) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(CASE WHEN rn = 1 THEN lg = -1 ELSE true END), true) FROM (" + "SELECT lag(v, 1, -1) OVER w lg, row_number() OVER w rn " + "FROM (SELECT g, id, v FROM wn_t) t " + "WINDOW w AS (PARTITION BY g ORDER BY id)) s"); + + pg_prop_disconnect(conn); +} + +/* + * lead(v) reads one row forward and so is non-NULL on every row of a partition + * except the last. Over a non-NULL v, the count of non-NULL lead values in a + * partition must equal (partition size - 1). A forward-offset or + * partition-boundary error would leave the wrong number of non-NULL leads. + */ +PG_PROP(lead_lag_symmetry) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(nleads = sz - 1), true) FROM (" + "SELECT count(ld) nleads, count(*) sz FROM (" + "SELECT g, lead(v) OVER (PARTITION BY g ORDER BY id) ld " + "FROM wn_t) t GROUP BY g) s"); + + pg_prop_disconnect(conn); +} + +/* + * first_value(v) over a full frame (UNBOUNDED PRECEDING .. UNBOUNDED + * FOLLOWING) is constant across a partition and equals the v of the row with + * the minimum id in that partition. Comparing the per-row first_value with + * the min-by-id v (computed independently via a correlated aggregate) catches + * a frame-start or positional-read error. + */ +PG_PROP(first_value_constant) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(fv = minv), true) FROM (" + "SELECT first_value(v) OVER w fv, " + "(SELECT v FROM wn_t x WHERE x.g = t.g ORDER BY x.id LIMIT 1) minv " + "FROM (SELECT g, id, v FROM wn_t) t " + "WINDOW w AS (PARTITION BY g ORDER BY id " + "ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)) s"); + + pg_prop_disconnect(conn); +} + +/* + * nth_value(v, 1) over a full frame equals first_value(v) over the same frame + * on every row: both name the first row of the frame. This is a differential + * identity between two navigation functions, sound by construction; a + * one-based-offset bug in nth_value would break it. + */ +PG_PROP(nth_value_1_eq_first) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(nv = fv), true) FROM (" + "SELECT nth_value(v, 1) OVER w nv, first_value(v) OVER w fv " + "FROM (SELECT g, id, v FROM wn_t) t " + "WINDOW w AS (PARTITION BY g ORDER BY id " + "ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)) s"); + + pg_prop_disconnect(conn); +} + +/* + * last_value(v) over a full frame (UNBOUNDED PRECEDING .. UNBOUNDED FOLLOWING) + * is constant across a partition and equals the v of the row with the maximum + * id in that partition. Without the explicit full frame last_value tracks the + * default running frame and reads the current row; the full frame is what + * makes it the partition's last. A frame-end error would read the wrong row. + */ +PG_PROP(last_value_full_frame) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(lv = maxv), true) FROM (" + "SELECT last_value(v) OVER w lv, " + "(SELECT v FROM wn_t x WHERE x.g = t.g ORDER BY x.id DESC LIMIT 1) maxv " + "FROM (SELECT g, id, v FROM wn_t) t " + "WINDOW w AS (PARTITION BY g ORDER BY id " + "ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)) s"); + + pg_prop_disconnect(conn); +} + +/* + * lag(v) reads one row back and so is non-NULL on every row of a partition + * except the first. Over a non-NULL v, the count of non-NULL lag values in a + * partition must equal (partition size - 1) -- the mirror of lead's count. A + * backward-offset or partition-boundary error would leave the wrong count. + */ +PG_PROP(lag_lead_inverse_interior) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(nlags = sz - 1), true) FROM (" + "SELECT count(lg) nlags, count(*) sz FROM (" + "SELECT g, lag(v) OVER (PARTITION BY g ORDER BY id) lg " + "FROM wn_t) t GROUP BY g) s"); + + pg_prop_disconnect(conn); +} + +/* + * row_number() OVER (PARTITION BY g ORDER BY id) is a dense bijection onto + * 1..size in every partition: its maximum equals the partition size and it + * takes that many distinct values. Both max(rn)=size and + * count(distinct rn)=size must hold per partition. A gap, a duplicate, or a + * failure to reset at a partition boundary would break one of them. + */ +PG_PROP(row_number_dense) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(mrn = sz AND drn = sz), true) FROM (" + "SELECT max(rn) mrn, count(DISTINCT rn) drn, sz FROM (" + "SELECT g, row_number() OVER (PARTITION BY g ORDER BY id) rn, " + "count(*) OVER (PARTITION BY g) sz " + "FROM wn_t) t GROUP BY g, sz) u"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 11328354fd446..a1165d49e252d 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From ac10cf0654c662c93dff521ad8287048297e0092 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:21 -0400 Subject: [PATCH 185/338] prop: add window running/filter aggregate bounds property tests (windowdistinct) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_windowdistinct.c | 328 ++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 332 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_windowdistinct.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index ff7c28e2565aa..7e8e4c479a860 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -200,7 +200,8 @@ OBJS = \ prop_windowrank.o \ prop_windowexcl.o \ prop_windowagg.o \ - prop_windownav.o + prop_windownav.o \ + prop_windowdistinct.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 7475aa78d08f0..9ab9077527b3f 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -198,6 +198,7 @@ prop_area_sources = files( 'prop_windowexcl.c', 'prop_windowagg.c', 'prop_windownav.c', + 'prop_windowdistinct.c', ) diff --git a/src/test/prop/prop_windowdistinct.c b/src/test/prop/prop_windowdistinct.c new file mode 100644 index 0000000000000..233af6fda998c --- /dev/null +++ b/src/test/prop/prop_windowdistinct.c @@ -0,0 +1,328 @@ +/*------------------------------------------------------------------------- + * + * prop_windowdistinct.c + * Property-based tests for window-aggregate correctness. + * + * These properties exercise windowed aggregates (running, partitioned, and + * framed) and FILTER on window functions against invariants that must hold + * for every row of every input: a running COUNT equals the row position, a + * running SUM reaches the partition total at the last row, per-partition + * min/max/avg bound each value, a FILTERed count never exceeds the full + * count, and a 2-row frame sum never exceeds the whole-partition sum when all + * values are non-negative. A window-machinery bug (wrong frame edges, a + * mishandled partition boundary, a dropped FILTER) breaks one of these, so an + * "f" is a correctness bug, not a rounding artefact. + * + * The oracle shape (A): each property wraps a per-row window check in + * bool_and(...) over a subquery and coalesces to true on empty input, so the + * whole invariant collapses to a single scalar boolean that must be "t". + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_windowdistinct.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "windowdistinct" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the window machinery errored or produced NULL where the + * invariant demands a definite answer, which is itself a bug -- so either is a + * failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build the table under test fresh for each case: wd_t(id, g, v) with 0..25 + * rows, a unique PRIMARY KEY id, a group g in 0..2 and a value v in 0..9. No + * NULLs: every property below assumes v is non-NULL and v >= 0. ANALYZE so + * the planner has stats (the window path is the same either way, but this + * matches the real-use shape and keeps plans realistic). + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS wd_t; " + "CREATE TABLE wd_t (id int PRIMARY KEY, g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 25); + for (int i = 0; i < nrows; i++) + { + int g = (int) pg_prop_draw_int(tc, 0, 2); + int v = (int) pg_prop_draw_int(tc, 0, 9); + + appendStringInfo(&ddl, + "INSERT INTO wd_t VALUES (%d, %d, %d); ", + i, g, v); + } + appendStringInfoString(&ddl, "ANALYZE wd_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n SQL: %s", + err ? err : "(unknown)", ddl.data); + + pfree(ddl.data); +} + +/* + * running_count_increases: a running COUNT(*) over ROWS UNBOUNDED PRECEDING, + * taken in id order within each partition, equals the 1-based row position in + * that partition. The oracle is the same ordinal computed by row_number(): + * cnt = rn for every row. Catches a frame or partition-boundary bug where + * the running count fails to reset or miscounts at the first row. + */ +PG_PROP(running_count_increases) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT coalesce(bool_and(cnt = rn), true) FROM (" + " SELECT count(*) OVER (PARTITION BY g ORDER BY id " + " ROWS UNBOUNDED PRECEDING) AS cnt, " + " row_number() OVER (PARTITION BY g ORDER BY id) AS rn " + " FROM wd_t) s"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * running_sum_last_eq_total: within each partition the running SUM(v) taken + * in id order reaches the partition total exactly at the last row. The check + * compares, at the row where the running row-number equals the partition's + * row count, the running sum against the whole-partition sum. A frame bug + * that drops or double-counts the final row breaks this. + */ +PG_PROP(running_sum_last_eq_total) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT coalesce(bool_and(rn <> cnt OR rsum = total), true) FROM (" + " SELECT sum(v) OVER (PARTITION BY g ORDER BY id " + " ROWS UNBOUNDED PRECEDING) AS rsum, " + " sum(v) OVER (PARTITION BY g) AS total, " + " row_number() OVER (PARTITION BY g ORDER BY id) AS rn, " + " count(*) OVER (PARTITION BY g) AS cnt " + " FROM wd_t) s"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * window_min_le_value: the per-partition MIN(v) is <= v for every row in that + * partition. A trivial but load-bearing invariant for the partition-wide + * min: a boundary bug that leaks another partition's rows into the frame + * would push the min below (still passes) or above (fails) some value. + */ +PG_PROP(window_min_le_value) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT coalesce(bool_and(mn <= v), true) FROM (" + " SELECT v, min(v) OVER (PARTITION BY g) AS mn " + " FROM wd_t) s"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * window_max_ge_value: the per-partition MAX(v) is >= v for every row in that + * partition. The mirror of window_min_le_value; together they bracket every + * value by its partition's extremes. + */ +PG_PROP(window_max_ge_value) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT coalesce(bool_and(mx >= v), true) FROM (" + " SELECT v, max(v) OVER (PARTITION BY g) AS mx " + " FROM wd_t) s"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * avg_between_minmax: the per-partition AVG(v) lies between the per-partition + * MIN(v) and MAX(v). The three aggregates share a partition frame, so this + * cross-checks that avg is computed over the same row set as min/max; a frame + * discrepancy among them breaks the BETWEEN. + */ +PG_PROP(avg_between_minmax) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT coalesce(bool_and(av BETWEEN mn AND mx), true) FROM (" + "SELECT avg(v) OVER w av, min(v) OVER w mn, max(v) OVER w mx " + "FROM wd_t " + "WINDOW w AS (PARTITION BY g)) s"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * filter_window: a FILTERed window count never exceeds the unfiltered window + * count over the same partition. count(*) FILTER (WHERE v > 5) counts a + * subset of the rows count(*) counts, so the filtered value is always <= the + * full value. Catches a FILTER-on-window bug that lets excluded rows through. + */ +PG_PROP(filter_window) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT coalesce(bool_and(fcnt <= tcnt), true) FROM (" + " SELECT count(*) FILTER (WHERE v > 5) " + " OVER (PARTITION BY g) AS fcnt, " + " count(*) OVER (PARTITION BY g) AS tcnt " + " FROM wd_t) s"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * frame_sum_le_total: a 2-row trailing frame sum (1 PRECEDING .. CURRENT ROW) + * never exceeds the whole-partition sum, given all v >= 0. With non-negative + * values a sub-frame's sum cannot exceed the full-partition sum, so this is a + * sound bound; a frame-edge bug that over-counts rows into the trailing frame + * breaks it. + */ +PG_PROP(frame_sum_le_total) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT coalesce(bool_and(fsum <= total), true) FROM (" + " SELECT sum(v) OVER (PARTITION BY g ORDER BY id " + " ROWS BETWEEN 1 PRECEDING AND CURRENT ROW) AS fsum, " + " sum(v) OVER (PARTITION BY g) AS total " + " FROM wd_t) s"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index a1165d49e252d..ff8a3ed7f5e5a 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From c43db9e98a749c66d3894177fdd5a8f8d2914261 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:21 -0400 Subject: [PATCH 186/338] prop: add HAVING, GROUP BY expression, and agg equivalence property tests (grouphaving) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_grouphaving.c | 290 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 294 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_grouphaving.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 7e8e4c479a860..7225d38c5ca11 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -201,7 +201,8 @@ OBJS = \ prop_windowexcl.o \ prop_windowagg.o \ prop_windownav.o \ - prop_windowdistinct.o + prop_windowdistinct.o \ + prop_grouphaving.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 9ab9077527b3f..15fe8b505ec14 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -199,6 +199,7 @@ prop_area_sources = files( 'prop_windowagg.c', 'prop_windownav.c', 'prop_windowdistinct.c', + 'prop_grouphaving.c', ) diff --git a/src/test/prop/prop_grouphaving.c b/src/test/prop/prop_grouphaving.c new file mode 100644 index 0000000000000..396f191c6e8a8 --- /dev/null +++ b/src/test/prop/prop_grouphaving.c @@ -0,0 +1,290 @@ +/*------------------------------------------------------------------------- + * + * prop_grouphaving.c + * Property-based tests for GROUP BY / HAVING grouped-aggregate rewrites. + * + * The oracle is differential: each property runs two queries that must return + * the same rows, where one is the plain grouped form and the other an + * equivalent rewrite (HAVING vs an outer WHERE on a grouped subquery, a + * grouping expression vs a subquery that pre-computes it, per-group aggregate + * vs a correlated-subquery form, HAVING true vs no HAVING, DISTINCT vs + * GROUP BY with no aggregate). A grouped answer must not depend on which + * equivalent form the planner is handed, so any disagreement is a bug in + * grouping, HAVING placement, or aggregate pushdown. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_grouphaving.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "grouphaving" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small domains keep each generated table legible in a counterexample and + * keep cases fast while still covering empty/one-group/many-group shapes, a + * low-cardinality grouping key (g in 0..4), signed values spanning zero, and + * ~15% NULLs so NULL-grouping and NULL-aggregate behaviour is exercised. + */ +#define G_MIN 0 +#define G_MAX 4 +#define V_MIN (-5) +#define V_MAX 5 + +/* Draw v: SQL NULL ~15% of the time, else an int in [V_MIN, V_MAX]. */ +static void +draw_v(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "(%d)", (int) pg_prop_draw_int(tc, V_MIN, V_MAX)); +} + +/* + * Build gh_t(id int PRIMARY KEY, g int, v int) with 0..30 random rows. g is + * never NULL (0..4); v is NULL ~15% of the time. Rebuilt fresh in each + * property so cases are independent. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS gh_t; " + "CREATE TABLE gh_t (id int PRIMARY KEY, g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 30); + for (int r = 0; r < nrows; r++) + { + StringInfoData v; + + initStringInfo(&v); + draw_v(tc, &v); + appendStringInfo(&ddl, "INSERT INTO gh_t VALUES (%d, %d, %s); ", + r, (int) pg_prop_draw_int(tc, G_MIN, G_MAX), v.data); + pfree(v.data); + } + appendStringInfoString(&ddl, "ANALYZE gh_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + pfree(ddl.data); +} + +/* + * Differential result-set oracle: q1 and q2 must return identical rows in + * identical order. Both callers supply a TOTAL ORDER BY so the row-major + * string compare behaves as a multiset compare. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Scalar boolean oracle (shape A): a single SELECT must return "t". A NULL + * result or SQL error is a failure. Used for the empty-input-sensitive + * property where the two row-shapes legitimately differ on an empty table. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * HAVING on a grouped result equals an outer WHERE applied to the same + * grouping wrapped in a subquery. The planner may push the predicate down or + * leave it on the Agg node; both must select the same groups. + */ +PG_PROP(having_eq_outer_where) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT g, count(*) c FROM gh_t GROUP BY g " + "HAVING count(*) > 1 ORDER BY g", + "SELECT g, c FROM (SELECT g, count(*) c FROM gh_t " + "GROUP BY g) s WHERE s.c > 1 ORDER BY g"); + + pg_prop_disconnect(conn); +} + +/* + * Grouping by an expression equals grouping by that expression pre-computed in + * a subquery. (v>0) is NULL when v is NULL, so NULLS FIRST pins the NULL + * group deterministically in the total order. + */ +PG_PROP(group_by_expr_eq_subquery) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT (v>0) k, count(*) FROM gh_t GROUP BY (v>0) " + "ORDER BY k NULLS FIRST", + "SELECT k, count(*) FROM (SELECT (v>0) k FROM gh_t) s " + "GROUP BY k ORDER BY k NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +/* + * A per-group sum equals the same sum computed by a correlated subquery over + * the distinct grouping keys. IS NOT DISTINCT FROM makes the correlation + * match the GROUP BY semantics for a NULL key (though g is non-NULL here, the + * form stays sound if that ever changes). Both sides return NULL for an + * all-NULL-v group, so no coalesce is needed as long as both agree. + */ +PG_PROP(group_sum_eq_join_sum) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT g, sum(v) FROM gh_t GROUP BY g " + "ORDER BY g NULLS FIRST", + "SELECT g, (SELECT sum(u.v) FROM gh_t u " + "WHERE u.g IS NOT DISTINCT FROM t.g) " + "FROM (SELECT DISTINCT g FROM gh_t) t " + "ORDER BY g NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +/* + * HAVING true is equivalent to no HAVING at all: a constant-true qual filters + * nothing, so every group survives either way. + */ +PG_PROP(having_true_eq_no_having) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT g, count(*) FROM gh_t GROUP BY g HAVING true " + "ORDER BY g NULLS FIRST", + "SELECT g, count(*) FROM gh_t GROUP BY g " + "ORDER BY g NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +/* + * The empty grouping set GROUP BY () collapses the whole relation to exactly + * one group, and -- unlike a column GROUP BY -- it emits that one row even for + * an empty table. So the number of groups is always exactly 1, on every + * input. (A bare constant such as GROUP BY (true) is rejected by the parser, + * so the empty grouping set is the right way to express "one group".) + */ +PG_PROP(group_const_single_group) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "(SELECT count(*) FROM gh_t GROUP BY ()) s) = 1"); + + pg_prop_disconnect(conn); +} + +/* + * SELECT DISTINCT g equals GROUP BY g with no aggregate: both reduce to the + * set of distinct grouping keys. The planner may implement DISTINCT via a + * grouping path, so the two must agree exactly. + */ +PG_PROP(distinct_eq_group_no_agg) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT DISTINCT g FROM gh_t ORDER BY g NULLS FIRST", + "SELECT g FROM gh_t GROUP BY g ORDER BY g NULLS FIRST"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index ff8a3ed7f5e5a..2c6b7783b3997 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 5ba2e819245568f2f659b9c2714d5c89846904bb Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:21 -0400 Subject: [PATCH 187/338] prop: add GROUPING SETS/ROLLUP/CUBE vs UNION ALL property tests (groupingsets) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_groupingsets.c | 258 ++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 262 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_groupingsets.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 7225d38c5ca11..7b1a7c14f8390 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -202,7 +202,8 @@ OBJS = \ prop_windowagg.o \ prop_windownav.o \ prop_windowdistinct.o \ - prop_grouphaving.o + prop_grouphaving.o \ + prop_groupingsets.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 15fe8b505ec14..7ff717cf6c0e4 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -200,6 +200,7 @@ prop_area_sources = files( 'prop_windownav.c', 'prop_windowdistinct.c', 'prop_grouphaving.c', + 'prop_groupingsets.c', ) diff --git a/src/test/prop/prop_groupingsets.c b/src/test/prop/prop_groupingsets.c new file mode 100644 index 0000000000000..0dea43710d68a --- /dev/null +++ b/src/test/prop/prop_groupingsets.c @@ -0,0 +1,258 @@ +/*------------------------------------------------------------------------- + * + * prop_groupingsets.c + * Property-based tests for GROUPING SETS / ROLLUP / CUBE execution. + * + * By definition, GROUP BY GROUPING SETS (...) is the UNION ALL of the plain + * GROUP BYs for each component set, with the columns collapsed by a set + * returned as NULL. ROLLUP and CUBE are shorthands for particular grouping + * set lists. The oracle rewrites each grouping-set query as that explicit + * UNION ALL of plain GROUP BY queries and requires identical rows; a mismatch + * means the grouping-sets executor disagrees with its own definition, which + * is a wrong-results bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_groupingsets.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "groupingsets" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Build gs_t(id, x, y, v) with x,y in 0..2 and small v, no NULLs in x or y. + * Keeping x/y small guarantees most groups are populated, and keeping them + * non-NULL makes a NULL in a grouping-set result unambiguously the "all" + * marker (a collapsed column) rather than real data. ANALYZE so the planner + * may choose any grouping-sets strategy. 0..24 rows covers the empty table. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows = (int) pg_prop_draw_int(tc, 0, 24); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS gs_t; " + "CREATE TABLE gs_t (id int PRIMARY KEY, x int, y int, v int); "); + for (int r = 0; r < nrows; r++) + { + int x = (int) pg_prop_draw_int(tc, 0, 2); + int y = (int) pg_prop_draw_int(tc, 0, 2); + int v = (int) pg_prop_draw_int(tc, 0, 5); + + appendStringInfo(&ddl, "INSERT INTO gs_t VALUES (%d, %d, %d, %d); ", + r, x, y, v); + } + appendStringInfoString(&ddl, "ANALYZE gs_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build failed: %s\n SQL: %s", + err ? err : "(no message)", ddl.data); + pfree(ddl.data); +} + +/* + * Differential oracle: run both queries (each already carrying a total ORDER + * BY) and require identical row-major output. A SQL error on either side is + * itself a failure. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error is a failure. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * ROLLUP(x, y) is GROUPING SETS ((x,y), (x), ()): the UNION ALL of grouping + * by both columns, by x alone (y collapsed to NULL), and the grand total + * (both NULL). The ORDER BY on both sides is x then y with NULLS LAST, which + * is a total order because distinct (x,y) groups cannot tie. + */ +PG_PROP(rollup_two) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT x, y, count(*), sum(v) FROM gs_t " + "GROUP BY ROLLUP(x, y) " + "ORDER BY x NULLS LAST, y NULLS LAST", + "( SELECT x, y, count(*), sum(v) FROM gs_t GROUP BY x, y " + "UNION ALL SELECT x, NULL::int, count(*), sum(v) FROM gs_t GROUP BY x " + "UNION ALL SELECT NULL::int, NULL::int, count(*), sum(v) FROM gs_t ) " + "ORDER BY x NULLS LAST, y NULLS LAST"); + + pg_prop_disconnect(conn); +} + +/* + * CUBE(x, y) is GROUPING SETS ((x,y), (x), (y), ()): all four subsets of the + * two columns. Each UNION ALL arm selects NULL::int for the column its set + * collapses. + */ +PG_PROP(cube_two) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT x, y, count(*), sum(v) FROM gs_t " + "GROUP BY CUBE(x, y) " + "ORDER BY x NULLS LAST, y NULLS LAST", + "( SELECT x, y, count(*), sum(v) FROM gs_t GROUP BY x, y " + "UNION ALL SELECT x, NULL::int, count(*), sum(v) FROM gs_t GROUP BY x " + "UNION ALL SELECT NULL::int, y, count(*), sum(v) FROM gs_t GROUP BY y " + "UNION ALL SELECT NULL::int, NULL::int, count(*), sum(v) FROM gs_t ) " + "ORDER BY x NULLS LAST, y NULLS LAST"); + + pg_prop_disconnect(conn); +} + +/* + * An explicit GROUPING SETS list that is neither a ROLLUP nor a CUBE: + * ((x), (y)). Its definition is the UNION ALL of grouping by x alone (y + * collapsed) and by y alone (x collapsed); no grand-total or all-columns arm. + */ +PG_PROP(grouping_sets_explicit) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT * FROM (SELECT x, y, count(*), sum(v) FROM gs_t " + "GROUP BY GROUPING SETS ((x), (y))) s " + "ORDER BY 1 NULLS LAST, 2 NULLS LAST, 3, 4", + "SELECT * FROM " + "( SELECT x, NULL::int AS y, count(*), sum(v) FROM gs_t GROUP BY x " + "UNION ALL SELECT NULL::int AS x, y, count(*), sum(v) FROM gs_t GROUP BY y ) s " + "ORDER BY 1 NULLS LAST, 2 NULLS LAST, 3, 4"); + + pg_prop_disconnect(conn); +} + +/* + * The GROUPING(x) function marks collapsed columns: it is 1 exactly on the + * rows where the grouping set left x out. Since x is never NULL in the data, + * x comes back NULL in a ROLLUP(x) result iff it was collapsed, so + * (GROUPING(x) = 1) must equal (x IS NULL) on every result row. bool_and + * over those rows must be true; coalesce covers the empty table (ROLLUP still + * emits the grand-total row, but guard anyway). + */ +PG_PROP(grouping_function_marks_collapsed) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and((g = 1) = (x IS NULL)), true) " + "FROM (SELECT x, GROUPING(x) g FROM gs_t GROUP BY ROLLUP(x)) s"); + + pg_prop_disconnect(conn); +} + +/* + * The grand-total row of ROLLUP(x, y) (both x and y NULL) aggregates every + * row, so its count equals the whole table's row count. ROLLUP emits the + * grand-total row even for an empty table (with count 0), so this holds on + * every input; coalesce guards against the row somehow being absent. + */ +PG_PROP(rollup_total_row_matches_ungrouped) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM gs_t) = coalesce(" + "(SELECT c FROM (SELECT x, y, count(*) c FROM gs_t " + "GROUP BY ROLLUP(x, y)) s WHERE x IS NULL AND y IS NULL), 0)"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 2c6b7783b3997..d7c215458c28f 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 9679646d4817de144d2b7be506928751b966771d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:21 -0400 Subject: [PATCH 188/338] prop: add GROUPING SETS with FILTER and grouping_id property tests (groupsets2) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_groupsets2.c | 243 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 247 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_groupsets2.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 7b1a7c14f8390..32e96a7ed1a44 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -203,7 +203,8 @@ OBJS = \ prop_windownav.o \ prop_windowdistinct.o \ prop_grouphaving.o \ - prop_groupingsets.o + prop_groupingsets.o \ + prop_groupsets2.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 7ff717cf6c0e4..a78c3da916aa4 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -201,6 +201,7 @@ prop_area_sources = files( 'prop_windowdistinct.c', 'prop_grouphaving.c', 'prop_groupingsets.c', + 'prop_groupsets2.c', ) diff --git a/src/test/prop/prop_groupsets2.c b/src/test/prop/prop_groupsets2.c new file mode 100644 index 0000000000000..e8df544a82889 --- /dev/null +++ b/src/test/prop/prop_groupsets2.c @@ -0,0 +1,243 @@ +/*------------------------------------------------------------------------- + * + * prop_groupsets2.c + * Property-based tests for GROUPING SETS with FILTER and grouping_id. + * + * These are scalar-boolean algebra properties about GROUPING SETS output that + * are distinct from the UNION-ALL differential oracle in prop_groupingsets.c + * and the ROLLUP/CUBE rewrites in prop_rollup2.c: the grand-total subtotal + * equals the full row count, the grouping() bitmask stays in range, the + * result-row count equals the sum of each set's distinct-group count, a + * FILTERed count never exceeds the plain count within a set, GROUPING SETS + * (()) always yields exactly one row, and ROLLUP has no more rows than CUBE. + * Any "f" is a wrong-results bug in the grouping-sets executor. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_groupsets2.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "groupsets2" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Build gs2_t(id int PRIMARY KEY, x int, y int, v int) with x,y in 0..2 and + * small v, no NULLs in x or y. Keeping x/y small guarantees most groups are + * populated, and keeping them non-NULL makes a NULL in a grouping-set result + * unambiguously the collapsed-column marker rather than real data. ANALYZE + * so the planner may pick any grouping-sets strategy. 0..24 rows covers the + * empty table. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows = (int) pg_prop_draw_int(tc, 0, 24); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS gs2_t; " + "CREATE TABLE gs2_t (id int PRIMARY KEY, x int, y int, v int); "); + for (int r = 0; r < nrows; r++) + { + int x = (int) pg_prop_draw_int(tc, 0, 2); + int y = (int) pg_prop_draw_int(tc, 0, 2); + int v = (int) pg_prop_draw_int(tc, 0, 5); + + appendStringInfo(&ddl, "INSERT INTO gs2_t VALUES (%d, %d, %d, %d); ", + r, x, y, v); + } + appendStringInfoString(&ddl, "ANALYZE gs2_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build failed: %s\n SQL: %s", + err ? err : "(no message)", ddl.data); + pfree(ddl.data); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error is a failure. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * In GROUPING SETS ((x),(y),()) the grand-total row -- the one from the empty + * set () where both columns are collapsed -- aggregates every row, so its + * count equals the whole table's row count. grouping(x,y)=3 (both bits set) + * identifies that row. The outer CASE guards the empty table (where there is + * no row to drive a scalar from and count(*)=0), returning true. + */ +PG_PROP(grouping_sets_subtotal_consistency) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN count(*)=0 THEN true " + "ELSE (SELECT c FROM (SELECT x,y,count(*) c, grouping(x,y) g " + "FROM gs2_t GROUP BY GROUPING SETS ((x),(y),())) s " + "WHERE g = 3 LIMIT 1) = (SELECT count(*) FROM gs2_t) END " + "FROM gs2_t"); + + pg_prop_disconnect(conn); +} + +/* + * grouping(x,y) is a two-bit mask, so across any GROUPING SETS result it must + * stay in {0,1,2,3}. bool_and over the result rows must be true; coalesce + * covers the empty-input case (bool_and over 0 rows is NULL). + */ +PG_PROP(grouping_id_bits) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(g BETWEEN 0 AND 3), true) " + "FROM (SELECT grouping(x,y) g FROM gs2_t " + "GROUP BY GROUPING SETS ((x),(y),())) s"); + + pg_prop_disconnect(conn); +} + +/* + * The number of result rows for GROUPING SETS ((x),(y)) equals the sum over + * each set of its distinct-group count: count(distinct x) + count(distinct y). + * No set shares a group with another here (x-only groups vs y-only groups), + * so the counts simply add. + */ +PG_PROP(groupingsets_rows_sum) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "(SELECT x FROM gs2_t GROUP BY GROUPING SETS ((x),(y))) s) " + "= (SELECT count(DISTINCT x) FROM gs2_t) " + "+ (SELECT count(DISTINCT y) FROM gs2_t)"); + + pg_prop_disconnect(conn); +} + +/* + * Within any grouping set, a FILTERed count counts a subset of the rows the + * plain count does, so count(*) FILTER (WHERE v>0) <= count(*). bool_and over + * every result row of GROUPING SETS ((x),()) must be true; coalesce covers the + * empty-input case. + */ +PG_PROP(filter_in_groupingsets) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(cf <= c), true) FROM " + "(SELECT count(*) c, count(*) FILTER (WHERE v>0) cf " + "FROM gs2_t GROUP BY GROUPING SETS ((x),())) s"); + + pg_prop_disconnect(conn); +} + +/* + * GROUP BY GROUPING SETS (()) is a single empty grouping set, which yields + * exactly one row (the grand total) regardless of input -- even for an empty + * table, where the aggregate still produces its one grand-total row. + */ +PG_PROP(empty_groupingset_one_row) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "(SELECT count(*) FROM gs2_t GROUP BY GROUPING SETS (())) s) = 1"); + + pg_prop_disconnect(conn); +} + +/* + * ROLLUP(x,y) is GROUPING SETS ((x,y),(x),()) and CUBE(x,y) is + * ((x,y),(x),(y),()); ROLLUP's sets are a subset of CUBE's and no set emits + * fewer rows under ROLLUP than CUBE, so ROLLUP's result-row count is at most + * CUBE's. Both subqueries return exactly one count, so this holds on every + * input including the empty table. + */ +PG_PROP(rollup_subset_of_cube_rows) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "(SELECT x,y FROM gs2_t GROUP BY ROLLUP(x,y)) s) " + "<= (SELECT count(*) FROM " + "(SELECT x,y FROM gs2_t GROUP BY CUBE(x,y)) t)"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index d7c215458c28f..349f60612e3f4 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From c68c4e335e12c30c6d6e67067e60ef60af3afb35 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:22 -0400 Subject: [PATCH 189/338] prop: add multi-key GROUP BY hash vs sort equivalence property tests (grouphash) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_grouphash.c | 270 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 274 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_grouphash.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 32e96a7ed1a44..834a795fdbdbb 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -204,7 +204,8 @@ OBJS = \ prop_windowdistinct.o \ prop_grouphaving.o \ prop_groupingsets.o \ - prop_groupsets2.o + prop_groupsets2.o \ + prop_grouphash.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index a78c3da916aa4..c5badaa1084ad 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -202,6 +202,7 @@ prop_area_sources = files( 'prop_grouphaving.c', 'prop_groupingsets.c', 'prop_groupsets2.c', + 'prop_grouphash.c', ) diff --git a/src/test/prop/prop_grouphash.c b/src/test/prop/prop_grouphash.c new file mode 100644 index 0000000000000..2a7ccfac5c308 --- /dev/null +++ b/src/test/prop/prop_grouphash.c @@ -0,0 +1,270 @@ +/*------------------------------------------------------------------------- + * + * prop_grouphash.c + * Property-based tests for multi-column GROUP BY / DISTINCT strategy + * independence. + * + * The oracle is differential: a grouping query that keys on several columns + * (or several grouping expressions) must return identical rows whether the + * executor hashes the groups (enable_hashagg = on) or sorts them + * (enable_hashagg = off). This targets the multi-key grouping paths + * specifically -- composite hash keys, NULL groups across several keys, + * DISTINCT over multiple columns, and count(DISTINCT) under grouping -- which + * is distinct from prop_aggstrategies (single-key strategy independence) and + * prop_grouphaving (equivalent SQL rewrites). Any disagreement between the + * two strategies is a wrong-results bug in multi-key grouping, not a + * performance difference. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_grouphash.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "grouphash" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small domains keep each generated table legible in a counterexample and + * keep cases fast while still covering the multi-key grouping boundaries: the + * grouping keys a and b in 0..2 with ~15% NULL (so composite NULL groups and + * mixed NULL/non-NULL keys are exercised on both strategies), and v small so + * sum(v) and the (v>0) predicate have content without overflow concerns. + */ +#define K_MIN 0 +#define K_MAX 2 +#define V_MIN (-3) +#define V_MAX 3 + +/* Draw a grouping key: SQL NULL ~15% of the time, else an int in [K_MIN,K_MAX]. */ +static void +draw_key(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, K_MIN, K_MAX)); +} + +/* Draw v: SQL NULL ~15% of the time, else an int in [V_MIN, V_MAX]. */ +static void +draw_v(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "(%d)", (int) pg_prop_draw_int(tc, V_MIN, V_MAX)); +} + +/* + * (Re)build gx_t(id int PRIMARY KEY, a int, b int, v int) fresh with 0..30 + * random rows. a and b are grouping keys (0..2, ~15% NULL); v is small + * (~15% NULL). Rebuilt in each property so cases are independent. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS gx_t; " + "CREATE TABLE gx_t (id int PRIMARY KEY, a int, b int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 30); + for (int r = 0; r < nrows; r++) + { + StringInfoData aval; + StringInfoData bval; + StringInfoData vval; + + initStringInfo(&aval); + initStringInfo(&bval); + initStringInfo(&vval); + draw_key(tc, &aval); + draw_key(tc, &bval); + draw_v(tc, &vval); + appendStringInfo(&ddl, "INSERT INTO gx_t VALUES (%d, %s, %s, %s); ", + r, aval.data, bval.data, vval.data); + pfree(aval.data); + pfree(bval.data); + pfree(vval.data); + } + appendStringInfoString(&ddl, "ANALYZE gx_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + pfree(ddl.data); +} + +/* + * Differential oracle: run the SAME query twice -- once forcing hash + * aggregation (enable_hashagg = on) and once forcing sorted/grouped + * aggregation (enable_hashagg = off) -- and require identical rows. RESET ALL + * after. The query must already carry a TOTAL ORDER BY on all grouping keys + * with NULLS FIRST; because the grouping keys are distinct per group that + * makes the ordering total, so the row-major string compares as a multiset. + */ +static void +run_two(pg_prop_tc *tc, pg_prop_conn *conn, const char *query) +{ + char *err = NULL; + bool f1 = false, + f2 = false; + char *r1, + *r2; + + if (!pg_prop_sql_exec(conn, "SET enable_hashagg = on", &err)) + pg_prop_fail(tc, "SET enable_hashagg=on failed: %s", err ? err : ""); + r1 = pg_prop_sql_rows(conn, query, &f1, &err); + if (f1) + pg_prop_fail(tc, "hashagg query failed: %s\n SQL: %s", + err ? err : "", query); + + if (!pg_prop_sql_exec(conn, "SET enable_hashagg = off", &err)) + pg_prop_fail(tc, "SET enable_hashagg=off failed: %s", err ? err : ""); + r2 = pg_prop_sql_rows(conn, query, &f2, &err); + if (f2) + pg_prop_fail(tc, "sort/groupagg query failed: %s\n SQL: %s", + err ? err : "", query); + + pg_prop_sql_exec(conn, "RESET ALL", &err); + + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, + "grouping result depends on strategy\n" + " SQL: %s\n" + " hashagg on:\n%s\n" + " hashagg off:\n%s", + query, r1, r2); + + pg_free(r1); + pg_free(r2); + if (err) + pg_free(err); +} + +/* + * Property 1: GROUP BY two plain columns with count(*) and sum(v). The + * grouping keys a and b both carry NULLs, so this exercises composite groups + * where one or both keys are NULL on the hashed and sorted paths alike. + */ +PG_PROP(multikey_group_strategy) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + run_two(tc, conn, + "SELECT a, b, count(*), sum(v) FROM gx_t GROUP BY a, b " + "ORDER BY a NULLS FIRST, b NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +/* + * Property 2: GROUP BY a single grouping expression over two columns. The + * expression (a + coalesce(b,0)) is NULL exactly when a is NULL, so NULLS + * FIRST pins that group; both strategies must compute the same expression + * groups. + */ +PG_PROP(group_expr_strategy) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + run_two(tc, conn, + "SELECT (a + coalesce(b,0)) k, count(*) FROM gx_t " + "GROUP BY (a + coalesce(b,0)) ORDER BY k NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +/* + * Property 3: GROUP BY two boolean grouping expressions. (v>0) is NULL when + * v is NULL (so the first key spans true/false/NULL), while (a IS NULL) is a + * total boolean; the composite exercises boolean hash keys and a NULL key + * together. + */ +PG_PROP(group_bool_strategy) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + run_two(tc, conn, + "SELECT (v>0) p, (a IS NULL) q, count(*) FROM gx_t " + "GROUP BY (v>0), (a IS NULL) " + "ORDER BY p NULLS FIRST, q NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +/* + * Property 4: SELECT DISTINCT over two columns. DISTINCT may be implemented + * by hashing or by sorting; both must yield the same distinct (a,b) pairs, + * including the pairs with NULL in either column. + */ +PG_PROP(distinct_multikey) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + run_two(tc, conn, + "SELECT DISTINCT a, b FROM gx_t " + "ORDER BY a NULLS FIRST, b NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +/* + * Property 5: count(DISTINCT b) per group of a. The inner DISTINCT and the + * outer grouping can each be hashed or sorted; both strategies must agree on + * the per-group distinct count, including for the NULL-a group and groups + * whose b values include NULL (which count(DISTINCT) ignores). + */ +PG_PROP(group_count_distinct) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + run_two(tc, conn, + "SELECT a, count(DISTINCT b) FROM gx_t GROUP BY a " + "ORDER BY a NULLS FIRST"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 349f60612e3f4..db559d57c7d0b 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From bfe5441a4701e72c55385fbfcb0797a399713105 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:22 -0400 Subject: [PATCH 190/338] prop: add HAVING clause filtering semantics property tests (havingclause) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_havingclause.c | 272 ++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 276 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_havingclause.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 834a795fdbdbb..9fe3f5b350141 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -205,7 +205,8 @@ OBJS = \ prop_grouphaving.o \ prop_groupingsets.o \ prop_groupsets2.o \ - prop_grouphash.o + prop_grouphash.o \ + prop_havingclause.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index c5badaa1084ad..0c74839ae42b6 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -203,6 +203,7 @@ prop_area_sources = files( 'prop_groupingsets.c', 'prop_groupsets2.c', 'prop_grouphash.c', + 'prop_havingclause.c', ) diff --git a/src/test/prop/prop_havingclause.c b/src/test/prop/prop_havingclause.c new file mode 100644 index 0000000000000..8f688e859c890 --- /dev/null +++ b/src/test/prop/prop_havingclause.c @@ -0,0 +1,272 @@ +/*------------------------------------------------------------------------- + * + * prop_havingclause.c + * Property-based tests for HAVING-clause filtering semantics. + * + * The oracle is differential: HAVING filters whole groups by an aggregate + * predicate, and that filtering must equal applying the same predicate as an + * outer WHERE over a grouped subquery -- regardless of whether the planner + * keeps the qual on the Agg node or pushes it down. Two scalar properties + * pin the group-count boundaries HAVING must honour: HAVING false yields no + * groups, and HAVING on an aggregate with no GROUP BY always yields exactly + * one group (the whole table). Any disagreement is a bug in HAVING placement + * or in the "no GROUP BY means one group" rule. This area is distinct from + * grouphaving (general GROUP BY/HAVING rewrites) and filteragg (FILTER on + * aggregates): it isolates HAVING's group-filtering contract. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_havingclause.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "havingclause" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small domains keep a counterexample legible and cases fast while covering + * empty/one-group/many-group shapes: a low-cardinality grouping key (g in + * 0..4) and signed values spanning zero (v in -5..5). v is non-NULL here so + * sum(v) > 0 and count(*) comparisons have clean boundaries; NULL-aggregate + * behaviour is the grouphaving area's job. + */ +#define G_MIN 0 +#define G_MAX 4 +#define V_MIN (-5) +#define V_MAX 5 + +/* + * Build hc_t(id int PRIMARY KEY, g int, v int) with 0..30 random rows. g in + * 0..4, v in -5..5, both non-NULL. Rebuilt fresh in each property so cases + * are independent. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS hc_t; " + "CREATE TABLE hc_t (id int PRIMARY KEY, g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 30); + for (int r = 0; r < nrows; r++) + appendStringInfo(&ddl, "INSERT INTO hc_t VALUES (%d, %d, (%d)); ", + r, + (int) pg_prop_draw_int(tc, G_MIN, G_MAX), + (int) pg_prop_draw_int(tc, V_MIN, V_MAX)); + appendStringInfoString(&ddl, "ANALYZE hc_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + pfree(ddl.data); +} + +/* + * Differential result-set oracle (shape B): q1 and q2 must return identical + * rows in identical order. Both callers supply a TOTAL ORDER BY (g is the + * sole output column and is distinct per group, so ORDER BY g is total) so + * the row-major string compare behaves as a multiset compare. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Scalar boolean oracle (shape A): a single SELECT must return "t". A NULL + * result or SQL error is a failure. Used for the group-count boundary + * properties where a differential row shape is awkward (empty result vs + * single-group result). + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * HAVING sum(v) > 0 keeps exactly the groups whose grouped sum exceeds 0, the + * same groups an outer WHERE on a grouped subquery keeps. The predicate may + * ride the Agg node or be pushed below; both must select the same groups. + */ +PG_PROP(having_filters_groups) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT g FROM hc_t GROUP BY g HAVING sum(v) > 0 " + "ORDER BY g", + "SELECT g FROM (SELECT g, sum(v) s FROM hc_t GROUP BY g) t " + "WHERE s > 0 ORDER BY g"); + + pg_prop_disconnect(conn); +} + +/* + * HAVING count(*) >= 2 keeps groups with at least two rows, matching an outer + * WHERE on the per-group count. + */ +PG_PROP(having_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT g FROM hc_t GROUP BY g HAVING count(*) >= 2 " + "ORDER BY g", + "SELECT g FROM (SELECT g, count(*) c FROM hc_t GROUP BY g) t " + "WHERE c >= 2 ORDER BY g"); + + pg_prop_disconnect(conn); +} + +/* + * WHERE and HAVING compose: WHERE filters rows before grouping, HAVING filters + * groups after. The grouped-and-HAVING form must equal pushing the same + * row filter and group filter through a grouped subquery. + */ +PG_PROP(having_and_where) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT g FROM hc_t WHERE v > -3 GROUP BY g " + "HAVING count(*) > 1 ORDER BY g", + "SELECT g FROM (SELECT g, count(*) c FROM hc_t " + "WHERE v > -3 GROUP BY g) t WHERE c > 1 ORDER BY g"); + + pg_prop_disconnect(conn); +} + +/* + * HAVING false filters out every group: a constant-false qual survives no + * group, so the result is always empty, for every input. + */ +PG_PROP(having_false_empty) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "(SELECT g FROM hc_t GROUP BY g HAVING false) s) = 0"); + + pg_prop_disconnect(conn); +} + +/* + * HAVING true filters nothing: a constant-true qual keeps every group, so the + * grouped result equals the same GROUP BY with no HAVING at all. + */ +PG_PROP(having_true_all) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT g FROM hc_t GROUP BY g HAVING true ORDER BY g", + "SELECT g FROM hc_t GROUP BY g ORDER BY g"); + + pg_prop_disconnect(conn); +} + +/* + * HAVING with no GROUP BY groups the whole table into a single group, and that + * one group row is emitted iff the group passes HAVING -- but the group itself + * always exists. With count(*) >= 0 (always true), the result is exactly one + * group row on every input, including an empty table. This pins the "no + * GROUP BY means one implicit group" rule that HAVING filters over. + */ +PG_PROP(having_without_groupby) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "(SELECT 1 FROM hc_t HAVING count(*) >= 0) s) = 1"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index db559d57c7d0b..d3db7dd285369 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 5644d768506f7b8835a52df1c544c51dcb542151 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:22 -0400 Subject: [PATCH 191/338] prop: add ROLLUP/CUBE grand-total and subtotal relationships property tests (rollup2) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_rollup2.c | 293 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 297 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_rollup2.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 9fe3f5b350141..60eecbd1f846e 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -206,7 +206,8 @@ OBJS = \ prop_groupingsets.o \ prop_groupsets2.o \ prop_grouphash.o \ - prop_havingclause.o + prop_havingclause.o \ + prop_rollup2.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 0c74839ae42b6..103ec9faa6973 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -204,6 +204,7 @@ prop_area_sources = files( 'prop_groupsets2.c', 'prop_grouphash.c', 'prop_havingclause.c', + 'prop_rollup2.c', ) diff --git a/src/test/prop/prop_rollup2.c b/src/test/prop/prop_rollup2.c new file mode 100644 index 0000000000000..38b1dc86d7e2f --- /dev/null +++ b/src/test/prop/prop_rollup2.c @@ -0,0 +1,293 @@ +/*------------------------------------------------------------------------- + * + * prop_rollup2.c + * Property-based tests for ROLLUP/CUBE grand-total and subtotal algebra. + * + * ROLLUP(x,y) and CUBE(x,y) emit, in addition to the detail groups, the + * collapsed "subtotal" and "grand-total" rows that GROUP BY x,y does not. + * These rows carry structural guarantees: the grand-total aggregates the + * whole table, every x-subtotal aggregates exactly that x, CUBE emits a + * superset of ROLLUP's groups, GROUPING() reports which columns collapsed, + * and ROLLUP's detail rows match a plain GROUP BY x,y. Each property is a + * scalar boolean oracle: a wrong grouping-set result is a bug in the + * grouping-sets executor, not a rounding artefact. The data has no NULL in + * x/y, so any NULL in a result column means "collapsed by the grouping set". + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_rollup2.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "rollup2" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the grouping-set query errored or produced NULL where the + * oracle demands a definite true, which is itself a bug -- so either is a + * failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build r2_t(id int PRIMARY KEY, x int, y int, v int) with 0..24 rows. x and + * y are each drawn from {0,1,2} with no NULL, so a NULL in any ROLLUP/CUBE + * result column unambiguously marks a collapsed grouping column rather than a + * data NULL. v is a small integer so sums stay legible in a counterexample. + * The table is rebuilt fresh in every property and ANALYZEd. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS r2_t; " + "CREATE TABLE r2_t(id int PRIMARY KEY, x int, y int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 24); + for (int r = 0; r < nrows; r++) + { + int x = (int) pg_prop_draw_int(tc, 0, 2); + int y = (int) pg_prop_draw_int(tc, 0, 2); + int v = (int) pg_prop_draw_int(tc, 0, 9); + + appendStringInfo(&ddl, "INSERT INTO r2_t VALUES (%d, %d, %d, %d); ", + r, x, y, v); + } + appendStringInfoString(&ddl, "ANALYZE r2_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + pfree(ddl.data); +} + +/* + * Property 1: the grand-total row of ROLLUP(x,y) (x and y both NULL, i.e. both + * collapsed) aggregates the whole table: its sum(v) equals sum(v) over r2_t. + * coalesce guards the empty table (no grand-total row exists) to true. + */ +PG_PROP(rollup_grand_total) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT coalesce(" + "(SELECT g.s FROM " + " (SELECT x, y, sum(v) s FROM r2_t GROUP BY ROLLUP(x,y)) g " + " WHERE x IS NULL AND y IS NULL LIMIT 1) " + "IS NOT DISTINCT FROM (SELECT sum(v) FROM r2_t), true)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 2: each x-subtotal of ROLLUP(x,y) (y collapsed to NULL, x not NULL) + * equals the plain GROUP BY x sum for that same x. Join the ROLLUP subtotal + * rows to GROUP BY x on x and require every pair to agree; bool_and over the + * join, coalesce'd to true for the empty table (no subtotal rows). + */ +PG_PROP(rollup_subtotal_sums) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT coalesce(" + "(SELECT bool_and(g.s = h.s) FROM " + " (SELECT x, y, sum(v) s FROM r2_t GROUP BY ROLLUP(x,y)) g " + " JOIN (SELECT x, sum(v) s FROM r2_t GROUP BY x) h " + " ON g.x = h.x " + " WHERE g.x IS NOT NULL AND g.y IS NULL), true)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 3: CUBE(x,y) emits at least as many groups as ROLLUP(x,y). CUBE + * enumerates all 4 subsets of {x,y} while ROLLUP enumerates only the 3 nested + * prefixes, so for any data CUBE's group count >= ROLLUP's. Both counts are + * definite (count(*) over a grouped subquery is non-NULL), so no coalesce is + * needed; the empty table gives 1 >= 1. + */ +PG_PROP(cube_more_rows_than_rollup) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT " + "(SELECT count(*) FROM " + " (SELECT x, y FROM r2_t GROUP BY CUBE(x,y)) c) >= " + "(SELECT count(*) FROM " + " (SELECT x, y FROM r2_t GROUP BY ROLLUP(x,y)) r)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 4: the detail rows of ROLLUP(x,y) (both x and y non-NULL, i.e. + * nothing collapsed) match GROUP BY x,y exactly. Since x/y are never NULL in + * the data, ROLLUP's detail set is precisely the GROUP BY x,y set, so their + * group counts are equal. Both counts are definite; equal to 0 on empty. + */ +PG_PROP(rollup_detail_matches_groupby) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT " + "(SELECT count(*) FROM " + " (SELECT x, y FROM r2_t GROUP BY ROLLUP(x,y)) g " + " WHERE x IS NOT NULL AND y IS NOT NULL) = " + "(SELECT count(*) FROM " + " (SELECT x, y FROM r2_t GROUP BY x, y) h)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 5: GROUPING(x,y) over CUBE(x,y) yields only values in {0,1,2,3}. + * The 2-bit grouping bitmask (bit 1 for x collapsed, bit 0 for y collapsed) + * cannot exceed 3 for two columns. bool_and over the distinct grouping + * values, coalesce'd to true for the empty table. + */ +PG_PROP(grouping_bits) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT coalesce(" + "(SELECT bool_and(gb BETWEEN 0 AND 3) FROM " + " (SELECT GROUPING(x,y) gb FROM r2_t GROUP BY CUBE(x,y)) s), " + "true)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 6: the grand-total row of ROLLUP(x,y) counts every row of the + * table: its count(*) equals (SELECT count(*) FROM r2_t). Analogous to + * property 1 but on count rather than sum. coalesce guards the empty table + * (no grand-total row) to true. + */ +PG_PROP(rollup_total_count) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT coalesce(" + "(SELECT c FROM " + " (SELECT x, y, count(*) c FROM r2_t GROUP BY ROLLUP(x,y)) g " + " WHERE x IS NULL AND y IS NULL LIMIT 1) " + "IS NOT DISTINCT FROM (SELECT count(*) FROM r2_t), true)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index d3db7dd285369..2dca9dd8bebe1 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 98ce869d0ac771cb4f6d1ef6af27914ad8e54aa5 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:22 -0400 Subject: [PATCH 192/338] prop: add partprune property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_partprune.c | 355 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 359 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_partprune.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 60eecbd1f846e..edafd9b8c7117 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -207,7 +207,8 @@ OBJS = \ prop_groupsets2.o \ prop_grouphash.o \ prop_havingclause.o \ - prop_rollup2.o + prop_rollup2.o \ + prop_partprune.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 103ec9faa6973..c7c077458958c 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -205,6 +205,7 @@ prop_area_sources = files( 'prop_grouphash.c', 'prop_havingclause.c', 'prop_rollup2.c', + 'prop_partprune.c', ) diff --git a/src/test/prop/prop_partprune.c b/src/test/prop/prop_partprune.c new file mode 100644 index 0000000000000..3012ddba2f5b0 --- /dev/null +++ b/src/test/prop/prop_partprune.c @@ -0,0 +1,355 @@ +/*------------------------------------------------------------------------- + * + * prop_partprune.c + * Property-based tests for partition pruning (src/backend/partitioning). + * + * Target #1 in the design, and the one most likely to turn up a wrong-results + * bug: three such fixes landed in the three months before this was written + * (709dfd27f14, 6e5d5680b55, b9d9e9bcc57), all involving multi-column RANGE + * keys, MINVALUE/MAXVALUE bounds, the DEFAULT partition, and NULLs. + * + * The oracle is differential: for a randomly generated partitioned table and + * a plain (unpartitioned) table holding exactly the same rows, any query must + * return the same rows from both. Pruning is a performance optimisation that + * must never change results, so the partitioned answer is also compared with + * enable_partition_pruning turned off. Any disagreement is a pruning bug. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_partprune.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "partprune" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small integer domains keep the generated schema legible in a counterexample + * and keep each case fast, while still covering the boundary interactions + * (a value exactly on a bound, below the first bound, above the last, NULL) + * where the recent bugs lived. + */ +#define VAL_MIN (-4) +#define VAL_MAX 4 + +/* Draw an integer in [VAL_MIN, VAL_MAX], or SQL NULL (~15% of the time). */ +static void +draw_val(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); +} + +/* + * Build a RANGE-partitioned table on (a, b) with a random set of + * non-overlapping partitions plus (usually) a DEFAULT partition, and a plain + * table with the same columns. Populate both with the same random rows. + * Returns false (after noting why) if the random bounds could not form a + * legal partition set, so the caller can discard the case. + */ +static bool +build_tables(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nparts; + int lo; + bool ok = true; + + initStringInfo(&ddl); + + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS pp_part; " + "DROP TABLE IF EXISTS pp_plain; " + "CREATE TABLE pp_part (a int, b int) PARTITION BY RANGE (a, b); " + "CREATE TABLE pp_plain (a int, b int); "); + + /* + * Build the partition set. Two shapes, chosen at random: + * + * 1. Columns laid end to end along a (each partition a distinct a-range), + * optionally with MINVALUE/MAXVALUE on the outer bounds. + * + * 2. Several partitions that SHARE an a-range but split on b -- e.g. + * FROM (3, MINVALUE) TO (3, 0) and FROM (3, 0) TO (3, MAXVALUE). This + * multi-column-prefix shape is what a prefix-only qual (a = 3 AND b op k) + * must prune correctly; it is the shape behind 6e5d5680b55 and + * b9d9e9bcc57. Mixing a run of these with plain a-ranges stresses the + * boundary between the b-split block and its neighbours. + */ + nparts = (int) pg_prop_draw_int(tc, 1, 6); + lo = VAL_MIN; + for (int i = 0; i < nparts; i++) + { + bool first = (i == 0); + bool last = (i == nparts - 1); + bool split_block = (pg_prop_draw_int(tc, 0, 2) == 0); + + if (split_block) + { + /* + * A block of 2-3 partitions sharing a-value `lo`, split on b at + * random cut points, spanning MINVALUE..MAXVALUE on b. Advances a + * by 1 so the next partition starts cleanly above the block. + */ + int sub = (int) pg_prop_draw_int(tc, 2, 3); + int cut = (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX); + int cut2 = cut + (int) pg_prop_draw_int(tc, 1, 3); + + appendStringInfo(&ddl, + "CREATE TABLE pp_p%d PARTITION OF pp_part " + "FOR VALUES FROM (%d, MINVALUE) TO (%d, %d); ", + i, lo, lo, cut); + if (sub == 3) + { + appendStringInfo(&ddl, + "CREATE TABLE pp_p%d_b PARTITION OF pp_part " + "FOR VALUES FROM (%d, %d) TO (%d, %d); ", + i, lo, cut, lo, cut2); + appendStringInfo(&ddl, + "CREATE TABLE pp_p%d_c PARTITION OF pp_part " + "FOR VALUES FROM (%d, %d) TO (%d, MAXVALUE); ", + i, lo, cut2, lo); + } + else + appendStringInfo(&ddl, + "CREATE TABLE pp_p%d_b PARTITION OF pp_part " + "FOR VALUES FROM (%d, %d) TO (%d, MAXVALUE); ", + i, lo, cut, lo); + lo = lo + 1; + } + else + { + int width = (int) pg_prop_draw_int(tc, 1, 3); + int hi = lo + width; + + appendStringInfo(&ddl, + "CREATE TABLE pp_p%d PARTITION OF pp_part FOR VALUES FROM (", i); + if (first && pg_prop_draw_bool(tc)) + appendStringInfoString(&ddl, "MINVALUE, MINVALUE"); + else + appendStringInfo(&ddl, "%d, MINVALUE", lo); + appendStringInfoString(&ddl, ") TO ("); + if (last && pg_prop_draw_bool(tc)) + appendStringInfoString(&ddl, "MAXVALUE, MAXVALUE"); + else + appendStringInfo(&ddl, "%d, MINVALUE", hi); + appendStringInfoString(&ddl, "); "); + lo = hi; + } + } + + /* Usually add a DEFAULT partition -- its mispruning was the recent bug. */ + if (pg_prop_draw_int(tc, 0, 4) != 0) + appendStringInfoString(&ddl, + "CREATE TABLE pp_def PARTITION OF pp_part DEFAULT; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + { + /* + * A random bound set can be rejected by the backend (e.g. an empty + * range, or a partition that collides with another once MINVALUE is + * involved). That is not a pruning bug; discard the case. + */ + pg_prop_note(tc, "schema rejected: %s", err ? err : "(no message)"); + pg_prop_note(tc, "ddl: %s", ddl.data); + if (err) + pg_free(err); + pfree(ddl.data); + pg_prop_assume(tc, false); + return false; /* unreachable: assume(false) longjmps */ + } + pfree(ddl.data); + + /* Insert the same random rows into both tables. */ + { + StringInfoData ins; + int nrows = (int) pg_prop_draw_int(tc, 0, 24); + + initStringInfo(&ins); + for (int r = 0; r < nrows; r++) + { + StringInfoData vals; + + initStringInfo(&vals); + draw_val(tc, &vals); + appendStringInfoString(&vals, ", "); + draw_val(tc, &vals); + + appendStringInfo(&ins, "INSERT INTO pp_part VALUES (%s); ", vals.data); + appendStringInfo(&ins, "INSERT INTO pp_plain VALUES (%s); ", vals.data); + pfree(vals.data); + } + if (ins.len > 0 && !pg_prop_sql_exec(conn, ins.data, &err)) + { + /* + * A row can fall outside every partition when there is no DEFAULT. + * That is a legal INSERT failure, not a bug; discard the case. + */ + pg_prop_note(tc, "insert rejected (likely no partition for row): %s", + err ? err : ""); + if (err) + pg_free(err); + ok = false; + } + pfree(ins.data); + } + + if (!ok) + pg_prop_assume(tc, false); + return ok; +} + +/* Draw a random boolean predicate over (a, b). */ +static void +draw_predicate(pg_prop_tc *tc, StringInfo where) +{ + static const char *ops[] = {"=", "<>", "<", "<=", ">", ">="}; + + /* + * One case in three, emit the shape the recent pruning bugs needed: a + * prefix-only equality on the first key column combined with an + * inequality or IS NOT NULL on the second (e.g. a = 3 AND b >= -7). With + * the b-split partition blocks above, this forces pruning to reason about + * a partial key prefix across a MAXVALUE/MINVALUE boundary -- the trigger + * for 6e5d5680b55 / 709dfd27f14 / b9d9e9bcc57. + */ + if (pg_prop_draw_int(tc, 0, 2) == 0) + { + appendStringInfo(where, "a = %d AND ", + (int) pg_prop_draw_int(tc, VAL_MIN - 1, VAL_MAX + 1)); + if (pg_prop_draw_bool(tc)) + appendStringInfo(where, "b %s %d", + ops[pg_prop_draw_int(tc, 2, lengthof(ops) - 1)], + (int) pg_prop_draw_int(tc, VAL_MIN - 1, VAL_MAX + 1)); + else + appendStringInfoString(where, "b IS NOT NULL"); + return; + } + + { + int nterms = (int) pg_prop_draw_int(tc, 1, 3); + + for (int i = 0; i < nterms; i++) + { + int kind = (int) pg_prop_draw_int(tc, 0, 4); + const char *col = pg_prop_draw_bool(tc) ? "a" : "b"; + + if (i > 0) + appendStringInfoString(where, pg_prop_draw_bool(tc) ? " AND " : " OR "); + + switch (kind) + { + case 0: /* col IS NULL */ + appendStringInfo(where, "%s IS NULL", col); + break; + case 1: /* col IS NOT NULL */ + appendStringInfo(where, "%s IS NOT NULL", col); + break; + case 2: /* col op const */ + appendStringInfo(where, "%s %s %d", col, + ops[pg_prop_draw_int(tc, 0, lengthof(ops) - 1)], + (int) pg_prop_draw_int(tc, VAL_MIN - 1, VAL_MAX + 1)); + break; + case 3: /* col IN (list) */ + appendStringInfo(where, "%s IN (%d, %d)", col, + (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX), + (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); + break; + default: /* col = const (equality prefix) */ + appendStringInfo(where, "%s = %d", col, + (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); + break; + } + } + } +} + +PG_PROP(pruning_preserves_rows) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData where; + StringInfoData q; + char *part_on; + char *part_off; + char *plain; + bool f1, + f2, + f3; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!build_tables(tc, conn)) + { + pg_prop_disconnect(conn); + return; /* unreachable; build_tables assume()d */ + } + + initStringInfo(&where); + draw_predicate(tc, &where); + + /* + * ORDER BY a canonical key so the row multisets compare order-insensitively + * (NULLS treated consistently). Select both columns so a mis-pruned row + * shows up fully in the counterexample. + */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT a, b FROM pp_part WHERE %s ORDER BY a, b, (a IS NULL), (b IS NULL)", + where.data); + + /* partitioned, pruning on (default) */ + pg_prop_sql_exec(conn, "SET enable_partition_pruning = on", &err); + part_on = pg_prop_sql_rows(conn, q.data, &f1, &err); + + /* partitioned, pruning off: the trusted answer */ + pg_prop_sql_exec(conn, "SET enable_partition_pruning = off", &err); + part_off = pg_prop_sql_rows(conn, q.data, &f2, &err); + + /* plain table, same rows: an independent oracle */ + { + StringInfoData pq; + + initStringInfo(&pq); + appendStringInfo(&pq, + "SELECT a, b FROM pp_plain WHERE %s ORDER BY a, b, (a IS NULL), (b IS NULL)", + where.data); + plain = pg_prop_sql_rows(conn, pq.data, &f3, &err); + pfree(pq.data); + } + + if (f1 || f2 || f3) + pg_prop_fail(tc, "query error on WHERE %s: %s", where.data, + err ? err : "(unknown)"); + + if (strcmp(part_on, part_off) != 0) + pg_prop_fail(tc, + "pruning changed results for WHERE %s\n" + " pruning on: %s\n pruning off: %s", + where.data, part_on, part_off); + if (strcmp(part_on, plain) != 0) + pg_prop_fail(tc, + "partitioned != plain for WHERE %s\n" + " partitioned: %s\n plain: %s", + where.data, part_on, plain); + + pg_free(part_on); + pg_free(part_off); + pg_free(plain); + pfree(where.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 2dca9dd8bebe1..99998f78b25d6 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From d840d777cd12e1cf174c4c492b3ef9b62a42ceb5 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:22 -0400 Subject: [PATCH 193/338] prop: add nbtree property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_nbtree.c | 227 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 231 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_nbtree.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index edafd9b8c7117..3940d8ec092e4 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -208,7 +208,8 @@ OBJS = \ prop_grouphash.o \ prop_havingclause.o \ prop_rollup2.o \ - prop_partprune.o + prop_partprune.o \ + prop_nbtree.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index c7c077458958c..13dd5e71a4d58 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -206,6 +206,7 @@ prop_area_sources = files( 'prop_havingclause.c', 'prop_rollup2.c', 'prop_partprune.c', + 'prop_nbtree.c', ) diff --git a/src/test/prop/prop_nbtree.c b/src/test/prop/prop_nbtree.c new file mode 100644 index 0000000000000..b3fcf41416b36 --- /dev/null +++ b/src/test/prop/prop_nbtree.c @@ -0,0 +1,227 @@ +/*------------------------------------------------------------------------- + * + * prop_nbtree.c + * Property-based tests for nbtree index scans (src/backend/access/nbtree). + * + * Target #2 in the design. The skip-scan work (92fe23d93aa and the ~50 + * follow-up fixes: "premature array advancement", "NULL tuple values always + * advance skip arrays", "commute strategy for DESC keys", row-compare + * robustness) makes nbtree scan-key handling a dense bug surface. + * + * The oracle is differential: the same query answered by a forced index scan + * and by a forced sequential scan must return the same rows. The index is + * randomly shaped (column count, ASC/DESC, NULLS FIRST/LAST) and the query + * mixes equality, inequality, IN, and IS NULL, which is what drives skip + * arrays and array-key preprocessing. Backward scans are exercised by a + * random ORDER BY direction. + * + * Tier B: runs SQL against the cluster the TAP driver started; see pg_prop.h + * and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_nbtree.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "nbtree" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +#define VAL_MIN (-5) +#define VAL_MAX 5 +#define NCOLS 3 /* columns a, b, c */ + +static const char *const colnames[NCOLS] = {"a", "b", "c"}; + +/* Draw an integer in [VAL_MIN,VAL_MAX] or NULL (~1 in 7). */ +static void +draw_val(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); +} + +/* + * Create a table (a, b, c), a randomly shaped multi-column btree index over a + * prefix of its columns, and random rows. Returns false (via assume) if the + * schema is rejected. + */ +static void +build(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int ncols; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS nb_t; " + "CREATE TABLE nb_t (a int, b int, c int); "); + + /* index over the first ncols columns, each ASC/DESC and NULLS first/last */ + ncols = (int) pg_prop_draw_int(tc, 1, NCOLS); + appendStringInfoString(&ddl, "CREATE INDEX nb_i ON nb_t ("); + for (int i = 0; i < ncols; i++) + { + if (i > 0) + appendStringInfoString(&ddl, ", "); + appendStringInfo(&ddl, "%s %s NULLS %s", colnames[i], + pg_prop_draw_bool(tc) ? "ASC" : "DESC", + pg_prop_draw_bool(tc) ? "FIRST" : "LAST"); + } + appendStringInfoString(&ddl, "); "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + { + pg_prop_note(tc, "schema rejected: %s", err ? err : ""); + if (err) + pg_free(err); + pfree(ddl.data); + pg_prop_assume(tc, false); + } + pfree(ddl.data); + + /* random rows */ + { + StringInfoData ins; + + nrows = (int) pg_prop_draw_int(tc, 0, 60); + initStringInfo(&ins); + for (int r = 0; r < nrows; r++) + { + appendStringInfoString(&ins, "INSERT INTO nb_t VALUES ("); + for (int col = 0; col < NCOLS; col++) + { + if (col > 0) + appendStringInfoString(&ins, ", "); + draw_val(tc, &ins); + } + appendStringInfoString(&ins, "); "); + } + if (ins.len > 0 && !pg_prop_sql_exec(conn, ins.data, &err)) + { + pg_prop_note(tc, "insert rejected: %s", err ? err : ""); + if (err) + pg_free(err); + pfree(ins.data); + pg_prop_assume(tc, false); + } + pfree(ins.data); + } + + /* keep the planner honest: tiny tables otherwise never use the index */ + pg_prop_sql_exec(conn, "ANALYZE nb_t", &err); +} + +/* Draw a predicate that leans on index scan-key machinery. */ +static void +draw_predicate(pg_prop_tc *tc, StringInfo where) +{ + static const char *ops[] = {"=", "<", "<=", ">", ">=", "<>"}; + int nterms = (int) pg_prop_draw_int(tc, 1, 4); + + for (int i = 0; i < nterms; i++) + { + const char *col = colnames[pg_prop_draw_int(tc, 0, NCOLS - 1)]; + int kind = (int) pg_prop_draw_int(tc, 0, 5); + + if (i > 0) + appendStringInfoString(where, pg_prop_draw_bool(tc) ? " AND " : " OR "); + + switch (kind) + { + case 0: + appendStringInfo(where, "%s IS NULL", col); + break; + case 1: + appendStringInfo(where, "%s IS NOT NULL", col); + break; + case 2: /* = ANY, drives array keys / skip scan */ + appendStringInfo(where, "%s = ANY (ARRAY[%d, %d, %d])", col, + (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX), + (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX), + (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); + break; + case 3: /* row comparison, (a,b) < (x,y) */ + appendStringInfo(where, "(a, b) %s (%d, %d)", + pg_prop_draw_bool(tc) ? "<" : ">", + (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX), + (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); + break; + default: + appendStringInfo(where, "%s %s %d", col, + ops[pg_prop_draw_int(tc, 0, lengthof(ops) - 1)], + (int) pg_prop_draw_int(tc, VAL_MIN - 1, VAL_MAX + 1)); + break; + } + } +} + +PG_PROP(indexscan_matches_seqscan) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData where; + StringInfoData q; + const char *dir; + char *idx; + char *seq; + bool f1, + f2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build(tc, conn); + + initStringInfo(&where); + draw_predicate(tc, &where); + + /* a random scan direction exercises backward index scans */ + dir = pg_prop_draw_bool(tc) ? "ASC" : "DESC"; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT a, b, c FROM nb_t WHERE %s " + "ORDER BY a %s, b %s, c %s, (a IS NULL), (b IS NULL), (c IS NULL)", + where.data, dir, dir, dir); + + /* forced index scan */ + pg_prop_sql_exec(conn, + "SET enable_seqscan = off; SET enable_bitmapscan = off; " + "SET enable_indexscan = on; SET enable_indexonlyscan = on", + &err); + idx = pg_prop_sql_rows(conn, q.data, &f1, &err); + + /* forced sequential scan: the trusted oracle */ + pg_prop_sql_exec(conn, + "SET enable_seqscan = on; SET enable_indexscan = off; " + "SET enable_indexonlyscan = off; SET enable_bitmapscan = off", + &err); + seq = pg_prop_sql_rows(conn, q.data, &f2, &err); + + if (f1 || f2) + pg_prop_fail(tc, "query error on WHERE %s: %s", where.data, + err ? err : "(unknown)"); + + if (strcmp(idx, seq) != 0) + pg_prop_fail(tc, + "index scan != seq scan for WHERE %s (ORDER BY %s)\n" + " index: %s\n seq: %s", + where.data, dir, idx, seq); + + pg_free(idx); + pg_free(seq); + pfree(where.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 99998f78b25d6..d87967387d4b7 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 84446b2a8b13047bf822d6a07e5a364ae7be916f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:22 -0400 Subject: [PATCH 194/338] prop: add index vs bitmap vs sequential scan equivalence property tests (indexscan) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_indexscan.c | 289 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 293 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_indexscan.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 3940d8ec092e4..fe0510f1b162c 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -209,7 +209,8 @@ OBJS = \ prop_havingclause.o \ prop_rollup2.o \ prop_partprune.o \ - prop_nbtree.o + prop_nbtree.o \ + prop_indexscan.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 13dd5e71a4d58..718cafea8d23d 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -207,6 +207,7 @@ prop_area_sources = files( 'prop_rollup2.c', 'prop_partprune.c', 'prop_nbtree.c', + 'prop_indexscan.c', ) diff --git a/src/test/prop/prop_indexscan.c b/src/test/prop/prop_indexscan.c new file mode 100644 index 0000000000000..6c09064d553bd --- /dev/null +++ b/src/test/prop/prop_indexscan.c @@ -0,0 +1,289 @@ +/*------------------------------------------------------------------------- + * + * prop_indexscan.c + * Property-based tests for scan-path equivalence over a b-tree index. + * + * The oracle is differential across access paths: a query answered by a + * b-tree index scan, by a bitmap heap scan over that index, and by a plain + * sequential scan must return the identical rows in the identical order. The + * three paths are forced one at a time with enable_* planner GUCs. A scan + * path is a performance choice that must never change results, so any + * disagreement -- a missed row at a NULL boundary, a wrong range endpoint, a + * mishandled IN-list -- is an index-scan or bitmap-scan bug. + * + * Distinct from the "nbtree" area, which checks b-tree internals directly; + * this exercises the planner/executor scan paths that read through the index. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_indexscan.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "indexscan" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* k is drawn in [-5, 5] with ~15% NULL; v is a small int. */ +#define K_MIN (-5) +#define K_MAX 5 + +/* + * Rebuild ix_t fresh for each property so a case never sees state from a + * previous one. 40-row domain with k in [-5,5] ~15% NULL and small v keeps a + * counterexample legible while covering the equality/range/NULL boundaries + * the scan paths must agree on. id is a PRIMARY KEY (unique, non-NULL) so it + * serves as a total-order tiebreak in every query. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS ix_t; " + "CREATE TABLE ix_t (id int PRIMARY KEY, k int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 40); + for (int i = 0; i < nrows; i++) + { + appendStringInfo(&ddl, "INSERT INTO ix_t VALUES (%d, ", i); + if (pg_prop_draw_int(tc, 0, 6) == 0) /* ~15% NULL */ + appendStringInfoString(&ddl, "NULL"); + else + appendStringInfo(&ddl, "%d", (int) pg_prop_draw_int(tc, K_MIN, K_MAX)); + appendStringInfo(&ddl, ", %d); ", (int) pg_prop_draw_int(tc, 0, 9)); + } + + appendStringInfoString(&ddl, + "CREATE INDEX ix_k ON ix_t (k); " + "ANALYZE ix_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(no message)", ddl.data); + pfree(ddl.data); +} + +/* The three scan-path GUC presets, applied one at a time. */ +static const char *const scan_gucs[3] = { + /* index scan only */ + "SET enable_seqscan=off; SET enable_bitmapscan=off; " + "SET enable_indexscan=on; SET enable_indexonlyscan=on;", + /* bitmap scan only */ + "SET enable_seqscan=off; SET enable_indexscan=off; " + "SET enable_indexonlyscan=off; SET enable_bitmapscan=on;", + /* sequential scan only */ + "SET enable_seqscan=on; SET enable_indexscan=off; " + "SET enable_bitmapscan=off; SET enable_indexonlyscan=off;" +}; + +/* + * Run "query" under each of the three scan-path presets and require the full + * result set to be identical across all three (run1==run2, run1==run3). The + * query must carry a total ORDER BY so the row-major string compares as an + * ordered set. RESET ALL afterwards so no preset leaks to the next query. + */ +static void +run_three_scan(pg_prop_tc *tc, pg_prop_conn *conn, const char *query) +{ + char *rows[3] = {NULL, NULL, NULL}; + char *err = NULL; + + for (int i = 0; i < 3; i++) + { + bool failed = false; + + if (!pg_prop_sql_exec(conn, scan_gucs[i], &err)) + pg_prop_fail(tc, "GUC set failed (path %d): %s", i + 1, + err ? err : "(no message)"); + + rows[i] = pg_prop_sql_rows(conn, query, &failed, &err); + if (failed) + pg_prop_fail(tc, "query failed (path %d): %s\n SQL: %s", + i + 1, err ? err : "(no message)", query); + } + + pg_prop_sql_exec(conn, "RESET ALL;", &err); + + if (strcmp(rows[0], rows[1]) != 0) + pg_prop_fail(tc, + "index scan != bitmap scan\n SQL: %s\n" + " index: %s\n bitmap: %s", + query, rows[0], rows[1]); + if (strcmp(rows[0], rows[2]) != 0) + pg_prop_fail(tc, + "index scan != seq scan\n SQL: %s\n" + " index: %s\n seq: %s", + query, rows[0], rows[2]); + + pg_free(rows[0]); + pg_free(rows[1]); + pg_free(rows[2]); +} + +/* Draw a k constant in [K_MIN, K_MAX]. */ +static int +draw_k(pg_prop_tc *tc) +{ + return (int) pg_prop_draw_int(tc, K_MIN, K_MAX); +} + +/* + * equality_scan: WHERE k = . The index equality path must return the + * same rows as the bitmap and sequential paths. + */ +PG_PROP(equality_scan) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT id, k, v FROM ix_t WHERE k = %d ORDER BY id", + draw_k(tc)); + run_three_scan(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * range_scan: WHERE k BETWEEN AND . Exercises the b-tree range + * scan's endpoint handling against bitmap and seq scans. lo/hi are drawn + * independently; an empty range (lo > hi) is a legal input all three agree on. + */ +PG_PROP(range_scan) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT id, k, v FROM ix_t WHERE k BETWEEN %d AND %d " + "ORDER BY id", + draw_k(tc), draw_k(tc)); + run_three_scan(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * in_list_scan: WHERE k IN (a, b, c). A multi-value IN becomes a + * ScalarArrayOp the b-tree index scan handles natively; it must match the + * bitmap and seq paths. + */ +PG_PROP(in_list_scan) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT id, k, v FROM ix_t WHERE k IN (%d, %d, %d) " + "ORDER BY id", + draw_k(tc), draw_k(tc), draw_k(tc)); + run_three_scan(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * is_null_scan: WHERE k IS NULL. The b-tree stores NULLs; the index path must + * return exactly the NULL rows that the sequential scan does. + */ +PG_PROP(is_null_scan) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *q = "SELECT id, k, v FROM ix_t WHERE k IS NULL ORDER BY id"; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + run_three_scan(tc, conn, q); + + pg_prop_disconnect(conn); +} + +/* + * inequality_scan: WHERE k > OR k IS NULL. Combines an open-ended + * index range with the NULL path in one predicate; all three scans must agree + * on the union. + */ +PG_PROP(inequality_scan) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT id, k, v FROM ix_t WHERE k > %d OR k IS NULL " + "ORDER BY id", + draw_k(tc)); + run_three_scan(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ordered_by_indexed_col: SELECT id, k WHERE k IS NOT NULL ORDER BY k, id. + * The index scan can satisfy the ORDER BY k by walking the index in order, + * while the seq/bitmap paths must sort. (k, id) is a total order (id unique), + * so index-ordered retrieval must produce exactly the sorted seqscan output. + */ +PG_PROP(ordered_by_indexed_col) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *q = "SELECT id, k FROM ix_t WHERE k IS NOT NULL " + "ORDER BY k, id"; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + + run_three_scan(tc, conn, q); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index d87967387d4b7..1afa237234634 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 084e97daa98652ad1795d187648fbee2b1ec3ba3 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:23 -0400 Subject: [PATCH 195/338] prop: add GIN/GiST/partial/expression index vs seqscan equivalence property tests (indexdiff) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_indexdiff.c | 371 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 375 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_indexdiff.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index fe0510f1b162c..0404aeffee34d 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -210,7 +210,8 @@ OBJS = \ prop_rollup2.o \ prop_partprune.o \ prop_nbtree.o \ - prop_indexscan.o + prop_indexscan.o \ + prop_indexdiff.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 718cafea8d23d..9286c1c0b755d 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -208,6 +208,7 @@ prop_area_sources = files( 'prop_partprune.c', 'prop_nbtree.c', 'prop_indexscan.c', + 'prop_indexdiff.c', ) diff --git a/src/test/prop/prop_indexdiff.c b/src/test/prop/prop_indexdiff.c new file mode 100644 index 0000000000000..70fb5e05a8d76 --- /dev/null +++ b/src/test/prop/prop_indexdiff.c @@ -0,0 +1,371 @@ +/*------------------------------------------------------------------------- + * + * prop_indexdiff.c + * Property-based tests for index-vs-seqscan result equivalence. + * + * An index scan must return exactly the same rows as a sequential scan of the + * same table for the same WHERE clause: an index is a performance structure, + * never a different answer. This differential oracle builds a small table, + * creates one index per property (plain btree, GIN and GiST over an int via + * contrib btree_gin/btree_gist, partial, expression, multicolumn), then forces + * the planner onto the index path and onto a seqscan via enable_* GUCs and + * requires the two result sets to be identical under a total ORDER BY. Any + * mismatch is a wrong-results bug in an index access method or the planner. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_indexdiff.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "indexdiff" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small integer domains: k in [-5, 5] with ~15% NULL exercises the =, BETWEEN + * and IS NULL boundaries the AMs care about; v is a tiny non-negative value so + * the partial-index predicate (v > 0) splits the table roughly in half. + */ +#define K_MIN (-5) +#define K_MAX 5 + +/* + * Connect to the cluster and ensure the two contrib opclass extensions exist. + * CREATE EXTENSION IF NOT EXISTS is idempotent, so this is safe to run on + * every per-case connection. + */ +static pg_prop_conn * +connect_with_ext(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, + "CREATE EXTENSION IF NOT EXISTS btree_gin; " + "CREATE EXTENSION IF NOT EXISTS btree_gist", + &err)) + pg_prop_fail(tc, "setup failed (btree_gin/btree_gist): %s", + err ? err : ""); + + return conn; +} + +/* + * Build id_t(id int PRIMARY KEY, k int, v int) with 0..40 rows, k in [-5, 5] + * (~15% NULL), v a small non-negative value. Fresh in every property so no + * state leaks between cases. id is 0..nrows-1 so it is a total-order key. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS id_t; " + "CREATE TABLE id_t (id int PRIMARY KEY, k int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 40); + for (int i = 0; i < nrows; i++) + { + StringInfoData kval; + + initStringInfo(&kval); + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(&kval, "NULL"); + else + appendStringInfo(&kval, "%d", + (int) pg_prop_draw_int(tc, K_MIN, K_MAX)); + + appendStringInfo(&ddl, "INSERT INTO id_t VALUES (%d, %s, %d); ", + i, kval.data, (int) pg_prop_draw_int(tc, 0, 3)); + pfree(kval.data); + } + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build failed: %s\n SQL: %s", err ? err : "", ddl.data); + pfree(ddl.data); +} + +/* Shape B: two queries must return identical rows. */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Run "select" first with index scans forced on and seqscan off, then with + * seqscan forced and all index paths off, and require identical rows. Resets + * the GUCs afterwards. bitmap controls whether the "index" side keeps bitmap + * scans enabled (GIN produces only bitmap scans, so it must). + */ +static void +require_index_eq_seqscan(pg_prop_tc *tc, pg_prop_conn *conn, + const char *select, bool bitmap) +{ + char *err = NULL; + char *idx; + char *seq; + bool fi = false, + fs = false; + + /* Force the index path: seqscan off; keep (bitmap)indexscan on. */ + if (bitmap) + pg_prop_sql_exec(conn, + "SET enable_seqscan = off; " + "SET enable_indexscan = off; " + "SET enable_bitmapscan = on", + &err); + else + pg_prop_sql_exec(conn, + "SET enable_seqscan = off; " + "SET enable_indexscan = on; " + "SET enable_bitmapscan = off", + &err); + idx = pg_prop_sql_rows(conn, select, &fi, &err); + if (fi) + pg_prop_fail(tc, "index-forced query failed: %s\n SQL: %s", + err ? err : "", select); + + /* Force the seqscan path: all index paths off. */ + pg_prop_sql_exec(conn, + "SET enable_seqscan = on; " + "SET enable_indexscan = off; " + "SET enable_bitmapscan = off", + &err); + seq = pg_prop_sql_rows(conn, select, &fs, &err); + if (fs) + pg_prop_fail(tc, "seqscan-forced query failed: %s\n SQL: %s", + err ? err : "", select); + + pg_prop_sql_exec(conn, "RESET ALL", &err); + + if (strcmp(idx, seq) != 0) + pg_prop_fail(tc, + "index != seqscan for query:\n %s\n index:\n%s\n seqscan:\n%s", + select, idx, seq); + + pg_free(idx); + pg_free(seq); +} + +/* + * 1. gin_equality: a GIN index over int (btree_gin opclass) answering k = + * must match a seqscan. GIN yields only bitmap scans, so the index + * side keeps bitmapscan enabled. + */ +PG_PROP(gin_equality) +{ + pg_prop_conn *conn; + char *err = NULL; + int k; + StringInfoData q; + + conn = connect_with_ext(tc); + build_table(tc, conn); + + if (!pg_prop_sql_exec(conn, + "DROP INDEX IF EXISTS id_t_idx; " + "CREATE INDEX id_t_idx ON id_t USING gin (k); " + "ANALYZE id_t", + &err)) + pg_prop_fail(tc, "index build failed: %s", err ? err : ""); + + k = (int) pg_prop_draw_int(tc, K_MIN - 1, K_MAX + 1); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT id, k, v FROM id_t WHERE k = %d ORDER BY id", + k); + + require_index_eq_seqscan(tc, conn, q.data, true); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. gist_range: a GiST index over int (btree_gist opclass) answering + * k BETWEEN lo AND hi must match a seqscan. + */ +PG_PROP(gist_range) +{ + pg_prop_conn *conn; + char *err = NULL; + int lo, + hi, + t; + StringInfoData q; + + conn = connect_with_ext(tc); + build_table(tc, conn); + + if (!pg_prop_sql_exec(conn, + "DROP INDEX IF EXISTS id_t_idx; " + "CREATE INDEX id_t_idx ON id_t USING gist (k); " + "ANALYZE id_t", + &err)) + pg_prop_fail(tc, "index build failed: %s", err ? err : ""); + + lo = (int) pg_prop_draw_int(tc, K_MIN - 1, K_MAX + 1); + hi = (int) pg_prop_draw_int(tc, K_MIN - 1, K_MAX + 1); + if (lo > hi) + { + t = lo; + lo = hi; + hi = t; + } + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT id, k, v FROM id_t WHERE k BETWEEN %d AND %d ORDER BY id", + lo, hi); + + require_index_eq_seqscan(tc, conn, q.data, false); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. partial_index: a partial btree indexed WHERE v > 0, used by a query whose + * qual implies the predicate (k = AND v > 0), must match a seqscan. + */ +PG_PROP(partial_index) +{ + pg_prop_conn *conn; + char *err = NULL; + int k; + StringInfoData q; + + conn = connect_with_ext(tc); + build_table(tc, conn); + + if (!pg_prop_sql_exec(conn, + "DROP INDEX IF EXISTS id_t_idx; " + "CREATE INDEX id_t_idx ON id_t (k) WHERE v > 0; " + "ANALYZE id_t", + &err)) + pg_prop_fail(tc, "index build failed: %s", err ? err : ""); + + k = (int) pg_prop_draw_int(tc, K_MIN - 1, K_MAX + 1); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT id, k, v FROM id_t WHERE k = %d AND v > 0 ORDER BY id", + k); + + require_index_eq_seqscan(tc, conn, q.data, false); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. expr_index: an expression index on (k*k), used by a query matching the + * same expression (k*k = ), must match a seqscan. const2 is a square + * in [0, 25] so it is a value the expression can actually take. + */ +PG_PROP(expr_index) +{ + pg_prop_conn *conn; + char *err = NULL; + int r; + int sq; + StringInfoData q; + + conn = connect_with_ext(tc); + build_table(tc, conn); + + if (!pg_prop_sql_exec(conn, + "DROP INDEX IF EXISTS id_t_idx; " + "CREATE INDEX id_t_idx ON id_t ((k * k)); " + "ANALYZE id_t", + &err)) + pg_prop_fail(tc, "index build failed: %s", err ? err : ""); + + r = (int) pg_prop_draw_int(tc, 0, 5); + sq = r * r; /* a square in [0, 25] */ + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT id, k, v FROM id_t WHERE k * k = %d ORDER BY id", + sq); + + require_index_eq_seqscan(tc, conn, q.data, false); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. multicol_btree: a multicolumn btree on (k, v), used by a prefix equality + * (k = ) with ORDER BY k, v, id, must match a seqscan. The ORDER BY is + * a total order (id is unique) so the comparison is deterministic. + */ +PG_PROP(multicol_btree) +{ + pg_prop_conn *conn; + char *err = NULL; + int k; + StringInfoData q; + + conn = connect_with_ext(tc); + build_table(tc, conn); + + if (!pg_prop_sql_exec(conn, + "DROP INDEX IF EXISTS id_t_idx; " + "CREATE INDEX id_t_idx ON id_t (k, v); " + "ANALYZE id_t", + &err)) + pg_prop_fail(tc, "index build failed: %s", err ? err : ""); + + k = (int) pg_prop_draw_int(tc, K_MIN - 1, K_MAX + 1); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT id, k, v FROM id_t WHERE k = %d ORDER BY k, v, id", + k); + + require_index_eq_seqscan(tc, conn, q.data, false); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 1afa237234634..c4d61304d1cd0 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From ee5b8b480b6a9cc6b5f57b90e68d2aaba74c4faa Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:23 -0400 Subject: [PATCH 196/338] prop: add hash/merge/nestloop join method equivalence property tests (joinmethods) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_joinmethods.c | 340 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 344 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_joinmethods.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 0404aeffee34d..df570ec430061 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -211,7 +211,8 @@ OBJS = \ prop_partprune.o \ prop_nbtree.o \ prop_indexscan.o \ - prop_indexdiff.o + prop_indexdiff.o \ + prop_joinmethods.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 9286c1c0b755d..e3fc025ba119c 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -209,6 +209,7 @@ prop_area_sources = files( 'prop_nbtree.c', 'prop_indexscan.c', 'prop_indexdiff.c', + 'prop_joinmethods.c', ) diff --git a/src/test/prop/prop_joinmethods.c b/src/test/prop/prop_joinmethods.c new file mode 100644 index 0000000000000..6ec9f6f7933ad --- /dev/null +++ b/src/test/prop/prop_joinmethods.c @@ -0,0 +1,340 @@ +/*------------------------------------------------------------------------- + * + * prop_joinmethods.c + * Property-based tests for join-method equivalence (src/backend/executor). + * + * A join's result must not depend on the physical join method the planner + * picks: hash, merge and nestloop are three implementations of the same + * relational operator and must return the same rows. These properties build + * two random small tables and run the same join under each method forced on + * (and the other two off), requiring identical row sets. A disagreement is a + * wrong-results bug in a join executor path -- NULL-key handling, hash batch + * spill, or merge-join mark/restore. + * + * The oracle is differential: for a single generated pair of tables and a + * single query, run1 (hashjoin) == run2 (mergejoin) == run3 (nestloop). If a + * method cannot be used for a given query the planner may fall back to + * another; that is fine, because we only require the results to be equal, not + * that a particular method was chosen. The small key domain (-3..3 with ~20% + * NULL) is the point: it forces NULL-key join behaviour, where these bugs + * hide. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_joinmethods.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "joinmethods" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small integer domains keep each case fast and legible in a counterexample + * while still covering the interactions where join bugs live: a key value + * shared by several rows on both sides (so a method has to emit a cross + * product for that key), and a NULL key (which never equi-matches but does + * match under IS NOT DISTINCT FROM). + */ +#define K_MIN (-3) +#define K_MAX 3 +#define V_MIN (-2) +#define V_MAX 2 + +/* Draw a key value in [K_MIN, K_MAX], or SQL NULL (~20% of the time). */ +static void +draw_key(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 4) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, K_MIN, K_MAX)); +} + +/* + * Build one table "name"(id int, k int, v int) with a random number of rows + * (0..12). id is a dense 0-based sequence so it is unique per table and can + * serve as a tie-breaker in a total ORDER BY. k is drawn by draw_key (with + * NULLs); v is a small non-negative-or-negative int. Discards the case + * (via pg_prop_assume) if the DDL/DML is rejected, which should not happen + * for this fixed schema but keeps a surprise from masquerading as a bug. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn, const char *name) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfo(&ddl, + "DROP TABLE IF EXISTS %s; " + "CREATE TABLE %s (id int, k int, v int); ", + name, name); + + nrows = (int) pg_prop_draw_int(tc, 0, 12); + for (int r = 0; r < nrows; r++) + { + StringInfoData k; + + initStringInfo(&k); + draw_key(tc, &k); + appendStringInfo(&ddl, + "INSERT INTO %s VALUES (%d, %s, %d); ", + name, r, k.data, + (int) pg_prop_draw_int(tc, V_MIN, V_MAX)); + pfree(k.data); + } + + appendStringInfo(&ddl, "ANALYZE %s; ", name); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + { + pg_prop_note(tc, "table build rejected for %s: %s", + name, err ? err : "(no message)"); + pg_prop_note(tc, "ddl: %s", ddl.data); + if (err) + pg_free(err); + pfree(ddl.data); + pg_prop_assume(tc, false); /* longjmps */ + } + pfree(ddl.data); +} + +/* Build both tables fresh for a case. */ +static void +build_tables(pg_prop_tc *tc, pg_prop_conn *conn) +{ + build_table(tc, conn, "jm_a"); + build_table(tc, conn, "jm_b"); +} + +/* + * Run "query" three ways -- hashjoin only, mergejoin only, nestloop only -- + * and require all three to produce the identical ordered row set. Any SQL + * error, or any disagreement, is a failure reported with the query and both + * offending row dumps. RESET ALL is issued after the runs so the forced + * GUCs cannot leak out of this helper. + * + * "query" must carry its own total ORDER BY so pg_prop_sql_rows (which + * compares the newline-joined row-major rendering) is comparing an ordered + * multiset soundly. + */ +static void +run_three(pg_prop_tc *tc, pg_prop_conn *conn, const char *query) +{ + char *err = NULL; + char *run1; + char *run2; + char *run3; + bool f1, + f2, + f3; + + /* 1. hashjoin only */ + pg_prop_sql_exec(conn, + "SET enable_hashjoin = on; " + "SET enable_mergejoin = off; " + "SET enable_nestloop = off;", + &err); + run1 = pg_prop_sql_rows(conn, query, &f1, &err); + + /* 2. mergejoin only */ + pg_prop_sql_exec(conn, + "SET enable_hashjoin = off; " + "SET enable_mergejoin = on; " + "SET enable_nestloop = off;", + &err); + run2 = pg_prop_sql_rows(conn, query, &f2, &err); + + /* 3. nestloop only */ + pg_prop_sql_exec(conn, + "SET enable_hashjoin = off; " + "SET enable_mergejoin = off; " + "SET enable_nestloop = on;", + &err); + run3 = pg_prop_sql_rows(conn, query, &f3, &err); + + /* Restore defaults regardless of outcome. */ + pg_prop_sql_exec(conn, "RESET ALL;", &err); + + if (f1 || f2 || f3) + pg_prop_fail(tc, "query error for: %s\n error: %s", + query, err ? err : "(unknown)"); + + if (strcmp(run1, run2) != 0) + pg_prop_fail(tc, + "hashjoin != mergejoin for: %s\n" + " hashjoin: %s\n mergejoin: %s", + query, run1, run2); + if (strcmp(run1, run3) != 0) + pg_prop_fail(tc, + "hashjoin != nestloop for: %s\n" + " hashjoin: %s\n nestloop: %s", + query, run1, run3); + + pg_free(run1); + pg_free(run2); + pg_free(run3); + if (err) + pg_free(err); +} + +/* + * Inner equi-join on the key: SELECT the matched rows and compare across + * methods. This is the straight hash/merge/nestloop inner path; a shared key + * value yields a cross product that each method must enumerate identically. + */ +PG_PROP(inner_join_on_key) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + run_three(tc, conn, + "SELECT a.id, b.id, a.v, b.v " + "FROM jm_a a JOIN jm_b b ON a.k = b.k " + "ORDER BY a.id, b.id, a.v, b.v"); + + pg_prop_disconnect(conn); +} + +/* + * Left outer join on the key: every left row survives, with NULLs filled in + * where it has no match. Exercises the outer-join handling of each method + * (hash left join, merge left join, nestloop with the inner side empty). + */ +PG_PROP(left_join_on_key) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + run_three(tc, conn, + "SELECT a.id, b.id, a.v, b.v " + "FROM jm_a a LEFT JOIN jm_b b ON a.k = b.k " + "ORDER BY a.id, b.id, a.v, b.v"); + + pg_prop_disconnect(conn); +} + +/* + * Full outer join on the key: unmatched rows from both sides survive, so + * either id can be NULL. The merge-join and hash-join full-join paths are + * distinct code; nestloop cannot do a full join directly, so the planner + * falls back -- which is fine, we only require equal results. Order by + * coalesce(id, -1) on both sides (NULL ids sort together) plus v on both + * sides to make the ordering total. + */ +PG_PROP(full_join_on_key) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + run_three(tc, conn, + "SELECT a.id, b.id, a.v, b.v " + "FROM jm_a a FULL JOIN jm_b b ON a.k = b.k " + "ORDER BY coalesce(a.id, -1), coalesce(b.id, -1), " + "a.v, b.v"); + + pg_prop_disconnect(conn); +} + +/* + * NULL-aware join key: ON a.k IS NOT DISTINCT FROM b.k treats NULL as equal + * to NULL, so NULL-keyed rows on each side match each other. This is the + * case most likely to catch a method that mishandles NULL keys (equi-join + * drops them; IS NOT DISTINCT FROM must keep them matched). + */ +PG_PROP(join_is_not_distinct) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + run_three(tc, conn, + "SELECT a.id, b.id, a.v, b.v " + "FROM jm_a a JOIN jm_b b ON a.k IS NOT DISTINCT FROM b.k " + "ORDER BY a.id, b.id, a.v, b.v"); + + pg_prop_disconnect(conn); +} + +/* + * Inner equi-join with a residual filter: the ON clause joins on the key and + * a WHERE restricts the result. The filter can be pushed down or applied as + * a join qual differently per method, so this checks that the filtered result + * is still method-independent. + */ +PG_PROP(join_with_filter) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + run_three(tc, conn, + "SELECT a.id, b.id, a.v, b.v " + "FROM jm_a a JOIN jm_b b ON a.k = b.k " + "WHERE a.v > 0 OR b.v IS NULL " + "ORDER BY a.id, b.id, a.v, b.v"); + + pg_prop_disconnect(conn); +} + +/* + * Anti-join: left rows that have no match on the right. NOT EXISTS plans as + * an anti-join, which hash, merge and nestloop each implement separately; a + * NULL key on the right must not spuriously satisfy or defeat the anti-match. + */ +PG_PROP(anti_join) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + run_three(tc, conn, + "SELECT a.id " + "FROM jm_a a " + "WHERE NOT EXISTS (SELECT 1 FROM jm_b b WHERE b.k = a.k) " + "ORDER BY a.id"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index c4d61304d1cd0..af4c88b391f5d 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 49dadd2d23acb49104ff9e42fb2f70b687f0e5bd Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:23 -0400 Subject: [PATCH 197/338] prop: add outer-join identity and reordering property tests (outerjoin) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_outerjoin.c | 397 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 401 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_outerjoin.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index df570ec430061..046f00c1d4f2a 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -212,7 +212,8 @@ OBJS = \ prop_nbtree.o \ prop_indexscan.o \ prop_indexdiff.o \ - prop_joinmethods.o + prop_joinmethods.o \ + prop_outerjoin.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index e3fc025ba119c..efba04dcddad8 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -210,6 +210,7 @@ prop_area_sources = files( 'prop_indexscan.c', 'prop_indexdiff.c', 'prop_joinmethods.c', + 'prop_outerjoin.c', ) diff --git a/src/test/prop/prop_outerjoin.c b/src/test/prop/prop_outerjoin.c new file mode 100644 index 0000000000000..6683c9e6f0003 --- /dev/null +++ b/src/test/prop/prop_outerjoin.c @@ -0,0 +1,397 @@ +/*------------------------------------------------------------------------- + * + * prop_outerjoin.c + * Property-based tests for outer-join identities and reordering. + * + * Where prop_joinmethods checks that one join query is method-independent, + * this targets the relational identities that relate different join SHAPES: + * LEFT vs INNER row counts, LEFT = swapped RIGHT, FULL = LEFT + the B-only + * antijoin, and the commutativity of an inner join's result. The oracle is + * differential (two forms must return identical ordered rows) or a scalar + * boolean the backend must answer "t"; where a shape has distinct executor + * paths per method, run_three forces each method and requires agreement. A + * mismatch is a wrong-results bug in outer-join handling (NULL-extension of + * the unmatched side, full-join merge/hash state) or in join reordering. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_outerjoin.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "outerjoin" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small key domain with ~20% NULLs is the point: a plain = join never matches + * NULL k to NULL k, but an outer join still emits the unmatched (NULL-keyed) + * side NULL-extended. Values v give the generated schema a payload column so + * a mis-extended row is visible in a counterexample. + */ +#define K_MIN (-3) +#define K_MAX 3 +#define V_MIN (-2) +#define V_MAX 2 + +/* Draw a key value in [K_MIN, K_MAX], or SQL NULL (~20% of the time). */ +static void +draw_key(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 4) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, K_MIN, K_MAX)); +} + +/* + * Build one table "name"(id int PRIMARY KEY, k int, v int) with 0..12 rows. + * id is a dense 0-based sequence (unique, so it is a sound total-order tie + * breaker); k is drawn with NULLs; v is a small int. Discards the case if + * the DDL/DML is rejected, which should not happen for this fixed schema but + * keeps a surprise from masquerading as a bug. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn, const char *name) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfo(&ddl, + "DROP TABLE IF EXISTS %s; " + "CREATE TABLE %s (id int PRIMARY KEY, k int, v int); ", + name, name); + + nrows = (int) pg_prop_draw_int(tc, 0, 12); + for (int r = 0; r < nrows; r++) + { + StringInfoData k; + + initStringInfo(&k); + draw_key(tc, &k); + appendStringInfo(&ddl, + "INSERT INTO %s VALUES (%d, %s, %d); ", + name, r, k.data, + (int) pg_prop_draw_int(tc, V_MIN, V_MAX)); + pfree(k.data); + } + + appendStringInfo(&ddl, "ANALYZE %s; ", name); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + { + pg_prop_note(tc, "table build rejected for %s: %s", + name, err ? err : "(no message)"); + pg_prop_note(tc, "ddl: %s", ddl.data); + if (err) + pg_free(err); + pfree(ddl.data); + pg_prop_assume(tc, false); /* longjmps */ + } + pfree(ddl.data); +} + +/* Build both tables fresh for a case. */ +static void +build_tables(pg_prop_tc *tc, pg_prop_conn *conn) +{ + build_table(tc, conn, "oj_a"); + build_table(tc, conn, "oj_b"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error is itself a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Differential result-set oracle: run two queries that must return identical + * ordered rows and fail on any disagreement. Both queries MUST carry a total + * ORDER BY so the newline-joined row-major rendering compares as a multiset. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Run "query" three ways -- hashjoin only, mergejoin only, nestloop only -- + * and require all three to produce the identical ordered row set. RESET ALL + * after the runs so the forced GUCs cannot leak out. "query" must carry its + * own total ORDER BY. + */ +static void +run_three(pg_prop_tc *tc, pg_prop_conn *conn, const char *query) +{ + char *err = NULL; + char *run1; + char *run2; + char *run3; + bool f1, + f2, + f3; + + /* 1. hashjoin only */ + pg_prop_sql_exec(conn, + "SET enable_hashjoin = on; " + "SET enable_mergejoin = off; " + "SET enable_nestloop = off;", + &err); + run1 = pg_prop_sql_rows(conn, query, &f1, &err); + + /* 2. mergejoin only */ + pg_prop_sql_exec(conn, + "SET enable_hashjoin = off; " + "SET enable_mergejoin = on; " + "SET enable_nestloop = off;", + &err); + run2 = pg_prop_sql_rows(conn, query, &f2, &err); + + /* 3. nestloop only */ + pg_prop_sql_exec(conn, + "SET enable_hashjoin = off; " + "SET enable_mergejoin = off; " + "SET enable_nestloop = on;", + &err); + run3 = pg_prop_sql_rows(conn, query, &f3, &err); + + /* Restore defaults regardless of outcome. */ + pg_prop_sql_exec(conn, "RESET ALL;", &err); + + if (f1 || f2 || f3) + pg_prop_fail(tc, "query error for: %s\n error: %s", + query, err ? err : "(unknown)"); + + if (strcmp(run1, run2) != 0) + pg_prop_fail(tc, + "hashjoin != mergejoin for: %s\n" + " hashjoin: %s\n mergejoin: %s", + query, run1, run2); + if (strcmp(run1, run3) != 0) + pg_prop_fail(tc, + "hashjoin != nestloop for: %s\n" + " hashjoin: %s\n nestloop: %s", + query, run1, run3); + + pg_free(run1); + pg_free(run2); + pg_free(run3); + if (err) + pg_free(err); +} + +/* + * 1. A LEFT JOIN keeps every left row and adds the matched right rows, so it + * can never return fewer rows than the INNER JOIN on the same ON condition: + * the inner result is a subset of the left result. Scalar boolean oracle. + */ +PG_PROP(left_join_count_ge_inner) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM oj_a a " + "LEFT JOIN oj_b b ON a.k = b.k) " + ">= (SELECT count(*) FROM oj_a a " + "JOIN oj_b b ON a.k = b.k)"); + + pg_prop_disconnect(conn); +} + +/* + * 2. A LEFT JOIN B is exactly B RIGHT JOIN A: the right join preserves the + * same (right-hand, here A) rows the left join preserves, with matches from + * the other side filled identically. Compared with the default plan, not the + * three-method helper -- this is a rewrite identity, not an executor-method + * check. Both forms project (a.id, b.id) and carry a total order. + */ +PG_PROP(right_join_is_left_swapped) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT a.id, b.id FROM oj_a a " + "LEFT JOIN oj_b b ON a.k = b.k " + "ORDER BY a.id, coalesce(b.id, -1)", + "SELECT a.id, b.id FROM oj_b b " + "RIGHT JOIN oj_a a ON a.k = b.k " + "ORDER BY a.id, coalesce(b.id, -1)"); + + pg_prop_disconnect(conn); +} + +/* + * 3. A FULL JOIN B equals (A LEFT JOIN B) UNION ALL (the B rows with no match + * in A, NULL-extended on the A side). The left join supplies every A row + * plus its matches; the antijoin supplies exactly the unmatched B rows the + * full join adds on top. ORDER BY coalesce(a.id,-1), coalesce(b.id,-1) is a + * total order because (a.id, b.id) is unique within each surviving row. + */ +PG_PROP(full_join_is_left_union_right_antijoin) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT aid, bid FROM (" + "SELECT a.id AS aid, b.id AS bid FROM oj_a a " + "FULL JOIN oj_b b ON a.k = b.k) s " + "ORDER BY coalesce(aid, -1), coalesce(bid, -1)", + "SELECT aid, bid FROM (" + "SELECT a.id AS aid, b.id AS bid FROM oj_a a " + "LEFT JOIN oj_b b ON a.k = b.k " + "UNION ALL " + "SELECT NULL::int AS aid, b.id AS bid FROM oj_b b " + "WHERE NOT EXISTS (SELECT 1 FROM oj_a a " + "WHERE a.k = b.k)) s " + "ORDER BY coalesce(aid, -1), coalesce(bid, -1)"); + + pg_prop_disconnect(conn); +} + +/* + * 4. An INNER JOIN ON a.k = b.k must return the identical rows under all three + * forced join methods. This is the baseline executor-equivalence check for + * the inner path; a shared key yields a cross product each method enumerates. + */ +PG_PROP(inner_join_method_invariant) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + run_three(tc, conn, + "SELECT a.id, b.id, a.v, b.v " + "FROM oj_a a JOIN oj_b b ON a.k = b.k " + "ORDER BY a.id, b.id"); + + pg_prop_disconnect(conn); +} + +/* + * 5. A LEFT JOIN must likewise return the identical rows under all three + * forced methods. This exercises each method's NULL-extension of unmatched + * left rows (hash left join, merge left join, nestloop with empty inner). + */ +PG_PROP(left_join_method_invariant) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + run_three(tc, conn, + "SELECT a.id, b.id, a.v, b.v " + "FROM oj_a a LEFT JOIN oj_b b ON a.k = b.k " + "ORDER BY a.id, coalesce(b.id, -1)"); + + pg_prop_disconnect(conn); +} + +/* + * 6. An inner join's result is commutative: A JOIN B ON a.k = b.k returns the + * same (a.id, b.id) pairs as B JOIN A ON a.k = b.k. Compared on the default + * plan -- this is the reordering identity, where a bug would reorder rows into + * a different multiset. ORDER BY the unique id pair gives a total order. + */ +PG_PROP(join_reorder_commutes) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT a.id, b.id FROM oj_a a " + "JOIN oj_b b ON a.k = b.k " + "ORDER BY a.id, b.id", + "SELECT a.id, b.id FROM oj_b b " + "JOIN oj_a a ON a.k = b.k " + "ORDER BY a.id, b.id"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index af4c88b391f5d..44fa2f46f8d6a 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From a707067bc713e79e2d75a119c41a826258fba140 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:23 -0400 Subject: [PATCH 198/338] prop: add join cardinality and identity-join laws property tests (joincard) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_joincard.c | 293 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 297 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_joincard.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 046f00c1d4f2a..1b58c9e85597f 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -213,7 +213,8 @@ OBJS = \ prop_indexscan.o \ prop_indexdiff.o \ prop_joinmethods.o \ - prop_outerjoin.o + prop_outerjoin.o \ + prop_joincard.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index efba04dcddad8..65e6af6632e0d 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -211,6 +211,7 @@ prop_area_sources = files( 'prop_indexdiff.c', 'prop_joinmethods.c', 'prop_outerjoin.c', + 'prop_joincard.c', ) diff --git a/src/test/prop/prop_joincard.c b/src/test/prop/prop_joincard.c new file mode 100644 index 0000000000000..7b2a3b6d0249c --- /dev/null +++ b/src/test/prop/prop_joincard.c @@ -0,0 +1,293 @@ +/*------------------------------------------------------------------------- + * + * prop_joincard.c + * Property-based tests for join cardinality and empty/identity-join laws. + * + * The oracle is relational algebra on two small random tables, jc_a and jc_b: + * a cross join has exactly count(A)*count(B) rows, an equijoin can only be a + * subset of that cross, joining with an empty relation produces no rows, a + * LEFT JOIN against an empty relation preserves the left side exactly, an + * inner join commutes as a multiset, CROSS JOIN equals JOIN ON true, and a + * FULL JOIN keeps at least max(count(A), count(B)) rows. Any violation is a + * planner/executor join bug, since every law here is a theorem of the join + * semantics regardless of data or plan shape. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_joincard.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "joincard" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Build jc_a and jc_b fresh. Each is (id int PRIMARY KEY, k int) with 0..15 + * rows, k drawn from 0..3 and NULL ~15% of the time. Small join keys force + * many matches so an equijoin's cardinality is a non-trivial fraction of the + * cross product; the id is a dense 0..n-1 key that gives every oracle a total + * order to compare on. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn, const char *name) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfo(&ddl, + "DROP TABLE IF EXISTS %s; " + "CREATE TABLE %s (id int PRIMARY KEY, k int); ", + name, name); + + nrows = (int) pg_prop_draw_int(tc, 0, 15); + for (int i = 0; i < nrows; i++) + { + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfo(&ddl, "INSERT INTO %s VALUES (%d, NULL); ", + name, i); + else + appendStringInfo(&ddl, "INSERT INTO %s VALUES (%d, %d); ", + name, i, (int) pg_prop_draw_int(tc, 0, 3)); + } + appendStringInfo(&ddl, "ANALYZE %s; ", name); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build %s failed: %s\n ddl: %s", + name, err ? err : "(unknown)", ddl.data); + pfree(ddl.data); +} + +static void +build_tables(pg_prop_tc *tc, pg_prop_conn *conn) +{ + build_table(tc, conn, "jc_a"); + build_table(tc, conn, "jc_b"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error is itself a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Compare two result sets that must be identical multisets. Both queries + * MUST carry a total ORDER BY so the row-major string compares correctly. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * A: count(A CROSS JOIN B) = count(A) * count(B). The defining cardinality + * of a cross product; holds for every table including the empty ones. + */ +PG_PROP(cross_join_card) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM jc_a a CROSS JOIN jc_b b) " + "= (SELECT count(*) FROM jc_a) * (SELECT count(*) FROM jc_b)"); + + pg_prop_disconnect(conn); +} + +/* + * A: count(A JOIN B ON a.k=b.k) <= count(A) * count(B). An equijoin is a + * filtered subset of the cross product, so it can never have more rows. + */ +PG_PROP(inner_join_le_cross) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM jc_a a JOIN jc_b b ON a.k = b.k) " + "<= (SELECT count(*) FROM jc_a) * (SELECT count(*) FROM jc_b)"); + + pg_prop_disconnect(conn); +} + +/* + * A: A JOIN (empty) is empty. Inner-joining anything with a relation that + * has no rows yields no rows, regardless of the ON condition. + */ +PG_PROP(join_empty_is_empty) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM jc_a a " + "JOIN (SELECT * FROM jc_b WHERE false) b ON true) = 0"); + + pg_prop_disconnect(conn); +} + +/* + * A: A LEFT JOIN (empty) keeps the left side. Every left row survives with + * NULL-extended right columns, so the count equals count(A). + */ +PG_PROP(left_join_empty_keeps_left) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM jc_a a " + "LEFT JOIN (SELECT * FROM jc_b WHERE false) b ON true) " + "= (SELECT count(*) FROM jc_a)"); + + pg_prop_disconnect(conn); +} + +/* + * B: an inner join commutes as a multiset. A JOIN B and B JOIN A, projected + * to (a.id, b.id), must return identical rows in identical order. + */ +PG_PROP(join_commutes) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT a.id, b.id FROM jc_a a JOIN jc_b b ON a.k = b.k " + "ORDER BY a.id, b.id", + "SELECT a.id, b.id FROM jc_b b JOIN jc_a a ON a.k = b.k " + "ORDER BY a.id, b.id"); + + pg_prop_disconnect(conn); +} + +/* + * B: CROSS JOIN equals JOIN ON true. A cross product and an inner join whose + * condition is a tautology are the same relation. + */ +PG_PROP(cross_eq_join_true) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT a.id, b.id FROM jc_a a CROSS JOIN jc_b b " + "ORDER BY a.id, b.id", + "SELECT a.id, b.id FROM jc_a a JOIN jc_b b ON true " + "ORDER BY a.id, b.id"); + + pg_prop_disconnect(conn); +} + +/* + * A: count(A FULL JOIN B ON a.k=b.k) >= greatest(count(A), count(B)). A full + * outer join keeps every row of both inputs (matched together or NULL- + * extended), so it cannot be smaller than either side. + */ +PG_PROP(full_join_card_bounds) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM jc_a a FULL JOIN jc_b b " + "ON a.k = b.k) " + ">= greatest((SELECT count(*) FROM jc_a), " + "(SELECT count(*) FROM jc_b))"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 44fa2f46f8d6a..0be14f5459b82 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From f8615b5a4809be11c43d17955bb890b79af6fb22 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:23 -0400 Subject: [PATCH 199/338] prop: add self-join cardinality and method equivalence property tests (selfjoin) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_selfjoin.c | 315 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 319 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_selfjoin.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 1b58c9e85597f..dc0449ed98723 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -214,7 +214,8 @@ OBJS = \ prop_indexdiff.o \ prop_joinmethods.o \ prop_outerjoin.o \ - prop_joincard.o + prop_joincard.o \ + prop_selfjoin.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 65e6af6632e0d..1d6520d6c0558 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -212,6 +212,7 @@ prop_area_sources = files( 'prop_joinmethods.c', 'prop_outerjoin.c', 'prop_joincard.c', + 'prop_selfjoin.c', ) diff --git a/src/test/prop/prop_selfjoin.c b/src/test/prop/prop_selfjoin.c new file mode 100644 index 0000000000000..8dbd4f18310d7 --- /dev/null +++ b/src/test/prop/prop_selfjoin.c @@ -0,0 +1,315 @@ +/*------------------------------------------------------------------------- + * + * prop_selfjoin.c + * Property-based tests for self-join and join-cardinality invariants. + * + * A self-join of a table on an equi-key must return the same rows regardless + * of which join method the planner picks, and its cardinality is fixed by the + * key-group sizes (an equi-join on a key partitions rows into groups; the join + * emits every ordered pair within a group, so count = sum of group_size^2). + * These are exact combinatorial facts, so the oracles need no tolerance: a + * method that returns different rows, or a count that disagrees with the + * group-size formula, is a join bug (hashjoin/mergejoin/nestloop mismatch, a + * NULL-key join that should not match, or a bad cardinality). + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_selfjoin.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "selfjoin" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Build sj_t(id int PRIMARY KEY, k int): 0..20 rows, id dense from 0, k drawn + * in 0..3 with ~15% NULL. Small key domain forces many ties, so most cases + * exercise non-trivial group sizes (the interesting cardinality regime); the + * PRIMARY KEY gives a total order for the row-set oracles. ANALYZE so the + * planner has stats to pick among forced methods meaningfully. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS sj_t; " + "CREATE TABLE sj_t (id int PRIMARY KEY, k int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 20); + for (int r = 0; r < nrows; r++) + { + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfo(&ddl, "INSERT INTO sj_t VALUES (%d, NULL); ", r); + else + appendStringInfo(&ddl, "INSERT INTO sj_t VALUES (%d, %d); ", + r, (int) pg_prop_draw_int(tc, 0, 3)); + } + + appendStringInfoString(&ddl, "ANALYZE sj_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(no message)", ddl.data); + pfree(ddl.data); +} + +/* --- Oracle B: differential result-set compare (from area_spec.md) --- */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* --- Oracle A: scalar boolean compare (require_true, from prop_numeric.c) --- */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Force each of the three join methods in turn and require that "query" + * returns identical rows under all three. We leave the chosen method on and + * disable the other two with SET enable_*; a RESET ALL between methods clears + * the GUCs so each run starts from a known base. nestloop can never be fully + * disabled (the planner falls back to it), so forcing it means disabling only + * hashjoin and mergejoin. "query" must carry a TOTAL ORDER BY. + */ +static void +run_three(pg_prop_tc *tc, pg_prop_conn *conn, const char *query) +{ + static const char *setups[] = { + /* hashjoin */ + "SET enable_hashjoin = on; SET enable_mergejoin = off; SET enable_nestloop = off;", + /* mergejoin */ + "SET enable_hashjoin = off; SET enable_mergejoin = on; SET enable_nestloop = off;", + /* nestloop */ + "SET enable_hashjoin = off; SET enable_mergejoin = off; SET enable_nestloop = on;" + }; + char *err = NULL; + char *base = NULL; + + for (int i = 0; i < 3; i++) + { + bool f = false; + char *rows; + + if (!pg_prop_sql_exec(conn, "RESET ALL;", &err)) + pg_prop_fail(tc, "RESET ALL failed: %s", err ? err : ""); + if (!pg_prop_sql_exec(conn, setups[i], &err)) + pg_prop_fail(tc, "GUC setup %d failed: %s", i, err ? err : ""); + + rows = pg_prop_sql_rows(conn, query, &f, &err); + if (f) + pg_prop_fail(tc, "query failed (method %d): %s\n SQL: %s", + i, err ? err : "", query); + + if (base == NULL) + base = rows; + else + { + if (strcmp(base, rows) != 0) + pg_prop_fail(tc, + "join method %d differs:\n SQL: %s\n base:\n%s\n this:\n%s", + i, query, base, rows); + pg_free(rows); + } + } + pg_free(base); + pg_prop_sql_exec(conn, "RESET ALL;", &err); + if (err) + pg_free(err); +} + +/* + * 1. An equi self-join on k must return identical rows under hashjoin, + * mergejoin and nestloop -- the join method is a performance choice that must + * never change results. + */ +PG_PROP(equi_self_join_method_invariant) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + run_three(tc, conn, + "SELECT a.id, b.id FROM sj_t a JOIN sj_t b ON a.k = b.k " + "ORDER BY a.id, b.id"); + + pg_prop_disconnect(conn); +} + +/* + * 2. An equi self-join on k partitions non-NULL rows into groups by k; the + * join emits every ordered pair within a group, so its count equals the sum + * over non-NULL k-groups of group_size^2. NULL k never equi-joins, so it is + * excluded. + */ +PG_PROP(join_count_eq_sum_of_squares) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM sj_t a JOIN sj_t b ON a.k = b.k) " + "= (SELECT coalesce(sum(c*c), 0) FROM " + "(SELECT count(*) c FROM sj_t WHERE k IS NOT NULL GROUP BY k) g)"); + + pg_prop_disconnect(conn); +} + +/* + * 3. A CROSS JOIN forms every ordered pair of rows, so its count is the row + * count squared. + */ +PG_PROP(cross_join_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM sj_t a CROSS JOIN sj_t b) " + "= (SELECT count(*) * count(*) FROM sj_t)"); + + pg_prop_disconnect(conn); +} + +/* + * 4. A LEFT self-join keeps every row of the inner self-join plus any + * left-side rows with no match (NULL-k rows, and -- for an inner equi-join -- + * none are dropped beyond those), so the LEFT count is always >= the INNER + * count on the same ON condition. + */ +PG_PROP(left_self_join_ge) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM sj_t a LEFT JOIN sj_t b ON a.k = b.k) " + ">= (SELECT count(*) FROM sj_t a JOIN sj_t b ON a.k = b.k)"); + + pg_prop_disconnect(conn); +} + +/* + * 5. The set of keys that appear in an equi self-join is exactly the set of + * non-NULL k values present in the table (a key self-joins iff some row with + * that key exists; NULL never joins). Differential: the join's distinct keys + * must equal the table's distinct non-NULL keys. + */ +PG_PROP(join_distinct_keys) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT DISTINCT a.k FROM sj_t a JOIN sj_t b ON a.k = b.k ORDER BY a.k", + "SELECT DISTINCT k FROM sj_t WHERE k IS NOT NULL ORDER BY k"); + + pg_prop_disconnect(conn); +} + +/* + * 6. A self-join on "a.k IS NOT DISTINCT FROM b.k" treats NULL as a matchable + * value (NULL IS NOT DISTINCT FROM NULL is true), so it groups ALL rows + * including the NULL group; its count equals the sum over every k-group + * (NULLs included) of group_size^2. + */ +PG_PROP(not_distinct_join_includes_nulls) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM sj_t a JOIN sj_t b " + "ON a.k IS NOT DISTINCT FROM b.k) " + "= (SELECT coalesce(sum(c*c), 0) FROM " + "(SELECT count(*) c FROM sj_t GROUP BY k) g)"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 0be14f5459b82..086a4d24709fe 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From d01dd5d5764813e965e5c90fa8600adfce9021cb Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:23 -0400 Subject: [PATCH 200/338] prop: add USING/NATURAL/ON join equivalence property tests (jointypes) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_jointypes.c | 216 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 220 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_jointypes.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index dc0449ed98723..db7c7ee8b46ae 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -215,7 +215,8 @@ OBJS = \ prop_joinmethods.o \ prop_outerjoin.o \ prop_joincard.o \ - prop_selfjoin.o + prop_selfjoin.o \ + prop_jointypes.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 1d6520d6c0558..60bbe663b9549 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -213,6 +213,7 @@ prop_area_sources = files( 'prop_outerjoin.c', 'prop_joincard.c', 'prop_selfjoin.c', + 'prop_jointypes.c', ) diff --git a/src/test/prop/prop_jointypes.c b/src/test/prop/prop_jointypes.c new file mode 100644 index 0000000000000..417761ff1c04c --- /dev/null +++ b/src/test/prop/prop_jointypes.c @@ -0,0 +1,216 @@ +/*------------------------------------------------------------------------- + * + * prop_jointypes.c + * Property-based tests for USING / NATURAL / ON join equivalences. + * + * The syntactic forms of a join (JOIN ... USING (k), NATURAL JOIN, JOIN ... + * ON a.k = b.k) are defined to produce the same rows when they reference the + * same columns. These build two tables whose only common column is k and + * check those equivalences differentially: any disagreement is a bug in how + * the parser expands USING/NATURAL into join quals, or in the merged-column + * projection that USING/NATURAL produce. + * + * Tier B: runs SQL against the cluster the TAP driver started; see pg_prop.h + * and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_jointypes.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "jointypes" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a k value in [0,3] or SQL NULL (~15%). A small domain with NULLs + * exercises both matching and the NULL-key behaviour (a plain equi-join never + * matches NULL to NULL). + */ +static void +draw_k(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, 0, 3)); +} + +/* + * Build jt_a(ida, k, x) and jt_b(idb, k, y). The id columns are named + * differently so k is the ONLY column the two tables share -- that is what + * makes NATURAL JOIN join on k alone. + */ +static void +build_tables(pg_prop_tc *tc, pg_prop_conn *conn) +{ + char *err = NULL; + StringInfoData ddl; + int na, + nb, + r; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS jt_a; DROP TABLE IF EXISTS jt_b; " + "CREATE TABLE jt_a (ida int PRIMARY KEY, k int, x int); " + "CREATE TABLE jt_b (idb int PRIMARY KEY, k int, y int); "); + + na = (int) pg_prop_draw_int(tc, 0, 15); + for (r = 1; r <= na; r++) + { + appendStringInfo(&ddl, "INSERT INTO jt_a VALUES (%d, ", r); + draw_k(tc, &ddl); + appendStringInfo(&ddl, ", %d); ", (int) pg_prop_draw_int(tc, -5, 5)); + } + nb = (int) pg_prop_draw_int(tc, 0, 15); + for (r = 1; r <= nb; r++) + { + appendStringInfo(&ddl, "INSERT INTO jt_b VALUES (%d, ", r); + draw_k(tc, &ddl); + appendStringInfo(&ddl, ", %d); ", (int) pg_prop_draw_int(tc, -5, 5)); + } + appendStringInfoString(&ddl, "ANALYZE jt_a; ANALYZE jt_b;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s", err ? err : ""); + pfree(ddl.data); +} + +/* Two queries must return identical ordered rows. */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL; + char *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* USING (k) equals ON a.k = b.k for an inner join. */ +PG_PROP(using_eq_on) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT ida, idb FROM jt_a a JOIN jt_b b USING (k) " + "ORDER BY ida, idb", + "SELECT ida, idb FROM jt_a a JOIN jt_b b ON a.k = b.k " + "ORDER BY ida, idb"); + + pg_prop_disconnect(conn); +} + +/* NATURAL JOIN joins on the only common column k, so it equals USING (k). */ +PG_PROP(natural_eq_using) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT ida, idb FROM jt_a NATURAL JOIN jt_b " + "ORDER BY ida, idb", + "SELECT ida, idb FROM jt_a JOIN jt_b USING (k) " + "ORDER BY ida, idb"); + + pg_prop_disconnect(conn); +} + +/* USING (k) merges k into a single output column equal to a.k on an inner join. */ +PG_PROP(using_single_k_column) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT k FROM jt_a a JOIN jt_b b USING (k) ORDER BY k", + "SELECT a.k FROM jt_a a JOIN jt_b b ON a.k = b.k " + "ORDER BY a.k"); + + pg_prop_disconnect(conn); +} + +/* + * In a LEFT JOIN USING (k), the merged k column is the coalesce of the two + * sides; since every left row is kept and the right k is NULL only where + * there was no match (and the left k drove the match), the merged k equals the + * left k. + */ +PG_PROP(left_join_using_coalesces) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT k FROM jt_a LEFT JOIN jt_b USING (k) " + "ORDER BY k NULLS LAST", + "SELECT a.k FROM jt_a a LEFT JOIN jt_b b ON a.k = b.k " + "ORDER BY a.k NULLS LAST"); + + pg_prop_disconnect(conn); +} + +/* CROSS JOIN equals JOIN ... ON true. */ +PG_PROP(cross_eq_join_true) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT ida, idb FROM jt_a a CROSS JOIN jt_b b " + "ORDER BY ida, idb", + "SELECT ida, idb FROM jt_a a JOIN jt_b b ON true " + "ORDER BY ida, idb"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 086a4d24709fe..630e4baf867ef 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 46800728e4f5c13c61e989d34d9017041f4a83c5 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:23 -0400 Subject: [PATCH 201/338] prop: add semi-join/anti-join and EXISTS/IN equivalence property tests (antisemijoin) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_antisemijoin.c | 286 ++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 290 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_antisemijoin.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index db7c7ee8b46ae..48b7f0a7be6e5 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -216,7 +216,8 @@ OBJS = \ prop_outerjoin.o \ prop_joincard.o \ prop_selfjoin.o \ - prop_jointypes.o + prop_jointypes.o \ + prop_antisemijoin.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 60bbe663b9549..2562a240e687c 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -214,6 +214,7 @@ prop_area_sources = files( 'prop_joincard.c', 'prop_selfjoin.c', 'prop_jointypes.c', + 'prop_antisemijoin.c', ) diff --git a/src/test/prop/prop_antisemijoin.c b/src/test/prop/prop_antisemijoin.c new file mode 100644 index 0000000000000..d9c66ecb1ce4f --- /dev/null +++ b/src/test/prop/prop_antisemijoin.c @@ -0,0 +1,286 @@ +/*------------------------------------------------------------------------- + * + * prop_antisemijoin.c + * Property-based tests for semi-join / anti-join plan equivalences. + * + * EXISTS/IN produce semi-joins and NOT EXISTS/NOT IN produce anti-joins; the + * executor may realise either with a hash join or a nested loop, and the + * planner rewrites the subquery forms into one another. The oracle is + * differential: a semi/anti query must return the same rows however it is + * phrased (EXISTS vs IN) and however it is executed (hashjoin vs nestloop), + * and the semi and anti partitions must together reconstruct every row. Any + * disagreement is a planner or executor wrong-results bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_antisemijoin.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "antisemijoin" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Keep k in a small non-null domain (0..4). Non-null k is what makes + * NOT IN sound here: NOT IN against a subquery that yields a NULL returns + * UNKNOWN for every outer row, which would make the EXISTS/IN rewrites + * diverge. Holding b.k non-null keeps NOT EXISTS == NOT IN true. + */ +#define K_MIN 0 +#define K_MAX 4 + +/* + * Build as_a and as_b, each (id int PRIMARY KEY, k int) with 0..18 rows and + * k in 0..4. as_b is sometimes left empty (the anti-join-returns-everything + * / semi-join-returns-nothing boundary). ANALYZE both so the planner has + * stats to choose among the plan shapes. + */ +static void +build_tables(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int na; + int nb; + bool b_empty; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS as_a; " + "DROP TABLE IF EXISTS as_b; " + "CREATE TABLE as_a (id int PRIMARY KEY, k int); " + "CREATE TABLE as_b (id int PRIMARY KEY, k int); "); + + na = (int) pg_prop_draw_int(tc, 0, 18); + for (int i = 0; i < na; i++) + appendStringInfo(&ddl, "INSERT INTO as_a VALUES (%d, %d); ", + i, (int) pg_prop_draw_int(tc, K_MIN, K_MAX)); + + /* as_b empty one case in four. */ + b_empty = (pg_prop_draw_int(tc, 0, 3) == 0); + nb = b_empty ? 0 : (int) pg_prop_draw_int(tc, 0, 18); + for (int i = 0; i < nb; i++) + appendStringInfo(&ddl, "INSERT INTO as_b VALUES (%d, %d); ", + i, (int) pg_prop_draw_int(tc, K_MIN, K_MAX)); + + appendStringInfoString(&ddl, "ANALYZE as_a; ANALYZE as_b; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build failed: %s\n SQL: %s", err ? err : "", ddl.data); + + pfree(ddl.data); +} + +/* + * Shape B differential oracle: both queries must carry a total ORDER BY so + * the row-major string compares as an ordered multiset. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Run one query twice -- once forcing a hash join, once forcing a nested loop + * -- and require identical rows. Resets the GUCs afterward. The query must + * carry a total ORDER BY (see require_same_rows). + */ +static void +run_two(pg_prop_tc *tc, pg_prop_conn *conn, const char *q) +{ + char *err = NULL; + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + /* Force hash join. */ + pg_prop_sql_exec(conn, + "SET enable_hashjoin = on; " + "SET enable_nestloop = off; " + "SET enable_mergejoin = off;", &err); + r1 = pg_prop_sql_rows(conn, q, &f1, &e1); + + /* Force nested loop. */ + pg_prop_sql_exec(conn, + "SET enable_hashjoin = off; " + "SET enable_nestloop = on; " + "SET enable_mergejoin = off;", &err); + r2 = pg_prop_sql_rows(conn, q, &f2, &e2); + + pg_prop_sql_exec(conn, "RESET ALL;", &err); + + if (f1) + pg_prop_fail(tc, "hashjoin run failed: %s\n SQL: %s", e1 ? e1 : "", q); + if (f2) + pg_prop_fail(tc, "nestloop run failed: %s\n SQL: %s", e2 ? e2 : "", q); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, + "hashjoin != nestloop:\n SQL: %s\n hash:\n%s\n nestloop:\n%s", + q, r1, r2); + + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * EXISTS over a correlated equality is the same as IN over the (non-null) + * key column. k is 0..4 (non-null) so IN and EXISTS agree with no NULL + * three-valued-logic surprise. + */ +PG_PROP(exists_eq_in_nonnull) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT id FROM as_a a " + "WHERE EXISTS (SELECT 1 FROM as_b b WHERE b.k = a.k) " + "ORDER BY id", + "SELECT id FROM as_a " + "WHERE k IN (SELECT k FROM as_b WHERE k IS NOT NULL) " + "ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * NOT EXISTS is the same as NOT IN *only* when the inner key is non-null: + * a NULL in as_b.k would make NOT IN return no rows regardless. k is 0..4 + * (non-null), so NOT EXISTS == NOT IN holds. + */ +PG_PROP(not_exists_eq_not_in_nonnull) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT id FROM as_a a " + "WHERE NOT EXISTS (SELECT 1 FROM as_b b WHERE b.k = a.k) " + "ORDER BY id", + "SELECT id FROM as_a " + "WHERE k NOT IN (SELECT k FROM as_b) " + "ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * The semi-join (EXISTS) must return the same rows whether executed as a hash + * semi-join or a nested-loop semi-join. + */ +PG_PROP(semijoin_hash_vs_nestloop) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_tables(tc, conn); + + run_two(tc, conn, + "SELECT id FROM as_a a " + "WHERE EXISTS (SELECT 1 FROM as_b b WHERE b.k = a.k) " + "ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * The anti-join (NOT EXISTS) must return the same rows whether executed as a + * hash anti-join or a nested-loop anti-join. + */ +PG_PROP(antijoin_hash_vs_nestloop) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_tables(tc, conn); + + run_two(tc, conn, + "SELECT id FROM as_a a " + "WHERE NOT EXISTS (SELECT 1 FROM as_b b WHERE b.k = a.k) " + "ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * The semi-join and the anti-join partition the outer rows: every row of as_a + * either has a match (EXISTS) or has none (NOT EXISTS), never both and never + * neither. So (EXISTS rows) UNION ALL (NOT EXISTS rows) must reconstruct all + * of as_a exactly. + */ +PG_PROP(semi_plus_anti_eq_all) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT id FROM as_a ORDER BY id", + "SELECT id FROM (" + "SELECT id FROM as_a a " + "WHERE EXISTS (SELECT 1 FROM as_b b WHERE b.k = a.k) " + "UNION ALL " + "SELECT id FROM as_a a " + "WHERE NOT EXISTS (SELECT 1 FROM as_b b WHERE b.k = a.k)" + ") s ORDER BY id"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 630e4baf867ef..9f77b416ebe13 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From aac857950dc515af55b808ed3f13803ef8b7277b Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:23 -0400 Subject: [PATCH 202/338] prop: add setops property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_setops.c | 290 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 294 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_setops.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 48b7f0a7be6e5..ab73709aa6698 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -217,7 +217,8 @@ OBJS = \ prop_joincard.o \ prop_selfjoin.o \ prop_jointypes.o \ - prop_antisemijoin.o + prop_antisemijoin.o \ + prop_setops.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 2562a240e687c..a85a6576b6cd8 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -215,6 +215,7 @@ prop_area_sources = files( 'prop_selfjoin.c', 'prop_jointypes.c', 'prop_antisemijoin.c', + 'prop_setops.c', ) diff --git a/src/test/prop/prop_setops.c b/src/test/prop/prop_setops.c new file mode 100644 index 0000000000000..be219ee9edae9 --- /dev/null +++ b/src/test/prop/prop_setops.c @@ -0,0 +1,290 @@ +/*------------------------------------------------------------------------- + * + * prop_setops.c + * Property-based tests for SQL set operations (UNION/INTERSECT/EXCEPT). + * + * UNION, INTERSECT and EXCEPT must obey set algebra: UNION is commutative and + * idempotent, INTERSECT yields a subset of both inputs, EXCEPT is disjoint + * from its right operand, and the bare operators deduplicate while their ALL + * variants preserve duplicates. A De Morgan-ish relation ties the two + * together: UNION equals SELECT DISTINCT over UNION ALL. These properties + * check those laws over two randomly generated integer tables. + * + * The oracle is set algebra itself, expressed in SQL: result SETS are compared + * with the "symmetric difference is empty" idiom + * NOT EXISTS (X EXCEPT Y) AND NOT EXISTS (Y EXCEPT X) + * which is independent of row order, and bag semantics are checked by COUNT. + * The small value range (-10..10) and 0..20 rows per table make duplicates + * likely, so the bag-vs-set distinction the ALL variants encode is actually + * exercised. Tables may also be empty, exercising the edge of each law. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_setops.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "setops" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* Value domain small enough that duplicates collide across both tables. */ +#define VAL_MIN (-10) +#define VAL_MAX 10 + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the set operation errored or produced NULL, which is + * itself a bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build the two integer tables setops_a(v int) and setops_b(v int), each with + * a random number of rows (0..20) drawn from [VAL_MIN, VAL_MAX]. Duplicates + * are likely given the narrow value range, which is the point: the set-op + * laws must hold whether the inputs are sets or bags, and the ALL variants + * must keep the duplicates the bare operators drop. + */ +static void +build_tables(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int na; + int nb; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS setops_a; " + "DROP TABLE IF EXISTS setops_b; " + "CREATE TABLE setops_a (v int); " + "CREATE TABLE setops_b (v int); "); + + na = (int) pg_prop_draw_int(tc, 0, 20); + for (int i = 0; i < na; i++) + appendStringInfo(&ddl, "INSERT INTO setops_a VALUES (%d); ", + (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); + + nb = (int) pg_prop_draw_int(tc, 0, 20); + for (int i = 0; i < nb; i++) + appendStringInfo(&ddl, "INSERT INTO setops_b VALUES (%d); ", + (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + + pfree(ddl.data); +} + +/* Drop both tables so no state leaks between cases. */ +static void +drop_tables(pg_prop_tc *tc, pg_prop_conn *conn) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, + "DROP TABLE IF EXISTS setops_a; " + "DROP TABLE IF EXISTS setops_b; ", + &err)) + pg_prop_fail(tc, "table drop failed: %s", err ? err : "(unknown)"); +} + +/* + * UNION commutes: (A UNION B) equals (B UNION A) as sets. Checked with a + * two-way symmetric-difference test so the comparison does not depend on the + * order the two operands' rows happen to come out. + */ +PG_PROP(union_commutative) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT NOT EXISTS (" + "(SELECT v FROM setops_a UNION SELECT v FROM setops_b) " + "EXCEPT " + "(SELECT v FROM setops_b UNION SELECT v FROM setops_a)) " + "AND NOT EXISTS (" + "(SELECT v FROM setops_b UNION SELECT v FROM setops_a) " + "EXCEPT " + "(SELECT v FROM setops_a UNION SELECT v FROM setops_b))"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * UNION is idempotent: (A UNION A) equals SELECT DISTINCT v FROM A. UNION + * deduplicates, so unioning A with itself must collapse to A's distinct values + * and add nothing. + */ +PG_PROP(union_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT NOT EXISTS (" + "(SELECT v FROM setops_a UNION SELECT v FROM setops_a) " + "EXCEPT (SELECT DISTINCT v FROM setops_a)) " + "AND NOT EXISTS (" + "(SELECT DISTINCT v FROM setops_a) " + "EXCEPT (SELECT v FROM setops_a UNION SELECT v FROM setops_a))"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * INTERSECT yields a subset of both inputs: (A INTERSECT B) EXCEPT A is empty, + * and likewise with B. Every intersected value must appear in each operand's + * distinct set. + */ +PG_PROP(intersect_subset_of_both) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT NOT EXISTS (" + "(SELECT v FROM setops_a INTERSECT SELECT v FROM setops_b) " + "EXCEPT SELECT v FROM setops_a) " + "AND NOT EXISTS (" + "(SELECT v FROM setops_a INTERSECT SELECT v FROM setops_b) " + "EXCEPT SELECT v FROM setops_b)"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * EXCEPT is disjoint from its right operand: (A EXCEPT B) INTERSECT B is + * empty. No value removed-against B can survive and still be in B. + */ +PG_PROP(except_disjoint_from_b) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT NOT EXISTS (" + "(SELECT v FROM setops_a EXCEPT SELECT v FROM setops_b) " + "INTERSECT SELECT v FROM setops_b)"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * UNION ALL preserves duplicates: the row count of (A UNION ALL B) equals + * count(A) + count(B). This is the bag-semantics counterpart to the + * deduplicating UNION, and the small value range makes real duplicates likely. + */ +PG_PROP(union_all_count_additive) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "(SELECT v FROM setops_a UNION ALL SELECT v FROM setops_b) s) " + "= ((SELECT count(*) FROM setops_a) " + "+ (SELECT count(*) FROM setops_b))"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * UNION equals SELECT DISTINCT over UNION ALL: the deduplicating operator is + * exactly the bag concatenation with duplicates removed. This is the De + * Morgan-ish bridge between the set and bag variants, checked set-wise with a + * two-way symmetric-difference test. + */ +PG_PROP(union_is_distinct_union_all) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT NOT EXISTS (" + "(SELECT v FROM setops_a UNION SELECT v FROM setops_b) " + "EXCEPT (SELECT DISTINCT v FROM " + "(SELECT v FROM setops_a UNION ALL SELECT v FROM setops_b) s)) " + "AND NOT EXISTS (" + "(SELECT DISTINCT v FROM " + "(SELECT v FROM setops_a UNION ALL SELECT v FROM setops_b) s) " + "EXCEPT (SELECT v FROM setops_a UNION SELECT v FROM setops_b))"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 9f77b416ebe13..a1147b2016ff9 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 694f6519e1f5363112fdcf753d4e59a28fa98f51 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:24 -0400 Subject: [PATCH 203/338] prop: add hashed vs sorted set-operation equivalence property tests (setopstrat) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_setopstrat.c | 260 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 264 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_setopstrat.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index ab73709aa6698..9c02f6a027f08 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -218,7 +218,8 @@ OBJS = \ prop_selfjoin.o \ prop_jointypes.o \ prop_antisemijoin.o \ - prop_setops.o + prop_setops.o \ + prop_setopstrat.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index a85a6576b6cd8..d2c1329d4fba4 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -216,6 +216,7 @@ prop_area_sources = files( 'prop_jointypes.c', 'prop_antisemijoin.c', 'prop_setops.c', + 'prop_setopstrat.c', ) diff --git a/src/test/prop/prop_setopstrat.c b/src/test/prop/prop_setopstrat.c new file mode 100644 index 0000000000000..179194a050de2 --- /dev/null +++ b/src/test/prop/prop_setopstrat.c @@ -0,0 +1,260 @@ +/*------------------------------------------------------------------------- + * + * prop_setopstrat.c + * Property-based tests for set-operation strategy independence. + * + * UNION/INTERSECT/EXCEPT are specified by set semantics, not by the executor + * strategy used to evaluate them. PostgreSQL can implement a set operation + * either by hashing (Setop/HashAggregate paths) or by sorting; which one it + * picks is a cost decision gated by enable_hashagg. The result rows must be + * identical either way. The oracle is differential: for each query we run it + * once with enable_hashagg=on (hashed) and once with enable_hashagg=off + * (sorted) and require identical rows. Any disagreement is a bug in one of + * the two set-operation implementations (or in NULL handling, since NULL is a + * single distinct value for set-operation deduplication -- "not distinct"). + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_setopstrat.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "setopstrat" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small value domain: v in [-4, 4] with ~15% NULL, so the set operations + * routinely hit duplicate values, empty intersections, and the NULL-as-a- + * single-distinct-value case that distinguishes correct set-op dedup from + * ordinary equality. + */ +#define VAL_MIN (-4) +#define VAL_MAX 4 + +/* Draw an integer in [VAL_MIN, VAL_MAX], or SQL NULL (~15% of the time). */ +static void +draw_val(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); +} + +/* + * Build so_a(id int, v int) and so_b(id int, v int), each with 0..20 random + * rows. Rebuilt fresh in every property so cases do not leak state. + */ +static void +build_tables(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int na; + int nb; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS so_a; " + "DROP TABLE IF EXISTS so_b; " + "CREATE TABLE so_a (id int, v int); " + "CREATE TABLE so_b (id int, v int); "); + + na = (int) pg_prop_draw_int(tc, 0, 20); + for (int i = 0; i < na; i++) + { + StringInfoData val; + + initStringInfo(&val); + draw_val(tc, &val); + appendStringInfo(&ddl, "INSERT INTO so_a VALUES (%d, %s); ", i, val.data); + pfree(val.data); + } + + nb = (int) pg_prop_draw_int(tc, 0, 20); + for (int i = 0; i < nb; i++) + { + StringInfoData val; + + initStringInfo(&val); + draw_val(tc, &val); + appendStringInfo(&ddl, "INSERT INTO so_b VALUES (%d, %s); ", i, val.data); + pfree(val.data); + } + + appendStringInfoString(&ddl, "ANALYZE so_a; ANALYZE so_b; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(no message)", ddl.data); + pfree(ddl.data); +} + +/* + * Run q with enable_hashagg=on (hashed set op) and again with + * enable_hashagg=off (sorted set op); require identical rows. RESET ALL + * afterward so no planner GUC leaks into the next query or case. + * + * Each query already carries a TOTAL ORDER BY, so the row-major strings + * compare as ordered multisets. + */ +static void +run_two(pg_prop_tc *tc, pg_prop_conn *conn, const char *q) +{ + char *err = NULL; + bool f1 = false; + bool f2 = false; + char *e1 = NULL; + char *e2 = NULL; + char *r1; + char *r2; + + if (!pg_prop_sql_exec(conn, "SET enable_hashagg = on", &err)) + pg_prop_fail(tc, "SET enable_hashagg=on failed: %s", err ? err : ""); + r1 = pg_prop_sql_rows(conn, q, &f1, &e1); + + if (!pg_prop_sql_exec(conn, "SET enable_hashagg = off", &err)) + pg_prop_fail(tc, "SET enable_hashagg=off failed: %s", err ? err : ""); + r2 = pg_prop_sql_rows(conn, q, &f2, &e2); + + pg_prop_sql_exec(conn, "RESET ALL", &err); + + if (f1) + pg_prop_fail(tc, "hashed query failed: %s\n SQL: %s", e1 ? e1 : "", q); + if (f2) + pg_prop_fail(tc, "sorted query failed: %s\n SQL: %s", e2 ? e2 : "", q); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, + "hashed != sorted set-op result\n SQL: %s\n" + " hashed (enable_hashagg=on):\n%s\n" + " sorted (enable_hashagg=off):\n%s", + q, r1, r2); + + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * The distinct set operations (UNION/INTERSECT/EXCEPT) dedup to distinct + * values of v, so v alone -- with NULLS FIRST pinning the single NULL value + * -- is a total order over the result. + */ +PG_PROP(union_distinct) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + run_two(tc, conn, + "(SELECT v FROM so_a UNION SELECT v FROM so_b) ORDER BY v NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +PG_PROP(intersect_distinct) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + run_two(tc, conn, + "(SELECT v FROM so_a INTERSECT SELECT v FROM so_b) ORDER BY v NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +PG_PROP(except_distinct) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + run_two(tc, conn, + "(SELECT v FROM so_a EXCEPT SELECT v FROM so_b) ORDER BY v NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +/* + * The ALL variants preserve multiplicity, so a value can repeat and v alone + * is not a total order. Grouping by v and emitting (v, count(*)) collapses + * each value to one row, making (v) a total order again while still asserting + * that both strategies produce the same multiset of values. + */ +PG_PROP(union_all_counts) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + run_two(tc, conn, + "SELECT v, count(*) FROM " + "(SELECT v FROM so_a UNION ALL SELECT v FROM so_b) s " + "GROUP BY v ORDER BY v NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +PG_PROP(intersect_all_counts) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + run_two(tc, conn, + "SELECT v, count(*) FROM " + "(SELECT v FROM so_a INTERSECT ALL SELECT v FROM so_b) s " + "GROUP BY v ORDER BY v NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +PG_PROP(except_all_counts) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + run_two(tc, conn, + "SELECT v, count(*) FROM " + "(SELECT v FROM so_a EXCEPT ALL SELECT v FROM so_b) s " + "GROUP BY v ORDER BY v NULLS FIRST"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index a1147b2016ff9..dc862672209fe 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From adb60b9859a24b90bbe31665bb1461a1b5ade4bb Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:24 -0400 Subject: [PATCH 204/338] prop: add EXCEPT/INTERSECT vs semijoin/antijoin property tests (exceptintersect) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_exceptintersect.c | 279 +++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 283 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_exceptintersect.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 9c02f6a027f08..9932fc90907f9 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -219,7 +219,8 @@ OBJS = \ prop_jointypes.o \ prop_antisemijoin.o \ prop_setops.o \ - prop_setopstrat.o + prop_setopstrat.o \ + prop_exceptintersect.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index d2c1329d4fba4..83aa29d4bb6f7 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -217,6 +217,7 @@ prop_area_sources = files( 'prop_antisemijoin.c', 'prop_setops.c', 'prop_setopstrat.c', + 'prop_exceptintersect.c', ) diff --git a/src/test/prop/prop_exceptintersect.c b/src/test/prop/prop_exceptintersect.c new file mode 100644 index 0000000000000..9d3d52877dc22 --- /dev/null +++ b/src/test/prop/prop_exceptintersect.c @@ -0,0 +1,279 @@ +/*------------------------------------------------------------------------- + * + * prop_exceptintersect.c + * Property-based tests for EXCEPT / INTERSECT set operations. + * + * The oracle is differential: EXCEPT (distinct) is a NULL-aware antijoin, + * INTERSECT (distinct) a NULL-aware semijoin, and EXCEPT ALL is bag + * difference max(count_a - count_b, 0). Each set-op result is compared to an + * equivalent join-based reconstruction over the same two tables; a mismatch + * means the set-op executor (hashing or sorting, NULL grouping) produced + * wrong rows, which is a bug. All comparisons use IS NOT DISTINCT FROM so + * NULL is treated as a single groupable value, matching set-op semantics. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_exceptintersect.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "exceptintersect" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* Small value domain with boundary negatives; ~15% NULL. */ +#define VAL_MIN (-4) +#define VAL_MAX 4 + +/* Draw an int in [VAL_MIN, VAL_MAX] (parenthesized if negative) or NULL. */ +static void +draw_val(pg_prop_tc *tc, StringInfo buf) +{ + int v; + + if (pg_prop_draw_int(tc, 0, 6) == 0) + { + appendStringInfoString(buf, "NULL"); + return; + } + v = (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX); + if (v < 0) + appendStringInfo(buf, "(%d)", v); + else + appendStringInfo(buf, "%d", v); +} + +/* + * Build ei_a(id int, v int) and ei_b(id int, v int), each with 0..20 random + * rows, v in [-4,4] ~15% NULL. Rebuilt fresh for every case. + */ +static void +build_tables(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int na; + int nb; + int r; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS ei_a; " + "DROP TABLE IF EXISTS ei_b; " + "CREATE TABLE ei_a (id int, v int); " + "CREATE TABLE ei_b (id int, v int); "); + + na = (int) pg_prop_draw_int(tc, 0, 20); + for (r = 0; r < na; r++) + { + StringInfoData v; + + initStringInfo(&v); + draw_val(tc, &v); + appendStringInfo(&ddl, "INSERT INTO ei_a VALUES (%d, %s); ", r, v.data); + pfree(v.data); + } + + nb = (int) pg_prop_draw_int(tc, 0, 20); + for (r = 0; r < nb; r++) + { + StringInfoData v; + + initStringInfo(&v); + draw_val(tc, &v); + appendStringInfo(&ddl, "INSERT INTO ei_b VALUES (%d, %s); ", r, v.data); + pfree(v.data); + } + + appendStringInfoString(&ddl, "ANALYZE ei_a; ANALYZE ei_b; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "", ddl.data); + pfree(ddl.data); +} + +/* Compare two result-set queries row-major; both must carry a total order. */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false; + bool f2 = false; + char *e1 = NULL; + char *e2 = NULL; + char *r1; + char *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* Run a boolean SELECT and require "t"; NULL or error is a failure. */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * EXCEPT equals a NULL-aware distinct antijoin: the values in A that have no + * IS NOT DISTINCT FROM match in B, deduplicated. + */ +PG_PROP(except_eq_antijoin) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT v FROM " + "(SELECT v FROM ei_a EXCEPT SELECT v FROM ei_b) s " + "ORDER BY v NULLS FIRST", + "SELECT v FROM " + "(SELECT DISTINCT a.v FROM ei_a a WHERE NOT EXISTS " + "(SELECT 1 FROM ei_b b WHERE b.v IS NOT DISTINCT FROM a.v)) s " + "ORDER BY v NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +/* + * INTERSECT equals a NULL-aware distinct semijoin: the values in A that have + * at least one IS NOT DISTINCT FROM match in B, deduplicated. + */ +PG_PROP(intersect_eq_semijoin) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT v FROM " + "(SELECT v FROM ei_a INTERSECT SELECT v FROM ei_b) s " + "ORDER BY v NULLS FIRST", + "SELECT v FROM " + "(SELECT DISTINCT a.v FROM ei_a a WHERE EXISTS " + "(SELECT 1 FROM ei_b b WHERE b.v IS NOT DISTINCT FROM a.v)) s " + "ORDER BY v NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +/* + * EXCEPT ALL is bag difference: the multiplicity of each value equals + * max(count_a - count_b, 0). Compare per-value counts of EXCEPT ALL against + * a FULL JOIN of the grouped counts (NULL-grouped via IS NOT DISTINCT FROM). + */ +PG_PROP(except_all_minus) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT v, count(*) AS c FROM " + "(SELECT v FROM ei_a EXCEPT ALL SELECT v FROM ei_b) s " + "GROUP BY v ORDER BY v NULLS FIRST", + "SELECT v, c FROM (" + "SELECT dv.v AS v, greatest(" + "(SELECT count(*) FROM ei_a a WHERE a.v IS NOT DISTINCT FROM dv.v) " + "- (SELECT count(*) FROM ei_b b WHERE b.v IS NOT DISTINCT FROM dv.v), " + "0) AS c " + "FROM (SELECT DISTINCT v FROM ei_a) dv) m " + "WHERE c > 0 ORDER BY v NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +/* + * INTERSECT commutes as a set operation: A INTERSECT B and B INTERSECT A + * produce the same set of values. + */ +PG_PROP(intersect_commutes_as_sets) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT v FROM " + "(SELECT v FROM ei_a INTERSECT SELECT v FROM ei_b) s " + "ORDER BY v NULLS FIRST", + "SELECT v FROM " + "(SELECT v FROM ei_b INTERSECT SELECT v FROM ei_a) s " + "ORDER BY v NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +/* + * A EXCEPT A is empty: a set minus itself has no rows, even with NULLs and + * duplicates present. A non-zero count would be a set-op bug. + */ +PG_PROP(except_self_empty) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT count(*) = 0 FROM " + "(SELECT v FROM ei_a EXCEPT SELECT v FROM ei_a) s"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index dc862672209fe..7d6b56f8149ed 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From abc4a9180aeb6ca3668af5820c771689ce5344b6 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:24 -0400 Subject: [PATCH 205/338] prop: add UNION ALL multiplicity and MergeAppend ordering property tests (unionall) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_unionall.c | 357 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 361 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_unionall.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 9932fc90907f9..bebe554dc7f06 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -220,7 +220,8 @@ OBJS = \ prop_antisemijoin.o \ prop_setops.o \ prop_setopstrat.o \ - prop_exceptintersect.o + prop_exceptintersect.o \ + prop_unionall.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 83aa29d4bb6f7..d9dc78fcdc0ef 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -218,6 +218,7 @@ prop_area_sources = files( 'prop_setops.c', 'prop_setopstrat.c', 'prop_exceptintersect.c', + 'prop_unionall.c', ) diff --git a/src/test/prop/prop_unionall.c b/src/test/prop/prop_unionall.c new file mode 100644 index 0000000000000..63db01485b42f --- /dev/null +++ b/src/test/prop/prop_unionall.c @@ -0,0 +1,357 @@ +/*------------------------------------------------------------------------- + * + * prop_unionall.c + * Property-based tests for UNION ALL / Append / MergeAppend semantics. + * + * UNION ALL is bag concatenation: it preserves every row and every row's + * multiplicity, and the planner implements it with an Append (or, when a + * common ORDER BY is pushed down, a MergeAppend that keeps the result + * globally sorted). These properties pin down the laws that distinguish + * UNION ALL from the deduplicating set operators: additive count and sum, + * doubled multiplicity for self-concatenation, non-decreasing output under a + * common ORDER BY, multiset commutativity, and the empty/self edge cases. A + * failure means Append/MergeAppend dropped, duplicated, reordered or mis- + * summed rows -- a wrong-results bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_unionall.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "unionall" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* Narrow value domain so duplicates collide and multiplicity matters. */ +#define VAL_MIN (-10) +#define VAL_MAX 10 + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the operation errored or produced NULL, which is itself a + * bug -- so either is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Run two result-set queries and require identical rows in identical order. + * Both queries MUST carry a total ORDER BY so the row-major string compares + * as a multiset. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* Draw a value column cell: SQL NULL ~15% of the time, else [VAL_MIN,VAL_MAX]. */ +static void +draw_val(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); +} + +/* + * Build ua_a(id int, v int) and ua_b(id int, v int), each with a random + * number of rows (0..20). id is a dense 1..n key per table; v is drawn from + * [VAL_MIN, VAL_MAX] with ~15% NULLs so both duplicate values and NULL + * handling are exercised. Rebuilt fresh in each property. + */ +static void +build_tables(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int na; + int nb; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS ua_a; " + "DROP TABLE IF EXISTS ua_b; " + "CREATE TABLE ua_a (id int, v int); " + "CREATE TABLE ua_b (id int, v int); "); + + na = (int) pg_prop_draw_int(tc, 0, 20); + for (int i = 0; i < na; i++) + { + StringInfoData vv; + + initStringInfo(&vv); + draw_val(tc, &vv); + appendStringInfo(&ddl, "INSERT INTO ua_a VALUES (%d, %s); ", i + 1, vv.data); + pfree(vv.data); + } + + nb = (int) pg_prop_draw_int(tc, 0, 20); + for (int i = 0; i < nb; i++) + { + StringInfoData vv; + + initStringInfo(&vv); + draw_val(tc, &vv); + appendStringInfo(&ddl, "INSERT INTO ua_b VALUES (%d, %s); ", i + 1, vv.data); + pfree(vv.data); + } + + appendStringInfoString(&ddl, "ANALYZE ua_a; ANALYZE ua_b; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + + pfree(ddl.data); +} + +/* Drop both tables so no state leaks between cases. */ +static void +drop_tables(pg_prop_tc *tc, pg_prop_conn *conn) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, + "DROP TABLE IF EXISTS ua_a; " + "DROP TABLE IF EXISTS ua_b; ", + &err)) + pg_prop_fail(tc, "table drop failed: %s", err ? err : "(unknown)"); +} + +/* + * (A) count(A UNION ALL B) = count(A) + count(B). UNION ALL concatenates + * bags, so the Append must emit exactly every row of both inputs. + */ +PG_PROP(union_all_count_additive) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "(SELECT v FROM ua_a UNION ALL SELECT v FROM ua_b) s) " + "= ((SELECT count(*) FROM ua_a) " + "+ (SELECT count(*) FROM ua_b))"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * (A) sum(v) over (A UNION ALL B) = sum(A.v) + sum(B.v). coalesce guards the + * empty / all-NULL case where sum() is SQL NULL; UNION ALL preserving every + * row means the summed total must be additive across the two inputs. + */ +PG_PROP(union_all_sum_additive) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT coalesce((SELECT sum(v) FROM " + "(SELECT v FROM ua_a UNION ALL SELECT v FROM ua_b) s), 0) " + "= (coalesce((SELECT sum(v) FROM ua_a), 0) " + "+ coalesce((SELECT sum(v) FROM ua_b), 0))"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * (B) A UNION ALL A doubles every value's multiplicity: grouping over + * (ua_a UNION ALL ua_a) must give exactly 2*count(*) per value compared with + * grouping over ua_a alone. ORDER BY v NULLS FIRST gives a total order over + * the one grouping column so the row strings compare as a multiset. + */ +PG_PROP(union_all_preserves_multiplicity) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT v, count(*) FROM " + "(SELECT v FROM ua_a UNION ALL SELECT v FROM ua_a) s " + "GROUP BY v ORDER BY v NULLS FIRST", + "SELECT v, 2*count(*) FROM ua_a " + "GROUP BY v ORDER BY v NULLS FIRST"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * (B->A) A UNION ALL with a common ORDER BY is planned as a MergeAppend and + * its output must be globally sorted. Checked soundly with a window lag(): + * for every row, the previous value (in NULLS LAST order) is NULL, is <= the + * current value, or the current value is NULL (NULLs sort last) -- i.e. the + * sequence is non-decreasing. bool_and over the whole stream, coalesced to + * true for the empty case. + */ +PG_PROP(mergeappend_ordered) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(" + "prev IS NULL OR prev <= v OR v IS NULL), true) FROM " + "(SELECT v, lag(v) OVER (ORDER BY v NULLS LAST) prev FROM " + "(SELECT v FROM ua_a UNION ALL SELECT v FROM ua_b) s) q"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * (B) Append commutes as a multiset: (A UNION ALL B) and (B UNION ALL A) hold + * the same values with the same multiplicities, even though the Append may + * emit them in a different physical order. Grouped by value with a total + * ORDER BY v NULLS FIRST so the comparison is order-insensitive on rows but + * exact on counts. + */ +PG_PROP(append_commutes_as_multiset) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_same_rows(tc, conn, + "SELECT v, count(*) FROM " + "(SELECT v FROM ua_a UNION ALL SELECT v FROM ua_b) s " + "GROUP BY v ORDER BY v NULLS FIRST", + "SELECT v, count(*) FROM " + "(SELECT v FROM ua_b UNION ALL SELECT v FROM ua_a) s " + "GROUP BY v ORDER BY v NULLS FIRST"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * (A) An empty left branch is the identity for UNION ALL: concatenating an + * empty relation with ua_b yields exactly count(ua_b) rows. Guards the + * Append's handling of a zero-row child. + */ +PG_PROP(empty_union_all) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "((SELECT v FROM ua_a WHERE false) " + "UNION ALL SELECT v FROM ua_b) s) " + "= (SELECT count(*) FROM ua_b)"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * (A) count(A UNION ALL A) = 2*count(A). Self-concatenation doubles the row + * count exactly; a MergeAppend/Append that deduplicated or dropped a child + * would break this. + */ +PG_PROP(union_all_self_double_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "(SELECT v FROM ua_a UNION ALL SELECT v FROM ua_a) s) " + "= (2 * (SELECT count(*) FROM ua_a))"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 7d6b56f8149ed..f57c03aa584b4 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 14765bf22cc7427fab609f2a3d91103d5b876e6f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:24 -0400 Subject: [PATCH 206/338] prop: add subquery pull-up and flattening equivalence property tests (subquery) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_subquery.c | 290 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 294 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_subquery.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index bebe554dc7f06..39621c3b942d5 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -221,7 +221,8 @@ OBJS = \ prop_setops.o \ prop_setopstrat.o \ prop_exceptintersect.o \ - prop_unionall.o + prop_unionall.o \ + prop_subquery.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index d9dc78fcdc0ef..385ee2959bdc5 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -219,6 +219,7 @@ prop_area_sources = files( 'prop_setopstrat.c', 'prop_exceptintersect.c', 'prop_unionall.c', + 'prop_subquery.c', ) diff --git a/src/test/prop/prop_subquery.c b/src/test/prop/prop_subquery.c new file mode 100644 index 0000000000000..814b0c581ced5 --- /dev/null +++ b/src/test/prop/prop_subquery.c @@ -0,0 +1,290 @@ +/*------------------------------------------------------------------------- + * + * prop_subquery.c + * Property-based tests for subquery flattening and pull-up equivalences. + * + * The planner rewrites subqueries in many ways -- an EXISTS into a semijoin, + * an IN into an EXISTS, a NOT EXISTS into an antijoin, a correlated scalar + * subquery into a grouped join, a derived table or CTE into an inline scan. + * Every such rewrite must be result-preserving: two semantically equivalent + * forms of the same query must return exactly the same rows. The oracle here + * is differential -- two hand-written equivalent query forms, compared with + * require_same_rows under a total ORDER BY -- so any disagreement is a + * flattening/pull-up bug. + * + * NULL semantics are the trap in this area. Property in_vs_exists restricts + * the IN subquery to non-NULL values so that IN == EXISTS stays sound (a NULL + * in an IN list changes three-valued-logic semantics and is a different, much + * trickier property, deliberately kept out). not_exists_vs_antijoin compares + * NOT EXISTS against the set complement via EXCEPT rather than a hand-rolled + * LEFT JOIN ... IS NULL antijoin, which is only equivalent when the join key + * never produces a spurious NULL match. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_subquery.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "subquery" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* Small integer domain for a and b; narrow so matches between rows are dense. */ +#define VAL_MIN (-4) +#define VAL_MAX 4 + +/* Draw an integer in [VAL_MIN, VAL_MAX], or SQL NULL (~15% of the time). */ +static void +draw_val(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); +} + +/* + * Rebuild sq_t fresh: id int PRIMARY KEY (0..n-1), a int, b int, both in + * [-4,4] with ~15% NULL. 0..25 rows. ANALYZE so the planner has stats to + * pick whichever flattening/pull-up strategy it likes -- that choice is + * exactly what the differential oracle is probing. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS sq_t; " + "CREATE TABLE sq_t (id int PRIMARY KEY, a int, b int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 25); + for (int r = 0; r < nrows; r++) + { + StringInfoData a; + StringInfoData b; + + initStringInfo(&a); + initStringInfo(&b); + draw_val(tc, &a); + draw_val(tc, &b); + appendStringInfo(&ddl, "INSERT INTO sq_t VALUES (%d, %s, %s); ", + r, a.data, b.data); + pfree(a.data); + pfree(b.data); + } + appendStringInfoString(&ddl, "ANALYZE sq_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + pfree(ddl.data); +} + +/* + * Differential oracle: run two equivalent query forms and require their full + * result sets (newline-joined, row-major) to be byte-identical. Both queries + * must carry a total ORDER BY so the comparison is a sound multiset compare. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * 1. EXISTS == semijoin. WHERE EXISTS (SELECT 1 ... u.a = t.b) selects each + * t exactly once when at least one matching u exists; the DISTINCT join is the + * explicit semijoin form. These must agree for every table. + */ +PG_PROP(exists_vs_semijoin) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id FROM sq_t t " + "WHERE EXISTS (SELECT 1 FROM sq_t u WHERE u.a = t.b) " + "ORDER BY id", + "SELECT DISTINCT t.id FROM sq_t t " + "JOIN sq_t u ON u.a = t.b " + "ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * 2. IN == EXISTS, over a non-NULL IN subquery. WHERE a IN (SELECT b ... b + * IS NOT NULL) is equivalent to the correlated EXISTS form only once the IN + * list has no NULL (a NULL in the list would turn a non-match into UNKNOWN + * rather than false), so both sides restrict the inner b to non-NULL. + */ +PG_PROP(in_vs_exists) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id FROM sq_t " + "WHERE a IN (SELECT b FROM sq_t WHERE b IS NOT NULL) " + "ORDER BY id", + "SELECT id FROM sq_t t " + "WHERE EXISTS (SELECT 1 FROM sq_t u " + "WHERE u.b = t.a AND u.b IS NOT NULL) " + "ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * 3. NOT EXISTS == set complement. Rather than a hand-rolled LEFT JOIN ... + * IS NULL antijoin (unsound when the key can match NULL), compare NOT EXISTS + * against the complement of the matching set computed with EXCEPT: the rows + * NOT selected by the positive EXISTS are exactly the rows selected by NOT + * EXISTS. Both are grounded in the same table so EXCEPT's set semantics are + * sound here. + */ +PG_PROP(not_exists_vs_antijoin) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id FROM sq_t t " + "WHERE NOT EXISTS (SELECT 1 FROM sq_t u WHERE u.a = t.b) " + "ORDER BY id", + "SELECT id FROM sq_t " + "EXCEPT " + "SELECT t.id FROM sq_t t " + "WHERE EXISTS (SELECT 1 FROM sq_t u WHERE u.a = t.b) " + "ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * 4. Correlated scalar subquery == grouped left join count. (SELECT count(*) + * ... u.a = t.a) per outer row equals count(u.*) from a LEFT JOIN grouped by + * t.id: the LEFT JOIN keeps every t, count(u.*) counts only matching u rows + * (0 when none), matching count(*) over the correlated set. Both order by id. + */ +PG_PROP(scalar_subquery_pullup) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id, (SELECT count(*) FROM sq_t u WHERE u.a = t.a) AS c " + "FROM sq_t t ORDER BY id", + "SELECT t.id, count(u.*) FROM sq_t t " + "LEFT JOIN sq_t u ON u.a = t.a " + "GROUP BY t.id ORDER BY t.id"); + + pg_prop_disconnect(conn); +} + +/* + * 5. Derived table flatten. A subquery in FROM with its own predicate is + * equivalent to the same scan with the predicates ANDed inline; the planner + * routinely flattens the former into the latter. + */ +PG_PROP(derived_table_flatten) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id FROM (SELECT id, a FROM sq_t WHERE a > 0) s " + "WHERE s.a < 4 ORDER BY id", + "SELECT id FROM sq_t WHERE a > 0 AND a < 4 ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * 6. CTE == inline subquery. A non-recursive, single-reference CTE is now + * inlinable; the result must match the equivalent inline form with the CTE's + * predicate ANDed into the base scan. + */ +PG_PROP(cte_vs_subquery) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "WITH w AS (SELECT id, a FROM sq_t WHERE a IS NOT NULL) " + "SELECT id FROM w WHERE a > 0 ORDER BY id", + "SELECT id FROM sq_t WHERE a IS NOT NULL AND a > 0 " + "ORDER BY id"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index f57c03aa584b4..1f3e39c1fab80 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 1642fc9aabd92a2dd29413752085c49319189d10 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:24 -0400 Subject: [PATCH 207/338] prop: add scalar subquery single-row and NULL semantics property tests (subqscalar) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_subqscalar.c | 310 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 314 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_subqscalar.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 39621c3b942d5..19158fd20e591 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -222,7 +222,8 @@ OBJS = \ prop_setopstrat.o \ prop_exceptintersect.o \ prop_unionall.o \ - prop_subquery.o + prop_subquery.o \ + prop_subqscalar.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 385ee2959bdc5..aa3959c29a456 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -220,6 +220,7 @@ prop_area_sources = files( 'prop_exceptintersect.c', 'prop_unionall.c', 'prop_subquery.c', + 'prop_subqscalar.c', ) diff --git a/src/test/prop/prop_subqscalar.c b/src/test/prop/prop_subqscalar.c new file mode 100644 index 0000000000000..045a357a5f9ad --- /dev/null +++ b/src/test/prop/prop_subqscalar.c @@ -0,0 +1,310 @@ +/*------------------------------------------------------------------------- + * + * prop_subqscalar.c + * Property-based tests for scalar-subquery semantics. + * + * A scalar subquery must yield exactly one value: at most one row (else a + * runtime error) and SQL NULL when no row is produced. An aggregate scalar + * subquery is the one shape that always returns a row even over zero input + * rows, so count(*) over an empty filter is 0, not NULL. These properties + * pin that single-row/NULL contract and the correlated-scalar-in-SELECT and + * scalar-in-WHERE forms; a disagreement is a scalar-subquery evaluation bug. + * Distinct from prop_subquery.c (flattening/pull-up) and prop_corrsubq.c + * (correlated decorrelation into grouped joins): here the focus is the scalar + * contract itself -- one value, NULL on no rows -- not the rewrite. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_subqscalar.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "subqscalar" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* Narrow domains: g in 0..3 keeps groups dense; v in -4..4 for aggregates. */ +#define V_MIN (-4) +#define V_MAX 4 + +/* Draw an integer in [min, max], or SQL NULL (~15% of the time). */ +static void +draw_opt(pg_prop_tc *tc, StringInfo buf, int min, int max) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, min, max)); +} + +/* + * Rebuild ss_t fresh: id int PRIMARY KEY (0..n-1), g int in 0..3 (~15% NULL), + * v int in -4..4 (~15% NULL). 0..25 rows. ANALYZE so the planner has stats + * to choose whichever scalar-subquery evaluation strategy it likes -- that + * choice is what the oracle probes. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS ss_t; " + "CREATE TABLE ss_t (id int PRIMARY KEY, g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 25); + for (int r = 0; r < nrows; r++) + { + StringInfoData g; + StringInfoData v; + + initStringInfo(&g); + initStringInfo(&v); + draw_opt(tc, &g, 0, 3); + draw_opt(tc, &v, V_MIN, V_MAX); + appendStringInfo(&ddl, "INSERT INTO ss_t VALUES (%d, %s, %s); ", + r, g.data, v.data); + pfree(g.data); + pfree(v.data); + } + appendStringInfoString(&ddl, "ANALYZE ss_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + pfree(ddl.data); +} + +/* + * Shape A: run a boolean-valued SELECT and require "t". A NULL result or a + * SQL error is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Shape B: run two equivalent query forms and require their full result sets + * (newline-joined, row-major) to be byte-identical. Both queries must carry + * a total ORDER BY so the comparison is a sound multiset compare. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * 1. A scalar subquery over no rows is SQL NULL. (SELECT v FROM ss_t WHERE + * false) produces zero rows, and a scalar subquery that yields no row must + * evaluate to NULL -- so the IS NULL test is true for every table. + */ +PG_PROP(scalar_empty_is_null) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT v FROM ss_t WHERE false) IS NULL"); + + pg_prop_disconnect(conn); +} + +/* + * 2. An aggregate scalar subquery always returns exactly one row, even over + * zero input rows -- count(*) over an empty filter is 0, not NULL. This is + * the single shape that distinguishes an aggregate scalar subquery from a + * row-driven one, so the result must equal 0, never be NULL. + */ +PG_PROP(scalar_agg_always_one_row) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM ss_t WHERE false) = 0"); + + pg_prop_disconnect(conn); +} + +/* + * 3. Correlated scalar max in SELECT == grouped LEFT JOIN max. Per-outer-row + * (SELECT max(u.v) ... WHERE u.g = t.g) equals the group's max from a LEFT + * JOIN on g. Both sides use = on g, so the NULL-g outer rows get no match on + * either side (= is UNKNOWN against NULL) and yield NULL consistently; the + * LEFT JOIN keeps every t. max over an empty/all-NULL group is NULL, matching + * the correlated form. Total order by id. + */ +PG_PROP(correlated_max_in_select) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id, (SELECT max(u.v) FROM ss_t u WHERE u.g = t.g) m " + "FROM ss_t t ORDER BY id", + "SELECT t.id, mx.m FROM ss_t t " + "LEFT JOIN (SELECT g, max(v) m FROM ss_t GROUP BY g) mx " + "ON mx.g = t.g " + "ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * 4. Scalar subquery in WHERE pulled into a derived column. The uncorrelated + * (SELECT avg(v) FROM ss_t) is one value for every outer row; comparing v to + * it in WHERE must equal first materializing it as a derived column a and then + * filtering v >= a. avg over an empty/all-NULL table is NULL, so v >= NULL is + * UNKNOWN on both sides and selects nothing -- consistent. Total order by id. + */ +PG_PROP(scalar_in_where) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id FROM ss_t WHERE v >= (SELECT avg(v) FROM ss_t) " + "ORDER BY id", + "SELECT id FROM " + "(SELECT id, v, (SELECT avg(v) FROM ss_t) a FROM ss_t) s " + "WHERE v >= a ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * 5. EXISTS == scalar count > 0. A row exists for g=0 exactly when the + * aggregate scalar count of such rows is positive. count(*) is never NULL, + * so the comparison is two-valued and the equality is total. + */ +PG_PROP(exists_vs_scalar_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT EXISTS (SELECT 1 FROM ss_t WHERE g = 0) " + "= ((SELECT count(*) FROM ss_t WHERE g = 0) > 0)"); + + pg_prop_disconnect(conn); +} + +/* + * 6. A constant scalar subquery is its constant. (SELECT 42) returns one row + * of one column, so it must evaluate to 42. + */ +PG_PROP(scalar_subquery_const) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, "SELECT (SELECT 42) = 42"); + + pg_prop_disconnect(conn); +} + +/* + * 7. Nested scalar subqueries collapse to the innermost constant. Three + * levels of (SELECT (SELECT ...)) each return one row of one column, so the + * whole thing must evaluate to 7. + */ +PG_PROP(nested_scalar) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, "SELECT (SELECT (SELECT (SELECT 7))) = 7"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 1f3e39c1fab80..a3610000be4da 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From c203db09cbc3a0bad2e21783bab5ad9114537a94 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:24 -0400 Subject: [PATCH 208/338] prop: add correlated-subquery decorrelation equivalence property tests (corrsubq) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_corrsubq.c | 269 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 273 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_corrsubq.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 19158fd20e591..5e4811d5be0be 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -223,7 +223,8 @@ OBJS = \ prop_exceptintersect.o \ prop_unionall.o \ prop_subquery.o \ - prop_subqscalar.o + prop_subqscalar.o \ + prop_corrsubq.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index aa3959c29a456..e7d745c4d4bd3 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -221,6 +221,7 @@ prop_area_sources = files( 'prop_unionall.c', 'prop_subquery.c', 'prop_subqscalar.c', + 'prop_corrsubq.c', ) diff --git a/src/test/prop/prop_corrsubq.c b/src/test/prop/prop_corrsubq.c new file mode 100644 index 0000000000000..38754c78e950c --- /dev/null +++ b/src/test/prop/prop_corrsubq.c @@ -0,0 +1,269 @@ +/*------------------------------------------------------------------------- + * + * prop_corrsubq.c + * Property-based tests for correlated-subquery decorrelation equivalences. + * + * A correlated subquery references the outer row inside its WHERE clause; the + * planner routinely decorrelates such a subquery into a grouped join (one + * aggregate per group, joined back on the correlation key). That rewrite must + * be result-preserving, so each property pairs the correlated form against the + * decorrelated grouped-join form and requires identical rows. Distinct from + * prop_subquery.c (uncorrelated EXISTS/IN/CTE flattening): here the inner query + * is correlated on a group key g, and the trap is the NULL group -- the join + * back onto g must be NULL-safe (IS NOT DISTINCT FROM) so the NULL group's + * aggregate reaches its outer rows just as the correlated scan would. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_corrsubq.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "corrsubq" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* Narrow domains: g in 0..3 makes groups dense; v in -4..4 for aggregates. */ +#define V_MIN (-4) +#define V_MAX 4 + +/* Draw an integer in [min, max], or SQL NULL (~15% of the time). */ +static void +draw_opt(pg_prop_tc *tc, StringInfo buf, int min, int max) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, min, max)); +} + +/* + * Rebuild cs_t fresh: id int PRIMARY KEY (0..n-1), g int in 0..3 (~15% NULL), + * v int in -4..4 (~15% NULL). 0..25 rows. ANALYZE so the planner has stats + * to choose whichever decorrelation strategy it likes -- that choice is what + * the differential oracle probes. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS cs_t; " + "CREATE TABLE cs_t (id int PRIMARY KEY, g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 25); + for (int r = 0; r < nrows; r++) + { + StringInfoData g; + StringInfoData v; + + initStringInfo(&g); + initStringInfo(&v); + draw_opt(tc, &g, 0, 3); + draw_opt(tc, &v, V_MIN, V_MAX); + appendStringInfo(&ddl, "INSERT INTO cs_t VALUES (%d, %s, %s); ", + r, g.data, v.data); + pfree(g.data); + pfree(v.data); + } + appendStringInfoString(&ddl, "ANALYZE cs_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + pfree(ddl.data); +} + +/* + * Differential oracle: run two equivalent query forms and require their full + * result sets (newline-joined, row-major) to be byte-identical. Both queries + * must carry a total ORDER BY so the comparison is a sound multiset compare. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * 1. Correlated count == grouped-join count. Per-outer-row count(*) of the + * same-group rows equals the group count joined back on g. count(*) is never + * NULL (0 for an empty group, but a correlated group always has >=1 member -- + * the outer row itself), and the join is NULL-safe so the NULL group's count + * reaches the NULL-g outer rows. + */ +PG_PROP(corr_count_eq_groupjoin) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id, (SELECT count(*) FROM cs_t u WHERE u.g IS NOT DISTINCT FROM t.g) AS c " + "FROM cs_t t ORDER BY id", + "SELECT t.id, gc.c FROM cs_t t " + "JOIN (SELECT g, count(*) AS c FROM cs_t GROUP BY g) gc " + "ON gc.g IS NOT DISTINCT FROM t.g " + "ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * 2. Correlated EXISTS == IN, restricted to non-NULL v on both sides. With + * t.v IS NOT NULL and the IN subquery filtered to non-NULL v, the three-valued + * UNKNOWN that a NULL would introduce is removed, so EXISTS(u.v = t.v) and + * t.v IN (SELECT v ...) coincide exactly. + */ +PG_PROP(corr_exists_eq_in) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id FROM cs_t t " + "WHERE t.v IS NOT NULL " + "AND EXISTS (SELECT 1 FROM cs_t u WHERE u.v = t.v) " + "ORDER BY id", + "SELECT id FROM cs_t t " + "WHERE t.v IS NOT NULL " + "AND t.v IN (SELECT v FROM cs_t WHERE v IS NOT NULL) " + "ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * 3. Correlated max == grouped-join max. Per-outer-row max(u.v) over the + * same group equals the group's max joined back on g. max(v) is SQL NULL when + * every v in the group is NULL -- the same value the correlated subquery + * yields -- so no coalesce is needed; the NULL-safe join carries the NULL + * group's max to the NULL-g outer rows. + */ +PG_PROP(corr_max_eq_join) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id, (SELECT max(u.v) FROM cs_t u WHERE u.g IS NOT DISTINCT FROM t.g) AS m " + "FROM cs_t t ORDER BY id", + "SELECT t.id, gm.m FROM cs_t t " + "JOIN (SELECT g, max(v) AS m FROM cs_t GROUP BY g) gm " + "ON gm.g IS NOT DISTINCT FROM t.g " + "ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * 4. Correlated NOT EXISTS == set complement via EXCEPT. Rather than a + * hand-rolled antijoin, the rows with NO same-group u having u.v > t.v are + * exactly the full id set minus the rows that DO have such a u. Both operands + * are grounded in cs_t, so EXCEPT's set semantics are sound; q2 is the EXCEPT + * form and must agree with the direct NOT EXISTS. + */ +PG_PROP(corr_not_exists_eq_except) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id FROM cs_t t " + "WHERE NOT EXISTS " + "(SELECT 1 FROM cs_t u WHERE u.g = t.g AND u.v > t.v) " + "ORDER BY id", + "SELECT id FROM cs_t " + "EXCEPT " + "SELECT t.id FROM cs_t t " + "WHERE EXISTS " + "(SELECT 1 FROM cs_t u WHERE u.g = t.g AND u.v > t.v) " + "ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * 5. Correlated sum with default == grouped LEFT JOIN sum with default. + * coalesce((SELECT sum(u.v) ...),0) per outer row equals coalesce(gs.s,0) over + * a LEFT JOIN to the per-group sum. sum(v) is NULL for an all-NULL group; + * coalesce folds both the all-NULL-group case and any unmatched LEFT-JOIN row + * to 0 identically. The NULL-safe join carries the NULL group's sum across. + */ +PG_PROP(scalar_subq_default) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id, coalesce((SELECT sum(u.v) FROM cs_t u WHERE u.g IS NOT DISTINCT FROM t.g), 0) AS s " + "FROM cs_t t ORDER BY id", + "SELECT t.id, coalesce(gs.s, 0) FROM cs_t t " + "LEFT JOIN (SELECT g, sum(v) AS s FROM cs_t GROUP BY g) gs " + "ON gs.g IS NOT DISTINCT FROM t.g " + "ORDER BY id"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index a3610000be4da..bfb8af5872596 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 4bdf3480d0f0dc26196e4bfd978774041ba20751 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:24 -0400 Subject: [PATCH 209/338] prop: add CTE inlining, materialization, and recursion property tests (cte) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_cte.c | 333 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 337 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_cte.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 5e4811d5be0be..dddaada803bea 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -224,7 +224,8 @@ OBJS = \ prop_unionall.o \ prop_subquery.o \ prop_subqscalar.o \ - prop_corrsubq.o + prop_corrsubq.o \ + prop_cte.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index e7d745c4d4bd3..f559d95478914 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -222,6 +222,7 @@ prop_area_sources = files( 'prop_subquery.c', 'prop_subqscalar.c', 'prop_corrsubq.c', + 'prop_cte.c', ) diff --git a/src/test/prop/prop_cte.c b/src/test/prop/prop_cte.c new file mode 100644 index 0000000000000..f9a85bdd2e782 --- /dev/null +++ b/src/test/prop/prop_cte.c @@ -0,0 +1,333 @@ +/*------------------------------------------------------------------------- + * + * prop_cte.c + * Property-based tests for common table expressions (WITH / WITH RECURSIVE). + * + * The oracle is differential: a CTE is a syntactic convenience and a + * materialisation boundary, never a change in results. An inlinable CTE must + * return the same rows as the same CTE forced MATERIALIZED and as the + * equivalent subquery; a recursive CTE walking a forest (parent is always a + * strictly smaller id, or NULL for a root, so there are no cycles and the walk + * always terminates) must visit exactly every node, each once. Any + * disagreement is a planner/executor bug in CTE handling, not a legal + * difference. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_cte.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "cte" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Differential result-set oracle: q1 and q2 must return identical rows in + * identical order. Both queries must carry a TOTAL ORDER BY (here on id, the + * primary key) so the row-major string compares as a multiset. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Scalar boolean oracle: run a "SELECT " and require "t". A NULL result + * or a SQL error is itself a failure. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build ct_t(id int PRIMARY KEY, parent int, v int) with 0..20 rows forming a + * forest: ids are 1..n assigned densely, and each row's parent is either NULL + * (a root) or a strictly smaller id. parent < id guarantees acyclicity, so a + * recursive walk from the roots always terminates and reaches every node. + * + * We draw parent in [0, id-1]; 0 encodes NULL (a root). v is a small int in + * [-5, 5]. ANALYZE so the planner has stats. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS ct_t; " + "CREATE TABLE ct_t (id int PRIMARY KEY, parent int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 20); + for (int id = 1; id <= nrows; id++) + { + int p = (int) pg_prop_draw_int(tc, 0, id - 1); + int v = (int) pg_prop_draw_int(tc, -5, 5); + + if (p == 0) + appendStringInfo(&ddl, + "INSERT INTO ct_t VALUES (%d, NULL, %d); ", id, v); + else + appendStringInfo(&ddl, + "INSERT INTO ct_t VALUES (%d, %d, %d); ", id, p, v); + } + + appendStringInfoString(&ddl, "ANALYZE ct_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n SQL: %s", + err ? err : "", ddl.data); + pfree(ddl.data); +} + +/* + * A non-recursive CTE returns the same rows whether the planner inlines it or + * is forced to materialise it. MATERIALIZED is a hint that changes execution + * but never results. + */ +PG_PROP(inline_vs_materialized) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "WITH w AS (SELECT id, v FROM ct_t WHERE v > 0) " + "SELECT id FROM w ORDER BY id", + "WITH w AS MATERIALIZED (SELECT id, v FROM ct_t WHERE v > 0) " + "SELECT id FROM w ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * A simple CTE and the equivalent FROM-subquery must agree. Inlining a + * single-reference CTE is exactly this rewrite, so any difference is a bug. + */ +PG_PROP(cte_vs_subquery) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "WITH w AS (SELECT id FROM ct_t WHERE v <= 0) " + "SELECT id FROM w ORDER BY id", + "SELECT id FROM (SELECT id FROM ct_t WHERE v <= 0) s " + "ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * A recursive CTE walking children down from the roots reaches exactly every + * node. Because every row's parent is a strictly smaller id (or NULL), every + * node has a root ancestor, so the full table is reachable. + */ +PG_PROP(recursive_reaches_all_descendants) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "WITH RECURSIVE r AS (" + "SELECT id FROM ct_t WHERE parent IS NULL " + "UNION ALL " + "SELECT c.id FROM ct_t c JOIN r ON c.parent = r.id) " + "SELECT id FROM r ORDER BY id", + "SELECT id FROM ct_t ORDER BY id"); + + pg_prop_disconnect(conn); +} + +/* + * Every node's computed depth via a recursive CTE is >= 1, and a root (the + * base case) has depth 1. coalesce(...,true) makes the empty-forest case + * (bool_and over 0 rows is NULL) pass soundly. + */ +PG_PROP(recursive_depth_nonneg) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(depth >= 1), true) FROM (" + "WITH RECURSIVE r AS (" + "SELECT id, 1 AS depth FROM ct_t WHERE parent IS NULL " + "UNION ALL " + "SELECT c.id, r.depth + 1 FROM ct_t c JOIN r ON c.parent = r.id) " + "SELECT depth FROM r) s"); + + pg_prop_disconnect(conn); +} + +/* + * The recursive walk visits each node exactly once. UNION ALL (not UNION) + * keeps every emitted row, so a duplicate visit would make the walk's count + * exceed the table's count -- the equality catches it. Since it's a forest, + * each node has a unique parent chain to one root, so the counts must match. + */ +PG_PROP(recursive_count_matches) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM ct_t) = " + "(SELECT count(*) FROM (" + "WITH RECURSIVE r AS (" + "SELECT id FROM ct_t WHERE parent IS NULL " + "UNION ALL " + "SELECT c.id FROM ct_t c JOIN r ON c.parent = r.id) " + "SELECT id FROM r) q)"); + + pg_prop_disconnect(conn); +} + +/* + * A data-modifying CTE modifies its target exactly once: the DELETE ... CTE's + * RETURNING row count must equal the number of rows that matched before the + * delete. We capture the pre-count on a scratch copy, run the DELETE-CTE, and + * compare. This exercises the "modify once, snapshot the target once" + * semantics of data-modifying WITH. + */ +PG_PROP(data_modifying_cte_returning) +{ + pg_prop_conn *conn; + char *err = NULL; + char *expected; + bool failed = false; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + if (!pg_prop_sql_exec(conn, + "DROP TABLE IF EXISTS ct_copy; " + "CREATE TABLE ct_copy AS SELECT * FROM ct_t;", + &err)) + pg_prop_fail(tc, "ct_copy build failed: %s", err ? err : ""); + + expected = pg_prop_sql_scalar(conn, + "SELECT count(*) FROM ct_copy WHERE v > 0", + &failed, &err); + if (failed || expected == NULL) + pg_prop_fail(tc, "pre-count failed: %s", err ? err : "(null)"); + + initStringInfo(&q); + appendStringInfoString(&q, + "WITH d AS (DELETE FROM ct_copy WHERE v > 0 " + "RETURNING id) SELECT count(*) FROM d"); + + { + char *got; + bool gf = false; + char *ge = NULL; + + /* + * A data-modifying CTE must be the top-level statement, so run it + * directly and compare its RETURNING count in C to the pre-count. + */ + got = pg_prop_sql_scalar(conn, q.data, &gf, &ge); + if (gf || got == NULL) + pg_prop_fail(tc, "data-modifying CTE failed: %s", + ge ? ge : "(null)"); + if (strcmp(got, expected) != 0) + pg_prop_fail(tc, + "RETURNING count %s != pre-count %s of v>0 rows", + got, expected); + pg_free(got); + if (ge) + pg_free(ge); + } + + pfree(q.data); + pg_free(expected); + if (err) + pg_free(err); + + if (!pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS ct_copy;", &err)) + pg_prop_fail(tc, "ct_copy drop failed: %s", err ? err : ""); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index bfb8af5872596..03e6cbf253dfd 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 36c8c9090b3d318eda24d5dbaec0e4ca987cd8b4 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:25 -0400 Subject: [PATCH 210/338] prop: add recursive CTE forest traversal and closure property tests (recursivecte) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_recursivecte.c | 368 ++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 372 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_recursivecte.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index dddaada803bea..9c8a6bf1d22b5 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -225,7 +225,8 @@ OBJS = \ prop_subquery.o \ prop_subqscalar.o \ prop_corrsubq.o \ - prop_cte.o + prop_cte.o \ + prop_recursivecte.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index f559d95478914..7ac64492ec446 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -223,6 +223,7 @@ prop_area_sources = files( 'prop_subqscalar.c', 'prop_corrsubq.c', 'prop_cte.c', + 'prop_recursivecte.c', ) diff --git a/src/test/prop/prop_recursivecte.c b/src/test/prop/prop_recursivecte.c new file mode 100644 index 0000000000000..cc2bb1457a0c0 --- /dev/null +++ b/src/test/prop/prop_recursivecte.c @@ -0,0 +1,368 @@ +/*------------------------------------------------------------------------- + * + * prop_recursivecte.c + * Property-based tests for WITH RECURSIVE traversal semantics. + * + * Distinct from the "cte" area (which checks inlining/materialisation and a + * single descendant walk): this area targets the recursive-worktable engine + * itself -- UNION (dedup) vs UNION ALL, depth-limited recursion, and + * transitive closure. On a FOREST (parent is NULL or a strictly smaller id, + * so no cycles) every recursion must terminate and the dedup/non-dedup and + * capped/uncapped variants must agree. Any disagreement, non-termination, or + * missing edge is a bug in recursive-CTE execution, not a legal difference. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_recursivecte.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "recursivecte" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Differential result-set oracle: q1 and q2 must return identical rows in + * identical order. Both queries must carry a TOTAL ORDER BY so the row-major + * string compares as a multiset. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Scalar boolean oracle: run a "SELECT " and require "t". A NULL result + * or a SQL error is itself a failure. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build rc_t(id int PRIMARY KEY, parent int) with 0..20 rows forming a FOREST: + * ids are 1..n assigned densely, and each row's parent is either NULL (a root) + * or a strictly smaller id. parent < id guarantees acyclicity, so every + * recursive walk terminates and the depth of any node is bounded by the row + * count. + * + * We draw parent in [0, id-1]; 0 encodes NULL (a root). ANALYZE so the + * planner has stats. + */ +static void +build_forest(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS rc_t; " + "CREATE TABLE rc_t (id int PRIMARY KEY, parent int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 20); + for (int id = 1; id <= nrows; id++) + { + int p = (int) pg_prop_draw_int(tc, 0, id - 1); + + if (p == 0) + appendStringInfo(&ddl, + "INSERT INTO rc_t VALUES (%d, NULL); ", id); + else + appendStringInfo(&ddl, + "INSERT INTO rc_t VALUES (%d, %d); ", id, p); + } + + appendStringInfoString(&ddl, "ANALYZE rc_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "forest build failed: %s\n SQL: %s", + err ? err : "", ddl.data); + pfree(ddl.data); +} + +/* + * Property 1 (A): on a forest, UNION recursion from the roots reaches every + * node. Every id is connected to some root (follow parent links down to a + * NULL), so the reachable set is all ids; its count equals the total row + * count. A dedup bug or a worktable-swap bug would drop or duplicate nodes. + */ +PG_PROP(union_terminates_on_forest) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_forest(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT (SELECT count(*) FROM (" + " WITH RECURSIVE r AS (" + " SELECT id FROM rc_t WHERE parent IS NULL" + " UNION" + " SELECT c.id FROM rc_t c JOIN r ON c.parent = r.id" + " ) SELECT id FROM r" + ") s) = (SELECT count(*) FROM rc_t)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 2 (B): on a forest (no cycles, each node a single parent), UNION + * (dedup) and UNION ALL produce the same set of ids. With no shared + * descendants reached by two paths and no cycles, dedup removes nothing, so + * the two id multisets are identical. A divergence means UNION ALL + * re-enqueued a node it should not have, or UNION dropped one it should keep. + */ +PG_PROP(union_all_same_as_union_on_tree) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q1, + q2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_forest(tc, conn); + + initStringInfo(&q1); + appendStringInfoString(&q1, + "SELECT id FROM (" + " WITH RECURSIVE r AS (" + " SELECT id FROM rc_t WHERE parent IS NULL" + " UNION" + " SELECT c.id FROM rc_t c JOIN r ON c.parent = r.id" + " ) SELECT id FROM r" + ") s ORDER BY id"); + + initStringInfo(&q2); + appendStringInfoString(&q2, + "SELECT id FROM (" + " WITH RECURSIVE r AS (" + " SELECT id FROM rc_t WHERE parent IS NULL" + " UNION ALL" + " SELECT c.id FROM rc_t c JOIN r ON c.parent = r.id" + " ) SELECT id FROM r" + ") s ORDER BY id"); + + require_same_rows(tc, conn, q1.data, q2.data); + + pfree(q1.data); + pfree(q2.data); + pg_prop_disconnect(conn); +} + +/* + * Property 3 (A): walking from each node up to its root, the depth computed by + * the recursion is 1 + number of proper ancestors, hence always >= 1 for + * every node. bool_and over all nodes must be true; coalesce(...,true) keeps + * the empty forest sound (no rows -> vacuously true). A depth-column bug in + * the recursive term would make some node report depth < 1. + */ +PG_PROP(depth_matches_ancestors) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_forest(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT coalesce(bool_and(depth >= 1), true) FROM (" + " WITH RECURSIVE r AS (" + " SELECT id, parent, 1 AS depth FROM rc_t WHERE parent IS NULL" + " UNION ALL" + " SELECT c.id, c.parent, r.depth + 1" + " FROM rc_t c JOIN r ON c.parent = r.id" + " ) SELECT depth FROM r" + ") s"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 4 (A): the transitive closure of (ancestor, descendant) pairs + * contains every direct parent-child edge. Build the closure recursively from + * the direct edges (parent IS NOT NULL) and chain through, then assert the + * closure's pair count is at least the number of direct edges -- every direct + * edge is itself an (ancestor, descendant) pair in the closure, so the count + * can only be greater or equal. A missing-base-row or missing-chaining bug + * would drop pairs and break the bound. + */ +PG_PROP(transitive_closure_contains_edges) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_forest(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT (SELECT count(*) FROM (" + " WITH RECURSIVE closure AS (" + " SELECT parent AS ancestor, id AS descendant" + " FROM rc_t WHERE parent IS NOT NULL" + " UNION" + " SELECT cl.ancestor, c.id" + " FROM rc_t c JOIN closure cl ON c.parent = cl.descendant" + " ) SELECT ancestor, descendant FROM closure" + ") cc) >= (SELECT count(*) FROM rc_t WHERE parent IS NOT NULL)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Property 5 (A): a depth-capped recursion (WHERE depth < 100) returns the + * same node set as the uncapped recursion. On a forest of at most 20 rows the + * depth of any node is bounded by the row count (<< 100), so the cap never + * fires and both must enumerate exactly the same ids. Comparison is done in + * C: both counts of distinct ids must be equal, and both must equal the full + * row count (the cap must not hide anyone). A premature-cutoff bug would make + * the capped count smaller. + */ +PG_PROP(cycle_guard_terminates) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData capped, + uncapped; + char *rc, + *ru, + *rtot; + bool fc = false, + fu = false, + ft = false; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_forest(tc, conn); + + initStringInfo(&capped); + appendStringInfoString(&capped, + "SELECT count(DISTINCT id) FROM (" + " WITH RECURSIVE r AS (" + " SELECT id, 1 AS depth FROM rc_t WHERE parent IS NULL" + " UNION ALL" + " SELECT c.id, r.depth + 1" + " FROM rc_t c JOIN r ON c.parent = r.id" + " WHERE r.depth < 100" + " ) SELECT id FROM r" + ") s"); + + initStringInfo(&uncapped); + appendStringInfoString(&uncapped, + "SELECT count(DISTINCT id) FROM (" + " WITH RECURSIVE r AS (" + " SELECT id FROM rc_t WHERE parent IS NULL" + " UNION" + " SELECT c.id FROM rc_t c JOIN r ON c.parent = r.id" + " ) SELECT id FROM r" + ") s"); + + rc = pg_prop_sql_scalar(conn, capped.data, &fc, &err); + if (fc) + pg_prop_fail(tc, "capped query failed: %s\n SQL: %s", + err ? err : "", capped.data); + ru = pg_prop_sql_scalar(conn, uncapped.data, &fu, &err); + if (fu) + pg_prop_fail(tc, "uncapped query failed: %s\n SQL: %s", + err ? err : "", uncapped.data); + rtot = pg_prop_sql_scalar(conn, "SELECT count(*) FROM rc_t", &ft, &err); + if (ft) + pg_prop_fail(tc, "row-count query failed: %s", err ? err : ""); + + if (rc == NULL || ru == NULL || rtot == NULL) + pg_prop_fail(tc, "unexpected NULL count (capped=%s uncapped=%s total=%s)", + rc ? rc : "NULL", ru ? ru : "NULL", rtot ? rtot : "NULL"); + + if (strcmp(rc, ru) != 0) + pg_prop_fail(tc, + "depth cap changed the node set: capped=%s uncapped=%s", + rc, ru); + if (strcmp(rc, rtot) != 0) + pg_prop_fail(tc, + "recursion did not reach every node: reached=%s total=%s", + rc, rtot); + + pg_free(rc); + pg_free(ru); + pg_free(rtot); + pfree(capped.data); + pfree(uncapped.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 03e6cbf253dfd..beb29b29e1978 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 071b1f69e3b5b542ac2fe99c51fdd280838b8243 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:25 -0400 Subject: [PATCH 211/338] prop: add data-modifying CTE snapshot and chaining property tests (dmlcte) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_dmlcte.c | 329 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 333 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_dmlcte.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 9c8a6bf1d22b5..cb180eaa186fe 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -226,7 +226,8 @@ OBJS = \ prop_subqscalar.o \ prop_corrsubq.o \ prop_cte.o \ - prop_recursivecte.o + prop_recursivecte.o \ + prop_dmlcte.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 7ac64492ec446..c8c90d8869024 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -224,6 +224,7 @@ prop_area_sources = files( 'prop_corrsubq.c', 'prop_cte.c', 'prop_recursivecte.c', + 'prop_dmlcte.c', ) diff --git a/src/test/prop/prop_dmlcte.c b/src/test/prop/prop_dmlcte.c new file mode 100644 index 0000000000000..0723ea98ef287 --- /dev/null +++ b/src/test/prop/prop_dmlcte.c @@ -0,0 +1,329 @@ +/*------------------------------------------------------------------------- + * + * prop_dmlcte.c + * Property-based tests for data-modifying CTE semantics. + * + * A WITH clause may contain INSERT/UPDATE/DELETE ... RETURNING subqueries + * whose effects feed the rest of the statement. Two things make these + * distinct from plain CTEs and plain RETURNING: all data-modifying CTEs in one + * statement run against the single snapshot taken before the statement began, + * so a sibling read of the modified table does NOT see the modifications; and + * one data-modifying CTE's RETURNING rows can drive another (chained + * INSERT ... SELECT FROM ). The oracle is the statement's own row + * accounting -- counts compared in C against independently computed + * pre-statement counts -- plus the snapshot rule checked directly. Any + * disagreement is a bug in the executor's ModifyTable / CTE handling, not a + * legal difference. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_dmlcte.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "dmlcte" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a scalar query expected to yield a non-NULL integer, and return it as a + * long long. A SQL error or a NULL result is a failure (every count query + * here is count()/min() guarded to be non-NULL). (Copied from + * prop_returning.c.) + */ +static long long +scalar_int(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + long long val; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + val = atoll(res); + pg_free(res); + if (err) + pg_free(err); + return val; +} + +/* + * Run a side-effecting statement (CREATE/DROP/DML) and fail on error. + */ +static void +exec_ok(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "exec failed: %s\n SQL: %s", err ? err : "", sql); + if (err) + pg_free(err); +} + +/* + * Build dc_t(id int PRIMARY KEY, v int) with 0..20 rows: ids are 1..n assigned + * densely so the id+1000 / id+2000 offsets used by the insert CTEs never + * collide with them, and v is a small int in [-5, 5] so the v < 0 and v >= 0 + * filters each land on a mix of rows and the empty boundary. ANALYZE so the + * planner has stats. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS dc_t; " + "CREATE TABLE dc_t (id int PRIMARY KEY, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 20); + for (int id = 1; id <= nrows; id++) + { + int v = (int) pg_prop_draw_int(tc, -5, 5); + + appendStringInfo(&ddl, "INSERT INTO dc_t VALUES (%d, %d); ", id, v); + } + + appendStringInfoString(&ddl, "ANALYZE dc_t;"); + + exec_ok(tc, conn, ddl.data); + pfree(ddl.data); +} + +/* + * Create a fresh scratch copy dc_copy of dc_t. CREATE TABLE AS has no primary + * key, so add one explicitly; none of these properties strictly need it, but + * it mirrors dc_t and makes an accidental duplicate id an error rather than + * silent corruption of the count oracle. + */ +static void +make_scratch(pg_prop_tc *tc, pg_prop_conn *conn) +{ + exec_ok(tc, conn, + "DROP TABLE IF EXISTS dc_copy; " + "CREATE TABLE dc_copy AS SELECT * FROM dc_t; " + "ALTER TABLE dc_copy ADD PRIMARY KEY (id);"); +} + +static void +drop_scratch(pg_prop_tc *tc, pg_prop_conn *conn) +{ + exec_ok(tc, conn, "DROP TABLE dc_copy;"); +} + +/* + * 1. Two chained data-modifying CTEs: a DELETE feeds an INSERT. The DELETE + * removes every v < 0 row and RETURNING its ids; the INSERT consumes those ids + * (offset by 1000 so they cannot collide with the surviving rows) and + * RETURNING its own ids; the outer SELECT counts the insert's output. The + * executor must insert exactly as many rows as the delete removed, so the + * insert count equals the pre-statement count of v < 0 rows, verified in C. + */ +PG_PROP(two_modifying_ctes) +{ + pg_prop_conn *conn; + char *err = NULL; + long long pre_neg; + long long inserted; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + make_scratch(tc, conn); + + pre_neg = scalar_int(tc, conn, "SELECT count(*) FROM dc_copy WHERE v < 0"); + + inserted = scalar_int(tc, conn, + "WITH d AS (" + "DELETE FROM dc_copy WHERE v < 0 RETURNING id), " + "i AS (" + "INSERT INTO dc_copy SELECT id + 1000, 0 FROM d " + "RETURNING id) " + "SELECT count(*) FROM i"); + + if (inserted != pre_neg) + pg_prop_fail(tc, + "chained INSERT count %lld != DELETE count (v<0) %lld", + inserted, pre_neg); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 2. A data-modifying CTE runs against the pre-statement snapshot, so a + * sibling read in the same statement does NOT see its effects. The DELETE + * empties dc_copy, but the outer SELECT count(*) FROM dc_copy -- reading the + * same snapshot the statement started with -- still sees every original row. + * So the statement result equals the original count (verified in C), and only + * a FRESH statement afterward sees the table emptied (0 rows). + */ +PG_PROP(cte_sees_snapshot) +{ + pg_prop_conn *conn; + char *err = NULL; + long long orig; + long long seen; + long long after; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + make_scratch(tc, conn); + + orig = scalar_int(tc, conn, "SELECT count(*) FROM dc_copy"); + + /* Outer read runs on the pre-DELETE snapshot: it still sees orig rows. */ + seen = scalar_int(tc, conn, + "WITH d AS (DELETE FROM dc_copy RETURNING id) " + "SELECT count(*) FROM dc_copy"); + if (seen != orig) + pg_prop_fail(tc, + "snapshot read saw %lld rows, expected pre-DELETE %lld", + seen, orig); + + /* A fresh statement sees the committed effect: the table is empty. */ + after = scalar_int(tc, conn, "SELECT count(*) FROM dc_copy"); + if (after != 0) + pg_prop_fail(tc, "after the WITH, table has %lld rows, expected 0", + after); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 3. UPDATE ... RETURNING in a CTE returns the NEW column values. The UPDATE + * adds 100 to v on every v >= 0 row and RETURNING v; those returned values are + * the post-update values, so each is >= 100. Checked as min(v) >= 100 over + * the CTE output, but only when at least one v >= 0 row existed -- otherwise + * the UPDATE touches no rows, the CTE is empty and min() is NULL, which is not + * a counterexample. + */ +PG_PROP(update_returning_new) +{ + pg_prop_conn *conn; + char *err = NULL; + long long pre_nonneg; + long long minv; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + make_scratch(tc, conn); + + pre_nonneg = scalar_int(tc, conn, + "SELECT count(*) FROM dc_copy WHERE v >= 0"); + pg_prop_assume(tc, pre_nonneg > 0); + + minv = scalar_int(tc, conn, + "WITH u AS (" + "UPDATE dc_copy SET v = v + 100 WHERE v >= 0 " + "RETURNING v) " + "SELECT min(v) FROM u"); + + if (minv < 100) + pg_prop_fail(tc, + "UPDATE RETURNING min(v) %lld < 100 (not the new values)", + minv); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 4. INSERT ... SELECT ... RETURNING in a CTE emits one row per inserted row. + * The INSERT copies every dc_t row (ids offset by 2000 so they cannot collide + * with dc_copy's 1..20 ids) and RETURNING a constant; the outer SELECT counts + * that output. The count must equal the number of source rows, dc_t's row + * count, verified in C. + */ +PG_PROP(insert_select_count) +{ + pg_prop_conn *conn; + char *err = NULL; + long long src; + long long inserted; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + make_scratch(tc, conn); + + src = scalar_int(tc, conn, "SELECT count(*) FROM dc_t"); + + inserted = scalar_int(tc, conn, + "WITH i AS (" + "INSERT INTO dc_copy SELECT id + 2000, v FROM dc_t " + "RETURNING 1) " + "SELECT count(*) FROM i"); + + if (inserted != src) + pg_prop_fail(tc, "INSERT SELECT count %lld != source rows %lld", + inserted, src); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 5. DELETE ... RETURNING in a CTE returns one row per deleted row, and the + * delete removes every row. The outer SELECT counts the CTE output: it must + * equal the original row count (verified in C), and a fresh read afterward + * must see the table empty (also verified in C). + */ +PG_PROP(delete_all_returning) +{ + pg_prop_conn *conn; + char *err = NULL; + long long orig; + long long deleted; + long long after; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + make_scratch(tc, conn); + + orig = scalar_int(tc, conn, "SELECT count(*) FROM dc_copy"); + + deleted = scalar_int(tc, conn, + "WITH d AS (DELETE FROM dc_copy RETURNING id) " + "SELECT count(*) FROM d"); + if (deleted != orig) + pg_prop_fail(tc, "DELETE RETURNING count %lld != original rows %lld", + deleted, orig); + + after = scalar_int(tc, conn, "SELECT count(*) FROM dc_copy"); + if (after != 0) + pg_prop_fail(tc, "after DELETE, table has %lld rows, expected 0", + after); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index beb29b29e1978..99085fb0d5d15 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 0978a59165570b7d4da17859607fe601c510249a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:25 -0400 Subject: [PATCH 212/338] prop: add LATERAL join and set-returning-function property tests (lateral) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_lateral.c | 309 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 313 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_lateral.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index cb180eaa186fe..5d5d0cbf386d3 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -227,7 +227,8 @@ OBJS = \ prop_corrsubq.o \ prop_cte.o \ prop_recursivecte.o \ - prop_dmlcte.o + prop_dmlcte.o \ + prop_lateral.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index c8c90d8869024..5cbf5aa719860 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -225,6 +225,7 @@ prop_area_sources = files( 'prop_cte.c', 'prop_recursivecte.c', 'prop_dmlcte.c', + 'prop_lateral.c', ) diff --git a/src/test/prop/prop_lateral.c b/src/test/prop/prop_lateral.c new file mode 100644 index 0000000000000..72c599e48a08e --- /dev/null +++ b/src/test/prop/prop_lateral.c @@ -0,0 +1,309 @@ +/*------------------------------------------------------------------------- + * + * prop_lateral.c + * Property-based tests for LATERAL and set-returning-function joins. + * + * A LATERAL join (and an implicitly-lateral SRF-in-FROM, or an SRF in the + * target list) always has an equivalent non-lateral rewriting: a correlated + * aggregate equals a grouped join, LATERAL unnest equals a CROSS JOIN unnest, + * a LATERAL top-1 equals a correlated scalar subquery, VALUES equals a UNION + * ALL of singletons. The planner is free to rewrite LATERAL into a nestloop + * or a hashed form; if any rewrite changes the rows returned, that is a bug. + * + * The oracle is differential (require_same_rows): the lateral form and its + * hand-written non-lateral equivalent must return identical row multisets. + * Both sides carry a total ORDER BY so the row-major render compares exactly. + * The comma-LATERAL "drop the outer row when the lateral is empty" semantics + * are handled by using the aggregate-subquery / EXISTS-restricted forms where + * needed, so the two sides are genuinely equal on every generated table. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_lateral.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "lateral" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small int domain for a,b: 0..4 with ~15% NULL. Narrow enough that many + * rows collide on "a" (so the correlated/grouped forms actually have groups + * with several members) and that generate_series(1, greatest(a,0)) stays tiny. + */ +#define VAL_MIN 0 +#define VAL_MAX 4 + +/* Append an int in [VAL_MIN,VAL_MAX], or NULL ~15% of the time. */ +static void +draw_val(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); +} + +/* + * Rebuild lt_t(id int PRIMARY KEY, a int, b int) fresh with 0..20 rows. id is + * a dense 0..n-1 sequence so it is a unique total-order key; a,b are drawn by + * draw_val. pg_prop_fail on any DDL/DML error (the schema is fixed and legal, + * so a failure here is a harness/server problem, not a discardable case). + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS lt_t; " + "CREATE TABLE lt_t (id int PRIMARY KEY, a int, b int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 20); + for (int r = 0; r < nrows; r++) + { + appendStringInfo(&ddl, "INSERT INTO lt_t VALUES (%d, ", r); + draw_val(tc, &ddl); + appendStringInfoString(&ddl, ", "); + draw_val(tc, &ddl); + appendStringInfoString(&ddl, "); "); + } + appendStringInfoString(&ddl, "ANALYZE lt_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(no message)", ddl.data); + pfree(ddl.data); +} + +/* + * Differential oracle: q1 and q2 must return identical row multisets (both + * carry a total ORDER BY, so the row-major strings compare directly). A SQL + * error on either side, or a mismatch, is a failure printed with both queries. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Scalar boolean oracle (shape A): run a "SELECT " and require "t". A + * NULL result or a SQL error is a failure. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); +} + +/* + * Property 1: a correlated LATERAL aggregate equals a grouped left join. + * + * A comma-LATERAL with a correlated aggregate ("SELECT max(u.b) ... WHERE + * u.a = t.a") always produces exactly one row per outer row -- max over an + * empty set is SQL NULL, but it is still one row -- so the outer row is never + * dropped. The grouped form "GROUP BY a" with a LEFT JOIN therefore matches + * it row for row (unmatched outer rows get a NULL m from the LEFT JOIN). + */ +PG_PROP(lateral_subquery_eq_join) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT t.id, s.m FROM lt_t t, " + "LATERAL (SELECT max(u.b) m FROM lt_t u WHERE u.a = t.a) s " + "ORDER BY t.id", + "SELECT t.id, mx.m FROM lt_t t " + "LEFT JOIN (SELECT a, max(b) m FROM lt_t GROUP BY a) mx " + "ON mx.a = t.a ORDER BY t.id"); + + pg_prop_disconnect(conn); +} + +/* + * Property 2: LATERAL unnest equals CROSS JOIN unnest. An SRF in FROM is + * implicitly lateral, so both expand ARRAY[t.a, t.b] per outer row into the + * same two rows. NULLs in the array are preserved; order by t.id then e with + * NULLS LAST so the total order is deterministic on both sides. + */ +PG_PROP(lateral_unnest_eq_unnest) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT t.id, e FROM lt_t t, " + "LATERAL unnest(ARRAY[t.a, t.b]) e " + "ORDER BY t.id, e NULLS LAST", + "SELECT t.id, e FROM lt_t t " + "CROSS JOIN unnest(ARRAY[t.a, t.b]) e " + "ORDER BY t.id, e NULLS LAST"); + + pg_prop_disconnect(conn); +} + +/* + * Property 3: a LATERAL top-1 equals a correlated scalar subquery. + * + * Comma (CROSS) LATERAL drops the outer row when the lateral yields no rows, + * so q1 keeps only rows whose "a" has at least one match. The scalar-subquery + * form keeps every outer row (with NULL where there is no match), so q2 is + * restricted to EXISTS matches to make the two sides equal. The inner + * ORDER BY u.b, u.id is a total order (id is unique), so LIMIT 1 is + * deterministic. + */ +PG_PROP(lateral_limit_topn) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT t.id, s.b FROM lt_t t, " + "LATERAL (SELECT u.b FROM lt_t u WHERE u.a = t.a " + "ORDER BY u.b, u.id LIMIT 1) s ORDER BY t.id", + "SELECT t.id, (SELECT u.b FROM lt_t u WHERE u.a = t.a " + "ORDER BY u.b, u.id LIMIT 1) FROM lt_t t " + "WHERE EXISTS (SELECT 1 FROM lt_t u WHERE u.a = t.a) " + "ORDER BY t.id"); + + pg_prop_disconnect(conn); +} + +/* + * Property 4 (shape A): the count of rows produced by + * (t CROSS JOIN LATERAL generate_series(1, greatest(a,0))) equals the sum + * over rows of greatest(a,0). generate_series(1, k) yields k rows for k >= 1 + * and 0 rows for k <= 0, so a row contributes greatest(a,0) rows; NULL a is + * coalesced to 0. Both sides are coalesced so the empty-table case is 0 = 0. + */ +PG_PROP(cross_join_lateral_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM lt_t t, " + "LATERAL generate_series(1, greatest(coalesce(t.a,0),0)) g) " + "= (SELECT coalesce(sum(greatest(coalesce(a,0),0)),0) FROM lt_t)"); + + pg_prop_disconnect(conn); +} + +/* + * Property 5: a VALUES scan equals a UNION ALL of singletons. No table is + * needed; both forms produce the same three one-column rows and are given the + * same total order. + */ +PG_PROP(values_scan_eq_union) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_same_rows(tc, conn, + "SELECT * FROM (VALUES (1),(2),(3)) v(x) ORDER BY x", + "SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 ORDER BY 1"); + + pg_prop_disconnect(conn); +} + +/* + * Property 6: an SRF in the target list equals a LATERAL SRF in FROM. Under + * modern (9.6+) semantics an SRF in the SELECT list expands exactly like a + * LATERAL function call, so both yield the same (id, g) rows; a total order on + * (id, g) makes the comparison deterministic. + */ +PG_PROP(srf_in_select_eq_lateral) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id, generate_series(1, greatest(coalesce(a,0),0)) AS g " + "FROM lt_t ORDER BY id, g", + "SELECT t.id, g FROM lt_t t, " + "LATERAL generate_series(1, greatest(coalesce(t.a,0),0)) g " + "ORDER BY t.id, g"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 99085fb0d5d15..cdc922dba0f5c 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From ebc45c36f36a264bc48b79193c7033f1ac4a16ed Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:25 -0400 Subject: [PATCH 213/338] prop: add VALUES, unnest, and row-source equivalence property tests (valuesrows) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_valuesrows.c | 249 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 253 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_valuesrows.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 5d5d0cbf386d3..164f5f3415b83 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -228,7 +228,8 @@ OBJS = \ prop_cte.o \ prop_recursivecte.o \ prop_dmlcte.o \ - prop_lateral.o + prop_lateral.o \ + prop_valuesrows.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 5cbf5aa719860..27b72b96299da 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -226,6 +226,7 @@ prop_area_sources = files( 'prop_recursivecte.c', 'prop_dmlcte.c', 'prop_lateral.c', + 'prop_valuesrows.c', ) diff --git a/src/test/prop/prop_valuesrows.c b/src/test/prop/prop_valuesrows.c new file mode 100644 index 0000000000000..9c46c44d738bb --- /dev/null +++ b/src/test/prop/prop_valuesrows.c @@ -0,0 +1,249 @@ +/*------------------------------------------------------------------------- + * + * prop_valuesrows.c + * Property-based tests for VALUES lists, parallel unnest, and WITH + * ORDINALITY as relation sources. + * + * These exercise the executor's relation-generating nodes: ValuesScan, the + * SRF-in-FROM machinery (unnest, generate_series), WITH ORDINALITY, and the + * array <-> relation round-trip. The oracle is differential or a round-trip: + * a VALUES list must produce the same rows as an equivalent UNION ALL; + * parallel unnest of two arrays must pair elements positionally; array_agg of + * an unnested array (ordered by ordinality) must reconstruct the array. Any + * disagreement is a wrong-results bug in those nodes. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_valuesrows.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "valuesrows" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error is itself a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Run two result-set queries and require their row-major renderings to match. + * Both queries MUST carry a total ORDER BY so the comparison is a multiset + * comparison over a deterministic row order. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * A VALUES list equals the equivalent UNION ALL of single-row SELECTs. Both + * sides are the same three rows; the ValuesScan node must produce them just + * as the Append-of-Result plan does. + */ +PG_PROP(values_eq_union_all) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_same_rows(tc, conn, + "SELECT * FROM (VALUES (1,'a'),(2,'b'),(3,'c')) v(n,s) ORDER BY n", + "SELECT 1,'a' UNION ALL SELECT 2,'b' UNION ALL SELECT 3,'c' ORDER BY 1"); + + pg_prop_disconnect(conn); +} + +/* + * Parallel unnest of two arrays pairs elements positionally, so unnest of + * ARRAY[1,2,3] and ARRAY['a','b','c'] yields the same (n,s) rows as the + * equivalent VALUES list. + */ +PG_PROP(unnest_two_arrays_parallel) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_same_rows(tc, conn, + "SELECT * FROM unnest(ARRAY[1,2,3], ARRAY['a','b','c']) AS u(n,s) ORDER BY n", + "SELECT * FROM (VALUES (1,'a'),(2,'b'),(3,'c')) v(n,s) ORDER BY n"); + + pg_prop_disconnect(conn); +} + +/* + * WITH ORDINALITY numbers rows 1..N in element order, so the ordinality + * column equals the row_number() over that same order for every row. + */ +PG_PROP(unnest_ordinality) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT coalesce(bool_and(ord = pos), true) FROM " + "(SELECT ord, row_number() OVER (ORDER BY ord) pos " + "FROM unnest(ARRAY[10,20,30]) WITH ORDINALITY AS u(x,ord)) s"); + + pg_prop_disconnect(conn); +} + +/* + * unnest then re-aggregate round-trips: for a drawn int array a, array_agg of + * the unnested elements ordered by ordinality reconstructs a exactly. The + * empty array is guarded with coalesce so a 0-row array_agg (SQL NULL) + * compares equal to the empty-array literal. + */ +PG_PROP(array_unnest_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData arr; + StringInfoData q; + int nelem; + int i; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* Build an int array literal with 0..6 elements, each a drawn int. */ + initStringInfo(&arr); + appendStringInfoString(&arr, "ARRAY["); + nelem = (int) pg_prop_draw_int(tc, 0, 6); + for (i = 0; i < nelem; i++) + { + if (i > 0) + appendStringInfoString(&arr, ","); + appendStringInfo(&arr, "(%d)", (int) pg_prop_draw_int(tc, -1000, 1000)); + } + appendStringInfoString(&arr, "]::int[]"); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(array_agg(x ORDER BY ord), '{}'::int[]) = %s " + "FROM unnest(%s) WITH ORDINALITY AS u(x,ord)", + arr.data, arr.data); + + require_true(tc, conn, q.data); + + pfree(arr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A VALUES list with k rows has exactly k rows. k is drawn 1..6 and the k + * rows (each a single integer i) are built in C. + */ +PG_PROP(values_count) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int k; + int i; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + k = (int) pg_prop_draw_int(tc, 1, 6); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT (SELECT count(*) FROM (VALUES "); + for (i = 1; i <= k; i++) + { + if (i > 1) + appendStringInfoString(&q, ","); + appendStringInfo(&q, "(%d)", i); + } + appendStringInfo(&q, ") v) = %d", k); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * generate_series as a relation source equals the equivalent VALUES list. + * Both yield the rows 1,2,3; the FunctionScan over generate_series and the + * ValuesScan must agree row for row. + */ +PG_PROP(gen_series_eq_values) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_same_rows(tc, conn, + "SELECT g FROM generate_series(1,3) g ORDER BY g", + "SELECT * FROM (VALUES (1),(2),(3)) v(g) ORDER BY g"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index cdc922dba0f5c..8916b90166f41 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From f0d7e63997484e83695511119e22a66e2630add1 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:25 -0400 Subject: [PATCH 214/338] prop: add set-returning function position equivalence property tests (srf) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_srf.c | 399 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 403 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_srf.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 164f5f3415b83..12d9e233fa676 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -229,7 +229,8 @@ OBJS = \ prop_recursivecte.o \ prop_dmlcte.o \ prop_lateral.o \ - prop_valuesrows.o + prop_valuesrows.o \ + prop_srf.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 27b72b96299da..56585f4021122 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -227,6 +227,7 @@ prop_area_sources = files( 'prop_dmlcte.c', 'prop_lateral.c', 'prop_valuesrows.c', + 'prop_srf.c', ) diff --git a/src/test/prop/prop_srf.c b/src/test/prop/prop_srf.c new file mode 100644 index 0000000000000..629151f206d95 --- /dev/null +++ b/src/test/prop/prop_srf.c @@ -0,0 +1,399 @@ +/*------------------------------------------------------------------------- + * + * prop_srf.c + * Property-based tests for set-returning functions in various positions. + * + * SRFs (unnest, generate_series, string_to_table, regexp_split_to_table, + * jsonb_array_elements) must agree whether placed in the target list or in + * FROM, must count/sum consistently, and must handle empty input. A failure + * means an SRF produced a different row set depending on its syntactic + * position, or miscounted, which is an executor bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_srf.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "srf" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error on valid inputs is itself a bug, so either is a failure. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Differential result-set oracle: two queries must return identical rows in + * identical order. Both queries MUST carry a total ORDER BY so the row-major + * string compares as a multiset. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Build an int[] literal of m elements drawn from [-50, 50], appending each + * element to "arr" (an ARRAY[...] body) and accumulating the exact sum in C. + * Returns the sum; "arr" receives the comma-separated element list (no + * brackets). m may be 0 (empty list). + */ +static int64 +draw_int_array(pg_prop_tc *tc, StringInfo arr, int m) +{ + int64 sum = 0; + int i; + + for (i = 0; i < m; i++) + { + int64 v = pg_prop_draw_int(tc, -50, 50); + + if (i > 0) + appendStringInfoString(arr, ", "); + /* cast as (%d) so a negative literal is not misparsed */ + appendStringInfo(arr, "(%lld)", (long long) v); + sum += v; + } + return sum; +} + +/* + * string_to_table('a,b,c', ',') in FROM must yield exactly the elements of + * the equivalent string_to_array unnested in the target list. Build a random + * comma-joined word of a-c letters so the split is non-trivial. + */ +PG_PROP(string_to_table_eq_split_array) +{ + pg_prop_conn *conn; + char *err = NULL; + char *s; + size_t slen; + StringInfoData q1, + q2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + /* a string of a-c and commas; comma is a legal alphabet member here */ + s = pg_prop_draw_from(tc, "abc,", 0, 20, &slen); + + initStringInfo(&q1); + initStringInfo(&q2); + appendStringInfo(&q1, + "SELECT x FROM string_to_table('%s', ',') x ORDER BY x", + s); + appendStringInfo(&q2, + "SELECT unnest(string_to_array('%s', ',')) ORDER BY 1", + s); + + require_same_rows(tc, conn, q1.data, q2.data); + + pfree(q1.data); + pfree(q2.data); + free(s); + pg_prop_disconnect(conn); +} + +/* + * regexp_split_to_table(s, '') in FROM splits a string into its characters; + * it must match regexp_split_to_array(s, '') unnested in the target list. + * Draw a random a-c string so the character split is non-trivial. + */ +PG_PROP(regexp_split_to_table_eq_array) +{ + pg_prop_conn *conn; + char *err = NULL; + char *s; + size_t slen; + StringInfoData q1, + q2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + s = pg_prop_draw_from(tc, "abc", 0, 20, &slen); + + initStringInfo(&q1); + initStringInfo(&q2); + appendStringInfo(&q1, + "SELECT x FROM regexp_split_to_table('%s', '') x ORDER BY x", + s); + appendStringInfo(&q2, + "SELECT unnest(regexp_split_to_array('%s', '')) ORDER BY 1", + s); + + require_same_rows(tc, conn, q1.data, q2.data); + + pfree(q1.data); + pfree(q2.data); + free(s); + pg_prop_disconnect(conn); +} + +/* + * sum(unnest(ARRAY[...])) over m drawn ints equals the total computed in C. + * The empty array sums to SQL NULL, so coalesce to 0 and compare to the C sum + * (which is 0 for m = 0): the empty case stays sound. + */ +PG_PROP(unnest_sum) +{ + pg_prop_conn *conn; + char *err = NULL; + int m; + int64 sum; + StringInfoData arr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + m = (int) pg_prop_draw_int(tc, 0, 40); + initStringInfo(&arr); + sum = draw_int_array(tc, &arr, m); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce((SELECT sum(x) FROM unnest(ARRAY[%s]::int[]) x), 0) = %lld", + arr.data, (long long) sum); + + require_true(tc, conn, q.data); + + pfree(arr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * generate_series in the target list and in FROM must yield identical rows. + * Draw a small ordered span so the series is non-trivial but bounded. + */ +PG_PROP(generate_series_in_target_eq_from) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a, + b, + tmp; + StringInfoData q1, + q2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = pg_prop_draw_int(tc, -20, 20); + b = pg_prop_draw_int(tc, -20, 20); + if (a > b) + { + tmp = a; + a = b; + b = tmp; + } + + initStringInfo(&q1); + initStringInfo(&q2); + appendStringInfo(&q1, + "SELECT g FROM generate_series(%lld, %lld) g ORDER BY g", + (long long) a, (long long) b); + appendStringInfo(&q2, + "SELECT generate_series(%lld, %lld) ORDER BY 1", + (long long) a, (long long) b); + + require_same_rows(tc, conn, q1.data, q2.data); + + pfree(q1.data); + pfree(q2.data); + pg_prop_disconnect(conn); +} + +/* + * jsonb_array_elements over an m-element JSON array yields exactly m rows. + * Build the array as [v, v, ...] of m drawn ints; the count is m regardless + * of values (m = 0 gives an empty array and count 0). + */ +PG_PROP(json_array_elements_count) +{ + pg_prop_conn *conn; + char *err = NULL; + int m, + i; + StringInfoData js, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + m = (int) pg_prop_draw_int(tc, 0, 40); + initStringInfo(&js); + appendStringInfoChar(&js, '['); + for (i = 0; i < m; i++) + { + int64 v = pg_prop_draw_int(tc, -50, 50); + + if (i > 0) + appendStringInfoString(&js, ", "); + appendStringInfo(&js, "%lld", (long long) v); + } + appendStringInfoChar(&js, ']'); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM jsonb_array_elements('%s'::jsonb)) = %d", + js.data, m); + + require_true(tc, conn, q.data); + + pfree(js.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * unnest(...) WITH ORDINALITY numbers rows 1..n in emission order, so the sum + * of the ordinal column is n*(n+1)/2. Build an m-element array; the ordinal + * sum is the Gauss total of m, computed in C and compared. m = 0 gives an + * empty set whose ordinal sum is NULL, so coalesce to 0 (which equals the C + * total 0). + */ +PG_PROP(unnest_ordinality_sum) +{ + pg_prop_conn *conn; + char *err = NULL; + int m; + int64 ordsum; + StringInfoData arr, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + m = (int) pg_prop_draw_int(tc, 0, 40); + initStringInfo(&arr); + (void) draw_int_array(tc, &arr, m); + ordsum = (int64) m * (m + 1) / 2; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce((SELECT sum(ord) FROM unnest(ARRAY[%s]::int[]) " + "WITH ORDINALITY u(x, ord)), 0) = %lld", + arr.data, (long long) ordsum); + + require_true(tc, conn, q.data); + + pfree(arr.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A CROSS JOIN of two generate_series emits exactly a*b rows: for a in 0..5 + * and b in 0..5, generate_series(1, a) has a rows and generate_series(1, b) + * has b rows (an empty side gives 0). count(*) must equal a*b. + */ +PG_PROP(srf_cross_join_count) +{ + pg_prop_conn *conn; + char *err = NULL; + int64 a, + b; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + a = pg_prop_draw_int(tc, 0, 5); + b = pg_prop_draw_int(tc, 0, 5); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM generate_series(1, %lld) ga " + "CROSS JOIN generate_series(1, %lld) gb) = %lld", + (long long) a, (long long) b, (long long) (a * b)); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * unnest of an empty int[] yields zero rows. This is a fixed boundary case: + * count(*) over unnest('{}'::int[]) must be 0. + */ +PG_PROP(empty_array_unnest_empty) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM unnest('{}'::int[])) = 0"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 8916b90166f41..ce36d6374612e 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 8fe20b9381a8cb17e65b59efa9a7fd5645c21f7c Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:26 -0400 Subject: [PATCH 215/338] prop: add unnest in joins and array expansion property tests (unnestjoin) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_unnestjoin.c | 281 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 285 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_unnestjoin.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 12d9e233fa676..6a99c75e5508d 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -230,7 +230,8 @@ OBJS = \ prop_dmlcte.o \ prop_lateral.o \ prop_valuesrows.o \ - prop_srf.o + prop_srf.o \ + prop_unnestjoin.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 56585f4021122..b1ac7c4c2bef3 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -228,6 +228,7 @@ prop_area_sources = files( 'prop_lateral.c', 'prop_valuesrows.c', 'prop_srf.c', + 'prop_unnestjoin.c', ) diff --git a/src/test/prop/prop_unnestjoin.c b/src/test/prop/prop_unnestjoin.c new file mode 100644 index 0000000000000..55901e901e2ed --- /dev/null +++ b/src/test/prop/prop_unnestjoin.c @@ -0,0 +1,281 @@ +/*------------------------------------------------------------------------- + * + * prop_unnestjoin.c + * Property-based tests for unnest() in joins and WITH ORDINALITY. + * + * unnest() expands an array into a set, optionally WITH ORDINALITY, and that + * set is then joined (self-joined on a shared element) or re-aggregated. The + * oracle is the type/function algebra of unnest: the number of produced rows + * equals the summed array cardinality, re-aggregating an ordinality-tagged + * unnest reconstructs the original array, ordinals span 1..cardinality, empty + * arrays yield no rows, and the summed elements agree whether summed over the + * flattened set or per-row. Any "f" is a wrong-results bug in array + * expansion, ordinality, or the join over the expanded set. + * + * This area is kept distinct from arrayops/srf/valuesrows/lateral: it targets + * unnest specifically as a set-returning expansion feeding joins, re-aggregation + * and ordinality join keys, not generic array operators or generic SRFs. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_unnestjoin.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "unnestjoin" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Element domain for the tag arrays. Non-NULL elements only (the spec + * requires it): a NULL element in a join key would make a.t = b.t three-valued + * and the count arithmetic unsound, so we never emit NULL array elements. + */ +#define TAG_MIN 0 +#define TAG_MAX 5 + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the expansion/ordinality/join produced NULL or errored on + * valid input, which is itself a bug -- so either is a failure, reported with + * the SQL. (Copied from prop_numeric.c's require_true.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build un_t(id int PRIMARY KEY, tags int[]) fresh with 0..12 rows. Each + * row's tags is an array of 0..4 non-NULL ints drawn from [TAG_MIN, TAG_MAX]. + * ids are 0..nrows-1 so they are unique. Rebuilt in each property so no state + * leaks between cases. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows = (int) pg_prop_draw_int(tc, 0, 12); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS un_t; " + "CREATE TABLE un_t(id int PRIMARY KEY, tags int[]); "); + + for (int r = 0; r < nrows; r++) + { + int ntags = (int) pg_prop_draw_int(tc, 0, 4); + + appendStringInfo(&ddl, "INSERT INTO un_t VALUES (%d, ARRAY[", r); + for (int e = 0; e < ntags; e++) + { + if (e > 0) + appendStringInfoString(&ddl, ", "); + appendStringInfo(&ddl, "%d", + (int) pg_prop_draw_int(tc, TAG_MIN, TAG_MAX)); + } + /* empty ARRAY[] needs a type annotation */ + if (ntags == 0) + appendStringInfoString(&ddl, "]::int[]); "); + else + appendStringInfoString(&ddl, "]); "); + } + + appendStringInfoString(&ddl, "ANALYZE un_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + + pfree(ddl.data); +} + +/* + * Property 1: the number of rows produced by unnest(tags) over the table + * equals the sum of each row's cardinality(tags). This is the defining count + * identity of unnest-as-expansion. coalesce guards the empty-table case where + * sum(...) is NULL. + */ +PG_PROP(unnest_count_eq_sum_cardinality) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "(SELECT id, unnest(tags) t FROM un_t) s) " + "= (SELECT coalesce(sum(cardinality(tags)), 0) FROM un_t)"); + + pg_prop_disconnect(conn); +} + +/* + * Property 2: re-aggregating an ordinality-tagged unnest reconstructs the + * original array. array_agg(t ORDER BY ord) over unnest(tags) WITH ORDINALITY + * must rebuild tags exactly. array_agg over an empty unnest is NULL, so + * coalesce(reagg, '{}'::int[]) = tags handles the empty-array row. bool_and + * over 0 rows is NULL, coalesced to true so the empty-table case passes. + */ +PG_PROP(unnest_reagg_roundtrip) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(coalesce(reagg, '{}'::int[]) = tags), true) " + "FROM (SELECT id, tags, " + "(SELECT array_agg(t ORDER BY ord) " + "FROM unnest(tags) WITH ORDINALITY u(t, ord)) reagg " + "FROM un_t) s"); + + pg_prop_disconnect(conn); +} + +/* + * Property 3: a self-join of the unnested tags on the shared element produces + * at least as many rows as there are rows with a non-empty tags array -- every + * tag matches itself, so each (id, t) tuple joins at least to its own + * counterpart, and each row with cardinality > 0 contributes at least one such + * self-match. This bounds the self-join over the expanded set without + * depending on duplicate multiplicities, so it is sound for any contents. + * coalesce guards the empty-table/empty-join NULLs. + */ +PG_PROP(unnest_selfjoin_tag_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "(SELECT id, unnest(tags) t FROM un_t) a " + "JOIN (SELECT id, unnest(tags) t FROM un_t) b ON a.t = b.t) " + ">= (SELECT coalesce(count(*), 0) FROM un_t " + "WHERE cardinality(tags) > 0)"); + + pg_prop_disconnect(conn); +} + +/* + * Property 4: in unnest(tags) WITH ORDINALITY, the ordinal ranges 1.. + * cardinality(tags) for every produced row. bool_and over 0 rows (empty + * table or all-empty arrays) is NULL, coalesced to true. + */ +PG_PROP(ordinality_bounds) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(ord BETWEEN 1 AND cardinality(tags)), true) " + "FROM (SELECT tags, ord FROM un_t, " + "unnest(tags) WITH ORDINALITY AS u(t, ord)) s"); + + pg_prop_disconnect(conn); +} + +/* + * Property 5: a row with an empty tags array contributes no unnest rows, so + * the total count of unnest(tags) WITH ORDINALITY rows equals the summed + * cardinality (empty arrays add 0). Same identity as property 1 but driven + * through the ORDINALITY form, which cross-checks that ORDINALITY does not + * invent a row for an empty array. coalesce guards the empty-table NULL. + */ +PG_PROP(empty_array_no_rows) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "(SELECT id FROM un_t, unnest(tags) WITH ORDINALITY) s) " + "= (SELECT coalesce(sum(cardinality(tags)), 0) FROM un_t)"); + + pg_prop_disconnect(conn); +} + +/* + * Property 6: the sum of all elements across the flattened unnest set equals + * the sum over rows of (sum of that row's tags). Summing the expansion must + * agree with summing per row -- a differential check that unnest neither drops + * nor duplicates elements. Elements are non-NULL so no NULL arithmetic. Both + * coalesce layers guard the empty-set/empty-table NULLs. + */ +PG_PROP(unnest_sum_eq_array_sum) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT coalesce(sum(t), 0) FROM " + "(SELECT unnest(tags) t FROM un_t) s) " + "= (SELECT coalesce(sum(" + "(SELECT coalesce(sum(e), 0) FROM unnest(tags) e)), 0) " + "FROM un_t)"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index ce36d6374612e..4a450dc2ae1a3 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 5aded6320750e84ba15e568caf9c326cd5c94ad6 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:26 -0400 Subject: [PATCH 216/338] prop: add DISTINCT, DISTINCT ON, and GROUP BY equivalence property tests (distinct) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_distinct.c | 315 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 319 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_distinct.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 6a99c75e5508d..8fa1bfa1bc407 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -231,7 +231,8 @@ OBJS = \ prop_lateral.o \ prop_valuesrows.o \ prop_srf.o \ - prop_unnestjoin.o + prop_unnestjoin.o \ + prop_distinct.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index b1ac7c4c2bef3..cb2c12cb54eca 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -229,6 +229,7 @@ prop_area_sources = files( 'prop_valuesrows.c', 'prop_srf.c', 'prop_unnestjoin.c', + 'prop_distinct.c', ) diff --git a/src/test/prop/prop_distinct.c b/src/test/prop/prop_distinct.c new file mode 100644 index 0000000000000..6ee3a4047c56c --- /dev/null +++ b/src/test/prop/prop_distinct.c @@ -0,0 +1,315 @@ +/*------------------------------------------------------------------------- + * + * prop_distinct.c + * Property-based tests for DISTINCT / DISTINCT ON / GROUP BY equivalences. + * + * DISTINCT, DISTINCT ON and GROUP BY are three spellings of the same + * duplicate-elimination semantics, and the executor computes each by either a + * hash aggregate or a sort+unique. Those must all agree: a row that survives + * one spelling or one strategy must survive the others, with NULL treated as a + * single distinct value throughout. Any disagreement is a planner/executor + * bug. Oracles are differential (two spellings must return identical rows) or + * differential across enable_hashagg (hash distinct vs sort distinct). + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_distinct.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "distinct" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small integer domains keep each case fast and keep collisions (so DISTINCT + * actually removes something) frequent while still exercising NULL as a + * distinct value. + */ +#define VAL_MIN 0 +#define VAL_MAX 4 + +/* Draw an integer in [VAL_MIN, VAL_MAX], or SQL NULL (~15% of the time). */ +static void +draw_val(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); +} + +/* + * (Re)build di_t(id int PRIMARY KEY, a int, b int) with 0..30 random rows. + * id is 0..nrows-1 so it is a total key; a and b are small ints or NULL. + * Fresh in every property so cases never see each other's rows. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS di_t; " + "CREATE TABLE di_t (id int PRIMARY KEY, a int, b int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 30); + for (int r = 0; r < nrows; r++) + { + StringInfoData a, + b; + + initStringInfo(&a); + initStringInfo(&b); + draw_val(tc, &a); + draw_val(tc, &b); + appendStringInfo(&ddl, "INSERT INTO di_t VALUES (%d, %s, %s); ", + r, a.data, b.data); + pfree(a.data); + pfree(b.data); + } + appendStringInfoString(&ddl, "ANALYZE di_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build di_t failed: %s\n ddl: %s", + err ? err : "(no message)", ddl.data); + pfree(ddl.data); +} + +/* + * Differential result-set oracle: q1 and q2 must return identical rows in + * identical order. Each query MUST carry a total ORDER BY so the row-major + * string compares as an ordered multiset. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Run one DISTINCT query two ways -- hash distinct (enable_hashagg = on) and + * sort distinct (enable_hashagg = off) -- and require identical rows. Both + * strategies are optimisations that must never change results. RESET ALL + * after so a failed/longjmp'd case does not leak the GUC into the next. + */ +static void +run_two_distinct(pg_prop_tc *tc, pg_prop_conn *conn, const char *query) +{ + bool fh = false, + fs = false; + char *eh = NULL, + *es = NULL, + *rh, + *rs, + *err = NULL; + + pg_prop_sql_exec(conn, "SET enable_hashagg = on", &err); + rh = pg_prop_sql_rows(conn, query, &fh, &eh); + pg_prop_sql_exec(conn, "SET enable_hashagg = off", &err); + rs = pg_prop_sql_rows(conn, query, &fs, &es); + pg_prop_sql_exec(conn, "RESET ALL", &err); + + if (fh) + pg_prop_fail(tc, "hash distinct failed: %s\n SQL: %s", + eh ? eh : "", query); + if (fs) + pg_prop_fail(tc, "sort distinct failed: %s\n SQL: %s", + es ? es : "", query); + if (strcmp(rh, rs) != 0) + pg_prop_fail(tc, + "hash vs sort distinct mismatch:\n SQL: %s\n" + " hash:\n%s\n sort:\n%s", + query, rh, rs); + pg_free(rh); + pg_free(rs); + if (eh) + pg_free(eh); + if (es) + pg_free(es); + if (err) + pg_free(err); +} + +/* + * SELECT DISTINCT a == SELECT a GROUP BY a. DISTINCT on one column and + * GROUP BY on that column must produce the same set of values, NULL included. + */ +PG_PROP(distinct_eq_groupby) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT DISTINCT a FROM di_t ORDER BY a NULLS FIRST", + "SELECT a FROM di_t GROUP BY a ORDER BY a NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +/* + * Multi-column SELECT DISTINCT a, b computed as a hash aggregate and as a + * sort+unique must agree. + */ +PG_PROP(distinct_hash_vs_sort) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + run_two_distinct(tc, conn, + "SELECT DISTINCT a, b FROM di_t " + "ORDER BY a NULLS FIRST, b NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +/* + * DISTINCT ON (a): exactly one row per distinct a, and the ORDER BY starting + * with a (then b) makes it the row with the smallest b for each a -- a + * deterministic choice. Hash vs sort must agree on that choice. + */ +PG_PROP(distinct_on_first) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + run_two_distinct(tc, conn, + "SELECT DISTINCT ON (a) a, b FROM di_t " + "ORDER BY a NULLS FIRST, b NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +/* + * The number of DISTINCT values of a equals the number of GROUP BY a groups. + * NULL counts as one distinct value in both, so the counts must be identical. + * Shape A: a scalar boolean oracle requiring "t". + */ +PG_PROP(distinct_count_eq_group_count) +{ + pg_prop_conn *conn; + char *err = NULL; + bool failed = false; + char *res; + const char *sql = + "SELECT (SELECT count(*) FROM (SELECT DISTINCT a FROM di_t) s) " + "= (SELECT count(*) FROM (SELECT a FROM di_t GROUP BY a) t)"; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "distinct count != group count (got %s) for: %s", + res, sql); + pg_free(res); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} + +/* + * UNION (which de-duplicates) equals DISTINCT over the corresponding + * UNION ALL. Columns are aliased to x so both sides order by the same name. + */ +PG_PROP(union_distinct_eq_distinct_union_all) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT * FROM " + "(SELECT a AS x FROM di_t UNION SELECT b FROM di_t) s " + "ORDER BY x NULLS FIRST", + "SELECT DISTINCT x FROM " + "(SELECT a AS x FROM di_t UNION ALL SELECT b FROM di_t) s " + "ORDER BY x NULLS FIRST"); + + pg_prop_disconnect(conn); +} + +/* + * DISTINCT is idempotent: applying it twice yields the same rows as once. + */ +PG_PROP(distinct_idempotent) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT DISTINCT a FROM (SELECT DISTINCT a FROM di_t) s " + "ORDER BY a NULLS FIRST", + "SELECT DISTINCT a FROM di_t ORDER BY a NULLS FIRST"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 4a450dc2ae1a3..1e0e5646c7168 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 1084c1d4972e3c36ae18a4516f1a215a7ce61701 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:26 -0400 Subject: [PATCH 217/338] prop: add DISTINCT ON row-selection semantics property tests (distincton) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_distincton.c | 282 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 286 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_distincton.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 8fa1bfa1bc407..3539b1027d9e7 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -232,7 +232,8 @@ OBJS = \ prop_valuesrows.o \ prop_srf.o \ prop_unnestjoin.o \ - prop_distinct.o + prop_distinct.o \ + prop_distincton.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index cb2c12cb54eca..538dec829c422 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -230,6 +230,7 @@ prop_area_sources = files( 'prop_srf.c', 'prop_unnestjoin.c', 'prop_distinct.c', + 'prop_distincton.c', ) diff --git a/src/test/prop/prop_distincton.c b/src/test/prop/prop_distincton.c new file mode 100644 index 0000000000000..f4849bbb48b5a --- /dev/null +++ b/src/test/prop/prop_distincton.c @@ -0,0 +1,282 @@ +/*------------------------------------------------------------------------- + * + * prop_distincton.c + * Property-based tests for DISTINCT ON row-selection semantics. + * + * DISTINCT ON (keys) keeps exactly one row per distinct value of the ON keys, + * and when the query carries a leading ORDER BY on those keys it keeps the + * first row in that order -- so ORDER BY key, tiebreak selects, per group, the + * row with the smallest tiebreak. This is distinct from plain DISTINCT (whole + * output row de-dup) and from NULL distinctness: here the differential oracle + * is against the per-group min()/GROUP BY that DISTINCT ON must coincide with, + * plus scalar algebra (one row per group, subset of the table, count bounds). + * Any disagreement is a planner/executor bug in the DISTINCT ON path. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_distincton.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "distincton" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small domains: g in 0..3 so there are at most a handful of groups and + * collisions are frequent; v in 0..9 so per-group min(v) ties and distinct + * values both occur. No NULLs -- DISTINCT ON null handling is covered by the + * nulldistinct area; here the leading ORDER BY on g (and the tiebreak) must be + * a total order for the "picks first" oracle to be deterministic, so keeping g + * and v non-null keeps every ORDER BY total without NULLS FIRST/LAST noise. + */ +#define G_MIN 0 +#define G_MAX 3 +#define V_MIN 0 +#define V_MAX 9 + +/* + * (Re)build do_t(id int PRIMARY KEY, g int, v int) with 0..30 random rows. + * id is 0..nrows-1 so it is a total key (used as the final tiebreak and for + * the two-key min(id) oracle); g and v are small non-null ints. Fresh in + * every property so cases never see each other's rows. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS do_t; " + "CREATE TABLE do_t (id int PRIMARY KEY, g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 30); + for (int r = 0; r < nrows; r++) + { + int g = (int) pg_prop_draw_int(tc, G_MIN, G_MAX); + int v = (int) pg_prop_draw_int(tc, V_MIN, V_MAX); + + appendStringInfo(&ddl, "INSERT INTO do_t VALUES (%d, %d, %d); ", + r, g, v); + } + appendStringInfoString(&ddl, "ANALYZE do_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build do_t failed: %s\n ddl: %s", + err ? err : "(no message)", ddl.data); + pfree(ddl.data); +} + +/* + * Differential result-set oracle: q1 and q2 must return identical rows in + * identical order. Each query MUST carry a total ORDER BY so the row-major + * string compares as an ordered multiset. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Scalar boolean oracle: run "SELECT " and require it returns "t". + * A NULL result or a SQL error is a failure. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "expected t, got %s for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * (A) DISTINCT ON (g) picks exactly one row per distinct g: the number of + * rows kept equals the number of distinct g values. The ORDER BY g makes + * DISTINCT ON well-defined. + */ +PG_PROP(distinct_on_one_per_group) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "(SELECT DISTINCT ON (g) g FROM do_t ORDER BY g) s) " + "= (SELECT count(DISTINCT g) FROM do_t)"); + + pg_prop_disconnect(conn); +} + +/* + * (B) DISTINCT ON (g) with ORDER BY g, v picks the first row in that order + * per group -- the row with the smallest v -- so (g, v) must equal the + * per-group GROUP BY g / min(v). Both sides carry a total ORDER BY g. + */ +PG_PROP(distinct_on_picks_first) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT DISTINCT ON (g) g, v FROM do_t ORDER BY g, v", + "SELECT g, min(v) FROM do_t GROUP BY g ORDER BY g"); + + pg_prop_disconnect(conn); +} + +/* + * (A) Every (g, v) pair returned by DISTINCT ON (g) exists in the table: the + * result selects real rows, never fabricated ones. bool_and over the kept + * rows; coalesce(..., true) so the empty-table case (zero kept rows -> + * bool_and is NULL) passes soundly. + */ +PG_PROP(distinct_on_subset) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(" + "EXISTS(SELECT 1 FROM do_t t WHERE t.g = s.g AND t.v = s.v))" + ", true) FROM " + "(SELECT DISTINCT ON (g) g, v FROM do_t ORDER BY g, v) s"); + + pg_prop_disconnect(conn); +} + +/* + * (B) DISTINCT ON (g, v) with ORDER BY g, v, id keeps one row per (g, v) pair, + * the one with the smallest id, so (g, v, id) must equal GROUP BY g, v / + * min(id). id is a PRIMARY KEY so g, v, id is a total order. + */ +PG_PROP(distinct_on_two_keys) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT DISTINCT ON (g, v) g, v, id FROM do_t " + "ORDER BY g, v, id", + "SELECT g, v, min(id) FROM do_t GROUP BY g, v " + "ORDER BY g, v"); + + pg_prop_disconnect(conn); +} + +/* + * (A) The DISTINCT ON result can only shrink the table: the number of rows + * kept is at most the number of rows in the table. ORDER BY g, id is total + * so the choice is deterministic (immaterial to the count, but keeps the + * query well-defined). + */ +PG_PROP(distinct_on_count_le) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "(SELECT DISTINCT ON (g) * FROM do_t ORDER BY g, id) s) " + "<= (SELECT count(*) FROM do_t)"); + + pg_prop_disconnect(conn); +} + +/* + * (A) DISTINCT ON (g) keeps the same number of rows as SELECT DISTINCT g: both + * collapse to one row per distinct g value, so their row counts must match. + */ +PG_PROP(distinct_on_vs_distinct) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM " + "(SELECT DISTINCT ON (g) g FROM do_t ORDER BY g) s) " + "= (SELECT count(*) FROM " + "(SELECT DISTINCT g FROM do_t) t)"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 1e0e5646c7168..fdc1a89bde253 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From dfbe3f651ad67f7d2e7ee39b0203f428c4c79f90 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:26 -0400 Subject: [PATCH 218/338] prop: add ORDER BY, LIMIT, OFFSET, and top-N property tests (sortlimit) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_sortlimit.c | 306 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 310 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_sortlimit.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 3539b1027d9e7..e602860a2d7f0 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -233,7 +233,8 @@ OBJS = \ prop_srf.o \ prop_unnestjoin.o \ prop_distinct.o \ - prop_distincton.o + prop_distincton.o \ + prop_sortlimit.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 538dec829c422..021cc3122ceeb 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -231,6 +231,7 @@ prop_area_sources = files( 'prop_unnestjoin.c', 'prop_distinct.c', 'prop_distincton.c', + 'prop_sortlimit.c', ) diff --git a/src/test/prop/prop_sortlimit.c b/src/test/prop/prop_sortlimit.c new file mode 100644 index 0000000000000..b8a79d3972c36 --- /dev/null +++ b/src/test/prop/prop_sortlimit.c @@ -0,0 +1,306 @@ +/*------------------------------------------------------------------------- + * + * prop_sortlimit.c + * Property-based tests for ORDER BY / LIMIT / OFFSET and top-N heapsort. + * + * The oracle is differential and algebraic: a top-N sort (ORDER BY ... LIMIT k) + * must return exactly the k-row prefix of the corresponding full sort, LIMIT + * must bound cardinality at least(k, n), OFFSET past the end must be empty, + * ORDER BY must preserve the multiset of rows, and LIMIT ALL must equal no + * LIMIT. A disagreement is a bug in nodeLimit / tuplesort's bounded (top-N + * heapsort) path, which is a performance optimisation that must never change + * results. Every LIMIT query carries a TOTAL order (v NULLS LAST, id) so the + * row-major comparison is deterministic. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_sortlimit.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "sortlimit" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* v domain: small ints -5..5, ~15% NULL. */ +#define VAL_MIN (-5) +#define VAL_MAX 5 + +/* + * Build sl_t(id int PRIMARY KEY, v int) with 0..30 rows. id is a dense + * sequence 0..n-1 so it is a unique total-order tiebreaker; v is drawn from + * [-5, 5] with ~15% NULL. Rebuilt fresh in every property so each case is + * independent. pg_prop_fail (not assume) on a build error: a failed DDL/DML + * here is a harness problem, not a discardable input. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows = (int) pg_prop_draw_int(tc, 0, 30); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS sl_t; " + "CREATE TABLE sl_t (id int PRIMARY KEY, v int); "); + for (int i = 0; i < nrows; i++) + { + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfo(&ddl, "INSERT INTO sl_t VALUES (%d, NULL); ", i); + else + appendStringInfo(&ddl, "INSERT INTO sl_t VALUES (%d, %d); ", + i, (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); + } + appendStringInfoString(&ddl, "ANALYZE sl_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + + pfree(ddl.data); + if (err) + pg_free(err); +} + +/* Differential result-set oracle: two queries must return identical rows. */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* Scalar boolean oracle: a SELECT must return "t". */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * 1. top-N (ORDER BY ... LIMIT k) equals the k-prefix of the full sort. + * + * q1 is the direct top-N form (tuplesort takes its bounded heapsort path); + * q2 wraps the full sort in a subquery and LIMITs the outer query, so the + * inner sort is unbounded. The total order (v NULLS LAST, id) makes both + * deterministic, so require_same_rows is sound. k drawn 0..30. + */ +PG_PROP(topn_eq_full_sort_limit) +{ + pg_prop_conn *conn; + char *err = NULL; + int k; + StringInfoData q1, + q2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + k = (int) pg_prop_draw_int(tc, 0, 30); + + initStringInfo(&q1); + initStringInfo(&q2); + appendStringInfo(&q1, + "SELECT id FROM sl_t ORDER BY v NULLS LAST, id LIMIT %d", k); + appendStringInfo(&q2, + "SELECT id FROM (SELECT id FROM sl_t " + "ORDER BY v NULLS LAST, id) s LIMIT %d", k); + + require_same_rows(tc, conn, q1.data, q2.data); + + pfree(q1.data); + pfree(q2.data); + pg_prop_disconnect(conn); +} + +/* + * 2. LIMIT bounds cardinality at least(k, n). + * + * Sound shape-A form: the count of a LIMIT k query equals least(k, total). + * k drawn 0..35 so it straddles the row count (0..30). Both subselects are + * aggregates over the table, so each returns exactly one row even when the + * table is empty -- no NULL-from-no-rows hazard. + */ +PG_PROP(limit_bounds_cardinality) +{ + pg_prop_conn *conn; + char *err = NULL; + int k; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + k = (int) pg_prop_draw_int(tc, 0, 35); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM " + "(SELECT id FROM sl_t ORDER BY v NULLS LAST, id LIMIT %d) s) " + "= least(%d, (SELECT count(*) FROM sl_t))", k, k); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. ORDER BY never changes cardinality. + * + * Sound shape-A form: the count over an ordered subquery equals the count of + * the base table. Both sides are aggregates, so one row each even when empty. + */ +PG_PROP(order_preserves_cardinality) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT (SELECT count(*) FROM " + "(SELECT * FROM sl_t ORDER BY v) a) " + "= (SELECT count(*) FROM sl_t)"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. OFFSET at or beyond the row count returns zero rows. + * + * Sound shape-A form: a LIMIT-less query with OFFSET = count yields an empty + * set, so wrapping it in count(*) gives 0. The outer count aggregate always + * returns one row. + */ +PG_PROP(offset_beyond_end_empty) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT (SELECT count(*) FROM " + "(SELECT id FROM sl_t ORDER BY id " + "OFFSET (SELECT count(*) FROM sl_t)) s) = 0"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. ORDER BY preserves the multiset of rows. + * + * Differential: the per-value counts of the ordered subquery equal those of + * the base table. Both sides are grouped and ORDER BY v NULLS FIRST so the + * row-major strings compare equal; the NULL group is included and lines up. + */ +PG_PROP(sort_stable_cardinality) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *q1 = + "SELECT v, count(*) FROM (SELECT v FROM sl_t ORDER BY v) s " + "GROUP BY v ORDER BY v NULLS FIRST"; + const char *q2 = + "SELECT v, count(*) FROM sl_t GROUP BY v ORDER BY v NULLS FIRST"; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, q1, q2); + + pg_prop_disconnect(conn); +} + +/* + * 6. LIMIT ALL equals no LIMIT. + * + * Differential: ORDER BY id (a total order on the PRIMARY KEY) with LIMIT ALL + * returns the same rows in the same order as the same query with no LIMIT. + */ +PG_PROP(limit_all_eq_no_limit) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *q1 = "SELECT id FROM sl_t ORDER BY id LIMIT ALL"; + const char *q2 = "SELECT id FROM sl_t ORDER BY id"; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, q1, q2); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index fdc1a89bde253..2b8792ec8dc74 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 23f41a31fa587be8df9b12b415a791c4d0322373 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:26 -0400 Subject: [PATCH 219/338] prop: add multi-key sort and incremental-sort equivalence property tests (sortmerge) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_sortmerge.c | 360 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 364 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_sortmerge.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index e602860a2d7f0..b5f177090c419 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -234,7 +234,8 @@ OBJS = \ prop_unnestjoin.o \ prop_distinct.o \ prop_distincton.o \ - prop_sortlimit.o + prop_sortlimit.o \ + prop_sortmerge.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 021cc3122ceeb..809259ee4953a 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -232,6 +232,7 @@ prop_area_sources = files( 'prop_distinct.c', 'prop_distincton.c', 'prop_sortlimit.c', + 'prop_sortmerge.c', ) diff --git a/src/test/prop/prop_sortmerge.c b/src/test/prop/prop_sortmerge.c new file mode 100644 index 0000000000000..60ce4fa19fe79 --- /dev/null +++ b/src/test/prop/prop_sortmerge.c @@ -0,0 +1,360 @@ +/*------------------------------------------------------------------------- + * + * prop_sortmerge.c + * Property-based tests for sort-key ordering, multi-column sort, and + * incremental-sort equivalence. + * + * The oracle is differential and algebraic: an incremental sort (enabled over + * a leading indexed prefix) must return exactly the same rows in the same + * order as a full sort; a multi-key DESC-mixed sort must be idempotent under a + * subquery wrapper; ORDER BY must preserve the multiset; an index-ordered scan + * must agree with a seqscan+sort; the first row of an ordered scan must carry + * the minimum key; and ties on the leading key must be broken by id. Every + * query carries a TOTAL order (ending in id) so the row-major comparison is + * deterministic. A disagreement is a wrong-results bug in the incremental + * sort / tuplesort / index-ordering paths, distinct from LIMIT top-N + * (sortlimit) and NULLS placement (orderbynulls). + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_sortmerge.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "sortmerge" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * a is drawn from a tiny domain (0..3) so there are few distinct values on the + * indexed leading key -- this is what makes an incremental sort possible (sort + * the leading indexed prefix for free, then sort each group by the trailing + * keys). b is drawn from -5..5 with ~15% NULL so NULLS placement interacts + * with the trailing sort key. + */ +#define A_MIN 0 +#define A_MAX 3 +#define B_MIN (-5) +#define B_MAX 5 + +/* Draw b: an integer in [B_MIN, B_MAX], or SQL NULL (~15% of the time). */ +static void +draw_b(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, B_MIN, B_MAX)); +} + +/* + * Build sm_t(id int PRIMARY KEY, a int, b int) with 0..30 rows. id is a dense + * sequence 0..n-1 so it is a unique total-order tiebreaker; a is drawn from + * [A_MIN, A_MAX] (few distinct values, so an incremental sort over the index + * on (a) is possible); b is drawn from [B_MIN, B_MAX] with ~15% NULL. A btree + * index on (a) is created so the planner can produce an index-ordered scan on + * the leading key, and ANALYZE primes the stats. Rebuilt fresh in every + * property so each case is independent. pg_prop_fail (not assume) on a build + * error: a failed DDL/DML here is a harness bug, not a discarded case. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows = (int) pg_prop_draw_int(tc, 0, 30); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS sm_t; " + "CREATE TABLE sm_t (id int PRIMARY KEY, a int, b int); "); + + for (int r = 0; r < nrows; r++) + { + StringInfoData b; + + initStringInfo(&b); + draw_b(tc, &b); + appendStringInfo(&ddl, + "INSERT INTO sm_t VALUES (%d, %d, %s); ", + r, (int) pg_prop_draw_int(tc, A_MIN, A_MAX), b.data); + pfree(b.data); + } + + appendStringInfoString(&ddl, + "CREATE INDEX sm_t_a_idx ON sm_t (a); " + "ANALYZE sm_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build failed: %s\n ddl: %s", + err ? err : "(no message)", ddl.data); + + pfree(ddl.data); +} + +/* + * Differential result-set oracle: run two queries that must return identical + * rows in identical order, and fail on any mismatch. Both queries must carry + * a TOTAL order (ending in id) so the row-major string compares as a multiset. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Scalar boolean oracle: run a SELECT and require it returns "t". A + * NULL result or a SQL error is a failure (the oracle is written to never + * yield NULL on any input via coalesce/CASE guards). + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * incsort_vs_fullsort (B): ORDER BY a, b NULLS LAST, id -- a has an index and + * few distinct values, so an incremental sort over the leading prefix is + * possible. The result must be identical with incremental sort enabled vs + * disabled. Incremental sort is a performance optimisation that must never + * change the row order. + */ +PG_PROP(incsort_vs_fullsort) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *q = + "SELECT id, a, b FROM sm_t ORDER BY a, b NULLS LAST, id"; + char *on, + *off; + bool f1 = false, + f2 = false; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + pg_prop_sql_exec(conn, "SET enable_incremental_sort = on", &err); + on = pg_prop_sql_rows(conn, q, &f1, &err); + if (f1) + pg_prop_fail(tc, "incsort-on query failed: %s", err ? err : ""); + + pg_prop_sql_exec(conn, "SET enable_incremental_sort = off", &err); + off = pg_prop_sql_rows(conn, q, &f2, &err); + if (f2) + pg_prop_fail(tc, "incsort-off query failed: %s", err ? err : ""); + + pg_prop_sql_exec(conn, "RESET ALL", &err); + + if (strcmp(on, off) != 0) + pg_prop_fail(tc, + "incremental sort changed results\n on:\n%s\n off:\n%s", + on, off); + + pg_free(on); + pg_free(off); + pg_prop_disconnect(conn); +} + +/* + * multikey_desc_mix (B): a mixed ASC/DESC multi-key sort must be idempotent -- + * wrapping the projection in a subquery and re-sorting identically yields the + * same rows in the same order. Exercises the comparator for a DESC key with + * explicit NULLS LAST alongside ASC keys. + */ +PG_PROP(multikey_desc_mix) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id FROM sm_t " + "ORDER BY a, b DESC NULLS LAST, id", + "SELECT id FROM (SELECT id, a, b FROM sm_t) s " + "ORDER BY a, b DESC NULLS LAST, id"); + + pg_prop_disconnect(conn); +} + +/* + * sort_nondestructive (B): ORDER BY does not change the multiset of rows. + * Grouping the a-values coming out of an ORDER BY must match grouping them + * straight from the table. A sort that dropped or duplicated a row would + * break this. + */ +PG_PROP(sort_nondestructive) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT a, count(*) FROM (SELECT a FROM sm_t ORDER BY a) s " + "GROUP BY a ORDER BY a", + "SELECT a, count(*) FROM sm_t GROUP BY a ORDER BY a"); + + pg_prop_disconnect(conn); +} + +/* + * first_key_dominant (A): the first row of ORDER BY a, id (ASC, NULLS LAST by + * default -- non-NULL a sorts first) carries the minimum non-NULL a, i.e. it + * agrees with min(a). Guarded with CASE so the empty table and the all-NULL-a + * table (where min(a) is NULL and the first ordered row is NULL) pass soundly: + * only when a non-NULL a exists do we require the leading row to equal min(a). + */ +PG_PROP(first_key_dominant) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN (SELECT count(a) FROM sm_t) = 0 THEN true " + "ELSE (SELECT a FROM sm_t ORDER BY a, id LIMIT 1) " + "IS NOT DISTINCT FROM (SELECT min(a) FROM sm_t) END"); + + pg_prop_disconnect(conn); +} + +/* + * index_sort_matches (B): ORDER BY a (indexed) must return identical rows + * whether the plan uses a forced index-ordered scan or a seqscan+sort. A + * total order (a, id) makes the comparison deterministic. A mismatch is a + * bug in the index-ordering path. + */ +PG_PROP(index_sort_matches) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *q = "SELECT id, a FROM sm_t ORDER BY a, id"; + char *idx, + *seq; + bool f1 = false, + f2 = false; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + /* forced index scan */ + pg_prop_sql_exec(conn, "SET enable_seqscan = off", &err); + pg_prop_sql_exec(conn, "SET enable_bitmapscan = off", &err); + idx = pg_prop_sql_rows(conn, q, &f1, &err); + if (f1) + pg_prop_fail(tc, "index-scan query failed: %s", err ? err : ""); + + /* seqscan + sort */ + pg_prop_sql_exec(conn, "RESET ALL", &err); + pg_prop_sql_exec(conn, "SET enable_indexscan = off", &err); + pg_prop_sql_exec(conn, "SET enable_indexonlyscan = off", &err); + seq = pg_prop_sql_rows(conn, q, &f2, &err); + if (f2) + pg_prop_fail(tc, "seqscan query failed: %s", err ? err : ""); + + pg_prop_sql_exec(conn, "RESET ALL", &err); + + if (strcmp(idx, seq) != 0) + pg_prop_fail(tc, + "index-ordered scan != seqscan+sort\n idx:\n%s\n seq:\n%s", + idx, seq); + + pg_free(idx); + pg_free(seq); + pg_prop_disconnect(conn); +} + +/* + * stable_within_key (A): under ORDER BY a, id, rows with equal a must be + * ordered by increasing id -- the trailing total-order key fully breaks ties + * on the leading key. Using a window LAG over the same ordering, every row + * must either start a new a-group or have a strictly larger id than its + * predecessor. coalesce(..., true) makes the empty table pass soundly + * (bool_and over 0 rows is NULL). + */ +PG_PROP(stable_within_key) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(bool_and(" + "prev_id IS NULL OR prev_a <> a OR prev_id < id), true) " + "FROM (SELECT a, id, " + "lag(a) OVER (ORDER BY a, id) prev_a, " + "lag(id) OVER (ORDER BY a, id) prev_id " + "FROM sm_t) s"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 2b8792ec8dc74..cf26b00c58eac 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From a37985f811bda1ee129029f13ef91c4311bf8876 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:26 -0400 Subject: [PATCH 220/338] prop: add ORDER BY NULLS ordering and sort/index equivalence property tests (orderbynulls) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_orderbynulls.c | 324 ++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 328 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_orderbynulls.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index b5f177090c419..2915a507f122c 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -235,7 +235,8 @@ OBJS = \ prop_distinct.o \ prop_distincton.o \ prop_sortlimit.o \ - prop_sortmerge.o + prop_sortmerge.o \ + prop_orderbynulls.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 809259ee4953a..090f06a050c47 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -233,6 +233,7 @@ prop_area_sources = files( 'prop_distincton.c', 'prop_sortlimit.c', 'prop_sortmerge.c', + 'prop_orderbynulls.c', ) diff --git a/src/test/prop/prop_orderbynulls.c b/src/test/prop/prop_orderbynulls.c new file mode 100644 index 0000000000000..e7c9ef6b04b53 --- /dev/null +++ b/src/test/prop/prop_orderbynulls.c @@ -0,0 +1,324 @@ +/*------------------------------------------------------------------------- + * + * prop_orderbynulls.c + * Property-based tests for ORDER BY NULLS FIRST/LAST ordering. + * + * The oracle is the SQL standard's rule that ASC orders NULLS LAST and DESC + * orders NULLS FIRST by default, that NULLS placement and multi-key ordering + * are independent of the plan (forced sort vs incremental-sort/index), and + * that an ordered scan's first row agrees with min()/LIMIT 1. A mismatch + * means a sort or index-ordering path placed NULLs or ordered keys wrongly, + * which is a wrong-results bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_orderbynulls.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "orderbynulls" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small integer domain for the sort keys, with ~20% NULL, so NULLS-placement + * boundaries (all NULL, no NULL, ties on a) are hit often while the table + * stays legible in a counterexample. + */ +#define VAL_MIN (-4) +#define VAL_MAX 4 + +/* Draw an integer in [VAL_MIN, VAL_MAX], or SQL NULL (~20% of the time). */ +static void +draw_val(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 4) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); +} + +/* + * Build ob_t(id int PRIMARY KEY, a int, b int) with 0..30 rows, a and b drawn + * from the small NULL-bearing domain, an index on (a), and fresh statistics. + * id is a dense 0..n-1 sequence so it is a guaranteed unique tiebreak for a + * total order. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows = (int) pg_prop_draw_int(tc, 0, 30); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS ob_t; " + "CREATE TABLE ob_t (id int PRIMARY KEY, a int, b int); "); + + for (int r = 0; r < nrows; r++) + { + StringInfoData a, + b; + + initStringInfo(&a); + initStringInfo(&b); + draw_val(tc, &a); + draw_val(tc, &b); + appendStringInfo(&ddl, "INSERT INTO ob_t VALUES (%d, %s, %s); ", + r, a.data, b.data); + pfree(a.data); + pfree(b.data); + } + + appendStringInfoString(&ddl, + "CREATE INDEX ob_t_a_idx ON ob_t (a); " + "ANALYZE ob_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n SQL: %s", + err ? err : "(no message)", ddl.data); + pfree(ddl.data); +} + +/* + * Differential result-set oracle: the two queries, each carrying a total + * ORDER BY, must return identical rows in identical order. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the ordering oracle errored or produced NULL, which is + * itself a failure. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Property 1: ASC default places NULLs last. "ORDER BY a, id" must return + * exactly the same rows in the same order as the explicit "ORDER BY a NULLS + * LAST, id". The id tiebreak makes both a total order. + */ +PG_PROP(nulls_last_default_asc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id FROM ob_t ORDER BY a, id", + "SELECT id FROM ob_t ORDER BY a NULLS LAST, id"); + + pg_prop_disconnect(conn); +} + +/* + * Property 2: DESC default places NULLs first. "ORDER BY a DESC, id" must + * equal the explicit "ORDER BY a DESC NULLS FIRST, id". + */ +PG_PROP(nulls_first_default_desc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id FROM ob_t ORDER BY a DESC, id", + "SELECT id FROM ob_t ORDER BY a DESC NULLS FIRST, id"); + + pg_prop_disconnect(conn); +} + +/* + * Property 3: the ordered answer must not depend on the plan. The same + * "ORDER BY a NULLS LAST, id" must return identical rows whether forced into a + * seqscan + sort or into an index scan. SET the planner GUCs around each run + * and RESET ALL afterward so no state leaks to the next case. + */ +PG_PROP(sort_vs_index_scan) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *q = "SELECT id FROM ob_t ORDER BY a NULLS LAST, id"; + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r_sort, + *r_index; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + /* Forced seqscan + sort. */ + pg_prop_sql_exec(conn, "SET enable_indexscan = off; " + "SET enable_indexonlyscan = off; " + "SET enable_bitmapscan = off; " + "SET enable_seqscan = on", &err); + r_sort = pg_prop_sql_rows(conn, q, &f1, &e1); + + /* Forced index scan. */ + pg_prop_sql_exec(conn, "SET enable_indexscan = on; " + "SET enable_indexonlyscan = on; " + "SET enable_seqscan = off; " + "SET enable_sort = off", &err); + r_index = pg_prop_sql_rows(conn, q, &f2, &e2); + + pg_prop_sql_exec(conn, "RESET ALL", &err); + + if (f1) + pg_prop_fail(tc, "forced-sort query failed: %s\n SQL: %s", + e1 ? e1 : "", q); + if (f2) + pg_prop_fail(tc, "forced-index query failed: %s\n SQL: %s", + e2 ? e2 : "", q); + if (strcmp(r_sort, r_index) != 0) + pg_prop_fail(tc, "plan changed ordered result:\n sort:\n%s\n index:\n%s", + r_sort, r_index); + + pg_free(r_sort); + pg_free(r_index); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); + pg_prop_disconnect(conn); +} + +/* + * Property 4: multi-key NULLS ordering is stable and plan-independent. The + * three-key total order "a NULLS FIRST, b NULLS FIRST, id" must equal the same + * ORDER BY applied over a subquery wrapper (which blocks a direct index path + * and typically forces a sort), proving the two orderings agree. + */ +PG_PROP(multikey_stable) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id FROM ob_t " + "ORDER BY a NULLS FIRST, b NULLS FIRST, id", + "SELECT id FROM (SELECT id, a, b FROM ob_t) s " + "ORDER BY a NULLS FIRST, b NULLS FIRST, id"); + + pg_prop_disconnect(conn); +} + +/* + * Property 5: when any non-null a exists, the first row of "ORDER BY a NULLS + * LAST, id" has a = min(a) over the non-null values. Guarded with CASE on the + * count of non-null a so an all-null or empty table passes soundly (true). + */ +PG_PROP(reverse_all_keys) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN (SELECT count(a) FROM ob_t) = 0 THEN true " + "ELSE (SELECT a FROM ob_t ORDER BY a NULLS LAST, id LIMIT 1) " + "= (SELECT min(a) FROM ob_t) END"); + + pg_prop_disconnect(conn); +} + +/* + * Property 6: LIMIT 1 of "ORDER BY a NULLS LAST, id" returns min(a) when a + * non-null a exists. IS NOT DISTINCT FROM compares safely; CASE on the + * non-null count guards the empty / all-null table so it passes soundly. + */ +PG_PROP(limit1_is_min) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT CASE WHEN (SELECT count(a) FROM ob_t) = 0 THEN true " + "ELSE (SELECT a FROM ob_t ORDER BY a NULLS LAST, id LIMIT 1) " + "IS NOT DISTINCT FROM (SELECT min(a) FROM ob_t) END"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index cf26b00c58eac..7a220557d0b29 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 13164f4f7479ac7ed0985453bdf20450769079ed Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:26 -0400 Subject: [PATCH 221/338] prop: add FETCH WITH TIES, LIMIT, and OFFSET property tests (fetchties) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_fetchties.c | 296 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 300 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_fetchties.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 2915a507f122c..162c16644efcb 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -236,7 +236,8 @@ OBJS = \ prop_distincton.o \ prop_sortlimit.o \ prop_sortmerge.o \ - prop_orderbynulls.o + prop_orderbynulls.o \ + prop_fetchties.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 090f06a050c47..d72c40b364817 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -234,6 +234,7 @@ prop_area_sources = files( 'prop_sortlimit.c', 'prop_sortmerge.c', 'prop_orderbynulls.c', + 'prop_fetchties.c', ) diff --git a/src/test/prop/prop_fetchties.c b/src/test/prop/prop_fetchties.c new file mode 100644 index 0000000000000..8e68a5dac5777 --- /dev/null +++ b/src/test/prop/prop_fetchties.c @@ -0,0 +1,296 @@ +/*------------------------------------------------------------------------- + * + * prop_fetchties.c + * Property-based tests for FETCH FIRST ... WITH TIES, LIMIT and OFFSET. + * + * The LIMIT/OFFSET executor node (nodeLimit.c) backs several surface syntaxes + * that must agree: LIMIT k and FETCH FIRST k ROWS ONLY are the same thing, as + * are OFFSET o FETCH NEXT l and OFFSET o LIMIT l. FETCH FIRST k ROWS WITH + * TIES additionally keeps every row tied with the k-th on the ORDER BY key, so + * its result is a superset of the ONLY result and never pushes a strictly + * larger key past the boundary. A disagreement between two spellings, or a + * WITH TIES set that drops below k rows or admits a larger key, is an executor + * bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see pg_prop.h + * and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_fetchties.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "fetchties" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Build ft_t(id int PRIMARY KEY, v int) with 0..30 rows: id is a dense 0-based + * sequence (a total order), v is drawn from 0..3 so ties on v are frequent -- + * exactly what the WITH TIES boundary logic has to get right. Rebuilt fresh + * in each property, then ANALYZEd so the planner has stats. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows = (int) pg_prop_draw_int(tc, 0, 30); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS ft_t; " + "CREATE TABLE ft_t(id int PRIMARY KEY, v int); "); + for (int i = 0; i < nrows; i++) + appendStringInfo(&ddl, "INSERT INTO ft_t VALUES (%d, %d); ", + i, (int) pg_prop_draw_int(tc, 0, 3)); + appendStringInfoString(&ddl, "ANALYZE ft_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + pfree(ddl.data); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * SQL error is itself a failure. (Same shape as prop_numeric.c.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Run two queries and require identical row-major result strings. Both + * queries must carry a TOTAL ORDER BY (ORDER BY v, id here) so the comparison + * is deterministic. (Same shape as the area spec's differential oracle.) + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * LIMIT k and FETCH FIRST k ROWS ONLY are the same construct; over a total + * order (v, id) they must return exactly the same rows. k in 0..30. + */ +PG_PROP(limit_eq_fetch_first) +{ + pg_prop_conn *conn; + char *err = NULL; + int k; + StringInfoData q1, + q2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + k = (int) pg_prop_draw_int(tc, 0, 30); + + initStringInfo(&q1); + initStringInfo(&q2); + appendStringInfo(&q1, "SELECT id FROM ft_t ORDER BY v, id LIMIT %d", k); + appendStringInfo(&q2, + "SELECT id FROM ft_t ORDER BY v, id FETCH FIRST %d ROWS ONLY", + k); + + require_same_rows(tc, conn, q1.data, q2.data); + + pfree(q1.data); + pfree(q2.data); + pg_prop_disconnect(conn); +} + +/* + * FETCH FIRST k ROWS WITH TIES returns at least k rows (or all of them if the + * table has fewer than k): ties can only add rows, never drop below the ONLY + * count. ORDER BY v only, so ties on v are what matter. k in 1..10. + */ +PG_PROP(with_ties_superset) +{ + pg_prop_conn *conn; + char *err = NULL; + int k; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + k = (int) pg_prop_draw_int(tc, 1, 10); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM " + "(SELECT v FROM ft_t ORDER BY v FETCH FIRST %d ROWS WITH TIES) s) " + ">= least(%d, (SELECT count(*) FROM ft_t))", + k, k); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * WITH TIES never admits a strictly larger ORDER BY key than ONLY: the max v + * in the WITH TIES set equals the max v in the ONLY set (ties sit exactly at + * the boundary, never above it). coalesce(..., true) makes the empty-table + * case (both maxima NULL) pass soundly. ORDER BY v only. k in 1..10. + */ +PG_PROP(with_ties_all_tied_at_boundary) +{ + pg_prop_conn *conn; + char *err = NULL; + int k; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + k = (int) pg_prop_draw_int(tc, 1, 10); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(" + "(SELECT max(v) FROM " + "(SELECT v FROM ft_t ORDER BY v FETCH FIRST %d ROWS WITH TIES) a) " + "= (SELECT max(v) FROM " + "(SELECT v FROM ft_t ORDER BY v FETCH FIRST %d ROWS ONLY) b), true)", + k, k); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * OFFSET o LIMIT l over a total order returns exactly greatest(0, least(l, + * total - o)) rows. o, l in 0..35 so both can exceed the row count. + */ +PG_PROP(offset_limit_total) +{ + pg_prop_conn *conn; + char *err = NULL; + int o, + l; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + o = (int) pg_prop_draw_int(tc, 0, 35); + l = (int) pg_prop_draw_int(tc, 0, 35); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM " + "(SELECT id FROM ft_t ORDER BY v, id OFFSET %d LIMIT %d) s) " + "= greatest(0, least(%d, (SELECT count(*) FROM ft_t) - %d))", + o, l, l, o); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * LIMIT 0 returns no rows: the count is exactly 0, independent of the table. + */ +PG_PROP(limit_zero_empty) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT (SELECT count(*) FROM " + "(SELECT id FROM ft_t ORDER BY v, id LIMIT 0) s) = 0"); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * OFFSET o FETCH NEXT l ROWS ONLY and OFFSET o LIMIT l are the same construct; + * over a total order they must return exactly the same rows (fixed o=2, l=3). + */ +PG_PROP(fetch_next_eq_offset) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id FROM ft_t ORDER BY v, id OFFSET 2 FETCH NEXT 3 ROWS ONLY", + "SELECT id FROM ft_t ORDER BY v, id OFFSET 2 LIMIT 3"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 7a220557d0b29..90c119a7d15db 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 3129639fa08dd8298196323ee927bd806b650a9f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:26 -0400 Subject: [PATCH 222/338] prop: add OFFSET/LIMIT and keyset pagination equivalence property tests (pagination) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_pagination.c | 313 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 317 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_pagination.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 162c16644efcb..110c77a737e56 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -237,7 +237,8 @@ OBJS = \ prop_sortlimit.o \ prop_sortmerge.o \ prop_orderbynulls.o \ - prop_fetchties.o + prop_fetchties.o \ + prop_pagination.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index d72c40b364817..218feffeb9b2d 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -235,6 +235,7 @@ prop_area_sources = files( 'prop_sortmerge.c', 'prop_orderbynulls.c', 'prop_fetchties.c', + 'prop_pagination.c', ) diff --git a/src/test/prop/prop_pagination.c b/src/test/prop/prop_pagination.c new file mode 100644 index 0000000000000..a78b8edf50d78 --- /dev/null +++ b/src/test/prop/prop_pagination.c @@ -0,0 +1,313 @@ +/*------------------------------------------------------------------------- + * + * prop_pagination.c + * Property-based tests for OFFSET/LIMIT and keyset pagination. + * + * The oracle is differential and algebraic: paging a result set with + * LIMIT/OFFSET must reconstruct the full ordered set, keyset pagination + * (WHERE (v,id) > (last_v,last_id)) must agree with the equivalent OFFSET, + * LIMIT must bound the first page at least(p, n), OFFSET past the end must be + * empty, LIMIT is monotone in its bound, and the first row of DESC order is + * the last of ASC order. A disagreement is a bug in nodeLimit / the planner's + * ordering or boundary handling. Every ordered query carries a TOTAL order + * (v, id) so the row-major comparison is deterministic and the oracle sound. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_pagination.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "pagination" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* v domain: small ints -5..5 (no NULL -- (v,id) must be a total order). */ +#define VAL_MIN (-5) +#define VAL_MAX 5 + +/* + * Build pg_t(id int PRIMARY KEY, v int) with a FIXED row count drawn once per + * case (n in 5..40). id is a dense sequence 0..n-1, so (v, id) is a total + * order with id as a unique tiebreaker. v is drawn from [-5, 5], never NULL, + * so (v, id) comparisons and row-compare keyset predicates are well defined. + * Rebuilt fresh in every property. Returns n. pg_prop_fail (not assume) on a + * build error: a failed DDL/DML here is a harness problem, not a bad input. + */ +static int +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 5, 40); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS pg_t; " + "CREATE TABLE pg_t (id int PRIMARY KEY, v int); "); + for (int i = 0; i < n; i++) + appendStringInfo(&ddl, "INSERT INTO pg_t VALUES (%d, %d); ", + i, (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); + appendStringInfoString(&ddl, "ANALYZE pg_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + + pfree(ddl.data); + if (err) + pg_free(err); + return n; +} + +/* Differential result-set oracle: two queries must return identical rows. */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* Scalar boolean oracle: a SELECT must return "t". */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * 1. pages_reconstruct (B): paging the ordered set reproduces it. + * + * A single huge-LIMIT page (OFFSET 0 LIMIT 1000000) covers the whole table, + * and re-sorting it by the total order (v, id) must yield the same rows in + * the same order as the direct full ordered scan. (A page partition + * concatenated back in page order is the full order; the huge-limit page is + * the degenerate one-page partition.) Both carry a total order, so + * require_same_rows is sound. + */ +PG_PROP(pages_reconstruct) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *q1 = "SELECT id FROM pg_t ORDER BY v, id"; + const char *q2 = + "SELECT id FROM (SELECT id, v FROM pg_t " + "ORDER BY v, id OFFSET 0 LIMIT 1000000) s ORDER BY v, id"; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + require_same_rows(tc, conn, q1, q2); + + pg_prop_disconnect(conn); +} + +/* + * 2. page_sizes_sum (A): the first page holds least(p, n) rows. + * + * For a page size p (1..10) the count of the first LIMIT-p page equals + * least(p, n). The inner query is wrapped in count(*), so it returns exactly + * one row even when p forces a short page. n drawn 5..40 so p straddles it. + */ +PG_PROP(page_sizes_sum) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + int p; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = build_table(tc, conn); + p = (int) pg_prop_draw_int(tc, 1, 10); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM " + "(SELECT id FROM pg_t ORDER BY v, id LIMIT %d) s) " + "= least(%d, %d)", p, p, n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. offset_past_end_empty (A): OFFSET n returns no rows. + * + * Skipping all n rows leaves an empty page, so wrapping it in count(*) gives + * 0. The outer count aggregate always returns one row. + */ +PG_PROP(offset_past_end_empty) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = build_table(tc, conn); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM " + "(SELECT id FROM pg_t ORDER BY v, id OFFSET %d) s) = 0", n); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. keyset_eq_offset (B): keyset pagination equals OFFSET. + * + * Keyset pagination past the first row -- WHERE (v, id) > (first_v, first_id) + * -- returns exactly the rows an OFFSET 1 page returns. The boundary is the + * first row of the total order (v, id), so "strictly greater than row 1" and + * "skip 1" denote the same suffix. Both carry the total order; sound. + */ +PG_PROP(keyset_eq_offset) +{ + pg_prop_conn *conn; + char *err = NULL; + const char *q1 = + "SELECT id FROM pg_t " + "WHERE (v, id) > (SELECT v, id FROM pg_t ORDER BY v, id LIMIT 1) " + "ORDER BY v, id"; + const char *q2 = "SELECT id FROM pg_t ORDER BY v, id OFFSET 1"; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + require_same_rows(tc, conn, q1, q2); + + pg_prop_disconnect(conn); +} + +/* + * 5. limit_monotone (A): LIMIT count is monotone in the bound. + * + * For k1 <= k2 the LIMIT-k1 page is a prefix of the LIMIT-k2 page, so its row + * count cannot exceed the larger page's. k1, k2 drawn 0..40 and ordered so + * k1 <= k2. Both inner queries are wrapped in count(*), so each returns one + * row even for a short or empty page. + */ +PG_PROP(limit_monotone) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + b, + k1, + k2; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + a = (int) pg_prop_draw_int(tc, 0, 40); + b = (int) pg_prop_draw_int(tc, 0, 40); + k1 = a < b ? a : b; + k2 = a < b ? b : a; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM " + "(SELECT id FROM pg_t ORDER BY v, id LIMIT %d) x) " + "<= (SELECT count(*) FROM " + "(SELECT id FROM pg_t ORDER BY v, id LIMIT %d) y)", k1, k2); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 6. reverse_pagination (A): first of DESC order is last of ASC order. + * + * The row with the greatest (v, id) is the first row of ORDER BY v DESC, id + * DESC and also the last row of ORDER BY v, id (OFFSET n-1 LIMIT 1). n is + * fixed >= 5, so n-1 >= 4 and the OFFSET lands on the final row. Each side is + * a one-row scalar, compared for equality. + */ +PG_PROP(reverse_pagination) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = build_table(tc, conn); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT id FROM pg_t ORDER BY v DESC, id DESC LIMIT 1) " + "= (SELECT id FROM pg_t ORDER BY v, id OFFSET %d LIMIT 1)", + n - 1); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 90c119a7d15db..d6861494f32c9 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From f763fcf06c0d81c9f959ebcb6be4eb1709d53329 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:27 -0400 Subject: [PATCH 223/338] prop: add predicate placement and pushdown equivalence property tests (predpushdown) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_predpushdown.c | 206 ++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 210 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_predpushdown.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 110c77a737e56..88e7744683ec8 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -238,7 +238,8 @@ OBJS = \ prop_sortmerge.o \ prop_orderbynulls.o \ prop_fetchties.o \ - prop_pagination.o + prop_pagination.o \ + prop_predpushdown.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 218feffeb9b2d..4fe79cabaa7e9 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -236,6 +236,7 @@ prop_area_sources = files( 'prop_orderbynulls.c', 'prop_fetchties.c', 'prop_pagination.c', + 'prop_predpushdown.c', ) diff --git a/src/test/prop/prop_predpushdown.c b/src/test/prop/prop_predpushdown.c new file mode 100644 index 0000000000000..f4191a5b335b7 --- /dev/null +++ b/src/test/prop/prop_predpushdown.c @@ -0,0 +1,206 @@ +/*------------------------------------------------------------------------- + * + * prop_predpushdown.c + * Property-based tests for predicate pushdown / qual placement. + * + * A predicate's syntactic position must not change which rows a query + * returns: whether a qual sits in a subquery, in the outer WHERE, in an + * INNER JOIN's ON clause, below a UNION ALL, or below a GROUP BY on the + * grouping key, the planner is free to move it but the result set must be + * identical. The oracle is differential (require_same_rows): two queries + * that place the same qual differently must return identical, totally- + * ordered rows. Any mismatch is a qual-placement / pushdown bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_predpushdown.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "predpushdown" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small integer domains keep each case fast and keep a counterexample + * legible, while still crossing the predicate boundaries (0, 4, NULL) the + * properties pivot on. + */ +#define VAL_MIN (-4) +#define VAL_MAX 4 + +/* Draw an integer in [VAL_MIN, VAL_MAX], or SQL NULL (~15% of the time). */ +static void +draw_val(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); +} + +/* + * Build pp_t(id int PRIMARY KEY, a int, b int) with 0..30 rows. id is dense + * (0..nrows-1) so it is a usable total-order key; a and b are drawn in + * [-4,4] with ~15% NULL. Rebuilt fresh in every property. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS pp_t; " + "CREATE TABLE pp_t (id int PRIMARY KEY, a int, b int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 30); + for (int r = 0; r < nrows; r++) + { + appendStringInfo(&ddl, "INSERT INTO pp_t VALUES (%d, ", r); + draw_val(tc, &ddl); + appendStringInfoString(&ddl, ", "); + draw_val(tc, &ddl); + appendStringInfoString(&ddl, "); "); + } + appendStringInfoString(&ddl, "ANALYZE pp_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(no message)", ddl.data); + pfree(ddl.data); +} + +/* Shape B: two queries that must return identical, totally-ordered rows. */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * A property body: connect, build the table, run the two formulations, done. + * conn/err are local so each property is self-contained. + */ +#define PP_SETUP() \ + pg_prop_conn *conn; \ + char *err = NULL; \ + conn = pg_prop_connect(&err); \ + if (conn == NULL) \ + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); \ + build_table(tc, conn) + +/* + * 1. A qual that lives in a subquery vs. the same qual folded into the outer + * WHERE of a flattened query. Subquery pullup must not change rows. + */ +PG_PROP(where_in_subquery_eq_outer) +{ + PP_SETUP(); + require_same_rows(tc, conn, + "SELECT id FROM (SELECT id, a FROM pp_t WHERE a > 0) s " + "WHERE s.a < 4 ORDER BY id", + "SELECT id FROM pp_t WHERE a > 0 AND a < 4 ORDER BY id"); + pg_prop_disconnect(conn); +} + +/* + * 2. For an INNER join, a qual in the ON clause and the same qual in WHERE are + * equivalent; the planner moves it freely. + */ +PG_PROP(join_on_vs_where_inner) +{ + PP_SETUP(); + require_same_rows(tc, conn, + "SELECT t.id FROM pp_t t JOIN pp_t u " + "ON t.id = u.id AND t.a > 0 ORDER BY t.id", + "SELECT t.id FROM pp_t t JOIN pp_t u " + "ON t.id = u.id WHERE t.a > 0 ORDER BY t.id"); + pg_prop_disconnect(conn); +} + +/* + * 3. A qual above a UNION ALL vs. pushed into both arms. Pushdown through a + * set operation must distribute over both legs identically. + */ +PG_PROP(pushdown_through_union) +{ + PP_SETUP(); + require_same_rows(tc, conn, + "SELECT id FROM (SELECT id, a FROM pp_t " + "UNION ALL SELECT id, a FROM pp_t) s " + "WHERE s.a > 0 ORDER BY id", + "(SELECT id FROM pp_t WHERE a > 0 " + "UNION ALL SELECT id FROM pp_t WHERE a > 0) ORDER BY id"); + pg_prop_disconnect(conn); +} + +/* + * 4. A qual on a grouping key above a GROUP BY vs. pushed below the aggregate. + * A restriction on the grouping column may legally move under the group. + */ +PG_PROP(pushdown_through_groupby) +{ + PP_SETUP(); + require_same_rows(tc, conn, + "SELECT g, c FROM (SELECT a g, count(*) c FROM pp_t GROUP BY a) s " + "WHERE s.g > 0 ORDER BY g", + "SELECT a g, count(*) c FROM pp_t WHERE a > 0 GROUP BY a ORDER BY a"); + pg_prop_disconnect(conn); +} + +/* + * 5. Constant folding of a qual expression must not change rows: a > (1+1-2) + * is a > 0. + */ +PG_PROP(const_fold_eq) +{ + PP_SETUP(); + require_same_rows(tc, conn, + "SELECT id FROM pp_t WHERE a > (1+1-2) ORDER BY id", + "SELECT id FROM pp_t WHERE a > 0 ORDER BY id"); + pg_prop_disconnect(conn); +} + +/* + * 6. Removing a duplicate qual (a>0 AND a>0) must not change rows. + */ +PG_PROP(redundant_qual) +{ + PP_SETUP(); + require_same_rows(tc, conn, + "SELECT id FROM pp_t WHERE a > 0 AND a > 0 ORDER BY id", + "SELECT id FROM pp_t WHERE a > 0 ORDER BY id"); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index d6861494f32c9..125456b27a806 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 84717e28c07e400bebff70b56e1fb05a6287d48a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:27 -0400 Subject: [PATCH 224/338] prop: add memory-bounded spill-path result equivalence property tests (spillpaths) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_spillpaths.c | 192 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 196 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_spillpaths.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 88e7744683ec8..4553b25bba081 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -239,7 +239,8 @@ OBJS = \ prop_orderbynulls.o \ prop_fetchties.o \ prop_pagination.o \ - prop_predpushdown.o + prop_predpushdown.o \ + prop_spillpaths.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 4fe79cabaa7e9..fd238821b8df2 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -237,6 +237,7 @@ prop_area_sources = files( 'prop_fetchties.c', 'prop_pagination.c', 'prop_predpushdown.c', + 'prop_spillpaths.c', ) diff --git a/src/test/prop/prop_spillpaths.c b/src/test/prop/prop_spillpaths.c new file mode 100644 index 0000000000000..6ce16fc9f5c23 --- /dev/null +++ b/src/test/prop/prop_spillpaths.c @@ -0,0 +1,192 @@ +/*------------------------------------------------------------------------- + * + * prop_spillpaths.c + * Property-based tests for memory-bounded executor spill paths. + * + * A query's result must not depend on how much working memory the executor + * is given. With a tiny work_mem the sort spills to tape, the hash + * aggregate and hash join spill their batches to disk, and SELECT DISTINCT / + * UNION fall back to their external paths; with ample work_mem they all run + * in memory. The oracle is differential: run each query under work_mem = + * '64kB' (spill-prone) and under work_mem = '64MB' (in-memory) and require + * byte-identical, fully ordered result sets. Any disagreement is a bug in a + * spill path (tape merge, batch repartitioning, HashAgg disk spill, ...). + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_spillpaths.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "spillpaths" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run "query" twice: once with a tiny work_mem that forces external sort / + * hash spill / hashagg spill, once with ample work_mem that keeps everything + * in memory. RESET ALL between (and after) so no GUC leaks across the two + * runs or to the next property. Require the two result sets to be identical. + * Every caller's query carries a TOTAL ORDER BY so the comparison is sound. + */ +static void +run_two_mem(pg_prop_tc *tc, pg_prop_conn *conn, const char *query) +{ + char *err = NULL; + char *r_small; + char *r_big; + bool fs = false; + bool fb = false; + + if (!pg_prop_sql_exec(conn, "SET work_mem = '64kB'", &err)) + pg_prop_fail(tc, "SET work_mem=64kB failed: %s", err ? err : ""); + r_small = pg_prop_sql_rows(conn, query, &fs, &err); + if (fs) + pg_prop_fail(tc, "spill-prone query failed: %s\n SQL: %s", + err ? err : "", query); + if (!pg_prop_sql_exec(conn, "RESET ALL", &err)) + pg_prop_fail(tc, "RESET ALL failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, "SET work_mem = '64MB'", &err)) + pg_prop_fail(tc, "SET work_mem=64MB failed: %s", err ? err : ""); + r_big = pg_prop_sql_rows(conn, query, &fb, &err); + if (fb) + pg_prop_fail(tc, "in-memory query failed: %s\n SQL: %s", + err ? err : "", query); + if (!pg_prop_sql_exec(conn, "RESET ALL", &err)) + pg_prop_fail(tc, "RESET ALL failed: %s", err ? err : ""); + + if (strcmp(r_small, r_big) != 0) + pg_prop_fail(tc, + "spill vs in-memory mismatch:\n SQL: %s\n" + " work_mem=64kB:\n%s\n work_mem=64MB:\n%s", + query, r_small, r_big); + + pg_free(r_small); + pg_free(r_big); + if (err) + pg_free(err); +} + +/* + * Build sp_t(id int PRIMARY KEY, g int, v int) with 0..60 rows. id is dense + * 0..n-1; g in 0..9 (so GROUP BY g makes a handful of groups that fit many + * rows each); v in -10..10 with ~10% NULL (so NULLS FIRST/LAST ordering and + * NULL-in-aggregate behaviour are exercised). ANALYZE so the planner has + * stats and may actually choose hash/sort plans worth spilling. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows = (int) pg_prop_draw_int(tc, 0, 60); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS sp_t; " + "CREATE TABLE sp_t(id int PRIMARY KEY, g int, v int); "); + + for (int i = 0; i < nrows; i++) + { + int g = (int) pg_prop_draw_int(tc, 0, 9); + + appendStringInfo(&ddl, "INSERT INTO sp_t VALUES (%d, %d, ", i, g); + if (pg_prop_draw_int(tc, 0, 9) == 0) + appendStringInfoString(&ddl, "NULL); "); + else + appendStringInfo(&ddl, "%d); ", + (int) pg_prop_draw_int(tc, -10, 10)); + } + appendStringInfoString(&ddl, "ANALYZE sp_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build sp_t failed: %s\n DDL: %s", + err ? err : "", ddl.data); + pfree(ddl.data); +} + +/* Shared setup: connect and build the table. Caller disconnects. */ +static pg_prop_conn * +setup(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + build_table(tc, conn); + return conn; +} + +/* 1. External sort vs in-memory sort. */ +PG_PROP(sort_spill) +{ + pg_prop_conn *conn = setup(tc); + + run_two_mem(tc, conn, + "SELECT id, v FROM sp_t ORDER BY v NULLS LAST, id"); + pg_prop_disconnect(conn); +} + +/* 2. HashAgg disk spill vs in-memory aggregate. */ +PG_PROP(hashagg_spill) +{ + pg_prop_conn *conn = setup(tc); + + run_two_mem(tc, conn, + "SELECT g, count(*), sum(v) FROM sp_t " + "GROUP BY g ORDER BY g NULLS FIRST"); + pg_prop_disconnect(conn); +} + +/* 3. DISTINCT hash/sort spill vs in-memory. */ +PG_PROP(distinct_spill) +{ + pg_prop_conn *conn = setup(tc); + + run_two_mem(tc, conn, + "SELECT DISTINCT g, (v > 0) FROM sp_t " + "ORDER BY g NULLS FIRST, 2 NULLS FIRST"); + pg_prop_disconnect(conn); +} + +/* 4. Hash join batch spill vs in-memory (bounded a side, full b side). */ +PG_PROP(hashjoin_spill) +{ + pg_prop_conn *conn = setup(tc); + + run_two_mem(tc, conn, + "SELECT a.id, b.id FROM sp_t a JOIN sp_t b ON a.g = b.g " + "WHERE a.id <= 10 ORDER BY a.id, b.id"); + pg_prop_disconnect(conn); +} + +/* 5. Set-operation (UNION) spill vs in-memory. */ +PG_PROP(setop_spill) +{ + pg_prop_conn *conn = setup(tc); + + run_two_mem(tc, conn, + "SELECT x FROM (SELECT v AS x FROM sp_t " + "UNION SELECT v + 1 FROM sp_t) s ORDER BY x NULLS FIRST"); + pg_prop_disconnect(conn); +} + +/* 6. GROUP BY with count(DISTINCT) (grouped-agg distinct) spill vs in-memory. */ +PG_PROP(groupagg_distinct_spill) +{ + pg_prop_conn *conn = setup(tc); + + run_two_mem(tc, conn, + "SELECT g, count(DISTINCT v) FROM sp_t " + "GROUP BY g ORDER BY g NULLS FIRST"); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 125456b27a806..b992b36eef48c 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 5a0a22e49ab1966117c0669ea35b7f18bdea6f81 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:27 -0400 Subject: [PATCH 225/338] prop: add TABLESAMPLE subset and determinism property tests (tablesample) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_tablesample.c | 227 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 231 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_tablesample.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 4553b25bba081..6f277c1871404 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -240,7 +240,8 @@ OBJS = \ prop_fetchties.o \ prop_pagination.o \ prop_predpushdown.o \ - prop_spillpaths.o + prop_spillpaths.o \ + prop_tablesample.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index fd238821b8df2..2d6a8b88d97a1 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -238,6 +238,7 @@ prop_area_sources = files( 'prop_pagination.c', 'prop_predpushdown.c', 'prop_spillpaths.c', + 'prop_tablesample.c', ) diff --git a/src/test/prop/prop_tablesample.c b/src/test/prop/prop_tablesample.c new file mode 100644 index 0000000000000..b732e19c1d879 --- /dev/null +++ b/src/test/prop/prop_tablesample.c @@ -0,0 +1,227 @@ +/*------------------------------------------------------------------------- + * + * prop_tablesample.c + * Property-based tests for TABLESAMPLE and sampling invariants. + * + * The oracle is the definition of sampling itself: a sample is a subset of + * the table (never invents rows), BERNOULLI(100) keeps every row and + * BERNOULLI(0) keeps none, a REPEATABLE(seed) sample is deterministic across + * two evaluations, and no sample is larger than the table. REPEATABLE(seed) + * is used everywhere so each run is reproducible; a failing "t" oracle means + * the sampling method (TsmRoutine) returned a row not in the table, dropped + * or duplicated rows at the 0/100 boundaries, or was non-deterministic for a + * fixed seed -- any of which is a bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_tablesample.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "tablesample" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the query errored or produced NULL, which is itself a + * failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build ts_t(id int PRIMARY KEY, v int) with a FIXED number of rows: n is + * drawn once (5..40) so each case has a deterministic, non-empty table. ids + * are 1..n and v = id * 2. pg_prop_fail on any DDL/DML error. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 5, 40); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS ts_t; " + "CREATE TABLE ts_t(id int PRIMARY KEY, v int); " + "INSERT INTO ts_t " + "SELECT g, g * 2 FROM generate_series(1, "); + appendStringInfo(&ddl, "%d", n); + appendStringInfoString(&ddl, ") g; ANALYZE ts_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build failed: %s\n SQL: %s", + err ? err : "(unknown)", ddl.data); + + pfree(ddl.data); + if (err) + pg_free(err); +} + +/* + * A BERNOULLI sample is a subset of the table: no sampled id is absent from + * the table. REPEATABLE(42) fixes the draw. + */ +PG_PROP(bernoulli_subset) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT NOT EXISTS (" + "SELECT id FROM ts_t TABLESAMPLE BERNOULLI(50) REPEATABLE(42) " + "EXCEPT SELECT id FROM ts_t)"); + + pg_prop_disconnect(conn); +} + +/* + * A SYSTEM sample is likewise a subset: every sampled id is in the table. + */ +PG_PROP(system_subset) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT NOT EXISTS (" + "SELECT id FROM ts_t TABLESAMPLE SYSTEM(50) REPEATABLE(42) " + "EXCEPT SELECT id FROM ts_t)"); + + pg_prop_disconnect(conn); +} + +/* + * BERNOULLI(100) returns every row: the sample count equals the table count. + */ +PG_PROP(sample_100_is_all) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM ts_t TABLESAMPLE BERNOULLI(100) " + "REPEATABLE(1)) = (SELECT count(*) FROM ts_t)"); + + pg_prop_disconnect(conn); +} + +/* + * BERNOULLI(0) returns no rows: the sample count is zero. + */ +PG_PROP(sample_0_is_empty) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM ts_t TABLESAMPLE BERNOULLI(0) " + "REPEATABLE(1)) = 0"); + + pg_prop_disconnect(conn); +} + +/* + * REPEATABLE(seed) is deterministic: the same BERNOULLI(50) REPEATABLE(7) + * sample twice yields the same id set (each direction of EXCEPT is empty). + */ +PG_PROP(repeatable_deterministic) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT NOT EXISTS (" + "(SELECT id FROM ts_t TABLESAMPLE BERNOULLI(50) REPEATABLE(7)) " + "EXCEPT " + "(SELECT id FROM ts_t TABLESAMPLE BERNOULLI(50) REPEATABLE(7))) " + "AND NOT EXISTS (" + "(SELECT id FROM ts_t TABLESAMPLE BERNOULLI(50) REPEATABLE(7)) " + "EXCEPT " + "(SELECT id FROM ts_t TABLESAMPLE BERNOULLI(50) REPEATABLE(7)))"); + + pg_prop_disconnect(conn); +} + +/* + * No sample is larger than the table: a BERNOULLI(37) count is <= the total. + */ +PG_PROP(sample_count_le_total) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM ts_t TABLESAMPLE BERNOULLI(37) " + "REPEATABLE(3)) <= (SELECT count(*) FROM ts_t)"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index b992b36eef48c..0180ed5ecfd90 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From b8a2019f4f8bb02bedbd2ebeedfbfede3b22b4ec Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:27 -0400 Subject: [PATCH 226/338] prop: add row-value comparison and IS DISTINCT property tests (rowcompare) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_rowcompare.c | 282 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 286 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_rowcompare.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 6f277c1871404..c2a178b9ff1fc 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -241,7 +241,8 @@ OBJS = \ prop_pagination.o \ prop_predpushdown.o \ prop_spillpaths.o \ - prop_tablesample.o + prop_tablesample.o \ + prop_rowcompare.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 2d6a8b88d97a1..c3ff43ad7273c 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -239,6 +239,7 @@ prop_area_sources = files( 'prop_predpushdown.c', 'prop_spillpaths.c', 'prop_tablesample.c', + 'prop_rowcompare.c', ) diff --git a/src/test/prop/prop_rowcompare.c b/src/test/prop/prop_rowcompare.c new file mode 100644 index 0000000000000..44d9150b1bfd6 --- /dev/null +++ b/src/test/prop/prop_rowcompare.c @@ -0,0 +1,282 @@ +/*------------------------------------------------------------------------- + * + * prop_rowcompare.c + * Property-based tests for row-value comparison and IS DISTINCT. + * + * The oracle is the defining expansion of each row-value construct: a + * ROW(...) equality/ordering compares componentwise/lexicographically, and + * ROW IS DISTINCT FROM is the NULL-aware negation of componentwise IS NOT + * DISTINCT. Each property reconstructs the row operator from its scalar + * components in SQL and requires the two sides to agree; a disagreement is a + * bug in the row-comparison code. Where NULLs appear we compare with IS NOT + * DISTINCT FROM so 3VL does not leak NULL into the boolean oracle. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_rowcompare.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "rowcompare" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +#define VAL_MIN (-4) +#define VAL_MAX 4 + +/* Draw a non-null int literal in [VAL_MIN, VAL_MAX], parenthesized if neg. */ +static void +draw_int(pg_prop_tc *tc, StringInfo buf) +{ + int v = (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX); + + if (v < 0) + appendStringInfo(buf, "(%d)", v); + else + appendStringInfo(buf, "%d", v); +} + +/* Draw an int-or-NULL literal (~15% NULL). */ +static void +draw_int_or_null(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + draw_int(tc, buf); +} + +/* Run a boolean SELECT and require "t"; NULL or error is a failure. */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Row equality is componentwise (all operands non-null): ROW(a,b) = ROW(c,d) + * iff a=c AND b=d. + */ +PG_PROP(row_eq_componentwise) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData a, + b, + c, + d, + q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + initStringInfo(&a); + initStringInfo(&b); + initStringInfo(&c); + initStringInfo(&d); + draw_int(tc, &a); + draw_int(tc, &b); + draw_int(tc, &c); + draw_int(tc, &d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (ROW(%s, %s) = ROW(%s, %s)) = (%s = %s AND %s = %s)", + a.data, b.data, c.data, d.data, + a.data, c.data, b.data, d.data); + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(d.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Row less-than is lexicographic (all operands non-null): ROW(a,b) < + * ROW(c,d) iff a Date: Mon, 12 Oct 2026 21:43:27 -0400 Subject: [PATCH 227/338] prop: add MVCC snapshot and transaction visibility property tests (mvcc) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_mvcc.c | 353 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 357 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_mvcc.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index c2a178b9ff1fc..878e7de24bb82 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -242,7 +242,8 @@ OBJS = \ prop_predpushdown.o \ prop_spillpaths.o \ prop_tablesample.o \ - prop_rowcompare.o + prop_rowcompare.o \ + prop_mvcc.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index c3ff43ad7273c..a118ccff63796 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -240,6 +240,7 @@ prop_area_sources = files( 'prop_spillpaths.c', 'prop_tablesample.c', 'prop_rowcompare.c', + 'prop_mvcc.c', ) diff --git a/src/test/prop/prop_mvcc.c b/src/test/prop/prop_mvcc.c new file mode 100644 index 0000000000000..fe05f9a4d013a --- /dev/null +++ b/src/test/prop/prop_mvcc.c @@ -0,0 +1,353 @@ +/*------------------------------------------------------------------------- + * + * prop_mvcc.c + * Property-based tests for MVCC snapshot/visibility invariants. + * + * The oracle is MVCC's own transaction semantics driven over a single + * connection with explicit transactions: a ROLLBACK must restore the prior + * state, a COMMIT must persist it, a ROLLBACK TO savepoint must undo only the + * work after the savepoint, a backend sees its own uncommitted writes, and a + * REPEATABLE READ snapshot is stable across statements. Any count or value + * that does not match the transaction's defined outcome is a visibility bug. + * + * These are single-session properties: multi-session concurrency cannot be + * driven deterministically from one connection, so each property exercises + * the snapshot/visibility machinery a single backend relies on. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_mvcc.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "mvcc" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Build mv_t(id int PRIMARY KEY, v int) fresh with a random number of rows in + * [0, 20]. ids are the contiguous run 1..n so a property can pick an existing + * id deterministically; v is a small signed int. Rebuilt in every property so + * no state leaks between cases. Returns the row count built. + */ +static int +build_mv_t(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 0, 20); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS mv_t; " + "CREATE TABLE mv_t (id int PRIMARY KEY, v int); "); + for (int i = 1; i <= n; i++) + appendStringInfo(&ddl, "INSERT INTO mv_t VALUES (%d, %d); ", + i, (int) pg_prop_draw_int(tc, -5, 5)); + appendStringInfoString(&ddl, "ANALYZE mv_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build mv_t failed: %s", err ? err : "(no message)"); + + pfree(ddl.data); + return n; +} + +/* Run SQL for side effects; a failure is a harness/setup bug, so fail hard. */ +static void +exec_ok(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "exec failed: %s\n SQL: %s", err ? err : "", sql); +} + +/* Read a scalar integer; a NULL or error is a bug (these queries never NULL). */ +static long +scalar_int(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + long val; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + val = atol(res); + pg_free(res); + return val; +} + +/* count(*) of mv_t. */ +static long +count_mv_t(pg_prop_tc *tc, pg_prop_conn *conn) +{ + return scalar_int(tc, conn, "SELECT count(*) FROM mv_t"); +} + +/* + * require_true (as in prop_numeric.c): a boolean SELECT must return "t"; a + * NULL or SQL error is a failure. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * ROLLBACK restores the row count. Capture c0; BEGIN; INSERT k rows (k in + * 0..5, with ids above the built range so no PK collision); the in-xact count + * must be c0+k; ROLLBACK; the count must be c0 again. A backend's own + * uncommitted inserts are visible to itself, and ROLLBACK must discard them. + */ +PG_PROP(rollback_restores_count) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + int k; + long c0; + long cmid; + long cafter; + StringInfoData ins; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = build_mv_t(tc, conn); + c0 = count_mv_t(tc, conn); + k = (int) pg_prop_draw_int(tc, 0, 5); + + exec_ok(tc, conn, "BEGIN"); + initStringInfo(&ins); + for (int i = 0; i < k; i++) + appendStringInfo(&ins, "INSERT INTO mv_t VALUES (%d, 0); ", n + 1000 + i); + if (ins.len > 0) + exec_ok(tc, conn, ins.data); + pfree(ins.data); + + cmid = count_mv_t(tc, conn); + if (cmid != c0 + k) + { + exec_ok(tc, conn, "ROLLBACK"); + pg_prop_fail(tc, "in-xact count %ld != c0+k %ld (c0=%ld k=%d)", + cmid, c0 + k, c0, k); + } + + exec_ok(tc, conn, "ROLLBACK"); + cafter = count_mv_t(tc, conn); + if (cafter != c0) + pg_prop_fail(tc, "post-rollback count %ld != c0 %ld", cafter, c0); + + pg_prop_disconnect(conn); +} + +/* + * COMMIT persists a delete. Capture c0; BEGIN; DELETE one existing row (if + * any); COMMIT; the count must be c0-1 when the table was non-empty, or c0 + * (== 0) when it was empty and no delete ran. + */ +PG_PROP(commit_persists_delete) +{ + pg_prop_conn *conn; + char *err = NULL; + long c0; + long cafter; + long expected; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_mv_t(tc, conn); + c0 = count_mv_t(tc, conn); + + exec_ok(tc, conn, "BEGIN"); + if (c0 > 0) + { + exec_ok(tc, conn, + "DELETE FROM mv_t WHERE id = (SELECT min(id) FROM mv_t)"); + expected = c0 - 1; + } + else + expected = c0; /* empty table: nothing to delete */ + exec_ok(tc, conn, "COMMIT"); + + cafter = count_mv_t(tc, conn); + if (cafter != expected) + pg_prop_fail(tc, "post-commit count %ld != expected %ld (c0=%ld)", + cafter, expected, c0); + + pg_prop_disconnect(conn); +} + +/* + * Savepoint partial rollback. BEGIN; INSERT A; SAVEPOINT s; INSERT B; + * ROLLBACK TO s; COMMIT. Only A survives, so the net change is +1. ROLLBACK + * TO must undo exactly the work after the savepoint, not before it. + */ +PG_PROP(savepoint_partial_rollback) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + long c0; + long cafter; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = build_mv_t(tc, conn); + c0 = count_mv_t(tc, conn); + + exec_ok(tc, conn, "BEGIN"); + { + StringInfoData s; + + initStringInfo(&s); + /* A persists; B is rolled back to the savepoint. */ + appendStringInfo(&s, "INSERT INTO mv_t VALUES (%d, 1); ", n + 2001); + appendStringInfoString(&s, "SAVEPOINT s; "); + appendStringInfo(&s, "INSERT INTO mv_t VALUES (%d, 2); ", n + 2002); + appendStringInfoString(&s, "ROLLBACK TO s;"); + exec_ok(tc, conn, s.data); + pfree(s.data); + } + exec_ok(tc, conn, "COMMIT"); + + cafter = count_mv_t(tc, conn); + if (cafter != c0 + 1) + pg_prop_fail(tc, "post-savepoint count %ld != c0+1 %ld (c0=%ld)", + cafter, c0 + 1, c0); + + pg_prop_disconnect(conn); +} + +/* + * A backend sees its own in-xact UPDATE, and ROLLBACK restores the old value. + * BEGIN; read old v; UPDATE v = v + 100 on an existing id; the same xact must + * now read old+100; ROLLBACK; the value is back to old. Vacuously passes when + * the table is empty (nothing to update). + */ +PG_PROP(self_update_visible_in_xact) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + long vold; + long vmid; + long vrestored; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = build_mv_t(tc, conn); + if (n == 0) + { + /* no row to update: the property is vacuously satisfied */ + pg_prop_disconnect(conn); + return; + } + + vold = scalar_int(tc, conn, "SELECT v FROM mv_t WHERE id = 1"); + + exec_ok(tc, conn, "BEGIN"); + exec_ok(tc, conn, "UPDATE mv_t SET v = v + 100 WHERE id = 1"); + vmid = scalar_int(tc, conn, "SELECT v FROM mv_t WHERE id = 1"); + if (vmid != vold + 100) + { + exec_ok(tc, conn, "ROLLBACK"); + pg_prop_fail(tc, "in-xact v %ld != old+100 %ld (old=%ld)", + vmid, vold + 100, vold); + } + exec_ok(tc, conn, "ROLLBACK"); + + vrestored = scalar_int(tc, conn, "SELECT v FROM mv_t WHERE id = 1"); + if (vrestored != vold) + pg_prop_fail(tc, "post-rollback v %ld != old %ld", vrestored, vold); + + pg_prop_disconnect(conn); +} + +/* + * A REPEATABLE READ snapshot is stable across statements. BEGIN ISOLATION + * LEVEL REPEATABLE READ; read the count twice; the two reads must agree. + * Single-session this is trivially true, but it exercises the RR snapshot + * path: a RR xact takes one snapshot at first query and reuses it. + */ +PG_PROP(repeatable_read_stable) +{ + pg_prop_conn *conn; + char *err = NULL; + long c1; + long c2; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_mv_t(tc, conn); + + exec_ok(tc, conn, "BEGIN ISOLATION LEVEL REPEATABLE READ"); + c1 = count_mv_t(tc, conn); + c2 = count_mv_t(tc, conn); + exec_ok(tc, conn, "COMMIT"); + + if (c1 != c2) + pg_prop_fail(tc, "RR count unstable: c1 %ld != c2 %ld", c1, c2); + + pg_prop_disconnect(conn); +} + +/* + * count(*) is independent of row ordering. Shape A: the plain count equals + * the count of a subquery that imposes an ORDER BY. Ordering is a + * presentation concern and must never change the cardinality a snapshot sees. + */ +PG_PROP(count_consistent_across_order) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_mv_t(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM mv_t) " + "= (SELECT count(*) FROM (SELECT * FROM mv_t ORDER BY v) s)"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 88cf9970d9eae..5364b545168e5 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 328178829460ca557666de8776181877db2a2058 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:27 -0400 Subject: [PATCH 228/338] prop: add savepoint and subtransaction rollback property tests (savepoint) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_savepoint.c | 331 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 335 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_savepoint.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 878e7de24bb82..7b80738376110 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -243,7 +243,8 @@ OBJS = \ prop_spillpaths.o \ prop_tablesample.o \ prop_rowcompare.o \ - prop_mvcc.o + prop_mvcc.o \ + prop_savepoint.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index a118ccff63796..15651f59bbb96 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -241,6 +241,7 @@ prop_area_sources = files( 'prop_tablesample.c', 'prop_rowcompare.c', 'prop_mvcc.c', + 'prop_savepoint.c', ) diff --git a/src/test/prop/prop_savepoint.c b/src/test/prop/prop_savepoint.c new file mode 100644 index 0000000000000..be26b668ab6ac --- /dev/null +++ b/src/test/prop/prop_savepoint.c @@ -0,0 +1,331 @@ +/*------------------------------------------------------------------------- + * + * prop_savepoint.c + * Property-based tests for savepoints and subtransaction rollback. + * + * The oracle is PostgreSQL's own subtransaction semantics driven over a + * single connection with explicit transactions and savepoints: ROLLBACK TO + * must undo exactly the work done since the named savepoint (and nothing + * before it), RELEASE must keep that work, a whole-transaction ROLLBACK must + * discard everything, and after an error inside a subtransaction a ROLLBACK TO + * must recover the session so later statements commit. Any net row-count + * delta that disagrees with the defined subtransaction outcome is a bug. + * + * This is distinct from the MVCC visibility properties: here the oracle is the + * nesting/undo behaviour of subtransactions, not snapshot visibility. Every + * property captures the count before BEGIN, runs a fixed savepoint sequence, + * captures the count after COMMIT/ROLLBACK, and compares the delta in C. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_savepoint.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "savepoint" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Build sv_t(id int PRIMARY KEY, v int) fresh with a random number of rows in + * [0, 15]. ids are the contiguous run 1..n; v is a small signed int. The + * savepoint sequences below use ids at 1000+ offsets so no insert collides + * with the built rows. Rebuilt in every property so no state leaks between + * cases. + */ +static void +build_sv_t(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 0, 15); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS sv_t; " + "CREATE TABLE sv_t (id int PRIMARY KEY, v int); "); + for (int i = 1; i <= n; i++) + appendStringInfo(&ddl, "INSERT INTO sv_t VALUES (%d, %d); ", + i, (int) pg_prop_draw_int(tc, -5, 5)); + appendStringInfoString(&ddl, "ANALYZE sv_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build sv_t failed: %s", err ? err : "(no message)"); + + pfree(ddl.data); +} + +/* Run SQL for side effects; a failure is a harness/setup bug, so fail hard. */ +static void +exec_ok(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "exec failed: %s\n SQL: %s", err ? err : "", sql); +} + +/* Read a scalar integer; a NULL or error is a bug (these queries never NULL). */ +static long +scalar_int(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + long val; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + val = atol(res); + pg_free(res); + return val; +} + +/* count(*) of sv_t. */ +static long +count_sv_t(pg_prop_tc *tc, pg_prop_conn *conn) +{ + return scalar_int(tc, conn, "SELECT count(*) FROM sv_t"); +} + +/* + * ROLLBACK TO undoes work after the savepoint, keeps work before it. + * BEGIN; INSERT A (1001); SAVEPOINT s1; INSERT B (1002); ROLLBACK TO s1; + * COMMIT. Only A survives: net +1. + */ +PG_PROP(savepoint_rollback_partial) +{ + pg_prop_conn *conn; + char *err = NULL; + long c0; + long cafter; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_sv_t(tc, conn); + c0 = count_sv_t(tc, conn); + + exec_ok(tc, conn, "BEGIN"); + exec_ok(tc, conn, "INSERT INTO sv_t VALUES (1001, 1)"); + exec_ok(tc, conn, "SAVEPOINT s1"); + exec_ok(tc, conn, "INSERT INTO sv_t VALUES (1002, 2)"); + exec_ok(tc, conn, "ROLLBACK TO s1"); + exec_ok(tc, conn, "COMMIT"); + + cafter = count_sv_t(tc, conn); + if (cafter != c0 + 1) + pg_prop_fail(tc, "net delta %ld != +1 (c0=%ld after=%ld)", + cafter - c0, c0, cafter); + + exec_ok(tc, conn, "DELETE FROM sv_t WHERE id >= 1000"); + pg_prop_disconnect(conn); +} + +/* + * Nested savepoints: ROLLBACK TO the inner savepoint undoes only its work. + * BEGIN; SAVEPOINT s1; INSERT (2001); SAVEPOINT s2; INSERT (2002); + * ROLLBACK TO s2; INSERT (2003); COMMIT. 2001 and 2003 survive, 2002 is + * undone: net +2. + */ +PG_PROP(nested_savepoints) +{ + pg_prop_conn *conn; + char *err = NULL; + long c0; + long cafter; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_sv_t(tc, conn); + c0 = count_sv_t(tc, conn); + + exec_ok(tc, conn, "BEGIN"); + exec_ok(tc, conn, "SAVEPOINT s1"); + exec_ok(tc, conn, "INSERT INTO sv_t VALUES (2001, 1)"); + exec_ok(tc, conn, "SAVEPOINT s2"); + exec_ok(tc, conn, "INSERT INTO sv_t VALUES (2002, 2)"); + exec_ok(tc, conn, "ROLLBACK TO s2"); + exec_ok(tc, conn, "INSERT INTO sv_t VALUES (2003, 3)"); + exec_ok(tc, conn, "COMMIT"); + + cafter = count_sv_t(tc, conn); + if (cafter != c0 + 2) + pg_prop_fail(tc, "net delta %ld != +2 (c0=%ld after=%ld)", + cafter - c0, c0, cafter); + + exec_ok(tc, conn, "DELETE FROM sv_t WHERE id >= 1000"); + pg_prop_disconnect(conn); +} + +/* + * RELEASE keeps the subtransaction's work. BEGIN; INSERT (3001); + * SAVEPOINT s1; INSERT (3002); RELEASE s1; COMMIT. Both rows survive: net +2. + * RELEASE merges the subtransaction up, it does not undo it. + */ +PG_PROP(release_savepoint_keeps) +{ + pg_prop_conn *conn; + char *err = NULL; + long c0; + long cafter; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_sv_t(tc, conn); + c0 = count_sv_t(tc, conn); + + exec_ok(tc, conn, "BEGIN"); + exec_ok(tc, conn, "INSERT INTO sv_t VALUES (3001, 1)"); + exec_ok(tc, conn, "SAVEPOINT s1"); + exec_ok(tc, conn, "INSERT INTO sv_t VALUES (3002, 2)"); + exec_ok(tc, conn, "RELEASE s1"); + exec_ok(tc, conn, "COMMIT"); + + cafter = count_sv_t(tc, conn); + if (cafter != c0 + 2) + pg_prop_fail(tc, "net delta %ld != +2 (c0=%ld after=%ld)", + cafter - c0, c0, cafter); + + exec_ok(tc, conn, "DELETE FROM sv_t WHERE id >= 1000"); + pg_prop_disconnect(conn); +} + +/* + * A savepoint is re-usable after ROLLBACK TO: the savepoint name survives the + * rollback and work can continue past it. BEGIN; INSERT (4001); SAVEPOINT s1; + * INSERT (4002); ROLLBACK TO s1; INSERT (4003); COMMIT. 4001 and 4003 + * survive, 4002 is undone: net +2. + */ +PG_PROP(rollback_to_rereleases) +{ + pg_prop_conn *conn; + char *err = NULL; + long c0; + long cafter; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_sv_t(tc, conn); + c0 = count_sv_t(tc, conn); + + exec_ok(tc, conn, "BEGIN"); + exec_ok(tc, conn, "INSERT INTO sv_t VALUES (4001, 1)"); + exec_ok(tc, conn, "SAVEPOINT s1"); + exec_ok(tc, conn, "INSERT INTO sv_t VALUES (4002, 2)"); + exec_ok(tc, conn, "ROLLBACK TO s1"); + exec_ok(tc, conn, "INSERT INTO sv_t VALUES (4003, 3)"); + exec_ok(tc, conn, "COMMIT"); + + cafter = count_sv_t(tc, conn); + if (cafter != c0 + 2) + pg_prop_fail(tc, "net delta %ld != +2 (c0=%ld after=%ld)", + cafter - c0, c0, cafter); + + exec_ok(tc, conn, "DELETE FROM sv_t WHERE id >= 1000"); + pg_prop_disconnect(conn); +} + +/* + * A whole-transaction ROLLBACK discards everything, savepoints included. + * BEGIN; INSERT (5001); SAVEPOINT s1; INSERT (5002); ROLLBACK. Nothing + * persists: net 0. + */ +PG_PROP(full_rollback) +{ + pg_prop_conn *conn; + char *err = NULL; + long c0; + long cafter; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_sv_t(tc, conn); + c0 = count_sv_t(tc, conn); + + exec_ok(tc, conn, "BEGIN"); + exec_ok(tc, conn, "INSERT INTO sv_t VALUES (5001, 1)"); + exec_ok(tc, conn, "SAVEPOINT s1"); + exec_ok(tc, conn, "INSERT INTO sv_t VALUES (5002, 2)"); + exec_ok(tc, conn, "ROLLBACK"); + + cafter = count_sv_t(tc, conn); + if (cafter != c0) + pg_prop_fail(tc, "net delta %ld != 0 (c0=%ld after=%ld)", + cafter - c0, c0, cafter); + + exec_ok(tc, conn, "DELETE FROM sv_t WHERE id >= 1000"); + pg_prop_disconnect(conn); +} + +/* + * After an error inside a subtransaction, ROLLBACK TO recovers the session. + * BEGIN; INSERT (6001); SAVEPOINT s1; INSERT a duplicate id (6001) to raise a + * PK violation (which aborts the subtransaction); ROLLBACK TO s1 recovers; + * INSERT (6002); COMMIT. 6001 and 6002 survive, the errored insert is + * undone: net +2. The duplicate INSERT's exec returns false; that is + * expected, so continue past it to ROLLBACK TO. + */ +PG_PROP(subxact_error_rollback) +{ + pg_prop_conn *conn; + char *err = NULL; + long c0; + long cafter; + bool dup_ok; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_sv_t(tc, conn); + c0 = count_sv_t(tc, conn); + + exec_ok(tc, conn, "BEGIN"); + exec_ok(tc, conn, "INSERT INTO sv_t VALUES (6001, 1)"); + exec_ok(tc, conn, "SAVEPOINT s1"); + + /* Duplicate PK: this MUST fail, aborting the subtransaction. */ + err = NULL; + dup_ok = pg_prop_sql_exec(conn, "INSERT INTO sv_t VALUES (6001, 9)", &err); + if (dup_ok) + { + exec_ok(tc, conn, "ROLLBACK"); + pg_prop_fail(tc, "duplicate PK insert unexpectedly succeeded"); + } + if (err) + pg_free(err); + + /* ROLLBACK TO must recover the aborted subtransaction. */ + exec_ok(tc, conn, "ROLLBACK TO s1"); + exec_ok(tc, conn, "INSERT INTO sv_t VALUES (6002, 2)"); + exec_ok(tc, conn, "COMMIT"); + + cafter = count_sv_t(tc, conn); + if (cafter != c0 + 2) + pg_prop_fail(tc, "net delta %ld != +2 (c0=%ld after=%ld)", + cafter - c0, c0, cafter); + + exec_ok(tc, conn, "DELETE FROM sv_t WHERE id >= 1000"); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 5364b545168e5..948219c2ce66b 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 7b29004b66ff1effcea28d03b37281e3a1dc2c4c Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:28 -0400 Subject: [PATCH 229/338] prop: add RETURNING, ON CONFLICT, and MERGE property tests (returning) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_returning.c | 405 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 409 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_returning.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 7b80738376110..34c5904a2f525 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -244,7 +244,8 @@ OBJS = \ prop_tablesample.o \ prop_rowcompare.o \ prop_mvcc.o \ - prop_savepoint.o + prop_savepoint.o \ + prop_returning.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 15651f59bbb96..17abd63122804 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -242,6 +242,7 @@ prop_area_sources = files( 'prop_rowcompare.c', 'prop_mvcc.c', 'prop_savepoint.c', + 'prop_returning.c', ) diff --git a/src/test/prop/prop_returning.c b/src/test/prop/prop_returning.c new file mode 100644 index 0000000000000..97379f656c9b9 --- /dev/null +++ b/src/test/prop/prop_returning.c @@ -0,0 +1,405 @@ +/*------------------------------------------------------------------------- + * + * prop_returning.c + * Property-based tests for data-modifying RETURNING, ON CONFLICT and MERGE. + * + * INSERT/UPDATE/DELETE ... RETURNING must emit exactly one output row per row + * the statement actually inserted, updated or deleted; ON CONFLICT must either + * update or skip the conflicting row (never a second insert); and a MERGE whose + * source matches every target row must update every row and insert none. The + * oracle is the statement's own row accounting: the RETURNING count is compared + * (in C) against an independently computed count of the affected rows, and the + * post-state (row counts, specific values) is checked with scalar booleans. + * Any disagreement is a bug in the executor's ModifyTable row handling, not a + * legal difference. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_returning.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "returning" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Scalar boolean oracle: run a "SELECT " and require "t". A NULL result + * or a SQL error is itself a failure. (Copied from prop_numeric.c.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Run a scalar query expected to yield a non-NULL integer, and return it as a + * long long. A SQL error or a NULL result is a failure (every count query + * here is count(*)/a scalar that is never NULL). + */ +static long long +scalar_int(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + long long val; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + val = atoll(res); + pg_free(res); + if (err) + pg_free(err); + return val; +} + +/* + * Run a side-effecting statement (CREATE/DROP/DML) and fail on error. + */ +static void +exec_ok(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "exec failed: %s\n SQL: %s", err ? err : "", sql); + if (err) + pg_free(err); +} + +/* + * Build rt_t(id int PRIMARY KEY, v int) with 0..20 rows: ids are 1..n assigned + * densely (so they never collide with the generate_series ids offset by 1000), + * and v is a small int in [-5, 5] so the v>0 / v<0 filters each land on a mix + * of rows and the empty boundary. ANALYZE so the planner has stats. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS rt_t; " + "CREATE TABLE rt_t (id int PRIMARY KEY, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 20); + for (int id = 1; id <= nrows; id++) + { + int v = (int) pg_prop_draw_int(tc, -5, 5); + + appendStringInfo(&ddl, "INSERT INTO rt_t VALUES (%d, %d); ", id, v); + } + + appendStringInfoString(&ddl, "ANALYZE rt_t;"); + + exec_ok(tc, conn, ddl.data); + pfree(ddl.data); +} + +/* Create a fresh scratch copy rt_c of rt_t (same columns + PK). */ +static void +make_scratch(pg_prop_tc *tc, pg_prop_conn *conn) +{ + exec_ok(tc, conn, + "DROP TABLE IF EXISTS rt_c; " + "CREATE TABLE rt_c AS SELECT * FROM rt_t; " + "ALTER TABLE rt_c ADD PRIMARY KEY (id);"); +} + +static void +drop_scratch(pg_prop_tc *tc, pg_prop_conn *conn) +{ + exec_ok(tc, conn, "DROP TABLE rt_c;"); +} + +/* + * 1. INSERT ... RETURNING emits one row per inserted row. Insert k new rows + * (ids offset by 1000 so they never collide with the 1..20 ids) and count the + * RETURNING output via a top-level data-modifying CTE; the count must equal k. + */ +PG_PROP(insert_returning_count) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int k; + long long returned; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + make_scratch(tc, conn); + + k = (int) pg_prop_draw_int(tc, 1, 6); + + initStringInfo(&q); + appendStringInfo(&q, + "WITH ins AS (" + "INSERT INTO rt_c SELECT g + 1000, g FROM generate_series(1, %d) g " + "RETURNING id) SELECT count(*) FROM ins", + k); + returned = scalar_int(tc, conn, q.data); + pfree(q.data); + + if (returned != (long long) k) + pg_prop_fail(tc, "INSERT RETURNING count %lld != k %d", returned, k); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 2. UPDATE ... RETURNING emits one row per updated row. The RETURNING count + * must equal the number of rows matching the WHERE filter, counted first. + */ +PG_PROP(update_returning_matches_filter) +{ + pg_prop_conn *conn; + char *err = NULL; + long long pre; + long long returned; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + make_scratch(tc, conn); + + pre = scalar_int(tc, conn, "SELECT count(*) FROM rt_c WHERE v > 0"); + returned = scalar_int(tc, conn, + "WITH u AS (" + "UPDATE rt_c SET v = v + 1 WHERE v > 0 RETURNING id) " + "SELECT count(*) FROM u"); + + if (returned != pre) + pg_prop_fail(tc, "UPDATE RETURNING count %lld != matching rows %lld", + returned, pre); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 3. DELETE ... RETURNING emits one row per deleted row. The RETURNING count + * must equal the number of rows matching the WHERE filter, counted first. + */ +PG_PROP(delete_returning_matches) +{ + pg_prop_conn *conn; + char *err = NULL; + long long pre; + long long returned; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + make_scratch(tc, conn); + + pre = scalar_int(tc, conn, "SELECT count(*) FROM rt_c WHERE v < 0"); + returned = scalar_int(tc, conn, + "WITH d AS (" + "DELETE FROM rt_c WHERE v < 0 RETURNING id) " + "SELECT count(*) FROM d"); + + if (returned != pre) + pg_prop_fail(tc, "DELETE RETURNING count %lld != matching rows %lld", + returned, pre); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 4. INSERT ... ON CONFLICT DO UPDATE either updates the conflicting row or + * inserts a new one, never both. If rt_c has rows, INSERT a value colliding + * with an existing id: the row count is unchanged and that row's v becomes 999. + * If rt_c is empty, there is no existing id to collide with, so a fresh insert + * grows the count by exactly 1 and the new row's v is 999. + */ +PG_PROP(upsert_insert_or_update) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + long long pre; + long long target; + long long post; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + make_scratch(tc, conn); + + pre = scalar_int(tc, conn, "SELECT count(*) FROM rt_c"); + + if (pre > 0) + { + /* Collide with the smallest existing id. */ + target = scalar_int(tc, conn, "SELECT min(id) FROM rt_c"); + + initStringInfo(&q); + appendStringInfo(&q, + "INSERT INTO rt_c VALUES (%lld, 999) " + "ON CONFLICT (id) DO UPDATE SET v = 999", + target); + exec_ok(tc, conn, q.data); + resetStringInfo(&q); + + post = scalar_int(tc, conn, "SELECT count(*) FROM rt_c"); + if (post != pre) + pg_prop_fail(tc, "upsert-update changed row count %lld -> %lld", + pre, post); + + appendStringInfo(&q, "SELECT v = 999 FROM rt_c WHERE id = %lld", target); + require_true(tc, conn, q.data); + pfree(q.data); + } + else + { + /* Empty table: no conflict, a plain insert grows the count by 1. */ + exec_ok(tc, conn, + "INSERT INTO rt_c VALUES (1, 999) " + "ON CONFLICT (id) DO UPDATE SET v = 999"); + + post = scalar_int(tc, conn, "SELECT count(*) FROM rt_c"); + if (post != pre + 1) + pg_prop_fail(tc, "upsert-insert row count %lld -> %lld (expected +1)", + pre, post); + + require_true(tc, conn, "SELECT v = 999 FROM rt_c WHERE id = 1"); + } + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 5. INSERT ... ON CONFLICT DO NOTHING leaves a conflicting row entirely + * unchanged: both the row count and that row's v are the same afterward. When + * rt_c is empty there is no row to conflict with, so this property only applies + * to the non-empty case; an empty table is discarded. + */ +PG_PROP(upsert_do_nothing) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + long long pre; + long long target; + long long oldv; + long long post; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + make_scratch(tc, conn); + + pre = scalar_int(tc, conn, "SELECT count(*) FROM rt_c"); + pg_prop_assume(tc, pre > 0); + + target = scalar_int(tc, conn, "SELECT min(id) FROM rt_c"); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT v FROM rt_c WHERE id = %lld", target); + oldv = scalar_int(tc, conn, q.data); + resetStringInfo(&q); + + appendStringInfo(&q, + "INSERT INTO rt_c VALUES (%lld, 999) " + "ON CONFLICT (id) DO NOTHING", + target); + exec_ok(tc, conn, q.data); + resetStringInfo(&q); + + post = scalar_int(tc, conn, "SELECT count(*) FROM rt_c"); + if (post != pre) + pg_prop_fail(tc, "DO NOTHING changed row count %lld -> %lld", pre, post); + + appendStringInfo(&q, "SELECT v = %lld FROM rt_c WHERE id = %lld", + oldv, target); + require_true(tc, conn, q.data); + pfree(q.data); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 6. MERGE whose source matches every target row updates every row and inserts + * none. rt_c starts as an exact copy of rt_t, so a MERGE keyed on id matches + * every row: WHEN MATCHED bumps v by 1, WHEN NOT MATCHED never fires. After + * the merge, every row's v equals its original v + 1 and the row count is + * unchanged. bool_and over 0 rows is NULL, so coalesce to true for the empty + * case (which holds vacuously). + */ +PG_PROP(merge_matches_manual) +{ + pg_prop_conn *conn; + char *err = NULL; + long long pre; + long long post; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + make_scratch(tc, conn); + + pre = scalar_int(tc, conn, "SELECT count(*) FROM rt_c"); + + exec_ok(tc, conn, + "MERGE INTO rt_c t USING (SELECT id, v FROM rt_t) s ON t.id = s.id " + "WHEN MATCHED THEN UPDATE SET v = s.v + 1 " + "WHEN NOT MATCHED THEN INSERT VALUES (s.id, s.v)"); + + post = scalar_int(tc, conn, "SELECT count(*) FROM rt_c"); + if (post != pre) + pg_prop_fail(tc, "MERGE changed row count %lld -> %lld", pre, post); + + /* Every row's v is its original (from rt_t) + 1; vacuously true if empty. */ + require_true(tc, conn, + "SELECT coalesce(bool_and(c.v = o.v + 1), true) " + "FROM rt_c c JOIN rt_t o USING (id)"); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 948219c2ce66b..adf149ccf6231 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From aaf404baf5eccea92f20237905621653e8f933f3 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:28 -0400 Subject: [PATCH 230/338] prop: add INSERT ON CONFLICT DO NOTHING/UPDATE semantics property tests (upsert) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_upsert.c | 423 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 427 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_upsert.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 34c5904a2f525..3eef6b3a25531 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -245,7 +245,8 @@ OBJS = \ prop_rowcompare.o \ prop_mvcc.o \ prop_savepoint.o \ - prop_returning.o + prop_returning.o \ + prop_upsert.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 17abd63122804..394a5a46a5172 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -243,6 +243,7 @@ prop_area_sources = files( 'prop_mvcc.c', 'prop_savepoint.c', 'prop_returning.c', + 'prop_upsert.c', ) diff --git a/src/test/prop/prop_upsert.c b/src/test/prop/prop_upsert.c new file mode 100644 index 0000000000000..8ea4efa816084 --- /dev/null +++ b/src/test/prop/prop_upsert.c @@ -0,0 +1,423 @@ +/*------------------------------------------------------------------------- + * + * prop_upsert.c + * Property-based tests for INSERT ... ON CONFLICT (upsert) semantics. + * + * ON CONFLICT must behave as a deterministic upsert: DO NOTHING leaves a + * conflicting row untouched (count and value unchanged); DO UPDATE SET + * v = EXCLUDED.v writes the proposed value; an EXCLUDED expression sees the + * row the statement proposed to insert; a conflict WHERE clause gates the + * update on the existing row; a brand-new key is inserted (count + 1); and a + * single statement that proposes the same conflicting key twice is an error + * ("ON CONFLICT DO UPDATE command cannot affect row a second time"). Each + * post-state is read back with scalar queries and the invariant is checked in + * C. Any disagreement is a bug in the executor's ON CONFLICT handling, not a + * legal difference. This area is distinct from prop_returning (RETURNING row + * accounting) and prop_mergecmd (MERGE); it only exercises the upsert state + * transition. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_upsert.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "upsert" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Read a single integer scalar; fail the case on SQL error or NULL (every + * query here is COUNT(*) or a value from a known-present row, so neither is + * expected). Returns the parsed long. + */ +static long +scalar_int(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + long val; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + val = atol(res); + pg_free(res); + if (err) + pg_free(err); + return val; +} + +/* + * Build up_copy(id int PRIMARY KEY, v int) as a copy of a freshly generated + * up_t with 0..15 rows (ids 0..N-1, v drawn from a small signed domain that + * includes negatives so the conflict-WHERE branch can be exercised). The + * per-property work table is up_copy; up_t is the generator source. + * + * Returns the row count (== number of distinct ids 0..count-1). Guarantees a + * non-empty table: if the draw produced 0 rows, one fresh row is inserted so + * that properties needing an existing id always have one. + */ +static int +build_copy(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + nrows = (int) pg_prop_draw_int(tc, 0, 15); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS up_copy; " + "DROP TABLE IF EXISTS up_t; " + "CREATE TABLE up_t (id int PRIMARY KEY, v int); "); + + for (int i = 0; i < nrows; i++) + { + int v = (int) pg_prop_draw_int(tc, -5, 5); + + appendStringInfo(&ddl, "INSERT INTO up_t VALUES (%d, (%d)); ", i, v); + } + + appendStringInfoString(&ddl, + "CREATE TABLE up_copy (id int PRIMARY KEY, v int); " + "INSERT INTO up_copy SELECT * FROM up_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build failed: %s\n ddl: %s", + err ? err : "(no message)", ddl.data); + pfree(ddl.data); + + /* Guard non-empty: ensure at least one existing id (id 0). */ + if (nrows == 0) + { + if (!pg_prop_sql_exec(conn, "INSERT INTO up_copy VALUES (0, 1);", &err)) + pg_prop_fail(tc, "seed insert failed: %s", err ? err : ""); + nrows = 1; + } + + return nrows; +} + +static void +drop_copy(pg_prop_conn *conn) +{ + char *err = NULL; + + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS up_copy; DROP TABLE IF EXISTS up_t;", + &err); + if (err) + pg_free(err); +} + +/* + * DO NOTHING on an existing id must leave both the row count and that row's v + * unchanged. + */ +PG_PROP(conflict_do_nothing_keeps) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + int id; + long cnt0, + v0, + cnt1, + v1; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = build_copy(tc, conn); + id = (int) pg_prop_draw_int(tc, 0, n - 1); + + cnt0 = scalar_int(tc, conn, "SELECT count(*) FROM up_copy"); + initStringInfo(&q); + appendStringInfo(&q, "SELECT v FROM up_copy WHERE id = %d", id); + v0 = scalar_int(tc, conn, q.data); + + resetStringInfo(&q); + appendStringInfo(&q, + "INSERT INTO up_copy VALUES (%d, 424242) ON CONFLICT DO NOTHING", + id); + if (!pg_prop_sql_exec(conn, q.data, &err)) + pg_prop_fail(tc, "DO NOTHING failed: %s\n SQL: %s", + err ? err : "", q.data); + + cnt1 = scalar_int(tc, conn, "SELECT count(*) FROM up_copy"); + resetStringInfo(&q); + appendStringInfo(&q, "SELECT v FROM up_copy WHERE id = %d", id); + v1 = scalar_int(tc, conn, q.data); + + if (cnt1 != cnt0) + pg_prop_fail(tc, "DO NOTHING changed count: %ld -> %ld (id %d)", + cnt0, cnt1, id); + if (v1 != v0) + pg_prop_fail(tc, "DO NOTHING changed v: %ld -> %ld (id %d)", + v0, v1, id); + + pfree(q.data); + drop_copy(conn); + pg_prop_disconnect(conn); +} + +/* + * DO UPDATE SET v = EXCLUDED.v on an existing id must set that row's v to the + * proposed value and leave the row count unchanged. + */ +PG_PROP(conflict_do_update_sets) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + int id; + long cnt0, + cnt1, + v1; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = build_copy(tc, conn); + id = (int) pg_prop_draw_int(tc, 0, n - 1); + + cnt0 = scalar_int(tc, conn, "SELECT count(*) FROM up_copy"); + + initStringInfo(&q); + appendStringInfo(&q, + "INSERT INTO up_copy VALUES (%d, 999) " + "ON CONFLICT (id) DO UPDATE SET v = EXCLUDED.v", + id); + if (!pg_prop_sql_exec(conn, q.data, &err)) + pg_prop_fail(tc, "DO UPDATE failed: %s\n SQL: %s", + err ? err : "", q.data); + + cnt1 = scalar_int(tc, conn, "SELECT count(*) FROM up_copy"); + resetStringInfo(&q); + appendStringInfo(&q, "SELECT v FROM up_copy WHERE id = %d", id); + v1 = scalar_int(tc, conn, q.data); + + if (cnt1 != cnt0) + pg_prop_fail(tc, "DO UPDATE changed count: %ld -> %ld (id %d)", + cnt0, cnt1, id); + if (v1 != 999) + pg_prop_fail(tc, "DO UPDATE SET v=EXCLUDED.v: expected 999, got %ld (id %d)", + v1, id); + + pfree(q.data); + drop_copy(conn); + pg_prop_disconnect(conn); +} + +/* + * A brand-new id with DO NOTHING is inserted: the row count increases by one + * and the row carries the proposed value. + */ +PG_PROP(no_conflict_inserts) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + int newid; + long cnt0, + cnt1, + v1; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = build_copy(tc, conn); + newid = n + (int) pg_prop_draw_int(tc, 0, 5); /* ids are 0..n-1, so >= n is new */ + + cnt0 = scalar_int(tc, conn, "SELECT count(*) FROM up_copy"); + + initStringInfo(&q); + appendStringInfo(&q, + "INSERT INTO up_copy VALUES (%d, 7) ON CONFLICT DO NOTHING", + newid); + if (!pg_prop_sql_exec(conn, q.data, &err)) + pg_prop_fail(tc, "insert failed: %s\n SQL: %s", err ? err : "", q.data); + + cnt1 = scalar_int(tc, conn, "SELECT count(*) FROM up_copy"); + resetStringInfo(&q); + appendStringInfo(&q, "SELECT v FROM up_copy WHERE id = %d", newid); + v1 = scalar_int(tc, conn, q.data); + + if (cnt1 != cnt0 + 1) + pg_prop_fail(tc, "new-id insert count: expected %ld, got %ld (id %d)", + cnt0 + 1, cnt1, newid); + if (v1 != 7) + pg_prop_fail(tc, "new-id insert value: expected 7, got %ld (id %d)", + v1, newid); + + pfree(q.data); + drop_copy(conn); + pg_prop_disconnect(conn); +} + +/* + * EXCLUDED refers to the proposed row: DO UPDATE SET v = EXCLUDED.v + 1 on a + * proposed (id, 50) must leave v = 51, regardless of the old value. + */ +PG_PROP(excluded_refers_proposed) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + int id; + long v1; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = build_copy(tc, conn); + id = (int) pg_prop_draw_int(tc, 0, n - 1); + + initStringInfo(&q); + appendStringInfo(&q, + "INSERT INTO up_copy VALUES (%d, 50) " + "ON CONFLICT (id) DO UPDATE SET v = EXCLUDED.v + 1", + id); + if (!pg_prop_sql_exec(conn, q.data, &err)) + pg_prop_fail(tc, "DO UPDATE failed: %s\n SQL: %s", + err ? err : "", q.data); + + resetStringInfo(&q); + appendStringInfo(&q, "SELECT v FROM up_copy WHERE id = %d", id); + v1 = scalar_int(tc, conn, q.data); + + if (v1 != 51) + pg_prop_fail(tc, "EXCLUDED.v + 1: expected 51, got %ld (id %d)", v1, id); + + pfree(q.data); + drop_copy(conn); + pg_prop_disconnect(conn); +} + +/* + * A conflict WHERE gates the update on the EXISTING row: DO UPDATE SET v = 777 + * WHERE up_copy.v < 0 only fires when the pre-existing v is negative. We set a + * known old value on a chosen id (one draw decides whether it is negative) and + * check both branches: negative -> 777, non-negative -> unchanged. + */ +PG_PROP(conflict_where_condition) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + int id; + int oldv; + long v1; + bool neg; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = build_copy(tc, conn); + id = (int) pg_prop_draw_int(tc, 0, n - 1); + + /* Force a known old v on this id so the WHERE branch is known. */ + neg = pg_prop_draw_bool(tc); + oldv = neg ? -(int) pg_prop_draw_int(tc, 1, 100) : (int) pg_prop_draw_int(tc, 0, 100); + + initStringInfo(&q); + appendStringInfo(&q, "UPDATE up_copy SET v = (%d) WHERE id = %d", oldv, id); + if (!pg_prop_sql_exec(conn, q.data, &err)) + pg_prop_fail(tc, "setup update failed: %s\n SQL: %s", + err ? err : "", q.data); + + resetStringInfo(&q); + appendStringInfo(&q, + "INSERT INTO up_copy VALUES (%d, 0) " + "ON CONFLICT (id) DO UPDATE SET v = 777 WHERE up_copy.v < 0", + id); + if (!pg_prop_sql_exec(conn, q.data, &err)) + pg_prop_fail(tc, "DO UPDATE WHERE failed: %s\n SQL: %s", + err ? err : "", q.data); + + resetStringInfo(&q); + appendStringInfo(&q, "SELECT v FROM up_copy WHERE id = %d", id); + v1 = scalar_int(tc, conn, q.data); + + if (neg) + { + if (v1 != 777) + pg_prop_fail(tc, + "WHERE v<0 should update: old v=%d, expected 777, got %ld (id %d)", + oldv, v1, id); + } + else + { + if (v1 != oldv) + pg_prop_fail(tc, + "WHERE v<0 should skip: old v=%d, expected unchanged, got %ld (id %d)", + oldv, v1, id); + } + + pfree(q.data); + drop_copy(conn); + pg_prop_disconnect(conn); +} + +/* + * A single INSERT that proposes the same new key twice with ON CONFLICT DO + * UPDATE is an error ("ON CONFLICT DO UPDATE command cannot affect row a + * second time"). The statement must FAIL; pg_prop_sql_exec returning true + * (success) is the bug. + */ +PG_PROP(multi_insert_last_wins) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + int newid; + StringInfoData q; + bool ok; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + n = build_copy(tc, conn); + newid = n + (int) pg_prop_draw_int(tc, 0, 5); /* a new key, so the two rows conflict with each other */ + + initStringInfo(&q); + appendStringInfo(&q, + "INSERT INTO up_copy VALUES (%d, 1), (%d, 2) " + "ON CONFLICT (id) DO UPDATE SET v = EXCLUDED.v", + newid, newid); + + ok = pg_prop_sql_exec(conn, q.data, &err); + if (ok) + pg_prop_fail(tc, + "expected ON CONFLICT DO UPDATE double-affect error, but statement succeeded\n SQL: %s", + q.data); + if (err) + pg_free(err); + + pfree(q.data); + drop_copy(conn); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index adf149ccf6231..56aac5bc21aec 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 20376577450382c4abd078a5665e1dafa9c38aae Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:28 -0400 Subject: [PATCH 231/338] prop: add MERGE command matched/not-matched semantics property tests (mergecmd) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_mergecmd.c | 338 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 342 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_mergecmd.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 3eef6b3a25531..783ebe1ad441b 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -246,7 +246,8 @@ OBJS = \ prop_mvcc.o \ prop_savepoint.o \ prop_returning.o \ - prop_upsert.o + prop_upsert.o \ + prop_mergecmd.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 394a5a46a5172..4703a712f4500 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -244,6 +244,7 @@ prop_area_sources = files( 'prop_savepoint.c', 'prop_returning.c', 'prop_upsert.c', + 'prop_mergecmd.c', ) diff --git a/src/test/prop/prop_mergecmd.c b/src/test/prop/prop_mergecmd.c new file mode 100644 index 0000000000000..281b2cab144b4 --- /dev/null +++ b/src/test/prop/prop_mergecmd.c @@ -0,0 +1,338 @@ +/*------------------------------------------------------------------------- + * + * prop_mergecmd.c + * Property-based tests for MERGE command semantics (not RETURNING). + * + * MERGE joins a target and a source on a key and fires WHEN MATCHED / + * WHEN NOT MATCHED actions. The oracle here is the post-MERGE state: after a + * MATCHED UPDATE every matched target row must carry the source value; after a + * NOT MATCHED INSERT every source key must be present; after a full MERGE the + * row count must equal |target-keys-before UNION source-keys|; a MATCHED DELETE + * must remove exactly the matched rows; and a MERGE over an empty source must + * leave the target untouched. Each is computed independently (in SQL via + * bool_and over the join, or in C from counts read with scalar queries) and + * compared to the state MERGE actually produced. Any disagreement is a bug in + * the executor's MERGE row handling, not a legal difference. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_mergecmd.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "mergecmd" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Scalar boolean oracle: run a "SELECT " and require "t". A NULL result + * or a SQL error is itself a failure. (Copied from prop_numeric.c.) + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Run a scalar query expected to yield a non-NULL integer, and return it as a + * long long. A SQL error or a NULL result is a failure (every count query + * here is a count(*) that is never NULL). + */ +static long long +scalar_int(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + long long val; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + val = atoll(res); + pg_free(res); + if (err) + pg_free(err); + return val; +} + +/* + * Run a side-effecting statement (CREATE/DROP/DML) and fail on error. + */ +static void +exec_ok(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "exec failed: %s\n SQL: %s", err ? err : "", sql); + if (err) + pg_free(err); +} + +/* + * Build the target mc_tgt(id int PRIMARY KEY, v int) and source + * mc_src(id int PRIMARY KEY, v int), each with 0..15 rows and ids 1..n. The + * two id ranges are drawn independently so they overlap partially: some ids + * are in both (MATCHED), some only in the target, some only in the source + * (NOT MATCHED). v is small ([-5, 5]) and differs between the two tables so a + * MATCHED UPDATE that copies s.v is observable. ANALYZE for planner stats. + */ +static void +build_tables(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + int ntgt; + int nsrc; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS mc_tgt; " + "DROP TABLE IF EXISTS mc_src; " + "CREATE TABLE mc_tgt (id int PRIMARY KEY, v int); " + "CREATE TABLE mc_src (id int PRIMARY KEY, v int); "); + + ntgt = (int) pg_prop_draw_int(tc, 0, 15); + for (int id = 1; id <= ntgt; id++) + appendStringInfo(&ddl, "INSERT INTO mc_tgt VALUES (%d, %d); ", + id, (int) pg_prop_draw_int(tc, -5, 5)); + + nsrc = (int) pg_prop_draw_int(tc, 0, 15); + for (int id = 1; id <= nsrc; id++) + appendStringInfo(&ddl, "INSERT INTO mc_src VALUES (%d, %d); ", + id, (int) pg_prop_draw_int(tc, -5, 5)); + + appendStringInfoString(&ddl, "ANALYZE mc_tgt; ANALYZE mc_src;"); + + exec_ok(tc, conn, ddl.data); + pfree(ddl.data); +} + +/* Create a fresh scratch copy mc_cp of mc_tgt (same columns + PK). */ +static void +make_scratch(pg_prop_tc *tc, pg_prop_conn *conn) +{ + exec_ok(tc, conn, + "DROP TABLE IF EXISTS mc_cp; " + "CREATE TABLE mc_cp AS SELECT * FROM mc_tgt; " + "ALTER TABLE mc_cp ADD PRIMARY KEY (id);"); +} + +static void +drop_scratch(pg_prop_tc *tc, pg_prop_conn *conn) +{ + exec_ok(tc, conn, "DROP TABLE mc_cp;"); +} + +/* + * 1. WHEN MATCHED THEN UPDATE SET v = s.v copies the source value onto every + * matched target row. Afterward, joining the copy back to the source on id, + * every joined row must satisfy copy.v = source.v. bool_and over 0 joined + * rows is NULL (no match at all), so coalesce to true -- that case holds + * vacuously. + */ +PG_PROP(merge_matched_updates) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + make_scratch(tc, conn); + + exec_ok(tc, conn, + "MERGE INTO mc_cp t USING mc_src s ON t.id = s.id " + "WHEN MATCHED THEN UPDATE SET v = s.v"); + + require_true(tc, conn, + "SELECT coalesce(bool_and(c.v = s.v), true) " + "FROM mc_cp c JOIN mc_src s USING (id)"); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 2. WHEN NOT MATCHED THEN INSERT (id, v) VALUES (s.id, s.v) adds every source + * row whose id is absent from the target. Afterward every source id must be + * present in the copy. bool_and over an empty source is NULL, so coalesce to + * true -- the no-source case holds vacuously. + */ +PG_PROP(merge_not_matched_inserts) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + make_scratch(tc, conn); + + exec_ok(tc, conn, + "MERGE INTO mc_cp t USING mc_src s ON t.id = s.id " + "WHEN NOT MATCHED THEN INSERT (id, v) VALUES (s.id, s.v)"); + + require_true(tc, conn, + "SELECT coalesce(bool_and(EXISTS " + "(SELECT 1 FROM mc_cp c WHERE c.id = s.id)), true) " + "FROM mc_src s"); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 3. A full MERGE (MATCHED UPDATE + NOT MATCHED INSERT) neither drops nor + * duplicates keys: the copy's row count afterward must equal the number of + * distinct ids in (copy-before UNION source). The union count is read before + * the MERGE (while the copy still holds its original ids) and compared in C to + * the post-MERGE row count. + */ +PG_PROP(merge_both_clauses_count) +{ + pg_prop_conn *conn; + char *err = NULL; + long long expected; + long long post; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + make_scratch(tc, conn); + + expected = scalar_int(tc, conn, + "SELECT count(*) FROM " + "(SELECT id FROM mc_cp UNION SELECT id FROM mc_src) u"); + + exec_ok(tc, conn, + "MERGE INTO mc_cp t USING mc_src s ON t.id = s.id " + "WHEN MATCHED THEN UPDATE SET v = s.v " + "WHEN NOT MATCHED THEN INSERT (id, v) VALUES (s.id, s.v)"); + + post = scalar_int(tc, conn, "SELECT count(*) FROM mc_cp"); + if (post != expected) + pg_prop_fail(tc, + "MERGE row count %lld != |copy UNION source| %lld", + post, expected); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 4. WHEN MATCHED THEN DELETE removes exactly the matched rows: the copy count + * afterward must equal the count before minus the number of ids present in + * both the copy and the source. Both the pre-count and the matched-id count + * are read before the MERGE and the arithmetic is checked in C. + */ +PG_PROP(merge_delete) +{ + pg_prop_conn *conn; + char *err = NULL; + long long pre; + long long matched; + long long post; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + make_scratch(tc, conn); + + pre = scalar_int(tc, conn, "SELECT count(*) FROM mc_cp"); + matched = scalar_int(tc, conn, + "SELECT count(*) FROM mc_cp c " + "WHERE EXISTS (SELECT 1 FROM mc_src s WHERE s.id = c.id)"); + + exec_ok(tc, conn, + "MERGE INTO mc_cp t USING mc_src s ON t.id = s.id " + "WHEN MATCHED THEN DELETE"); + + post = scalar_int(tc, conn, "SELECT count(*) FROM mc_cp"); + if (post != pre - matched) + pg_prop_fail(tc, + "MERGE DELETE count %lld != pre %lld - matched %lld", + post, pre, matched); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 5. A MERGE over an empty source is a no-op: no key can match and none can be + * NOT MATCHED (there are no source rows), so both the row count and the sum of + * v must be unchanged. The empty source is forced with USING (SELECT ... WHERE + * false); the pre-state (count, coalesced sum) is captured and compared in C. + * sum over 0 rows is NULL, so coalesce(sum, 0) keeps the empty-copy case sound. + */ +PG_PROP(merge_no_source_noop) +{ + pg_prop_conn *conn; + char *err = NULL; + long long pre_count; + long long pre_sum; + long long post_count; + long long post_sum; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_tables(tc, conn); + make_scratch(tc, conn); + + pre_count = scalar_int(tc, conn, "SELECT count(*) FROM mc_cp"); + pre_sum = scalar_int(tc, conn, "SELECT coalesce(sum(v), 0) FROM mc_cp"); + + exec_ok(tc, conn, + "MERGE INTO mc_cp t " + "USING (SELECT 1 AS id, 1 AS v WHERE false) s ON t.id = s.id " + "WHEN MATCHED THEN UPDATE SET v = s.v " + "WHEN NOT MATCHED THEN INSERT (id, v) VALUES (s.id, s.v)"); + + post_count = scalar_int(tc, conn, "SELECT count(*) FROM mc_cp"); + post_sum = scalar_int(tc, conn, "SELECT coalesce(sum(v), 0) FROM mc_cp"); + + if (post_count != pre_count) + pg_prop_fail(tc, "empty-source MERGE changed row count %lld -> %lld", + pre_count, post_count); + if (post_sum != pre_sum) + pg_prop_fail(tc, "empty-source MERGE changed sum(v) %lld -> %lld", + pre_sum, post_sum); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 56aac5bc21aec..8cdbb13e040ef 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From d5fcf3d3800486feef381908980e6d262763d696 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:28 -0400 Subject: [PATCH 232/338] prop: add UPDATE SET, FROM-join, and RETURNING semantics property tests (updatesem) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_updatesem.c | 410 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 414 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_updatesem.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 783ebe1ad441b..37484e262b4b9 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -247,7 +247,8 @@ OBJS = \ prop_savepoint.o \ prop_returning.o \ prop_upsert.o \ - prop_mergecmd.o + prop_mergecmd.o \ + prop_updatesem.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 4703a712f4500..090661f88fc48 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -245,6 +245,7 @@ prop_area_sources = files( 'prop_returning.c', 'prop_upsert.c', 'prop_mergecmd.c', + 'prop_updatesem.c', ) diff --git a/src/test/prop/prop_updatesem.c b/src/test/prop/prop_updatesem.c new file mode 100644 index 0000000000000..ce0cf5391a090 --- /dev/null +++ b/src/test/prop/prop_updatesem.c @@ -0,0 +1,410 @@ +/*------------------------------------------------------------------------- + * + * prop_updatesem.c + * Property-based tests for UPDATE statement semantics. + * + * UPDATE must change exactly the matching rows and exactly the named columns, + * leaving everything else alone. The oracle is the statement's own arithmetic + * accounting: pre-UPDATE sums/counts are captured with scalar queries, the + * UPDATE is run, and the post-UPDATE sums/counts are compared in C against the + * value the semantics require (sum grows by count, a filtered subset empties, + * a join adds the matched delta, a no-match UPDATE is a no-op, RETURNING counts + * the matched rows). Any disagreement is a bug in the executor's ModifyTable + * handling, not a legal difference. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_updatesem.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "updatesem" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a scalar query expected to yield a non-NULL integer, and return it as a + * long long. A SQL error or a NULL result is a failure (every scalar here is + * count()/coalesce(sum())/min() guarded to be non-NULL). + */ +static long long +scalar_int(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + long long val; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + val = atoll(res); + pg_free(res); + if (err) + pg_free(err); + return val; +} + +/* + * Run a side-effecting statement (CREATE/DROP/DML) and fail on error. + */ +static void +exec_ok(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "exec failed: %s\n SQL: %s", err ? err : "", sql); + if (err) + pg_free(err); +} + +/* + * Build us_t(id int PRIMARY KEY, g int, v int) with 0..20 rows: ids are 1..n + * assigned densely, g is a small group key in [0, 3] so the g = 0 filter lands + * on a mix of rows and the empty boundary, and v is a small int in [-5, 5] so + * the v > 0 filter straddles zero. ANALYZE so the planner has stats. + */ +static void +build_us_t(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS us_t; " + "CREATE TABLE us_t (id int PRIMARY KEY, g int, v int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 20); + for (int i = 1; i <= nrows; i++) + appendStringInfo(&ddl, + "INSERT INTO us_t VALUES (%d, %d, %d); ", + i, + (int) pg_prop_draw_int(tc, 0, 3), + (int) pg_prop_draw_int(tc, -5, 5)); + + appendStringInfoString(&ddl, "ANALYZE us_t; "); + exec_ok(tc, conn, ddl.data); + pfree(ddl.data); +} + +/* + * Make a fresh scratch copy us_copy of us_t (no PK needed for these checks), + * dropping any leftover from an earlier case first. + */ +static void +make_copy(pg_prop_tc *tc, pg_prop_conn *conn) +{ + exec_ok(tc, conn, + "DROP TABLE IF EXISTS us_copy; " + "CREATE TABLE us_copy AS SELECT * FROM us_t;"); +} + +static void +drop_scratch(pg_prop_tc *tc, pg_prop_conn *conn) +{ + exec_ok(tc, conn, + "DROP TABLE IF EXISTS us_copy; " + "DROP TABLE IF EXISTS us_src; " + "DROP TABLE IF EXISTS us_t;"); +} + +/* + * 1. UPDATE copy SET v = v + 1 touches every row, so the new sum must be the + * old sum plus one per row: new_sum = old_sum + count(*). + */ +PG_PROP(update_all_increments) +{ + pg_prop_conn *conn; + char *err = NULL; + long long oldsum, + cnt, + newsum; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_us_t(tc, conn); + make_copy(tc, conn); + + oldsum = scalar_int(tc, conn, "SELECT coalesce(sum(v), 0) FROM us_copy"); + cnt = scalar_int(tc, conn, "SELECT count(*) FROM us_copy"); + + exec_ok(tc, conn, "UPDATE us_copy SET v = v + 1"); + + newsum = scalar_int(tc, conn, "SELECT coalesce(sum(v), 0) FROM us_copy"); + + if (newsum != oldsum + cnt) + pg_prop_fail(tc, + "SET v = v + 1 broke sum: old=%lld count=%lld new=%lld " + "(expected %lld)", + oldsum, cnt, newsum, oldsum + cnt); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 2. UPDATE copy SET v = 0 WHERE v > 0 clears exactly the positive rows: after + * it, no row has v > 0, and the total row count is unchanged (UPDATE never + * adds or removes rows). + */ +PG_PROP(update_where_subset) +{ + pg_prop_conn *conn; + char *err = NULL; + long long oldcnt, + newcnt, + pos; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_us_t(tc, conn); + make_copy(tc, conn); + + oldcnt = scalar_int(tc, conn, "SELECT count(*) FROM us_copy"); + + exec_ok(tc, conn, "UPDATE us_copy SET v = 0 WHERE v > 0"); + + pos = scalar_int(tc, conn, "SELECT count(*) FROM us_copy WHERE v > 0"); + newcnt = scalar_int(tc, conn, "SELECT count(*) FROM us_copy"); + + if (pos != 0) + pg_prop_fail(tc, "WHERE v > 0 UPDATE left %lld rows with v > 0", pos); + if (newcnt != oldcnt) + pg_prop_fail(tc, "UPDATE changed row count: old=%lld new=%lld", + oldcnt, newcnt); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 3. UPDATE copy c SET v = v + s.delta FROM us_src s WHERE c.id = s.id adds the + * source delta to exactly the ids present in both tables. us_src holds ids + * overlapping us_copy (a random subset of 1..n) with random deltas; since + * us_src.id is unique here, each matched row's new v = old v + delta. The + * oracle: the total increase over matched rows equals sum(delta over matched + * ids), and unmatched rows are unchanged. Both are captured as one scalar: + * sum over the join of (new v - (old v + delta)) must be 0, and the overall + * sum must grow by exactly the matched-delta sum. + */ +PG_PROP(update_from_join) +{ + pg_prop_conn *conn; + char *err = NULL; + int n; + StringInfoData src; + long long oldsum, + deltasum, + newsum, + mismatch; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_us_t(tc, conn); + make_copy(tc, conn); + + n = (int) scalar_int(tc, conn, "SELECT count(*) FROM us_copy"); + + /* + * Build us_src with unique ids overlapping us_copy: for each id 1..n, + * include it ~half the time with a random delta. Unique ids keep the join + * one-to-one so each matched row gets exactly one delta. + */ + initStringInfo(&src); + appendStringInfoString(&src, + "DROP TABLE IF EXISTS us_src; " + "CREATE TABLE us_src (id int, delta int); "); + for (int i = 1; i <= n; i++) + if (pg_prop_draw_bool(tc)) + appendStringInfo(&src, "INSERT INTO us_src VALUES (%d, %d); ", + i, (int) pg_prop_draw_int(tc, -5, 5)); + appendStringInfoString(&src, "ANALYZE us_src; "); + exec_ok(tc, conn, src.data); + pfree(src.data); + + /* record old copy sum and the sum of deltas over ids that will match */ + oldsum = scalar_int(tc, conn, "SELECT coalesce(sum(v), 0) FROM us_copy"); + deltasum = scalar_int(tc, conn, + "SELECT coalesce(sum(s.delta), 0) " + "FROM us_src s JOIN us_copy c ON c.id = s.id"); + + /* + * Snapshot each copy row's pre-UPDATE v keyed by id, so after the UPDATE we + * can check that every matched row moved by exactly its delta and no + * unmatched row moved at all. us_pre lives only for this check. + */ + exec_ok(tc, conn, + "DROP TABLE IF EXISTS us_pre; " + "CREATE TABLE us_pre AS SELECT id, v AS oldv FROM us_copy;"); + + exec_ok(tc, conn, + "UPDATE us_copy c SET v = v + s.delta " + "FROM us_src s WHERE c.id = s.id"); + + newsum = scalar_int(tc, conn, "SELECT coalesce(sum(v), 0) FROM us_copy"); + + /* + * mismatch counts rows whose post value differs from the required value: + * matched rows must be oldv + delta, unmatched rows must be oldv. + */ + mismatch = scalar_int(tc, conn, + "SELECT count(*) " + "FROM us_copy c " + "JOIN us_pre p ON p.id = c.id " + "LEFT JOIN us_src s ON s.id = c.id " + "WHERE c.v <> p.oldv + coalesce(s.delta, 0)"); + + exec_ok(tc, conn, "DROP TABLE IF EXISTS us_pre;"); + + if (mismatch != 0) + pg_prop_fail(tc, + "UPDATE ... FROM moved %lld rows by the wrong amount", + mismatch); + if (newsum != oldsum + deltasum) + pg_prop_fail(tc, + "UPDATE ... FROM broke sum: old=%lld deltasum=%lld " + "new=%lld (expected %lld)", + oldsum, deltasum, newsum, oldsum + deltasum); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 4. UPDATE copy SET v = v + 1, g = g + 10 changes both named columns on every + * row: new sum(v) = old sum(v) + count, and new sum(g) = old sum(g) + + * 10*count. Verifying both together confirms a multi-target SET applies + * every assignment. + */ +PG_PROP(update_multiple_set) +{ + pg_prop_conn *conn; + char *err = NULL; + long long oldv, + oldg, + cnt, + newv, + newg; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_us_t(tc, conn); + make_copy(tc, conn); + + oldv = scalar_int(tc, conn, "SELECT coalesce(sum(v), 0) FROM us_copy"); + oldg = scalar_int(tc, conn, "SELECT coalesce(sum(g), 0) FROM us_copy"); + cnt = scalar_int(tc, conn, "SELECT count(*) FROM us_copy"); + + exec_ok(tc, conn, "UPDATE us_copy SET v = v + 1, g = g + 10"); + + newv = scalar_int(tc, conn, "SELECT coalesce(sum(v), 0) FROM us_copy"); + newg = scalar_int(tc, conn, "SELECT coalesce(sum(g), 0) FROM us_copy"); + + if (newv != oldv + cnt) + pg_prop_fail(tc, + "multi-SET v wrong: old=%lld count=%lld new=%lld " + "(expected %lld)", + oldv, cnt, newv, oldv + cnt); + if (newg != oldg + 10 * cnt) + pg_prop_fail(tc, + "multi-SET g wrong: old=%lld count=%lld new=%lld " + "(expected %lld)", + oldg, cnt, newg, oldg + 10 * cnt); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 5. UPDATE copy SET v = 999 WHERE id = -1 matches nothing (ids are 1..n), so + * it is a no-op: both count(*) and sum(v) are unchanged. + */ +PG_PROP(update_no_match_noop) +{ + pg_prop_conn *conn; + char *err = NULL; + long long oldcnt, + oldsum, + newcnt, + newsum; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_us_t(tc, conn); + make_copy(tc, conn); + + oldcnt = scalar_int(tc, conn, "SELECT count(*) FROM us_copy"); + oldsum = scalar_int(tc, conn, "SELECT coalesce(sum(v), 0) FROM us_copy"); + + exec_ok(tc, conn, "UPDATE us_copy SET v = 999 WHERE id = (-1)"); + + newcnt = scalar_int(tc, conn, "SELECT count(*) FROM us_copy"); + newsum = scalar_int(tc, conn, "SELECT coalesce(sum(v), 0) FROM us_copy"); + + if (newcnt != oldcnt) + pg_prop_fail(tc, "no-match UPDATE changed count: old=%lld new=%lld", + oldcnt, newcnt); + if (newsum != oldsum) + pg_prop_fail(tc, "no-match UPDATE changed sum: old=%lld new=%lld", + oldsum, newsum); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * 6. WITH u AS (UPDATE copy SET v = v WHERE g = 0 RETURNING id) SELECT count(*) + * FROM u returns exactly one id per updated row, so the count must equal the + * pre-UPDATE count of g = 0 rows. (v = v keeps values stable so the test + * isolates the RETURNING row count from any value change.) + */ +PG_PROP(update_returning_count) +{ + pg_prop_conn *conn; + char *err = NULL; + long long preg0, + returned; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_us_t(tc, conn); + make_copy(tc, conn); + + preg0 = scalar_int(tc, conn, "SELECT count(*) FROM us_copy WHERE g = 0"); + + returned = scalar_int(tc, conn, + "WITH u AS (UPDATE us_copy SET v = v WHERE g = 0 " + "RETURNING id) SELECT count(*) FROM u"); + + if (returned != preg0) + pg_prop_fail(tc, + "RETURNING count %lld != pre-count of g=0 rows %lld", + returned, preg0); + + drop_scratch(tc, conn); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 8cdbb13e040ef..fef5a911f5f61 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From fa7fe395180ede6b157469361005edbfac949010 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:28 -0400 Subject: [PATCH 233/338] prop: add column DEFAULT, generated columns, NOT NULL property tests (defaults) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_defaults.c | 290 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 294 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_defaults.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 37484e262b4b9..80588e80db8e5 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -248,7 +248,8 @@ OBJS = \ prop_returning.o \ prop_upsert.o \ prop_mergecmd.o \ - prop_updatesem.o + prop_updatesem.o \ + prop_defaults.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 090661f88fc48..71fd664abaecb 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -246,6 +246,7 @@ prop_area_sources = files( 'prop_upsert.c', 'prop_mergecmd.c', 'prop_updatesem.c', + 'prop_defaults.c', ) diff --git a/src/test/prop/prop_defaults.c b/src/test/prop/prop_defaults.c new file mode 100644 index 0000000000000..fcfa04b57efd5 --- /dev/null +++ b/src/test/prop/prop_defaults.c @@ -0,0 +1,290 @@ +/*------------------------------------------------------------------------- + * + * prop_defaults.c + * Property-based tests for column DEFAULTs, generated columns and NOT NULL. + * + * The oracle is the stated semantics of each column feature: a column DEFAULT + * supplies its value when an INSERT omits the column; a GENERATED ALWAYS AS + * STORED column equals its expression over the base columns on both INSERT + * and UPDATE; a NOT NULL constraint rejects a NULL; an IDENTITY column yields + * strictly increasing values. Each property builds a fresh scratch table, + * drives one INSERT/UPDATE, and checks the resulting value with a scalar + * SELECT (or, for NOT NULL, that the INSERT fails and the table stays empty). + * A mismatch is a bug in default/generated/constraint handling. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_defaults.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "defaults" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error is itself a failure (the oracle expects a definite true). + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Run DDL/DML and require success; a failure here is a setup bug, not a + * property result, so report it with the SQL. + */ +static void +require_exec(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "exec failed: %s\n SQL: %s", err ? err : "", sql); + if (err) + pg_free(err); +} + +/* + * Draw an integer in [-100, 100]. Returned as an int; callers that embed it + * in SQL must parenthesize when it may be negative so a leading '-' does not + * misparse in a larger expression. + */ +static int +draw_k(pg_prop_tc *tc) +{ + return (int) pg_prop_draw_int(tc, -100, 100); +} + +/* + * A column DEFAULT is applied when the INSERT omits the column: + * df(id int, v int DEFAULT 42); INSERT (id) only => v = 42. + */ +PG_PROP(default_applied) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_exec(tc, conn, + "DROP TABLE IF EXISTS df; " + "CREATE TABLE df (id int, v int DEFAULT 42); " + "INSERT INTO df (id) VALUES (1);"); + + require_true(tc, conn, "SELECT v = 42 FROM df WHERE id = 1"); + + require_exec(tc, conn, "DROP TABLE df;"); + pg_prop_disconnect(conn); +} + +/* + * A DEFAULT expression is evaluated: v int DEFAULT (10+5) => inserted v = 15. + */ +PG_PROP(default_expression) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_exec(tc, conn, + "DROP TABLE IF EXISTS df; " + "CREATE TABLE df (id int, v int DEFAULT (10+5)); " + "INSERT INTO df (id) VALUES (1);"); + + require_true(tc, conn, "SELECT v = 15 FROM df WHERE id = 1"); + + require_exec(tc, conn, "DROP TABLE df;"); + pg_prop_disconnect(conn); +} + +/* + * A GENERATED ALWAYS AS (a*2) STORED column equals 2k when a = k is inserted. + */ +PG_PROP(generated_stored) +{ + pg_prop_conn *conn; + char *err = NULL; + int k = draw_k(tc); + StringInfoData sql; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_exec(tc, conn, + "DROP TABLE IF EXISTS dg; " + "CREATE TABLE dg (a int, g int GENERATED ALWAYS AS (a*2) STORED);"); + + initStringInfo(&sql); + appendStringInfo(&sql, "INSERT INTO dg (a) VALUES ((%d));", k); + require_exec(tc, conn, sql.data); + + resetStringInfo(&sql); + appendStringInfo(&sql, "SELECT g = (%d) FROM dg", 2 * k); + require_true(tc, conn, sql.data); + + require_exec(tc, conn, "DROP TABLE dg;"); + pfree(sql.data); + pg_prop_disconnect(conn); +} + +/* + * A STORED generated column recomputes on UPDATE: insert a = k, then + * UPDATE a = m => g = 2m, independent of the original k. + */ +PG_PROP(generated_updates_with_base) +{ + pg_prop_conn *conn; + char *err = NULL; + int k = draw_k(tc); + int m = draw_k(tc); + StringInfoData sql; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_exec(tc, conn, + "DROP TABLE IF EXISTS dg; " + "CREATE TABLE dg (a int, g int GENERATED ALWAYS AS (a*2) STORED);"); + + initStringInfo(&sql); + appendStringInfo(&sql, "INSERT INTO dg (a) VALUES ((%d));", k); + require_exec(tc, conn, sql.data); + + resetStringInfo(&sql); + appendStringInfo(&sql, "UPDATE dg SET a = (%d);", m); + require_exec(tc, conn, sql.data); + + resetStringInfo(&sql); + appendStringInfo(&sql, "SELECT g = (%d) FROM dg", 2 * m); + require_true(tc, conn, sql.data); + + require_exec(tc, conn, "DROP TABLE dg;"); + pfree(sql.data); + pg_prop_disconnect(conn); +} + +/* + * A NOT NULL column rejects a NULL insert: the INSERT must fail + * (pg_prop_sql_exec returns false) and the table must stay empty. + */ +PG_PROP(not_null_rejects) +{ + pg_prop_conn *conn; + char *err = NULL; + char *ierr = NULL; + bool ok; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_exec(tc, conn, + "DROP TABLE IF EXISTS dn; " + "CREATE TABLE dn (a int NOT NULL);"); + + ok = pg_prop_sql_exec(conn, "INSERT INTO dn (a) VALUES (NULL);", &ierr); + if (ok) + pg_prop_fail(tc, "NULL insert into NOT NULL column unexpectedly succeeded"); + if (ierr) + pg_free(ierr); + + require_true(tc, conn, "SELECT count(*) = 0 FROM dn"); + + require_exec(tc, conn, "DROP TABLE dn;"); + pg_prop_disconnect(conn); +} + +/* + * An IDENTITY column increases: insert two rows, the two ids differ and the + * second is strictly greater than the first. + */ +PG_PROP(serial_increments) +{ + pg_prop_conn *conn; + char *err = NULL; + int k1 = draw_k(tc); + int k2 = draw_k(tc); + StringInfoData sql; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_exec(tc, conn, + "DROP TABLE IF EXISTS ds; " + "CREATE TABLE ds (id int GENERATED ALWAYS AS IDENTITY, v int);"); + + initStringInfo(&sql); + appendStringInfo(&sql, + "INSERT INTO ds (v) VALUES ((%d)); " + "INSERT INTO ds (v) VALUES ((%d));", + k1, k2); + require_exec(tc, conn, sql.data); + + /* + * Two rows, ordered by insertion value is not reliable; order by id and + * require the lower id strictly less than the higher. min < max also + * proves they differ. + */ + require_true(tc, conn, + "SELECT min(id) < max(id) FROM ds"); + + require_exec(tc, conn, "DROP TABLE ds;"); + pfree(sql.data); + pg_prop_disconnect(conn); +} + +/* + * A column with no default inserts NULL: dz(id int, v int); INSERT id only + * => v IS NULL. + */ +PG_PROP(default_null) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + require_exec(tc, conn, + "DROP TABLE IF EXISTS dz; " + "CREATE TABLE dz (id int, v int); " + "INSERT INTO dz (id) VALUES (1);"); + + require_true(tc, conn, "SELECT v IS NULL FROM dz WHERE id = 1"); + + require_exec(tc, conn, "DROP TABLE dz;"); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index fef5a911f5f61..10f00e0bdfaba 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 6b8f3c749ff99e5f27b8568d6bedf94d1443881a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:28 -0400 Subject: [PATCH 234/338] prop: add CHECK and domain constraint enforcement property tests (checkconstraint) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_checkconstraint.c | 216 +++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 220 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_checkconstraint.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 80588e80db8e5..a473323a6928d 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -249,7 +249,8 @@ OBJS = \ prop_upsert.o \ prop_mergecmd.o \ prop_updatesem.o \ - prop_defaults.o + prop_defaults.o \ + prop_checkconstraint.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 71fd664abaecb..762baac32fc46 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -247,6 +247,7 @@ prop_area_sources = files( 'prop_mergecmd.c', 'prop_updatesem.c', 'prop_defaults.c', + 'prop_checkconstraint.c', ) diff --git a/src/test/prop/prop_checkconstraint.c b/src/test/prop/prop_checkconstraint.c new file mode 100644 index 0000000000000..f29ec3bb3cbc0 --- /dev/null +++ b/src/test/prop/prop_checkconstraint.c @@ -0,0 +1,216 @@ +/*------------------------------------------------------------------------- + * + * prop_checkconstraint.c + * Property-based tests for CHECK and domain constraint enforcement. + * + * The oracle is the SQL standard semantics of constraint enforcement: a row + * is rejected (INSERT fails) iff a CHECK predicate evaluates to FALSE, and is + * accepted when the predicate is TRUE or UNKNOWN (NULL). Domain CHECKs and + * NOT NULL follow the same rule. Each property builds a fresh scratch table + * (and domain, where relevant) with a unique name, then asserts the exact + * pg_prop_sql_exec outcome for allowed vs rejected rows. A wrong outcome -- + * a violating row accepted, or a legal row rejected -- is a constraint bug. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_checkconstraint.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "checkconstraint" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run "sql" for side effects and require it to succeed (want==true) or to be + * rejected by a constraint (want==false). A mismatch is the property + * failure: a violating row accepted, or a legal row wrongly rejected. + */ +static void +require_exec(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql, bool want) +{ + char *err = NULL; + bool ok; + + ok = pg_prop_sql_exec(conn, sql, &err); + if (ok != want) + { + pg_prop_fail(tc, + "expected %s but got %s for: %s\n error: %s", + want ? "success" : "rejection", + ok ? "success" : "rejection", + sql, + err ? err : "(none)"); + } + if (err) + pg_free(err); +} + +/* Setup DDL/DML that must succeed; a failure here is a harness/env problem. */ +static void +must_setup(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "setup failed: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (err) + pg_free(err); +} + +/* + * CHECK (v > 0): a violating insert (v = 0) must be rejected and a satisfying + * insert (v = 5) must succeed. + */ +PG_PROP(check_rejects_violation) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + must_setup(tc, conn, + "DROP TABLE IF EXISTS cc_rv; " + "CREATE TABLE cc_rv (v int CHECK (v > 0));"); + + require_exec(tc, conn, "INSERT INTO cc_rv VALUES (0)", false); + require_exec(tc, conn, "INSERT INTO cc_rv VALUES (5)", true); + + must_setup(tc, conn, "DROP TABLE cc_rv;"); + pg_prop_disconnect(conn); +} + +/* + * CHECK (v > 0) must accept NULL: the predicate is UNKNOWN on NULL, and a + * CHECK fails only on FALSE, so INSERT NULL succeeds. + */ +PG_PROP(check_allows_null) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + must_setup(tc, conn, + "DROP TABLE IF EXISTS cc_null; " + "CREATE TABLE cc_null (v int CHECK (v > 0));"); + + require_exec(tc, conn, "INSERT INTO cc_null VALUES (NULL)", true); + + must_setup(tc, conn, "DROP TABLE cc_null;"); + pg_prop_disconnect(conn); +} + +/* + * A domain CHECK (VALUE > 0) enforces the same way as a table CHECK: a column + * of that domain rejects values <= 0 and accepts positive values. + */ +PG_PROP(domain_constraint) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + must_setup(tc, conn, + "DROP TABLE IF EXISTS cc_dom; " + "DROP DOMAIN IF EXISTS cc_posint; " + "CREATE DOMAIN cc_posint AS int CHECK (VALUE > 0); " + "CREATE TABLE cc_dom (v cc_posint);"); + + require_exec(tc, conn, "INSERT INTO cc_dom VALUES (0)", false); + require_exec(tc, conn, "INSERT INTO cc_dom VALUES (5)", true); + + must_setup(tc, conn, + "DROP TABLE cc_dom; " + "DROP DOMAIN cc_posint;"); + pg_prop_disconnect(conn); +} + +/* + * CHECK (v BETWEEN 1 AND 10): a value inside the range is accepted; values + * below (0) and above (11) the range are rejected. + */ +PG_PROP(multi_check) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + must_setup(tc, conn, + "DROP TABLE IF EXISTS cc_mc; " + "CREATE TABLE cc_mc (v int CHECK (v BETWEEN 1 AND 10));"); + + require_exec(tc, conn, "INSERT INTO cc_mc VALUES (5)", true); + require_exec(tc, conn, "INSERT INTO cc_mc VALUES (0)", false); + require_exec(tc, conn, "INSERT INTO cc_mc VALUES (11)", false); + + must_setup(tc, conn, "DROP TABLE cc_mc;"); + pg_prop_disconnect(conn); +} + +/* + * CHECK (a + b > 0) over two columns: (3, -1) sums to 2 (> 0) and is + * accepted; (1, -5) sums to -4 and is rejected. + */ +PG_PROP(check_on_expression) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + must_setup(tc, conn, + "DROP TABLE IF EXISTS cc_ce; " + "CREATE TABLE cc_ce (a int, b int, CHECK (a + b > 0));"); + + require_exec(tc, conn, "INSERT INTO cc_ce VALUES (3, (-1))", true); + require_exec(tc, conn, "INSERT INTO cc_ce VALUES (1, (-5))", false); + + must_setup(tc, conn, "DROP TABLE cc_ce;"); + pg_prop_disconnect(conn); +} + +/* + * A column that is both NOT NULL and CHECK (v > 0): a NULL is rejected by the + * NOT NULL constraint (the CHECK alone would allow it), while a valid positive + * value is accepted. + */ +PG_PROP(not_null_and_check) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + must_setup(tc, conn, + "DROP TABLE IF EXISTS cc_nn; " + "CREATE TABLE cc_nn (v int NOT NULL CHECK (v > 0));"); + + require_exec(tc, conn, "INSERT INTO cc_nn VALUES (NULL)", false); + require_exec(tc, conn, "INSERT INTO cc_nn VALUES (5)", true); + + must_setup(tc, conn, "DROP TABLE cc_nn;"); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 10f00e0bdfaba..c4a84cc151881 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From e97cd32fe8bd300604be104723eb8d86bf0efc1d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:29 -0400 Subject: [PATCH 235/338] prop: add FOREIGN KEY enforcement and cascade actions property tests (foreignkey) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_foreignkey.c | 339 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 343 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_foreignkey.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index a473323a6928d..978df0e534333 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -250,7 +250,8 @@ OBJS = \ prop_mergecmd.o \ prop_updatesem.o \ prop_defaults.o \ - prop_checkconstraint.o + prop_checkconstraint.o \ + prop_foreignkey.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 762baac32fc46..e664c61531a0c 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -248,6 +248,7 @@ prop_area_sources = files( 'prop_updatesem.c', 'prop_defaults.c', 'prop_checkconstraint.c', + 'prop_foreignkey.c', ) diff --git a/src/test/prop/prop_foreignkey.c b/src/test/prop/prop_foreignkey.c new file mode 100644 index 0000000000000..2b1d968b4104c --- /dev/null +++ b/src/test/prop/prop_foreignkey.c @@ -0,0 +1,339 @@ +/*------------------------------------------------------------------------- + * + * prop_foreignkey.c + * Property-based tests for FOREIGN KEY enforcement and referential actions. + * + * The oracle is the referential-integrity contract itself: a FK constraint + * must reject a child row that references a non-existent parent, must permit + * a NULL FK column, and must apply the declared ON DELETE / ON UPDATE action + * (CASCADE, RESTRICT, SET NULL) exactly when the referenced parent changes. + * Each property builds a fresh parent/child pair, drives DDL/DML through + * pg_prop_sql_exec, asserts the exec succeeds or fails as the contract + * requires, and verifies the post-state row counts in C. A deviation is a + * referential-integrity bug (trigger firing, constraint check, or action + * propagation), not a rounding artefact. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_foreignkey.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "foreignkey" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Connect or fail. Every property opens one connection and closes it at the + * end; a failure/assume longjmps over the disconnect, and the driver's + * per-case cleanup reclaims the handle. + */ +static pg_prop_conn * +connect_or_fail(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + return conn; +} + +/* + * Run DDL/DML that MUST succeed; fail the case (with the SQL and error) if it + * does not. Used for setup and for operations the FK contract must allow. + */ +static void +exec_ok(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + { + pg_prop_fail(tc, "expected success but failed: %s\n SQL: %s", + err ? err : "(unknown)", sql); + } + if (err) + pg_free(err); +} + +/* + * Run DML that MUST be rejected by the FK constraint; fail the case if it + * unexpectedly succeeds. The error is discarded -- a rejection is the + * expected outcome. + */ +static void +exec_rejected(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (pg_prop_sql_exec(conn, sql, &err)) + { + pg_prop_fail(tc, "expected FK rejection but succeeded: %s", sql); + } + if (err) + pg_free(err); +} + +/* + * Read a single integer count via a "SELECT count(*) ..." (never NULL: count + * over zero rows is 0, so the scalar is always a well-defined integer). Fail + * the case on SQL error or NULL. + */ +static long +scalar_count(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + long n; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "count query failed: %s\n SQL: %s", + err ? err : "(unknown)", sql); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL count for: %s", sql); + n = atol(res); + pg_free(res); + if (err) + pg_free(err); + return n; +} + +/* + * Build a fresh parent (fk_p) and child (fk_c) pair. The child FK clause is + * caller-supplied so each property declares the referential action it tests. + * The parent id is a PRIMARY KEY so it can be referenced; the child has its + * own serial-free id plus a pid FK column. + */ +static void +build_tables(pg_prop_tc *tc, pg_prop_conn *conn, const char *fk_clause) +{ + StringInfoData ddl; + + initStringInfo(&ddl); + appendStringInfo(&ddl, + "DROP TABLE IF EXISTS fk_c; " + "DROP TABLE IF EXISTS fk_p; " + "CREATE TABLE fk_p (id int PRIMARY KEY); " + "CREATE TABLE fk_c (id int, pid int %s); ", + fk_clause); + exec_ok(tc, conn, ddl.data); + pfree(ddl.data); +} + +/* Drop both tables so a leaked schema cannot leak into the next case. */ +static void +drop_tables(pg_prop_tc *tc, pg_prop_conn *conn) +{ + exec_ok(tc, conn, "DROP TABLE IF EXISTS fk_c; DROP TABLE IF EXISTS fk_p;"); +} + +/* + * fk_rejects_orphan: a FK rejects a child referencing a non-existent parent. + * Insert parent id=1; inserting a child with a random pid that is NOT a parent + * id must be rejected, while pid=1 (an existing parent) must succeed. + */ +PG_PROP(fk_rejects_orphan) +{ + pg_prop_conn *conn = connect_or_fail(tc); + StringInfoData q; + int orphan; + + build_tables(tc, conn, "REFERENCES fk_p(id)"); + exec_ok(tc, conn, "INSERT INTO fk_p(id) VALUES (1)"); + + /* A pid that is deliberately not the single parent id (1). */ + orphan = (int) pg_prop_draw_int(tc, 2, 1000); + + initStringInfo(&q); + appendStringInfo(&q, "INSERT INTO fk_c(id, pid) VALUES (1, %d)", orphan); + exec_rejected(tc, conn, q.data); + pfree(q.data); + + /* Referencing the existing parent must be allowed. */ + exec_ok(tc, conn, "INSERT INTO fk_c(id, pid) VALUES (2, 1)"); + + if (scalar_count(tc, conn, "SELECT count(*) FROM fk_c") != 1) + pg_prop_fail(tc, "expected exactly 1 child row after orphan rejection"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * fk_allows_null: a nullable FK column accepts NULL (no parent required). + * The parent may be empty or not; a NULL pid must insert regardless. + */ +PG_PROP(fk_allows_null) +{ + pg_prop_conn *conn = connect_or_fail(tc); + + build_tables(tc, conn, "REFERENCES fk_p(id)"); + + /* Optionally populate a parent; it must not matter for a NULL FK. */ + if (pg_prop_draw_bool(tc)) + exec_ok(tc, conn, "INSERT INTO fk_p(id) VALUES (1)"); + + exec_ok(tc, conn, "INSERT INTO fk_c(id, pid) VALUES (1, NULL)"); + + if (scalar_count(tc, conn, + "SELECT count(*) FROM fk_c WHERE pid IS NULL") != 1) + pg_prop_fail(tc, "expected 1 child row with NULL pid"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * cascade_delete: ON DELETE CASCADE removes children when the parent is + * deleted. Insert one parent and N (1..10) children referencing it, delete + * the parent, and require the child count to be 0. + */ +PG_PROP(cascade_delete) +{ + pg_prop_conn *conn = connect_or_fail(tc); + StringInfoData ins; + int nchild; + + build_tables(tc, conn, "REFERENCES fk_p(id) ON DELETE CASCADE"); + exec_ok(tc, conn, "INSERT INTO fk_p(id) VALUES (1)"); + + nchild = (int) pg_prop_draw_int(tc, 1, 10); + initStringInfo(&ins); + for (int i = 0; i < nchild; i++) + appendStringInfo(&ins, "INSERT INTO fk_c(id, pid) VALUES (%d, 1); ", i); + exec_ok(tc, conn, ins.data); + pfree(ins.data); + + if (scalar_count(tc, conn, "SELECT count(*) FROM fk_c") != nchild) + pg_prop_fail(tc, "expected %d children before delete", nchild); + + exec_ok(tc, conn, "DELETE FROM fk_p WHERE id = 1"); + + if (scalar_count(tc, conn, "SELECT count(*) FROM fk_c") != 0) + pg_prop_fail(tc, "ON DELETE CASCADE left children behind"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * restrict_delete: ON DELETE RESTRICT blocks deleting a referenced parent. + * With at least one child referencing the parent, DELETE of the parent must + * be rejected and the parent must still be present. + */ +PG_PROP(restrict_delete) +{ + pg_prop_conn *conn = connect_or_fail(tc); + StringInfoData ins; + int nchild; + + build_tables(tc, conn, "REFERENCES fk_p(id) ON DELETE RESTRICT"); + exec_ok(tc, conn, "INSERT INTO fk_p(id) VALUES (1)"); + + nchild = (int) pg_prop_draw_int(tc, 1, 10); + initStringInfo(&ins); + for (int i = 0; i < nchild; i++) + appendStringInfo(&ins, "INSERT INTO fk_c(id, pid) VALUES (%d, 1); ", i); + exec_ok(tc, conn, ins.data); + pfree(ins.data); + + exec_rejected(tc, conn, "DELETE FROM fk_p WHERE id = 1"); + + if (scalar_count(tc, conn, "SELECT count(*) FROM fk_p WHERE id = 1") != 1) + pg_prop_fail(tc, "ON DELETE RESTRICT allowed the parent to be deleted"); + if (scalar_count(tc, conn, "SELECT count(*) FROM fk_c") != nchild) + pg_prop_fail(tc, "children changed despite blocked parent delete"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * set_null_delete: ON DELETE SET NULL nulls the child FK when the parent is + * deleted. After deleting the parent, every child must survive with pid NULL. + */ +PG_PROP(set_null_delete) +{ + pg_prop_conn *conn = connect_or_fail(tc); + StringInfoData ins; + int nchild; + + build_tables(tc, conn, "REFERENCES fk_p(id) ON DELETE SET NULL"); + exec_ok(tc, conn, "INSERT INTO fk_p(id) VALUES (1)"); + + nchild = (int) pg_prop_draw_int(tc, 1, 10); + initStringInfo(&ins); + for (int i = 0; i < nchild; i++) + appendStringInfo(&ins, "INSERT INTO fk_c(id, pid) VALUES (%d, 1); ", i); + exec_ok(tc, conn, ins.data); + pfree(ins.data); + + exec_ok(tc, conn, "DELETE FROM fk_p WHERE id = 1"); + + /* All children survive, all with pid now NULL. */ + if (scalar_count(tc, conn, "SELECT count(*) FROM fk_c") != nchild) + pg_prop_fail(tc, "ON DELETE SET NULL removed children (expected %d)", + nchild); + if (scalar_count(tc, conn, + "SELECT count(*) FROM fk_c WHERE pid IS NULL") != nchild) + pg_prop_fail(tc, "ON DELETE SET NULL did not null all child pids"); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} + +/* + * cascade_update: ON UPDATE CASCADE propagates a parent id change to the + * children. Insert parent id=1 and children referencing it, update the + * parent id to a fresh value, and require every child's pid to follow. + */ +PG_PROP(cascade_update) +{ + pg_prop_conn *conn = connect_or_fail(tc); + StringInfoData ins; + StringInfoData q; + int nchild; + int newid; + + build_tables(tc, conn, "REFERENCES fk_p(id) ON UPDATE CASCADE"); + exec_ok(tc, conn, "INSERT INTO fk_p(id) VALUES (1)"); + + nchild = (int) pg_prop_draw_int(tc, 1, 10); + initStringInfo(&ins); + for (int i = 0; i < nchild; i++) + appendStringInfo(&ins, "INSERT INTO fk_c(id, pid) VALUES (%d, 1); ", i); + exec_ok(tc, conn, ins.data); + pfree(ins.data); + + /* A new parent id distinct from the old one (1). */ + newid = (int) pg_prop_draw_int(tc, 2, 1000); + + initStringInfo(&q); + appendStringInfo(&q, "UPDATE fk_p SET id = %d WHERE id = 1", newid); + exec_ok(tc, conn, q.data); + pfree(q.data); + + /* No child should still reference the old id. */ + if (scalar_count(tc, conn, "SELECT count(*) FROM fk_c WHERE pid = 1") != 0) + pg_prop_fail(tc, "ON UPDATE CASCADE left a child on the old parent id"); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT count(*) FROM fk_c WHERE pid = %d", newid); + if (scalar_count(tc, conn, q.data) != nchild) + pg_prop_fail(tc, "ON UPDATE CASCADE did not move all %d children to %d", + nchild, newid); + pfree(q.data); + + drop_tables(tc, conn); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index c4a84cc151881..f3ef63fbb0db5 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 22154cff03e26e40cad1ec8f58d62f8d7309017c Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:29 -0400 Subject: [PATCH 236/338] prop: add views and the rule-system rewrite property tests (views) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_views.c | 332 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 336 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_views.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 978df0e534333..930c9e95d90a4 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -251,7 +251,8 @@ OBJS = \ prop_updatesem.o \ prop_defaults.o \ prop_checkconstraint.o \ - prop_foreignkey.o + prop_foreignkey.o \ + prop_views.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index e664c61531a0c..b2a1aaa9a256f 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -249,6 +249,7 @@ prop_area_sources = files( 'prop_defaults.c', 'prop_checkconstraint.c', 'prop_foreignkey.c', + 'prop_views.c', ) diff --git a/src/test/prop/prop_views.c b/src/test/prop/prop_views.c new file mode 100644 index 0000000000000..e99af1ee4b2a8 --- /dev/null +++ b/src/test/prop/prop_views.c @@ -0,0 +1,332 @@ +/*------------------------------------------------------------------------- + * + * prop_views.c + * Property-based tests for views and the rule system (src/backend/rewrite). + * + * A view is syntactic sugar over a query implemented by an ON SELECT rewrite + * rule, and a simple updatable view maps INSERT/UPDATE/DELETE straight through + * to its base relation. The oracles here are therefore definitional: a view's + * rows must equal its defining query (differential, shape B), and a change + * made through a simple updatable view must appear in the base table (scalar, + * shape A). A materialized view instead snapshots its query at CREATE/REFRESH + * time, so its stored value must equal the query result as of the last refresh. + * Any disagreement is a rewrite-rule bug, not a legal result. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_views.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "views" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Small integer domains keep each generated table legible in a counterexample + * and cover the boundaries the view predicates key on (v > 0, g = 0): v and g + * straddle zero, ids are a dense unique key. + */ +#define VAL_MIN (-5) +#define VAL_MAX 5 + +/* + * Build a table named "name" shaped (id int PRIMARY KEY, g int, v int) with + * 0..25 random rows (ids 1..nrows, g and v in [VAL_MIN, VAL_MAX]). Drops any + * prior table of that name first so each property starts fresh. Fails the + * case on any DDL/DML error (a build error is a harness problem, not a view + * property, so surface it loudly). + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn, const char *name) +{ + StringInfoData ddl; + char *err = NULL; + int nrows = (int) pg_prop_draw_int(tc, 0, 25); + + initStringInfo(&ddl); + appendStringInfo(&ddl, + "DROP TABLE IF EXISTS %s CASCADE; " + "CREATE TABLE %s (id int PRIMARY KEY, g int, v int); ", + name, name); + for (int r = 1; r <= nrows; r++) + appendStringInfo(&ddl, + "INSERT INTO %s VALUES (%d, %d, %d); ", + name, r, + (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX), + (int) pg_prop_draw_int(tc, VAL_MIN, VAL_MAX)); + appendStringInfo(&ddl, "ANALYZE %s; ", name); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n SQL: %s", + err ? err : "", ddl.data); + pfree(ddl.data); +} + +/* + * Shape B: two result-set queries must return byte-identical rows. Each MUST + * carry a total ORDER BY so the row-major render compares as an ordered list. + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *q1, const char *q2) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL, + *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); + if (f1) + pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); + r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); + if (f2) + pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", + q1, q2, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Shape A: run a boolean SELECT and require "t". A NULL result or SQL error + * is a failure, reported with the SQL. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Run a DDL/DML statement, failing the case on error. Used for CREATE/DROP + * VIEW and the through-view writes. + */ +static void +run_ddl(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "DDL failed: %s\n SQL: %s", err ? err : "", sql); +} + +/* + * Property 1 (B): a view's rows equal its defining query. + * + * CREATE VIEW vw_p1_v1 AS SELECT id, v FROM vw_p1 WHERE v > 0; selecting from + * the view must equal selecting the same projection+filter from the base. + */ +PG_PROP(view_eq_query) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn, "vw_p1"); + run_ddl(tc, conn, "DROP VIEW IF EXISTS vw_p1_v1; " + "CREATE VIEW vw_p1_v1 AS SELECT id, v FROM vw_p1 WHERE v > 0;"); + + require_same_rows(tc, conn, + "SELECT id, v FROM vw_p1_v1 ORDER BY id", + "SELECT id, v FROM vw_p1 WHERE v > 0 ORDER BY id"); + + run_ddl(tc, conn, "DROP VIEW IF EXISTS vw_p1_v1;"); + run_ddl(tc, conn, "DROP TABLE IF EXISTS vw_p1 CASCADE;"); + pg_prop_disconnect(conn); +} + +/* + * Property 2 (B): a view on a view equals the composed query. + * + * vw_p2_v1 = rows of vw_p2 with v>0; vw_p2_v2 = ids from vw_p2_v1 with g=0. + * The composed base query (v>0 AND g=0) must match vw_p2_v2. + */ +PG_PROP(view_of_view) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn, "vw_p2"); + run_ddl(tc, conn, + "DROP VIEW IF EXISTS vw_p2_v2; " + "DROP VIEW IF EXISTS vw_p2_v1; " + "CREATE VIEW vw_p2_v1 AS SELECT * FROM vw_p2 WHERE v > 0; " + "CREATE VIEW vw_p2_v2 AS SELECT id FROM vw_p2_v1 WHERE g = 0;"); + + require_same_rows(tc, conn, + "SELECT id FROM vw_p2_v2 ORDER BY id", + "SELECT id FROM vw_p2 WHERE v > 0 AND g = 0 ORDER BY id"); + + run_ddl(tc, conn, "DROP VIEW IF EXISTS vw_p2_v2; DROP VIEW IF EXISTS vw_p2_v1;"); + run_ddl(tc, conn, "DROP TABLE IF EXISTS vw_p2 CASCADE;"); + pg_prop_disconnect(conn); +} + +/* + * Property 3 (A): INSERT through a simple updatable view hits the base. + * + * vw_p3_v1 = SELECT id, v FROM vw_p3; inserting (9999, 5) through the view + * must create exactly that row in the base table (id 9999 is above the 1..25 + * ids build_table uses, so it never collides with the PRIMARY KEY). + */ +PG_PROP(updatable_view_insert) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn, "vw_p3"); + run_ddl(tc, conn, "DROP VIEW IF EXISTS vw_p3_v1; " + "CREATE VIEW vw_p3_v1 AS SELECT id, v FROM vw_p3;"); + run_ddl(tc, conn, "INSERT INTO vw_p3_v1 VALUES (9999, 5);"); + + /* exactly one base row with id 9999 and v 5 (g defaults to NULL). */ + require_true(tc, conn, + "SELECT count(*) = 1 FROM vw_p3 " + "WHERE id = 9999 AND v = 5 AND g IS NULL"); + + run_ddl(tc, conn, "DROP VIEW IF EXISTS vw_p3_v1;"); + run_ddl(tc, conn, "DROP TABLE IF EXISTS vw_p3 CASCADE;"); + pg_prop_disconnect(conn); +} + +/* + * Property 4 (A): UPDATE through a simple updatable view changes the base. + * + * Insert a known row (9999, 7) through the view, UPDATE its v to 0 through the + * view, then require the base row's v is 0. Using a self-inserted id avoids + * depending on which random ids exist. + */ +PG_PROP(updatable_view_update) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn, "vw_p4"); + run_ddl(tc, conn, "DROP VIEW IF EXISTS vw_p4_v1; " + "CREATE VIEW vw_p4_v1 AS SELECT id, v FROM vw_p4;"); + run_ddl(tc, conn, "INSERT INTO vw_p4_v1 VALUES (9999, 7);"); + run_ddl(tc, conn, "UPDATE vw_p4_v1 SET v = 0 WHERE id = 9999;"); + + require_true(tc, conn, + "SELECT count(*) = 1 FROM vw_p4 WHERE id = 9999 AND v = 0"); + + run_ddl(tc, conn, "DROP VIEW IF EXISTS vw_p4_v1;"); + run_ddl(tc, conn, "DROP TABLE IF EXISTS vw_p4 CASCADE;"); + pg_prop_disconnect(conn); +} + +/* + * Property 5 (A): DELETE through a simple updatable view removes from the base. + * + * Insert a known row (9999, 3) through the view, DELETE it through the view, + * then require no base row with id 9999 remains. + */ +PG_PROP(updatable_view_delete) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn, "vw_p5"); + run_ddl(tc, conn, "DROP VIEW IF EXISTS vw_p5_v1; " + "CREATE VIEW vw_p5_v1 AS SELECT id, v FROM vw_p5;"); + run_ddl(tc, conn, "INSERT INTO vw_p5_v1 VALUES (9999, 3);"); + run_ddl(tc, conn, "DELETE FROM vw_p5_v1 WHERE id = 9999;"); + + require_true(tc, conn, + "SELECT count(*) = 0 FROM vw_p5 WHERE id = 9999"); + + run_ddl(tc, conn, "DROP VIEW IF EXISTS vw_p5_v1;"); + run_ddl(tc, conn, "DROP TABLE IF EXISTS vw_p5 CASCADE;"); + pg_prop_disconnect(conn); +} + +/* + * Property 6 (A): a materialized view snapshots its query until REFRESH. + * + * mv.c = count(*) of vw_p6 at creation. After inserting one extra row the mv + * is stale (still equals the old count), and REFRESH makes it equal the new + * count. Both the staleness and the post-refresh equality are asserted so a + * refresh that silently tracked live changes (a bug) would fail the first + * check and an incorrect refresh would fail the second. + */ +PG_PROP(materialized_view_snapshot) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn, "vw_p6"); + run_ddl(tc, conn, "DROP MATERIALIZED VIEW IF EXISTS vw_p6_mv; " + "CREATE MATERIALIZED VIEW vw_p6_mv AS SELECT count(*) AS c FROM vw_p6;"); + + /* at creation the snapshot equals the live count. */ + require_true(tc, conn, + "SELECT (SELECT c FROM vw_p6_mv) = (SELECT count(*) FROM vw_p6)"); + + /* insert one more row (id 9999 avoids the 1..25 PK range). */ + run_ddl(tc, conn, "INSERT INTO vw_p6 VALUES (9999, 0, 1);"); + + /* now stale: the snapshot still equals the old (pre-insert) count. */ + require_true(tc, conn, + "SELECT (SELECT c FROM vw_p6_mv) = (SELECT count(*) - 1 FROM vw_p6)"); + + /* refresh re-snapshots: the mv equals the new live count. */ + run_ddl(tc, conn, "REFRESH MATERIALIZED VIEW vw_p6_mv;"); + require_true(tc, conn, + "SELECT (SELECT c FROM vw_p6_mv) = (SELECT count(*) FROM vw_p6)"); + + run_ddl(tc, conn, "DROP MATERIALIZED VIEW IF EXISTS vw_p6_mv;"); + run_ddl(tc, conn, "DROP TABLE IF EXISTS vw_p6 CASCADE;"); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index f3ef63fbb0db5..19fd50b01f333 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 146ff9c294fe6ce51f45a889e4765741a97c862c Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:29 -0400 Subject: [PATCH 237/338] prop: add count(*) vs count(col) vs count(DISTINCT) property tests (countsem) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_countsem.c | 284 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 288 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_countsem.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 930c9e95d90a4..4ee5c7a354d1f 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -252,7 +252,8 @@ OBJS = \ prop_defaults.o \ prop_checkconstraint.o \ prop_foreignkey.o \ - prop_views.o + prop_views.o \ + prop_countsem.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index b2a1aaa9a256f..15e20b80ea198 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -250,6 +250,7 @@ prop_area_sources = files( 'prop_checkconstraint.c', 'prop_foreignkey.c', 'prop_views.c', + 'prop_countsem.c', ) diff --git a/src/test/prop/prop_countsem.c b/src/test/prop/prop_countsem.c new file mode 100644 index 0000000000000..5569cea51248f --- /dev/null +++ b/src/test/prop/prop_countsem.c @@ -0,0 +1,284 @@ +/*------------------------------------------------------------------------- + * + * prop_countsem.c + * Property-based tests for COUNT semantics: count(*) vs count(col) vs + * count(DISTINCT col) and their NULL handling. + * + * The oracle is the defined meaning of COUNT: count(*) counts rows, + * count(col) counts non-NULL values of col, and count(DISTINCT col) counts + * distinct non-NULL values -- so count(DISTINCT col) <= count(col) <= + * count(*), the rows count(col) drops are exactly the NULL-col rows (count(*) + * - count(col) = count(*) FILTER (WHERE col IS NULL)), count(DISTINCT col) + * excludes NULL, and count of any non-NULL constant equals count(*). A "f" + * result is a wrong-results bug in COUNT, DISTINCT de-duplication, or the + * aggregate's NULL skipping -- not a rounding artefact. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_countsem.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "countsem" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means COUNT errored or produced NULL where it must not, which is + * itself a bug -- so either is a failure, reported with the SQL. count() is + * never NULL (0 over empty input), so no coalesce is needed anywhere here. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build cs2_t(id int PRIMARY KEY, a int, b int) with 0..30 rows: id is 0..n-1, + * a and b each drawn in 0..4 with ~20% NULL independently. The small 0..4 + * domain over up to 30 rows guarantees frequent duplicates so count(DISTINCT) + * is a meaningful de-dup, and ~20% NULL in each column exercises both the + * NULL-skipping of count(col) and the NULL exclusion of count(DISTINCT). + * Fresh each property so a counterexample is self-contained; ANALYZE so plan + * choice (hash vs sorted vs partial agg) is free to vary and the oracle holds. + */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS cs2_t; " + "CREATE TABLE cs2_t (id int PRIMARY KEY, a int, b int); "); + + nrows = (int) pg_prop_draw_int(tc, 0, 30); + for (int i = 0; i < nrows; i++) + { + appendStringInfo(&ddl, "INSERT INTO cs2_t VALUES (%d, ", i); + if (pg_prop_draw_int(tc, 0, 4) == 0) + appendStringInfoString(&ddl, "NULL, "); + else + appendStringInfo(&ddl, "%d, ", (int) pg_prop_draw_int(tc, 0, 4)); + if (pg_prop_draw_int(tc, 0, 4) == 0) + appendStringInfoString(&ddl, "NULL); "); + else + appendStringInfo(&ddl, "%d); ", (int) pg_prop_draw_int(tc, 0, 4)); + } + appendStringInfoString(&ddl, "ANALYZE cs2_t; "); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s\n ddl: %s", + err ? err : "(unknown)", ddl.data); + pfree(ddl.data); +} + +/* + * count(*) equals the row count computed a second way (count(*) over a + * one-column projection of the same rows). A tautology that pins the simple + * count(*) path: both must report the same number of rows. + */ +PG_PROP(count_star_eq_rows) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(*) FROM cs2_t) " + "= (SELECT count(*) FROM (SELECT 1 FROM cs2_t) s)"); + + pg_prop_disconnect(conn); +} + +/* + * count(a) <= count(*). count(a) counts only non-NULL a, which can never + * exceed the total row count. + */ +PG_PROP(count_col_le_star) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(a) <= count(*) FROM cs2_t"); + + pg_prop_disconnect(conn); +} + +/* + * count(DISTINCT a) <= count(a). De-duplicating the non-NULL values can only + * reduce (or keep) the count, never increase it. + */ +PG_PROP(count_distinct_le_count) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(DISTINCT a) <= count(a) FROM cs2_t"); + + pg_prop_disconnect(conn); +} + +/* + * count(*) - count(a) = count(*) FILTER (WHERE a IS NULL). The rows count(a) + * drops are exactly the NULL-a rows, so the deficit equals the NULL count. + */ +PG_PROP(count_null_complement) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(*) - count(a) " + "= count(*) FILTER (WHERE a IS NULL) FROM cs2_t"); + + pg_prop_disconnect(conn); +} + +/* + * count(DISTINCT (a,b)) <= count(*). A row-value count(DISTINCT) over the + * pair counts distinct (a,b) tuples (NULL tuples are skipped only when both + * are NULL -- a row-value is NULL iff every element is NULL), which can never + * exceed the row count. + */ +PG_PROP(count_distinct_pair_le) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(DISTINCT (a,b)) <= count(*) FROM cs2_t"); + + pg_prop_disconnect(conn); +} + +/* + * count(a) and count(b) are each <= count(*), and count(a) partitions + * count(*) with the NULL-a rows: count(a) + count(*) FILTER (WHERE a IS NULL) + * = count(*). Two per-column non-NULL counts bounded by the row count, and + * the exact non-NULL/NULL split for a. + */ +PG_PROP(count_two_cols) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(a) <= count(*) " + "AND count(b) <= count(*) " + "AND count(a) + count(*) FILTER (WHERE a IS NULL) = count(*) " + "FROM cs2_t"); + + pg_prop_disconnect(conn); +} + +/* + * count(DISTINCT a) excludes NULL: it equals the number of distinct non-NULL + * a values computed independently via SELECT DISTINCT over the non-NULL rows. + * A differential check that DISTINCT in the aggregate drops NULL exactly as a + * SELECT DISTINCT ... WHERE a IS NOT NULL would. + */ +PG_PROP(count_distinct_nulls_excluded) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(DISTINCT a) FROM cs2_t) " + "= (SELECT count(*) FROM " + "(SELECT DISTINCT a FROM cs2_t WHERE a IS NOT NULL) s)"); + + pg_prop_disconnect(conn); +} + +/* + * count(*) = count(1) = count(42): count of a non-NULL constant equals the + * row count, since the constant is non-NULL in every row and so is never + * skipped. Pins that count(const) behaves as count(*), not as count of a + * nullable column. + */ +PG_PROP(count_star_const) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + build_table(tc, conn); + + require_true(tc, conn, + "SELECT count(*) = count(1) AND count(1) = count(42) " + "FROM cs2_t"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 19fd50b01f333..f2d0153b3b321 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From e6bfd21c8f6eb3529c4068a55e3ff9174f2fde81 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:29 -0400 Subject: [PATCH 238/338] prop: add hstore property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_hstore.c | 350 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 354 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_hstore.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 4ee5c7a354d1f..92102c68eb9e8 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -253,7 +253,8 @@ OBJS = \ prop_checkconstraint.o \ prop_foreignkey.o \ prop_views.o \ - prop_countsem.o + prop_countsem.o \ + prop_hstore.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 15e20b80ea198..a5bffb15d20dd 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -251,6 +251,7 @@ prop_area_sources = files( 'prop_foreignkey.c', 'prop_views.c', 'prop_countsem.c', + 'prop_hstore.c', ) diff --git a/src/test/prop/prop_hstore.c b/src/test/prop/prop_hstore.c new file mode 100644 index 0000000000000..71fbbd9bf10cb --- /dev/null +++ b/src/test/prop/prop_hstore.c @@ -0,0 +1,350 @@ +/*------------------------------------------------------------------------- + * + * prop_hstore.c + * Property-based tests for the hstore contrib key/value map type. + * + * hstore is a key/value map stored as a single value; this exercises its + * text round-trip, the || merge (right operand wins on key conflict), + * key/value extraction (akeys, exist, ->), key deletion (delete), and + * containment (@>), over generated maps with distinct keys. + * + * The oracle is hstore's own documented algebra: a map parsed back from its + * text output must equal the original, a merge must prefer the right + * operand's value on a shared key, akeys must report every distinct key, + * exist must agree with whether a key is defined, delete must drop exactly + * one key, and @> must hold reflexively. Keys are generated distinct (k0, + * k1, ...) so there is no dedup ambiguity and cardinality(akeys()) equals the + * number of keys drawn. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. connect_hstore() creates the hstore + * extension (built in the test tree) before any hstore SQL runs. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_hstore.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "hstore" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the hstore operation errored or produced NULL on valid + * inputs, which is itself a bug -- so either is a failure, reported with the + * SQL. Mirrors require_true in prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Connect to the test cluster and ensure the hstore extension exists. Each + * property opens its own connection (the Tier-B pattern); CREATE EXTENSION IF + * NOT EXISTS is idempotent, so it is safe to run per connection. The hstore + * contrib module is built in the test tree, so this must succeed; on any + * failure the property is failed with the message. + */ +static pg_prop_conn * +connect_hstore(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, "CREATE EXTENSION IF NOT EXISTS hstore", &err)) + pg_prop_fail(tc, "setup failed (hstore extension): %s", + err ? err : ""); + + return conn; +} + +/* + * Draw an hstore literal into buf with nkeys DISTINCT keys named k0..k(n-1) + * and single-character [a-z] values, rendered as + * + * '"k0"=>"va", "k1"=>"vb"'::hstore + * + * The whole literal is single-quoted; the key/value text uses only [a-z] and + * the k key names, so no single quote ever appears inside and the + * literal is always well-formed. Keys are distinct by construction, so there + * is no dedup ambiguity and cardinality(akeys()) equals nkeys. + * + * If vals is non-NULL, vals[i] (a caller-provided buffer of at least 2 bytes) + * receives the drawn [a-z] value character for ki, NUL-terminated, so a + * property can cross-check H -> 'ki' against the exact value stored. + */ +static void +draw_hstore(pg_prop_tc *tc, StringInfo buf, int nkeys, char (*vals)[2]) +{ + int i; + + appendStringInfoChar(buf, '\''); + for (i = 0; i < nkeys; i++) + { + char v = (char) ('a' + pg_prop_draw_int(tc, 0, 25)); + + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "\"k%d\"=>\"%c\"", i, v); + + if (vals != NULL) + { + vals[i][0] = v; + vals[i][1] = '\0'; + } + } + appendStringInfoString(buf, "'::hstore"); +} + +/* + * Text round-trip: an hstore cast to text and parsed back is the same map. + * hstore_out must render the map so hstore_in parses it to an equal value; a + * quoting or escaping bug breaks this. + * + * SELECT (H)::text::hstore = H + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + StringInfoData h, + q; + int nkeys = (int) pg_prop_draw_int(tc, 0, 6); + + conn = connect_hstore(tc); + + initStringInfo(&h); + draw_hstore(tc, &h, nkeys, NULL); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)::text::hstore = (%s)", h.data, h.data); + + require_true(tc, conn, q.data); + + pfree(h.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Concat right operand wins: in A || B, every key present in B maps to B's + * value. B is built to contain at least k0, so k0 is a shared/overriding key + * and (A || B) -> 'k0' must equal B -> 'k0': + * + * SELECT (A || B) -> 'k0' IS NOT DISTINCT FROM (B -> 'k0') + * + * IS NOT DISTINCT FROM so a NULL value compares equal (values are [a-z] here, + * but this keeps the comparison two-valued regardless). + */ +PG_PROP(concat_right_wins) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + int na = (int) pg_prop_draw_int(tc, 1, 6); + int nb = (int) pg_prop_draw_int(tc, 1, 6); + + conn = connect_hstore(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_hstore(tc, &a, na, NULL); + draw_hstore(tc, &b, nb, NULL); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s) || (%s)) -> 'k0' " + "IS NOT DISTINCT FROM ((%s) -> 'k0')", + a.data, b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * akeys cardinality: akeys(H) returns the key array, whose cardinality equals + * the number of distinct keys drawn. Keys are distinct by construction, so + * no dedup collapses the count: + * + * SELECT cardinality(akeys(H)) = + */ +PG_PROP(akeys_count_matches) +{ + pg_prop_conn *conn; + StringInfoData h, + q; + int nkeys = (int) pg_prop_draw_int(tc, 0, 6); + + conn = connect_hstore(tc); + + initStringInfo(&h); + draw_hstore(tc, &h, nkeys, NULL); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT cardinality(akeys(%s)) = %d", h.data, nkeys); + + require_true(tc, conn, q.data); + + pfree(h.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * exist agrees with definedness: for a key k0 present in H, exist(H, 'k0') is + * true, and for 'zzz' (never generated -- keys are k) it is false: + * + * SELECT exist(H, 'k0') AND NOT exist(H, 'zzz') + * + * nkeys >= 1 so k0 is always present; 'zzz' lies outside the k key + * namespace, so it is guaranteed absent. + */ +PG_PROP(exist_matches_defined) +{ + pg_prop_conn *conn; + StringInfoData h, + q; + int nkeys = (int) pg_prop_draw_int(tc, 1, 6); + + conn = connect_hstore(tc); + + initStringInfo(&h); + draw_hstore(tc, &h, nkeys, NULL); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT exist(%s, 'k0') AND NOT exist(%s, 'zzz')", + h.data, h.data); + + require_true(tc, conn, q.data); + + pfree(h.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Containment is reflexive: every hstore contains itself. + * + * SELECT H @> H + */ +PG_PROP(contains_self) +{ + pg_prop_conn *conn; + StringInfoData h, + q; + int nkeys = (int) pg_prop_draw_int(tc, 0, 6); + + conn = connect_hstore(tc); + + initStringInfo(&h); + draw_hstore(tc, &h, nkeys, NULL); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s) @> (%s)", h.data, h.data); + + require_true(tc, conn, q.data); + + pfree(h.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * delete removes exactly one key: delete(H, 'k0') no longer has k0 and has + * one fewer key. nkeys >= 1 so k0 is present and the expected count is n-1: + * + * SELECT NOT exist(delete(H, 'k0'), 'k0') + * AND cardinality(akeys(delete(H, 'k0'))) = - 1 + */ +PG_PROP(delete_removes_key) +{ + pg_prop_conn *conn; + StringInfoData h, + q; + int nkeys = (int) pg_prop_draw_int(tc, 1, 6); + + conn = connect_hstore(tc); + + initStringInfo(&h); + draw_hstore(tc, &h, nkeys, NULL); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT NOT exist(delete(%s, 'k0'), 'k0') " + "AND cardinality(akeys(delete(%s, 'k0'))) = %d", + h.data, h.data, nkeys - 1); + + require_true(tc, conn, q.data); + + pfree(h.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Arrow get matches the stored value: H -> 'k0' equals the [a-z] value drawn + * for k0. The value is captured by draw_hstore into vals[0] and substituted + * into the query, so this cross-checks hstore's -> against the exact literal + * that was stored rather than comparing H to itself: + * + * SELECT (H -> 'k0') = '' + */ +PG_PROP(get_matches_arrow) +{ + pg_prop_conn *conn; + StringInfoData h, + q; + int nkeys = (int) pg_prop_draw_int(tc, 1, 6); + char vals[6][2]; + + conn = connect_hstore(tc); + + initStringInfo(&h); + draw_hstore(tc, &h, nkeys, vals); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s) -> 'k0') = '%s'", h.data, vals[0]); + + require_true(tc, conn, q.data); + + pfree(h.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index f2d0153b3b321..d618aab9894c7 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 62f3e23581c5b2c8548b3833b213a28d21d9053f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:29 -0400 Subject: [PATCH 239/338] prop: add hstore slice, delete, akeys, and json round-trip property tests (hstore2) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_hstore2.c | 376 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 380 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_hstore2.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 92102c68eb9e8..a44bf4bd512ab 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -254,7 +254,8 @@ OBJS = \ prop_foreignkey.o \ prop_views.o \ prop_countsem.o \ - prop_hstore.o + prop_hstore.o \ + prop_hstore2.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index a5bffb15d20dd..9f2435ae4feb6 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -252,6 +252,7 @@ prop_area_sources = files( 'prop_views.c', 'prop_countsem.c', 'prop_hstore.c', + 'prop_hstore2.c', ) diff --git a/src/test/prop/prop_hstore2.c b/src/test/prop/prop_hstore2.c new file mode 100644 index 0000000000000..d9d8345ae0e1e --- /dev/null +++ b/src/test/prop/prop_hstore2.c @@ -0,0 +1,376 @@ +/*------------------------------------------------------------------------- + * + * prop_hstore2.c + * Property-based tests for a second set of hstore contrib operations. + * + * This complements prop_hstore.c with hstore functions distinct from those + * exercised there: slice (restrict to a key set), the key/value projections + * akeys/avals/skeys/svals, each() as a set-returning function, hstore_to_json, + * defined, and @> containment viewed from these angles. + * + * The oracle is hstore's own documented algebra over maps built with DISTINCT + * keys k0..k(n-1) and single-character [a-c] values: akeys cardinality equals + * the key count (no dedup), delete drops exactly the named key, slice keeps + * only the requested keys that exist, || prefers the right operand, the JSON + * projection carries the same key->value mapping, defined agrees with the ? + * exist operator, each() yields one row per key, and @> is reflexive. A + * failure of any of these is a bug in hstore. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. connect_hstore() creates the hstore + * extension (built in the test tree) before any hstore SQL runs. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_hstore2.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "hstore2" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the hstore operation errored or produced NULL on valid + * inputs, which is itself a bug -- so either is a failure, reported with the + * SQL. Mirrors require_true in prop_numeric.c / prop_hstore.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Connect to the test cluster and ensure the hstore extension exists. Each + * property opens its own connection (the Tier-B pattern); CREATE EXTENSION IF + * NOT EXISTS is idempotent, so it is safe to run per connection. + */ +static pg_prop_conn * +connect_hstore(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, "CREATE EXTENSION IF NOT EXISTS hstore", &err)) + pg_prop_fail(tc, "setup failed (hstore extension): %s", + err ? err : ""); + + return conn; +} + +/* + * Draw an hstore literal into buf with nkeys DISTINCT keys named k0..k(n-1) + * and single-character [a-c] values, rendered as + * + * '"k0"=>"a", "k1"=>"b"'::hstore + * + * The whole literal is single-quoted; the key/value text uses only [a-c] and + * the k key names, so no single quote ever appears inside and the + * literal is always well-formed. Keys are distinct by construction, so there + * is no dedup ambiguity and cardinality(akeys()) equals nkeys. + * + * nkeys is drawn 2..5 so k0 and k1 are always present, giving the delete and + * slice properties guaranteed targets. + * + * If vals is non-NULL, vals[i] (a buffer of at least 2 bytes) receives the + * drawn [a-c] value character for ki, NUL-terminated. + */ +static void +draw_hstore(pg_prop_tc *tc, StringInfo buf, int nkeys, char (*vals)[2]) +{ + int i; + + appendStringInfoChar(buf, '\''); + for (i = 0; i < nkeys; i++) + { + char v = (char) ('a' + pg_prop_draw_int(tc, 0, 2)); + + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "\"k%d\"=>\"%c\"", i, v); + + if (vals != NULL) + { + vals[i][0] = v; + vals[i][1] = '\0'; + } + } + appendStringInfoString(buf, "'::hstore"); +} + +/* + * akeys cardinality: array_length(akeys(H), 1) equals the number of distinct + * keys drawn. Keys are distinct by construction so no dedup collapses the + * count: + * + * SELECT array_length(akeys(H), 1) = + */ +PG_PROP(akeys_count) +{ + pg_prop_conn *conn; + StringInfoData h, + q; + int nkeys = (int) pg_prop_draw_int(tc, 2, 5); + + conn = connect_hstore(tc); + + initStringInfo(&h); + draw_hstore(tc, &h, nkeys, NULL); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT array_length(akeys(%s), 1) = %d", + h.data, nkeys); + + require_true(tc, conn, q.data); + + pfree(h.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * delete removes exactly the named key: after delete(H, 'k0') the result no + * longer defines k0, but still defines k1 (nkeys >= 2 so k1 is present): + * + * SELECT NOT (delete(H, 'k0') ? 'k0') AND (delete(H, 'k0') ? 'k1') + */ +PG_PROP(delete_removes) +{ + pg_prop_conn *conn; + StringInfoData h, + q; + int nkeys = (int) pg_prop_draw_int(tc, 2, 5); + + conn = connect_hstore(tc); + + initStringInfo(&h); + draw_hstore(tc, &h, nkeys, NULL); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT NOT (delete(%s, 'k0') ? 'k0') " + "AND (delete(%s, 'k0') ? 'k1')", + h.data, h.data); + + require_true(tc, conn, q.data); + + pfree(h.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * slice keeps only the requested keys that exist: slice(H, ARRAY['k0']) has + * exactly the single key k0 (k0 is always present, nkeys >= 2). Checked both + * ways -- the slice defines k0 and its only key is k0: + * + * SELECT (slice(H, ARRAY['k0']) ? 'k0') + * AND array_length(akeys(slice(H, ARRAY['k0'])), 1) = 1 + */ +PG_PROP(slice_subset) +{ + pg_prop_conn *conn; + StringInfoData h, + q; + int nkeys = (int) pg_prop_draw_int(tc, 2, 5); + + conn = connect_hstore(tc); + + initStringInfo(&h); + draw_hstore(tc, &h, nkeys, NULL); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (slice(%s, ARRAY['k0']) ? 'k0') " + "AND array_length(akeys(slice(%s, ARRAY['k0'])), 1) = 1", + h.data, h.data); + + require_true(tc, conn, q.data); + + pfree(h.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Concat right operand wins: (H || hstore('k0','ZZ')) -> 'k0' = 'ZZ'. The + * right operand overrides the k0 value regardless of what H held for k0: + * + * SELECT ((H || hstore('k0', 'ZZ')) -> 'k0') = 'ZZ' + */ +PG_PROP(concat_right_wins) +{ + pg_prop_conn *conn; + StringInfoData h, + q; + int nkeys = (int) pg_prop_draw_int(tc, 2, 5); + + conn = connect_hstore(tc); + + initStringInfo(&h); + draw_hstore(tc, &h, nkeys, NULL); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (((%s) || hstore('k0', 'ZZ')) -> 'k0') = 'ZZ'", + h.data); + + require_true(tc, conn, q.data); + + pfree(h.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * JSON projection preserves the mapping: hstore_to_json(H) ->> 'k0' equals + * H -> 'k0'. The JSON object must carry the same key->value association for + * every key; checked at k0 (always present). Values are [a-c] text, so the + * JSON string text and the hstore value compare equal: + * + * SELECT (hstore_to_json(H) ->> 'k0') = (H -> 'k0') + */ +PG_PROP(json_roundtrip_keys) +{ + pg_prop_conn *conn; + StringInfoData h, + q; + int nkeys = (int) pg_prop_draw_int(tc, 2, 5); + + conn = connect_hstore(tc); + + initStringInfo(&h); + draw_hstore(tc, &h, nkeys, NULL); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (hstore_to_json(%s) ->> 'k0') = ((%s) -> 'k0')", + h.data, h.data); + + require_true(tc, conn, q.data); + + pfree(h.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * defined agrees with the ? exist operator for both a present key (k0, always + * drawn) and an absent key ('zzz', outside the k namespace): + * + * SELECT (defined(H, 'k0') = (H ? 'k0')) + * AND (defined(H, 'zzz') = (H ? 'zzz')) + */ +PG_PROP(defined_vs_exists) +{ + pg_prop_conn *conn; + StringInfoData h, + q; + int nkeys = (int) pg_prop_draw_int(tc, 2, 5); + + conn = connect_hstore(tc); + + initStringInfo(&h); + draw_hstore(tc, &h, nkeys, NULL); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (defined(%s, 'k0') = ((%s) ? 'k0')) " + "AND (defined(%s, 'zzz') = ((%s) ? 'zzz'))", + h.data, h.data, h.data, h.data); + + require_true(tc, conn, q.data); + + pfree(h.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * each() yields one row per key: (SELECT count(*) FROM each(H)) equals + * array_length(akeys(H), 1). each expands the map to (key, value) rows, so + * its row count is the key count: + * + * SELECT (SELECT count(*) FROM each(H)) = array_length(akeys(H), 1) + */ +PG_PROP(each_count) +{ + pg_prop_conn *conn; + StringInfoData h, + q; + int nkeys = (int) pg_prop_draw_int(tc, 2, 5); + + conn = connect_hstore(tc); + + initStringInfo(&h); + draw_hstore(tc, &h, nkeys, NULL); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (SELECT count(*) FROM each(%s)) " + "= array_length(akeys(%s), 1)", + h.data, h.data); + + require_true(tc, conn, q.data); + + pfree(h.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Containment is reflexive: every hstore contains itself. + * + * SELECT H @> H + */ +PG_PROP(contains_self) +{ + pg_prop_conn *conn; + StringInfoData h, + q; + int nkeys = (int) pg_prop_draw_int(tc, 2, 5); + + conn = connect_hstore(tc); + + initStringInfo(&h); + draw_hstore(tc, &h, nkeys, NULL); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s) @> (%s)", h.data, h.data); + + require_true(tc, conn, q.data); + + pfree(h.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index d618aab9894c7..3fe01697e35af 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From b188505fee10a23436cac1becacd38961acb4ae9 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:29 -0400 Subject: [PATCH 240/338] prop: add ltree property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_ltree.c | 330 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 334 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_ltree.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index a44bf4bd512ab..46f061f9af75b 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -255,7 +255,8 @@ OBJS = \ prop_views.o \ prop_countsem.o \ prop_hstore.o \ - prop_hstore2.o + prop_hstore2.o \ + prop_ltree.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 9f2435ae4feb6..a3201bdae4bca 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -253,6 +253,7 @@ prop_area_sources = files( 'prop_countsem.c', 'prop_hstore.c', 'prop_hstore2.c', + 'prop_ltree.c', ) diff --git a/src/test/prop/prop_ltree.c b/src/test/prop/prop_ltree.c new file mode 100644 index 0000000000000..95e477a6b1ea2 --- /dev/null +++ b/src/test/prop/prop_ltree.c @@ -0,0 +1,330 @@ +/*------------------------------------------------------------------------- + * + * prop_ltree.c + * Property-based tests for the ltree contrib extension (label paths). + * + * ltree is a dotted label path (contrib/ltree); this exercises concatenation + * length, ancestor (@>) semantics, subpath, nlevel, and text round-trip. + * + * The oracle is the type's own structure: a path is a sequence of labels, so + * nlevel counts them, concatenation adds them, a prefix is an ancestor, and + * the two sub-extractors (subpath / subltree) must agree on a single label. + * These identities must hold with no slack; any "f" is a bug in ltree. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_ltree.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "ltree" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means an ltree function errored or produced NULL on valid + * inputs, which is itself a bug -- so either is a failure, reported with the + * SQL. Mirrors require_true in prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Connect to the test cluster and make sure the ltree extension exists. + * ltree is a contrib module built in the test tree; CREATE EXTENSION IF NOT + * EXISTS makes the DDL idempotent across the per-case connections. On any + * failure the property is failed with the message. Mirrors the connect + * helper pattern in prop_record.c. + */ +static pg_prop_conn * +connect_ltree(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, + "CREATE EXTENSION IF NOT EXISTS ltree", + &err)) + pg_prop_fail(tc, "setup failed (ltree): %s", err ? err : ""); + + return conn; +} + +/* + * Draw an ltree path literal into buf as 'l1.l2...'::ltree, with 1..5 labels + * each 1..4 characters from [a-z], joined by '.'. ltree labels are + * alphanumeric/underscore; [a-z] is a safe subset that needs no escaping, so + * the literal is always well-formed. *nlabels receives the label count. + */ +static void +draw_ltree(pg_prop_tc *tc, StringInfo buf, int *nlabels) +{ + int n = (int) pg_prop_draw_int(tc, 1, 5); + int i; + + appendStringInfoChar(buf, '\''); + for (i = 0; i < n; i++) + { + int nchars = (int) pg_prop_draw_int(tc, 1, 4); + int j; + + if (i > 0) + appendStringInfoChar(buf, '.'); + for (j = 0; j < nchars; j++) + appendStringInfoChar(buf, + (char) ('a' + pg_prop_draw_int(tc, 0, 25))); + } + appendStringInfoString(buf, "'::ltree"); + + *nlabels = n; +} + +/* + * Text round-trip: ltree output parses back equal, (L)::text::ltree = L. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + StringInfoData l, + q; + int n; + + conn = connect_ltree(tc); + + initStringInfo(&l); + draw_ltree(tc, &l, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)::text::ltree = (%s)", + l.data, l.data); + + require_true(tc, conn, q.data); + + pfree(l.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * nlevel(L) equals the number of labels n drawn into the path. + */ +PG_PROP(nlevel_matches_label_count) +{ + pg_prop_conn *conn; + StringInfoData l, + q; + int n; + + conn = connect_ltree(tc); + + initStringInfo(&l); + draw_ltree(tc, &l, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT nlevel(%s) = %d", l.data, n); + + require_true(tc, conn, q.data); + + pfree(l.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ltree concatenation adds levels: nlevel(A || B) = nlevel(A) + nlevel(B). + */ +PG_PROP(concat_nlevel_additive) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + int na, + nb; + + conn = connect_ltree(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_ltree(tc, &a, &na); + draw_ltree(tc, &b, &nb); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT nlevel(%s || %s) = nlevel(%s) + nlevel(%s)", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A path is an ancestor (@>) of itself concatenated with anything: + * A @> (A || B). @> means "is ancestor of or equal", so a prefix of a path + * is always an ancestor of it. + */ +PG_PROP(prefix_is_ancestor) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + int na, + nb; + + conn = connect_ltree(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_ltree(tc, &a, &na); + draw_ltree(tc, &b, &nb); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s @> (%s || %s)", + a.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The first k labels of a path form an ancestor of it: subpath(L, 0, k) @> L + * for k drawn in 1..nlevel(L). Offsets are 0-based, so subpath(L, 0, k) is + * the k-label prefix. + */ +PG_PROP(subpath_prefix_is_ancestor) +{ + pg_prop_conn *conn; + StringInfoData l, + q; + int n; + int k; + + conn = connect_ltree(tc); + + initStringInfo(&l); + draw_ltree(tc, &l, &n); + k = (int) pg_prop_draw_int(tc, 1, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT subpath(%s, 0, %d) @> %s", + l.data, k, l.data); + + require_true(tc, conn, q.data); + + pfree(l.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Ancestor is reflexive, and ancestry implies no-greater level: L @> L, and + * if A @> B then nlevel(A) <= nlevel(B). Boolean implication "a implies b" + * is written (a <= b) for booleans (false <= anything, true <= true), so the + * second clause is ((A @> B) <= (nlevel(A) <= nlevel(B))). A and B are two + * independent paths. + */ +PG_PROP(ancestor_reflexive_and_level) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + int na, + nb; + + conn = connect_ltree(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_ltree(tc, &a, &na); + draw_ltree(tc, &b, &nb); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s @> %s) " + "AND ((%s @> %s) <= (nlevel(%s) <= nlevel(%s)))", + a.data, a.data, + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The two sub-extractors agree on a single label: subpath(L, k, 1) extracts + * the label at position k and must equal subltree(L, k, k+1), for k in + * 0..n-1. Offsets are 0-based; subpath takes (offset, len) while subltree + * takes (start, end), so a one-off bug in either breaks this cross-check. + */ +PG_PROP(index_access_matches_subpath) +{ + pg_prop_conn *conn; + StringInfoData l, + q; + int n; + int k; + + conn = connect_ltree(tc); + + initStringInfo(&l); + draw_ltree(tc, &l, &n); + k = (int) pg_prop_draw_int(tc, 0, n - 1); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT subpath(%s, %d, 1) = subltree(%s, %d, %d)", + l.data, k, l.data, k, k + 1); + + require_true(tc, conn, q.data); + + pfree(l.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 3fe01697e35af..7a619a8c35ae2 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 7f6beb7330b7c8c7ce614448bc0dbf78abdebfc9 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:29 -0400 Subject: [PATCH 241/338] prop: add ltree subpath, lquery, and ancestor operators property tests (ltree2) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_ltree2.c | 340 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 344 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_ltree2.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 46f061f9af75b..418a3335ada8d 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -256,7 +256,8 @@ OBJS = \ prop_countsem.o \ prop_hstore.o \ prop_hstore2.o \ - prop_ltree.o + prop_ltree.o \ + prop_ltree2.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index a3201bdae4bca..0afaab987f2e3 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -254,6 +254,7 @@ prop_area_sources = files( 'prop_hstore.c', 'prop_hstore2.c', 'prop_ltree.c', + 'prop_ltree2.c', ) diff --git a/src/test/prop/prop_ltree2.c b/src/test/prop/prop_ltree2.c new file mode 100644 index 0000000000000..c9b93ac8c8b3b --- /dev/null +++ b/src/test/prop/prop_ltree2.c @@ -0,0 +1,340 @@ +/*------------------------------------------------------------------------- + * + * prop_ltree2.c + * Property-based tests for ltree query operators and sub-extractors. + * + * ltree is a dotted label path (contrib/ltree). This file is distinct from + * prop_ltree.c: it exercises the query side -- lquery matching (~), the + * subltree/subpath extractors, index() position lookup, and the ancestor + * operators (@>, <@) -- rather than concatenation/round-trip algebra. + * + * The oracle is the type's own structure: a prefix is an ancestor, the + * whole path is subpath from offset 0, nlevel counts labels, index finds a + * path within itself at 0, and the match-anything lquery '*' matches every + * ltree. These identities hold with no slack; any "f" is a bug in ltree. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_ltree2.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "ltree2" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means an ltree function errored or produced NULL on valid + * inputs, which is itself a bug -- so either is a failure, reported with the + * SQL. Mirrors require_true in prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Connect to the test cluster and make sure the ltree extension exists. + * CREATE EXTENSION IF NOT EXISTS makes the DDL idempotent across the per-case + * connections. On any failure the property is failed with the message. + */ +static pg_prop_conn * +connect_ltree(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, + "CREATE EXTENSION IF NOT EXISTS ltree", + &err)) + pg_prop_fail(tc, "setup failed (ltree): %s", err ? err : ""); + + return conn; +} + +/* + * Draw an ltree path literal into buf as 'l1.l2...'::ltree, with 1..5 labels + * each 1..3 characters from [a-c], joined by '.'. [a-c] always begins with a + * letter and needs no escaping, so the literal is always well-formed. + * *nlabels receives the label count. + */ +static void +draw_ltree(pg_prop_tc *tc, StringInfo buf, int *nlabels) +{ + int n = (int) pg_prop_draw_int(tc, 1, 5); + int i; + + appendStringInfoChar(buf, '\''); + for (i = 0; i < n; i++) + { + int nchars = (int) pg_prop_draw_int(tc, 1, 3); + int j; + + if (i > 0) + appendStringInfoChar(buf, '.'); + for (j = 0; j < nchars; j++) + appendStringInfoChar(buf, + (char) ('a' + pg_prop_draw_int(tc, 0, 2))); + } + appendStringInfoString(buf, "'::ltree"); + + *nlabels = n; +} + +/* + * subltree(L, 0, k) is the k-label prefix of L, and a prefix is an ancestor: + * subltree(L, 0, k) @> L for k drawn in 1..nlevel(L). Offsets are 0-based. + */ +PG_PROP(subltree_prefix) +{ + pg_prop_conn *conn; + StringInfoData l, + q; + int n; + int k; + + conn = connect_ltree(tc); + + initStringInfo(&l); + draw_ltree(tc, &l, &n); + k = (int) pg_prop_draw_int(tc, 1, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT subltree(%s, 0, %d) @> %s", + l.data, k, l.data); + + require_true(tc, conn, q.data); + + pfree(l.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * subpath from offset 0 with no length returns the whole path: + * subpath(L, 0) = L. + */ +PG_PROP(subpath_roundtrip) +{ + pg_prop_conn *conn; + StringInfoData l, + q; + int n; + + conn = connect_ltree(tc); + + initStringInfo(&l); + draw_ltree(tc, &l, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT subpath(%s, 0) = %s", l.data, l.data); + + require_true(tc, conn, q.data); + + pfree(l.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The k-label prefix has exactly k levels: nlevel(subpath(L, 0, k)) = k for k + * drawn in 0..nlevel(L). k = 0 yields the empty path with nlevel 0. + */ +PG_PROP(nlevel_subpath) +{ + pg_prop_conn *conn; + StringInfoData l, + q; + int n; + int k; + + conn = connect_ltree(tc); + + initStringInfo(&l); + draw_ltree(tc, &l, &n); + k = (int) pg_prop_draw_int(tc, 0, n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT nlevel(subpath(%s, 0, %d)) = %d", + l.data, k, k); + + require_true(tc, conn, q.data); + + pfree(l.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A path contains itself at position 0: index(L, L) = 0. index returns the + * 0-based position of the first match of the second path within the first. + */ +PG_PROP(index_of_self) +{ + pg_prop_conn *conn; + StringInfoData l, + q; + int n; + + conn = connect_ltree(tc); + + initStringInfo(&l); + draw_ltree(tc, &l, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT index(%s, %s) = 0", l.data, l.data); + + require_true(tc, conn, q.data); + + pfree(l.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ltree concatenation adds levels: nlevel(A || B) = nlevel(A) + nlevel(B). + */ +PG_PROP(concat_nlevel) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + int na, + nb; + + conn = connect_ltree(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_ltree(tc, &a, &na); + draw_ltree(tc, &b, &nb); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT nlevel(%s || %s) = nlevel(%s) + nlevel(%s)", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The match-anything lquery '*' matches every ltree: L ~ '*'::lquery. + */ +PG_PROP(lquery_star_matches) +{ + pg_prop_conn *conn; + StringInfoData l, + q; + int n; + + conn = connect_ltree(tc); + + initStringInfo(&l); + draw_ltree(tc, &l, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT %s ~ '*'::lquery", l.data); + + require_true(tc, conn, q.data); + + pfree(l.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The first label followed by anything matches the path: + * L ~ (subltree(L, 0, 1)::text || '.*')::lquery, for nlevel(L) >= 1 (always + * true here, paths have 1..5 labels). subltree(L, 0, 1) is the literal first + * label, which [a-c] guarantees is a valid lquery atom needing no escaping. + */ +PG_PROP(lquery_prefix_matches) +{ + pg_prop_conn *conn; + StringInfoData l, + q; + int n; + + conn = connect_ltree(tc); + + initStringInfo(&l); + draw_ltree(tc, &l, &n); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT %s ~ (subltree(%s, 0, 1)::text || '.*')::lquery", + l.data, l.data); + + require_true(tc, conn, q.data); + + pfree(l.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Ancestor is antisymmetric: if A @> B and B @> A then A = B. Tested on L + * and itself, where both directions hold: L @> L AND L <@ L. <@ is the + * descendant operator (the converse of @>). + */ +PG_PROP(ancestor_antisymmetric) +{ + pg_prop_conn *conn; + StringInfoData l, + q; + int n; + + conn = connect_ltree(tc); + + initStringInfo(&l); + draw_ltree(tc, &l, &n); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s @> %s) AND (%s <@ %s)", + l.data, l.data, l.data, l.data); + + require_true(tc, conn, q.data); + + pfree(l.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 7a619a8c35ae2..07a84b453df17 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From d658ac847272b7b78f491d36432cffe2ad380c47 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:30 -0400 Subject: [PATCH 242/338] prop: add citext property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_citext.c | 347 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 351 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_citext.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 418a3335ada8d..777ad2d26371b 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -257,7 +257,8 @@ OBJS = \ prop_hstore.o \ prop_hstore2.o \ prop_ltree.o \ - prop_ltree2.o + prop_ltree2.o \ + prop_citext.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 0afaab987f2e3..e0b0166e2c6d6 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -255,6 +255,7 @@ prop_area_sources = files( 'prop_hstore2.c', 'prop_ltree.c', 'prop_ltree2.c', + 'prop_citext.c', ) diff --git a/src/test/prop/prop_citext.c b/src/test/prop/prop_citext.c new file mode 100644 index 0000000000000..bbd29b2448aef --- /dev/null +++ b/src/test/prop/prop_citext.c @@ -0,0 +1,347 @@ +/*------------------------------------------------------------------------- + * + * prop_citext.c + * Property-based tests for the citext case-insensitive text type. + * + * The citext contrib extension compares and hashes case-insensitively while + * preserving the stored text; these check the case-insensitive equality and + * ordering contract against lower()-folded plain text as the oracle. + * + * The oracle is independent of citext: lower()/upper() on plain text fold + * case the same way citext's comparison does, so citext equality must equal + * plain-text equality after lower(), and citext ordering must equal lower() + * ordering. Casting a citext back to text must return the original bytes + * unchanged (folding happens only for comparison, not storage). Inputs are + * single-byte mixed-case ASCII ("abAB") so folding is non-trivial and + * lower()/upper()/length() are exact oracles. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_citext.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "citext" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Connect to the test cluster and ensure the citext extension exists. Each + * property opens its own connection (the Tier-B pattern); CREATE EXTENSION IF + * NOT EXISTS makes the DDL idempotent across connections. On any failure the + * property is failed with the message. + */ +static pg_prop_conn * +connect_citext(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, + "CREATE EXTENSION IF NOT EXISTS citext", + &err)) + pg_prop_fail(tc, "setup failed (citext): %s", err ? err : ""); + + return conn; +} + +/* + * Draw 0..12 characters of mixed-case ASCII from "abAB" into buf as a bare + * string (no quoting). The alphabet mixes upper and lower case so case + * folding is exercised, and every character is single-byte so length is exact + * and lower()/upper() are the precise oracles. + */ +static void +draw_mixed(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abAB"; + int n = (int) pg_prop_draw_int(tc, 0, 12); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 3)]); +} + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabet never contains a quote, but do it defensively + * so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means citext comparison/output errored or produced NULL on + * valid single-byte strings, which is itself a bug -- so either is a failure, + * reported with the SQL. Mirrors require_true in prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * citext equality folds case: a string equals both its uppercased and its + * lowercased forms when compared as citext. upper()/lower() are applied + * server-side on plain text so no C-side folding can disagree: + * + * S::citext = upper(S)::citext AND S::citext = lower(S)::citext + */ +PG_PROP(equality_is_case_insensitive) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + + conn = connect_citext(tc); + + initStringInfo(&s); + draw_mixed(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "::citext = upper("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")::citext) AND ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "::citext = lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")::citext)"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The core contract: citext equality is exactly case-folded text equality. + * For two strings S, T, comparing them as citext must agree with comparing + * their lower()-folded plain text: + * + * (S::citext = T::citext) = (lower(S) = lower(T)) + * + * lower() on plain text is the independent oracle, so a folding bug in + * citext's equality operator fails here. + */ +PG_PROP(equality_matches_lower_oracle) +{ + pg_prop_conn *conn; + StringInfoData s, + t, + q; + + conn = connect_citext(tc); + + initStringInfo(&s); + initStringInfo(&t); + draw_mixed(tc, &s); + draw_mixed(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "::citext = "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, "::citext) = (lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = lower("); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, "))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * citext preserves the stored text on output: casting back to text returns + * the original bytes, not a folded form (folding happens only for + * comparison): + * + * (S::citext)::text = S + */ +PG_PROP(preserves_stored_text) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + + conn = connect_citext(tc); + + initStringInfo(&s); + draw_mixed(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "::citext)::text = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * citext ordering is case-insensitive, matching lower()-folded comparison. + * For two strings S, T, the citext "<" operator must agree with "<" on their + * lower()-folded plain text: + * + * (S::citext < T::citext) = (lower(S) < lower(T)) + * + * A sign or short-circuit bug in citext's comparison routine fails here. + */ +PG_PROP(ordering_matches_lower) +{ + pg_prop_conn *conn; + StringInfoData s, + t, + q; + + conn = connect_citext(tc); + + initStringInfo(&s); + initStringInfo(&t); + draw_mixed(tc, &s); + draw_mixed(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "::citext < "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, "::citext) = (lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") < lower("); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, "))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Concatenation then compare folds case: concatenating S and T and comparing + * as citext must equal concatenating their uppercased forms and comparing as + * citext (both fold to the same value): + * + * (S || T)::citext = (upper(S) || upper(T))::citext + */ +PG_PROP(concat_is_case_insensitive) +{ + pg_prop_conn *conn; + StringInfoData s, + t, + q; + + conn = connect_citext(tc); + + initStringInfo(&s); + initStringInfo(&t); + draw_mixed(tc, &s); + draw_mixed(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT (("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " || "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ")::citext) = ((upper("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") || upper("); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, "))::citext)"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * citext length equals the plain-text length: case folding does not change + * the length for single-byte ASCII, so length() through citext must match + * length() on the plain text: + * + * length(S::citext) = length(S) + */ +PG_PROP(length_preserved) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + + conn = connect_citext(tc); + + initStringInfo(&s); + draw_mixed(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "::citext) = length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 07a84b453df17..61a07e6f8fd21 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From c087adfeb512f154a3e62f7fc68cf8d78daabab2 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:30 -0400 Subject: [PATCH 243/338] prop: add citext LIKE, grouping, and ordering property tests (citext2) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_citext2.c | 368 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 372 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_citext2.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 777ad2d26371b..ebd57e457629e 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -258,7 +258,8 @@ OBJS = \ prop_hstore2.o \ prop_ltree.o \ prop_ltree2.o \ - prop_citext.o + prop_citext.o \ + prop_citext2.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index e0b0166e2c6d6..4f59d7bb58a94 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -256,6 +256,7 @@ prop_area_sources = files( 'prop_ltree.c', 'prop_ltree2.c', 'prop_citext.c', + 'prop_citext2.c', ) diff --git a/src/test/prop/prop_citext2.c b/src/test/prop/prop_citext2.c new file mode 100644 index 0000000000000..6eed029bd469b --- /dev/null +++ b/src/test/prop/prop_citext2.c @@ -0,0 +1,368 @@ +/*------------------------------------------------------------------------- + * + * prop_citext2.c + * Property-based tests for citext operators and functions beyond the + * basic equality/ordering contract (LIKE, DISTINCT/GROUP BY folding, + * concat, aggregates). + * + * citext compares and hashes case-insensitively while storing the original + * bytes. The oracle throughout is lower()-folded plain text: LIKE on citext + * must match lower() LIKE lower(); counting distinct citext values must equal + * counting distinct lower() values; min/max fold the same way. Any + * disagreement is a bug in citext's operators or in the aggregate/sort paths + * that use its comparison function. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_citext2.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "citext2" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Connect to the test cluster and ensure the citext extension exists. Each + * property opens its own connection (the Tier-B pattern); CREATE EXTENSION IF + * NOT EXISTS makes the DDL idempotent across the thousands of per-case + * connections. On any failure the property is failed with the message. + */ +static pg_prop_conn * +connect_citext(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, + "CREATE EXTENSION IF NOT EXISTS citext", + &err)) + pg_prop_fail(tc, "setup failed (citext): %s", err ? err : ""); + + return conn; +} + +/* + * Draw 0..10 characters of mixed-case ASCII from "abcABC" into buf as a bare + * string (no quoting). The alphabet mixes upper and lower case so case + * folding is exercised; every character is single-byte so lower()/upper() are + * exact oracles and the strings contain no LIKE metacharacters (so LIKE + * reduces to case-insensitive equality). + */ +static void +draw_mixed(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abcABC"; + int n = (int) pg_prop_draw_int(tc, 0, 10); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 5)]); +} + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabet never contains a quote, but do it defensively + * so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a citext operator/function errored or produced NULL on + * valid single-byte strings, which is itself a bug -- so either is a failure, + * reported with the SQL. Mirrors require_true in prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Build a one-column table ci_t(x text) with 0..12 mixed-case rows drawn from + * "abcABC". Rebuilt fresh in each property that needs it. The rows are + * plain text; properties cast x to citext in their queries so the folding + * behaviour under aggregation/DISTINCT is exercised. Failure is a harness + * problem, not a property counterexample, so it fails the case directly. + */ +static void +build_ci_t(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int nrows = (int) pg_prop_draw_int(tc, 0, 12); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS ci_t; " + "CREATE TABLE ci_t (x text); "); + for (int r = 0; r < nrows; r++) + { + StringInfoData v; + + initStringInfo(&v); + draw_mixed(tc, &v); + appendStringInfoString(&ddl, "INSERT INTO ci_t VALUES ("); + append_sql_literal(&ddl, v.data); + appendStringInfoString(&ddl, "); "); + pfree(v.data); + } + appendStringInfoString(&ddl, "ANALYZE ci_t;"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build ci_t failed: %s", err ? err : ""); + + pfree(ddl.data); +} + +/* + * citext equality folds case: a string equals both its uppercased and its + * lowercased forms when compared as citext. + * + * S::citext = upper(S)::citext AND S::citext = lower(S)::citext + */ +PG_PROP(citext_eq_case_insensitive) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + + conn = connect_citext(tc); + + initStringInfo(&s); + draw_mixed(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "::citext = upper("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")::citext) AND ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "::citext = lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")::citext)"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * LIKE on citext is case-insensitive. With no wildcards in the pattern P, + * citext LIKE reduces to case-insensitive equality, so it must agree with + * plain-text LIKE on the lower()-folded strings: + * + * (S::citext LIKE P::citext) = (lower(S) LIKE lower(P)) + * + * Our alphabet contains no %, _ or backslash, so P is a literal pattern and + * the identity is sound. A folding bug in citext's LIKE operator fails here. + */ +PG_PROP(citext_like_insensitive) +{ + pg_prop_conn *conn; + StringInfoData s, + p, + q; + + conn = connect_citext(tc); + + initStringInfo(&s); + initStringInfo(&p); + draw_mixed(tc, &s); + draw_mixed(tc, &p); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "::citext LIKE "); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, "::citext) = (lower("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") LIKE lower("); + append_sql_literal(&q, p.data); + appendStringInfoString(&q, "))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(p.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * DISTINCT over citext folds case exactly like DISTINCT over lower()-folded + * text. Counting distinct x::citext must equal counting distinct lower(x) + * over the same rows: + * + * (SELECT count(DISTINCT x::citext) FROM ci_t) + * = (SELECT count(DISTINCT lower(x)) FROM ci_t) + * + * Both aggregates return exactly one row even when ci_t is empty (count is 0 + * on both sides), so the scalar oracle is sound at the empty boundary. A bug + * in citext's hash/equality used by the DISTINCT path fails here. + */ +PG_PROP(citext_distinct_folds) +{ + pg_prop_conn *conn; + + conn = connect_citext(tc); + build_ci_t(tc, conn); + + require_true(tc, conn, + "SELECT (SELECT count(DISTINCT x::citext) FROM ci_t) " + "= (SELECT count(DISTINCT lower(x)) FROM ci_t)"); + + pg_prop_disconnect(conn); +} + +/* + * citext ordering is case-insensitive, matching lower()-folded comparison: + * + * (A::citext < B::citext) = (lower(A) < lower(B)) + * + * A sign or short-circuit bug in citext's comparison routine (the one the + * sort/aggregate paths use) fails here. + */ +PG_PROP(citext_order_matches_lower) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + + conn = connect_citext(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_mixed(tc, &a); + draw_mixed(tc, &b); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, a.data); + appendStringInfoString(&q, "::citext < "); + append_sql_literal(&q, b.data); + appendStringInfoString(&q, "::citext) = (lower("); + append_sql_literal(&q, a.data); + appendStringInfoString(&q, ") < lower("); + append_sql_literal(&q, b.data); + appendStringInfoString(&q, "))"); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Concatenation is plain text, not folded: concatenating two citext values + * and casting the result back to text yields the raw concatenation of the + * stored bytes (folding happens only for comparison, never for storage or + * output): + * + * (S::citext || T::citext)::text = (S || T) + */ +PG_PROP(citext_concat_preserves) +{ + pg_prop_conn *conn; + StringInfoData s, + t, + q; + + conn = connect_citext(tc); + + initStringInfo(&s); + initStringInfo(&t); + draw_mixed(tc, &s); + draw_mixed(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT (("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "::citext || "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, "::citext)::text) = ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " || "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * min/max over citext use its comparison function; over any set of rows the + * minimum must be <= the maximum. Over an empty ci_t both aggregates are + * NULL, so coalesce the comparison to true to keep the empty case sound: + * + * coalesce(min(x::citext) <= max(x::citext), true) + * + * A comparison-direction bug in citext's min/max support would make min > max + * and fail here. + */ +PG_PROP(citext_min_max) +{ + pg_prop_conn *conn; + + conn = connect_citext(tc); + build_ci_t(tc, conn); + + require_true(tc, conn, + "SELECT coalesce(min(x::citext) <= max(x::citext), true) " + "FROM ci_t"); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 61a07e6f8fd21..d29280d5d24b8 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 58e6f543613da2310a02ba333d6a9ce61e1fa137 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:30 -0400 Subject: [PATCH 244/338] prop: add cube property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_cube.c | 406 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 410 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_cube.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index ebd57e457629e..a27ca19cfb129 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -259,7 +259,8 @@ OBJS = \ prop_ltree.o \ prop_ltree2.o \ prop_citext.o \ - prop_citext2.o + prop_citext2.o \ + prop_cube.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 4f59d7bb58a94..117688abe6f17 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -257,6 +257,7 @@ prop_area_sources = files( 'prop_ltree2.c', 'prop_citext.c', 'prop_citext2.c', + 'prop_cube.c', ) diff --git a/src/test/prop/prop_cube.c b/src/test/prop/prop_cube.c new file mode 100644 index 0000000000000..45051abf0f927 --- /dev/null +++ b/src/test/prop/prop_cube.c @@ -0,0 +1,406 @@ +/*------------------------------------------------------------------------- + * + * prop_cube.c + * Property-based tests for the cube contrib extension. + * + * cube represents points and boxes in N dimensions (contrib/cube); these + * check text round-trip, containment reflexivity, union/intersection bounds, + * and dimension/distance invariants. + * + * A cube is either a point in d-space or an axis-aligned box between a + * lower-left and an upper-right corner. Its coordinates are float8, but the + * literals here draw small integers (-20..20) so every comparison is exact. + * The oracle is cube's own algebra: a cube contains itself; a cube's text + * output parses back equal; cube_union bounds both operands; cube_inter is + * bounded by each operand; cube_distance is a non-negative, symmetric metric; + * and each lower-left coordinate is <= its matching upper-right coordinate. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_cube.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "cube" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a cube operation errored or produced NULL on valid + * inputs, which is itself a bug -- so either is a failure, reported with the + * SQL. Mirrors require_true in prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Connect to the test cluster and ensure the cube extension exists. Each + * property opens its own connection (the Tier-B pattern); CREATE EXTENSION IF + * NOT EXISTS makes it idempotent across connections. On any failure the + * property is failed with the message. + */ +static pg_prop_conn * +connect_cube(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, + "CREATE EXTENSION IF NOT EXISTS cube", + &err)) + pg_prop_fail(tc, "setup failed (cube): %s", err ? err : ""); + + return conn; +} + +/* + * Draw a cube literal of dimension d into buf as a single-quoted, cast + * expression: '...'::cube. With probability 1/2 it is a point + * '(x1,...,xd)'::cube; otherwise a box '(l1,...,ld),(h1,...,hd)'::cube with + * each lo <= hi (cube normalizes internally, but keeping the literal clean + * makes failures readable). Coordinates are small integers in -20..20, which + * float8 represents exactly, so every downstream comparison is exact. The + * literal only ever contains parentheses, commas, digits and minus signs, all + * of which are fine inside single quotes. + */ +static void +draw_cube(pg_prop_tc *tc, StringInfo buf, int d) +{ + bool is_box = pg_prop_draw_bool(tc); + int k; + + appendStringInfoChar(buf, '\''); + + if (!is_box) + { + /* point: (x1, x2, ..., xd) */ + appendStringInfoChar(buf, '('); + for (k = 0; k < d; k++) + { + if (k > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "%d", + (int) pg_prop_draw_int(tc, -20, 20)); + } + appendStringInfoChar(buf, ')'); + } + else + { + int lo[4]; + int hi[4]; + + /* box: draw both corners, then normalize so each lo <= hi */ + for (k = 0; k < d; k++) + { + int a = (int) pg_prop_draw_int(tc, -20, 20); + int b = (int) pg_prop_draw_int(tc, -20, 20); + + lo[k] = (a <= b) ? a : b; + hi[k] = (a <= b) ? b : a; + } + + appendStringInfoChar(buf, '('); + for (k = 0; k < d; k++) + { + if (k > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "%d", lo[k]); + } + appendStringInfoString(buf, "),("); + for (k = 0; k < d; k++) + { + if (k > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "%d", hi[k]); + } + appendStringInfoChar(buf, ')'); + } + + appendStringInfoString(buf, "'::cube"); +} + +/* + * Text round-trip: a cube's text output parses back to the identical cube. + * + * (C)::text::cube = C + * + * Any quoting or formatting bug in cube_out/cube_in breaks this. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + StringInfoData c, + q; + int d = (int) pg_prop_draw_int(tc, 1, 4); + + conn = connect_cube(tc); + + initStringInfo(&c); + draw_cube(tc, &c, d); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)::text::cube = (%s)", + c.data, c.data); + + require_true(tc, conn, q.data); + + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Containment reflexivity: every cube contains itself and is contained by + * itself. + * + * C @> C AND C <@ C + * + * @> is "contains" and <@ is "contained by"; a boundary bug in the + * containment test (strict vs. non-strict) breaks this. + */ +PG_PROP(contains_self) +{ + pg_prop_conn *conn; + StringInfoData c, + q; + int d = (int) pg_prop_draw_int(tc, 1, 4); + + conn = connect_cube(tc); + + initStringInfo(&c); + draw_cube(tc, &c, d); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s) @> (%s) AND (%s) <@ (%s)", + c.data, c.data, c.data, c.data); + + require_true(tc, conn, q.data); + + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Dimension matches: cube_dim reports the dimension the literal was built + * with. + * + * cube_dim(C) = d + * + * cube_dim counts the coordinates; a point and a box of the same dimension + * both report d. + */ +PG_PROP(dim_matches) +{ + pg_prop_conn *conn; + StringInfoData c, + q; + int d = (int) pg_prop_draw_int(tc, 1, 4); + + conn = connect_cube(tc); + + initStringInfo(&c); + draw_cube(tc, &c, d); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT cube_dim(%s) = %d", c.data, d); + + require_true(tc, conn, q.data); + + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Union bounds both operands: the bounding box of A and B contains each. + * + * cube_union(A, B) @> A AND cube_union(A, B) @> B + * + * cube_union requires same-dimension operands, so A and B are both drawn at + * the one drawn dimension d. + */ +PG_PROP(union_contains_both) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + int d = (int) pg_prop_draw_int(tc, 1, 4); + + conn = connect_cube(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_cube(tc, &a, d); + draw_cube(tc, &b, d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cube_union(%s, %s) @> (%s) " + "AND cube_union(%s, %s) @> (%s)", + a.data, b.data, a.data, + a.data, b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * When two cubes overlap, their intersection is contained in each. For + * DISJOINT cubes cube_inter does not return an empty cube -- it returns a box + * spanning the gap, contained in neither -- so the law is guarded by the + * overlap operator &&. + * + * A && B => cube_inter(A, B) <@ A AND cube_inter(A, B) <@ B + * + * cube_inter requires same-dimension operands, so A and B are both drawn at + * the one drawn dimension d. + */ +PG_PROP(inter_contained_in_both) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + int d = (int) pg_prop_draw_int(tc, 1, 4); + + conn = connect_cube(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_cube(tc, &a, d); + draw_cube(tc, &b, d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN (%s) && (%s) " + "THEN cube_inter(%s, %s) <@ (%s) AND cube_inter(%s, %s) <@ (%s) " + "ELSE true END", + a.data, b.data, + a.data, b.data, a.data, + a.data, b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Distance is a non-negative, symmetric metric: cube_distance is >= 0 and + * does not depend on operand order. + * + * cube_distance(A, B) >= 0 AND cube_distance(A, B) = cube_distance(B, A) + * + * cube_distance requires same-dimension operands, so A and B are both drawn + * at the one drawn dimension d. Integer coordinates keep the Euclidean + * distance's intermediate arithmetic exact, so the symmetric equality is + * exact. + */ +PG_PROP(distance_nonneg_symmetric) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + int d = (int) pg_prop_draw_int(tc, 1, 4); + + conn = connect_cube(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_cube(tc, &a, d); + draw_cube(tc, &b, d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cube_distance(%s, %s) >= 0 " + "AND cube_distance(%s, %s) = cube_distance(%s, %s)", + a.data, b.data, + a.data, b.data, b.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Lower-left <= upper-right per coordinate: for every dimension k in 1..d the + * lower-left corner coordinate does not exceed the upper-right one. + * + * bool_and(cube_ll_coord(C, k) <= cube_ur_coord(C, k)) + * FROM generate_series(1, d) k + * + * This is cube's internal normalization invariant surfaced through the + * coordinate accessors; it must hold for points (where ll = ur) and boxes + * alike. + */ +PG_PROP(ll_ur_bound_coordinates) +{ + pg_prop_conn *conn; + StringInfoData c, + q; + int d = (int) pg_prop_draw_int(tc, 1, 4); + + conn = connect_cube(tc); + + initStringInfo(&c); + draw_cube(tc, &c, d); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT bool_and(cube_ll_coord(%s, k) <= cube_ur_coord(%s, k)) " + "FROM generate_series(1, %d) k", + c.data, c.data, d); + + require_true(tc, conn, q.data); + + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index d29280d5d24b8..84531c3b86ebd 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From bdbbe4dd9503c021e0ce6180e2e0cf6b0e57e699 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:30 -0400 Subject: [PATCH 245/338] prop: add cube distance metric axioms property tests (cubedist) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_cubedist.c | 385 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 389 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_cubedist.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index a27ca19cfb129..797327b1146a8 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -260,7 +260,8 @@ OBJS = \ prop_ltree2.o \ prop_citext.o \ prop_citext2.o \ - prop_cube.o + prop_cube.o \ + prop_cubedist.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 117688abe6f17..643cc8c7f62ed 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -258,6 +258,7 @@ prop_area_sources = files( 'prop_citext.c', 'prop_citext2.c', 'prop_cube.c', + 'prop_cubedist.c', ) diff --git a/src/test/prop/prop_cubedist.c b/src/test/prop/prop_cubedist.c new file mode 100644 index 0000000000000..d22bfdd75ffaa --- /dev/null +++ b/src/test/prop/prop_cubedist.c @@ -0,0 +1,385 @@ +/*------------------------------------------------------------------------- + * + * prop_cubedist.c + * Property-based tests for the cube extension's distance metric. + * + * Where prop_cube.c covers containment and the box algebra, this area targets + * the metric surface of contrib/cube: cube_distance as a Euclidean metric + * over point cubes, plus the coordinate accessors (cube_ll_coord, + * cube_ur_coord) and cube_dim on single-point cubes. A failure means + * cube_distance is not a metric (negative, asymmetric, non-zero on a point to + * itself, or violating the triangle inequality) or that a coordinate accessor + * disagrees with the integer the point was built from -- each a bug in + * contrib/cube's float8 math or coordinate indexing. + * + * Points are built from integer coordinates in -50..50 over dimension 1..3. + * Integers are exact in float8, so each squared-difference sum is exact and + * the only inexact step is the final sqrt; the Euclidean-match property + * therefore uses a 1e-6 tolerance, and the triangle inequality a 1e-9 slack. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_cubedist.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "cubedist" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* Dimension and coordinate domains. */ +#define MAX_DIM 3 +#define COORD_MIN (-50) +#define COORD_MAX 50 + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means a cube operation errored or produced NULL on valid + * inputs, which is itself a bug -- so either is a failure, reported with the + * SQL. Mirrors require_true in prop_numeric.c / prop_cube.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Connect to the test cluster and ensure the cube extension exists. Each + * property opens its own connection (the Tier-B pattern); CREATE EXTENSION IF + * NOT EXISTS makes it idempotent across the thousands of per-case + * connections. On any failure the property is failed with the message. + */ +static pg_prop_conn * +connect_cube(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, + "CREATE EXTENSION IF NOT EXISTS cube", + &err)) + pg_prop_fail(tc, "setup failed (cube): %s", err ? err : ""); + + return conn; +} + +/* + * Draw a point of dimension d, filling coords[0..d-1] with integers in + * [COORD_MIN, COORD_MAX], and emit the point as a cube literal into buf: + * + * '(c1, c2, ..., cd)'::cube + * + * The text form is used (not cube(ARRAY[...])) because it is the most direct + * point constructor and needs no per-element casting; negative coordinates + * are harmless inside the single-quoted literal. cube normalizes a d-coord + * literal to a single point whose lower-left and upper-right corners coincide. + */ +static void +draw_point(pg_prop_tc *tc, StringInfo buf, int d, int *coords) +{ + int k; + + appendStringInfoString(buf, "'("); + for (k = 0; k < d; k++) + { + coords[k] = (int) pg_prop_draw_int(tc, COORD_MIN, COORD_MAX); + if (k > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "%d", coords[k]); + } + appendStringInfoString(buf, ")'::cube"); +} + +/* + * cube_distance(A, B) >= 0 for any two same-dimension point cubes. + */ +PG_PROP(distance_nonneg) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + int d = (int) pg_prop_draw_int(tc, 1, MAX_DIM); + int ca[MAX_DIM]; + int cb[MAX_DIM]; + + conn = connect_cube(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_point(tc, &a, d, ca); + draw_point(tc, &b, d, cb); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT cube_distance(%s, %s) >= 0", + a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * cube_distance is symmetric: cube_distance(A, B) = cube_distance(B, A). + * Integer coordinates make the squared-difference sum exact and sqrt + * deterministic, so this holds with no tolerance. + */ +PG_PROP(distance_symmetric) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + int d = (int) pg_prop_draw_int(tc, 1, MAX_DIM); + int ca[MAX_DIM]; + int cb[MAX_DIM]; + + conn = connect_cube(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_point(tc, &a, d, ca); + draw_point(tc, &b, d, cb); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cube_distance(%s, %s) = cube_distance(%s, %s)", + a.data, b.data, b.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * cube_distance(A, A) = 0: the distance from a point to itself is exactly + * zero. + */ +PG_PROP(distance_self_zero) +{ + pg_prop_conn *conn; + StringInfoData a, + q; + int d = (int) pg_prop_draw_int(tc, 1, MAX_DIM); + int ca[MAX_DIM]; + + conn = connect_cube(tc); + + initStringInfo(&a); + draw_point(tc, &a, d, ca); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT cube_distance(%s, %s) = 0", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Triangle inequality: cube_distance(A, C) <= cube_distance(A, B) + + * cube_distance(B, C) + 1e-9. The small slack absorbs the rounding of three + * independent sqrt results so a tight (equal) triple on collinear points + * cannot fail on a last-bit rounding difference. + */ +PG_PROP(triangle_inequality) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + c, + q; + int d = (int) pg_prop_draw_int(tc, 1, MAX_DIM); + int ca[MAX_DIM]; + int cb[MAX_DIM]; + int cc[MAX_DIM]; + + conn = connect_cube(tc); + + initStringInfo(&a); + initStringInfo(&b); + initStringInfo(&c); + draw_point(tc, &a, d, ca); + draw_point(tc, &b, d, cb); + draw_point(tc, &c, d, cc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT cube_distance(%s, %s) <= " + "cube_distance(%s, %s) + cube_distance(%s, %s) + 1e-9", + a.data, c.data, + a.data, b.data, b.data, c.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(c.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Coordinate accessors on a point: for a single-point cube P and every + * dimension k in 1..d, cube_ll_coord(P, k) = cube_ur_coord(P, k) = the k-th + * coordinate the point was built from. The expected integers are substituted + * directly so the accessor is checked against the source value, not against + * itself. + */ +PG_PROP(coord_access) +{ + pg_prop_conn *conn; + StringInfoData a, + q; + int d = (int) pg_prop_draw_int(tc, 1, MAX_DIM); + int ca[MAX_DIM]; + int k; + + conn = connect_cube(tc); + + initStringInfo(&a); + draw_point(tc, &a, d, ca); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + for (k = 0; k < d; k++) + { + if (k > 0) + appendStringInfoString(&q, " AND "); + /* emit negative expected coords as (%d) so the comparison parses */ + appendStringInfo(&q, + "cube_ll_coord(%s, %d) = (%d) " + "AND cube_ur_coord(%s, %d) = (%d)", + a.data, k + 1, ca[k], + a.data, k + 1, ca[k]); + } + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * cube_dim(P) = d: a point built from d coordinates reports dimension d. + */ +PG_PROP(dim_matches) +{ + pg_prop_conn *conn; + StringInfoData a, + q; + int d = (int) pg_prop_draw_int(tc, 1, MAX_DIM); + int ca[MAX_DIM]; + + conn = connect_cube(tc); + + initStringInfo(&a); + draw_point(tc, &a, d, ca); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT cube_dim(%s) = %d", a.data, d); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Euclidean match: cube_distance(A, B) equals the Euclidean distance computed + * independently in SQL from the integer coordinates, within 1e-6. + * + * abs(cube_distance(A, B) - sqrt(sum (ai - bi)^2)) < 1e-6 + * + * The squared-difference sum is formed from the drawn integers (each as a + * parenthesized float8 literal so a negative difference's square is exact), + * giving an independent oracle: cube_distance must agree with the textbook + * formula up to float8 sqrt rounding. + */ +PG_PROP(distance_matches_euclidean) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + sumsq, + q; + int d = (int) pg_prop_draw_int(tc, 1, MAX_DIM); + int ca[MAX_DIM]; + int cb[MAX_DIM]; + int k; + + conn = connect_cube(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_point(tc, &a, d, ca); + draw_point(tc, &b, d, cb); + + /* sum of squared coordinate differences, as a float8 expression */ + initStringInfo(&sumsq); + for (k = 0; k < d; k++) + { + int diff = ca[k] - cb[k]; + + if (k > 0) + appendStringInfoString(&sumsq, " + "); + /* (diff)^2; parenthesize diff so a negative value squares correctly */ + appendStringInfo(&sumsq, "(%d)::float8 * (%d)::float8", diff, diff); + } + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT abs(cube_distance(%s, %s) - sqrt(%s)) < 1e-6", + a.data, b.data, sumsq.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(sumsq.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 84531c3b86ebd..bca1a30ca8df8 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From c25dc17a40bcbd865cb5431978debd20936f24b0 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:30 -0400 Subject: [PATCH 246/338] prop: add seg interval containment and overlap property tests (seg) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_seg.c | 341 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 345 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_seg.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 797327b1146a8..9c72280dead5e 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -261,7 +261,8 @@ OBJS = \ prop_citext.o \ prop_citext2.o \ prop_cube.o \ - prop_cubedist.o + prop_cubedist.o \ + prop_seg.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 643cc8c7f62ed..a82a31783f529 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -259,6 +259,7 @@ prop_area_sources = files( 'prop_citext2.c', 'prop_cube.c', 'prop_cubedist.c', + 'prop_seg.c', ) diff --git a/src/test/prop/prop_seg.c b/src/test/prop/prop_seg.c new file mode 100644 index 0000000000000..cd873a200433d --- /dev/null +++ b/src/test/prop/prop_seg.c @@ -0,0 +1,341 @@ +/*------------------------------------------------------------------------- + * + * prop_seg.c + * Property-based tests for the seg contrib extension. + * + * seg represents a 1-D interval [lower, upper] with floating-point bounds and + * error tolerances (contrib/seg). These check text round-trip, containment + * reflexivity, the lower <= upper invariant, overlap symmetry, and that + * containment agrees with the stored bounds. (seg has no union/intersection + * operator, only comparison and containment.) Integer-valued bounds are used + * so the underlying float stores and compares exactly. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. connect_seg() creates the extension. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_seg.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "seg" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the seg operation errored or produced NULL on valid + * inputs, which is itself a bug -- so either is a failure, reported with the + * SQL. Mirrors require_true in prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Connect to the test cluster and ensure the seg extension exists. Each + * property opens its own connection (the Tier-B pattern); CREATE EXTENSION + * IF NOT EXISTS is idempotent across connections. On failure the property is + * failed with the message. + */ +static pg_prop_conn * +connect_seg(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, "CREATE EXTENSION IF NOT EXISTS seg", &err)) + pg_prop_fail(tc, "setup failed (seg): %s", err ? err : ""); + + return conn; +} + +/* + * Draw a seg literal into buf as ' .. '::seg, with two integers drawn + * in -100..100 and ordered so lo <= hi. The chosen lo and hi are returned + * through *lo / *hi so a caller can build a point value inside the interval. + * Integer bounds keep the underlying float exact for equality comparisons. + */ +static void +draw_seg(pg_prop_tc *tc, StringInfo buf, int *lo, int *hi) +{ + int a = (int) pg_prop_draw_int(tc, -100, 100); + int b = (int) pg_prop_draw_int(tc, -100, 100); + int l, + h; + + if (a <= b) + { + l = a; + h = b; + } + else + { + l = b; + h = a; + } + + appendStringInfo(buf, "'%d .. %d'::seg", l, h); + + *lo = l; + *hi = h; +} + +/* + * Text round-trip: (S)::text::seg = S. seg_out formats the interval and + * seg_in must parse that text back to the identical seg; integer-valued + * bounds round-trip exactly through the float store. + */ +PG_PROP(text_roundtrip) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + int lo, + hi; + + conn = connect_seg(tc); + + initStringInfo(&s); + draw_seg(tc, &s, &lo, &hi); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s)::text::seg = (%s)", s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Containment reflexivity: S @> S AND S <@ S. A seg always contains itself + * and is contained in itself. + */ +PG_PROP(contains_self) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + int lo, + hi; + + conn = connect_seg(tc); + + initStringInfo(&s); + draw_seg(tc, &s, &lo, &hi); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s) @> (%s) AND (%s) <@ (%s)", + s.data, s.data, s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Interval invariant: seg_lower(S) <= seg_upper(S). Every seg's lower bound + * is at most its upper bound. + */ +PG_PROP(lower_le_upper) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + int lo, + hi; + + conn = connect_seg(tc); + + initStringInfo(&s); + draw_seg(tc, &s, &lo, &hi); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT seg_lower(%s) <= seg_upper(%s)", + s.data, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * seg exposes comparison and containment operators but no union/intersection, + * so overlap (&&) is checked for symmetry instead. + */ +PG_PROP(overlap_is_symmetric) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + int alo, + ahi, + blo, + bhi; + + conn = connect_seg(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_seg(tc, &a, &alo, &ahi); + draw_seg(tc, &b, &blo, &bhi); + + /* seg has no union/intersection operator; && overlap is symmetric */ + initStringInfo(&q); + appendStringInfo(&q, "SELECT ((%s) && (%s)) = ((%s) && (%s))", + a.data, b.data, b.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Strict left/right (<< and >>) are mutually exclusive and imply no overlap: + * if A << B (A entirely left of B) then not (A && B), and not (B << A). + */ +PG_PROP(strictly_left_excludes_overlap) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + int alo, + ahi, + blo, + bhi; + + conn = connect_seg(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_seg(tc, &a, &alo, &ahi); + draw_seg(tc, &b, &blo, &bhi); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN (%s) << (%s) " + "THEN NOT ((%s) && (%s)) AND NOT ((%s) << (%s)) " + "ELSE true END", + a.data, b.data, + a.data, b.data, b.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Point containment: a degenerate single-point seg built from a value v with + * lo <= v <= hi is contained in S. v is drawn in C within the drawn seg's + * integer bounds, so the point lies inside the interval and S @> it. + */ +PG_PROP(contains_point_seg) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + int lo, + hi, + v; + + conn = connect_seg(tc); + + initStringInfo(&s); + draw_seg(tc, &s, &lo, &hi); + + v = (int) pg_prop_draw_int(tc, lo, hi); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s) @> (%d || ' .. ' || %d)::seg", + s.data, v, v); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Containment is consistent with the stored bounds: A @> B iff A's lower is + * <= B's lower and A's upper is >= B's upper. seg_lower/seg_upper are the + * independent oracle for the @> containment operator. + */ +PG_PROP(contains_matches_bounds) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + int alo, + ahi, + blo, + bhi; + + conn = connect_seg(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_seg(tc, &a, &alo, &ahi); + draw_seg(tc, &b, &blo, &bhi); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ((%s) @> (%s)) " + "= (seg_lower(%s) <= seg_lower(%s) " + "AND seg_upper(%s) >= seg_upper(%s))", + a.data, b.data, + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index bca1a30ca8df8..42d24a9b688ff 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 5a722fcbcbd26c68e65a236d94552eeabfb49b2c Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:31 -0400 Subject: [PATCH 247/338] prop: add intarray property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_intarray.c | 312 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 316 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_intarray.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 9c72280dead5e..06a670358a7ee 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -262,7 +262,8 @@ OBJS = \ prop_citext2.o \ prop_cube.o \ prop_cubedist.o \ - prop_seg.o + prop_seg.o \ + prop_intarray.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index a82a31783f529..f4edf8526a486 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -260,6 +260,7 @@ prop_area_sources = files( 'prop_cube.c', 'prop_cubedist.c', 'prop_seg.c', + 'prop_intarray.c', ) diff --git a/src/test/prop/prop_intarray.c b/src/test/prop/prop_intarray.c new file mode 100644 index 0000000000000..37b66e02ac2b3 --- /dev/null +++ b/src/test/prop/prop_intarray.c @@ -0,0 +1,312 @@ +/*------------------------------------------------------------------------- + * + * prop_intarray.c + * Property-based tests for the intarray contrib extension (int[] ops). + * + * intarray adds set/multiset operators on int[]; these check its sort/uniq + * normalization and the +/-/&/# set operators against core-SQL oracles. + * + * intarray redefines operators on int[] (sort, uniq, + union, - difference, + * & intersection, idx) that core SQL does not provide. The oracle here is + * core-SQL set machinery independent of intarray's own operators: multiset + * sorting via array_agg(x ORDER BY x), distinct-sorting via + * array_agg(DISTINCT x ORDER BY x), and element membership via core + * "= ANY(array)". Any "f" (or an unexpected NULL/error) is a bug in intarray, + * not a modelling slip -- empty-array edges stay crisp via + * coalesce(bool_and(...), true) and coalesce(..., '{}'::int[]). + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_intarray.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "intarray" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a random SQL integer-array literal into buf as an ARRAY[...] expression + * typed ::int[]. n is drawn in 0..8 elements, each a small int in -10..10 so + * duplicates are likely and the set operators have overlapping operands to + * work on. No NULL elements: intarray's operators are undefined for NULLs. + * The ::int[] cast is always emitted so an empty array (ARRAY[]::int[]) still + * carries a type. + */ +static void +draw_intarr(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 8); + + appendStringInfoString(buf, "ARRAY["); + for (int i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, -10, 10)); + } + appendStringInfoString(buf, "]::int[]"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means an intarray operator errored or produced NULL on valid + * input, which is itself a bug -- so either is a failure, reported with the + * SQL and any error text. Mirrors require_true in prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Connect to the test cluster and ensure the intarray extension exists. Each + * property opens its own connection (the Tier-B pattern); CREATE EXTENSION IF + * NOT EXISTS makes installing it idempotent across connections. On any + * failure the property is failed with the message. + */ +static pg_prop_conn * +connect_intarray(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, + "CREATE EXTENSION IF NOT EXISTS intarray", + &err)) + pg_prop_fail(tc, "setup failed (intarray): %s", err ? err : ""); + + return conn; +} + +/* + * sort(A) is an ordered permutation of A: it has the same elements in + * ascending order. array_agg(x ORDER BY x) over unnest(A) is an independent + * sort of the same multiset (core SQL, not intarray), so it must equal + * sort(A). This catches an out-of-order or element-dropping bug in intarray's + * sort. + */ +PG_PROP(sort_is_ordered_permutation) +{ + pg_prop_conn *conn; + StringInfoData a, + q; + + conn = connect_intarray(tc); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce((SELECT array_agg(x ORDER BY x) FROM unnest(%s) x), " + "'{}'::int[]) = sort(%s)", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * uniq(sort(A)) is the sorted distinct elements of A. uniq removes only + * ADJACENT duplicates, so it must be applied to sort(A); the oracle is + * core-SQL array_agg(DISTINCT x ORDER BY x). array_agg over no rows is NULL, + * so the empty case is reconciled with coalesce(..., '{}'::int[]) -- intarray + * yields '{}' for uniq(sort('{}')). + */ +PG_PROP(uniq_sort_is_distinct_sorted) +{ + pg_prop_conn *conn; + StringInfoData a, + q; + + conn = connect_intarray(tc); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT uniq(sort(%s)) " + "= coalesce((SELECT array_agg(DISTINCT x ORDER BY x) " + " FROM unnest(%s) x), '{}'::int[])", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The intarray union A + B contains every element of A (and, by symmetry of + * the drawn operands over many cases, of B). The oracle is core membership + * "= ANY (A + B)" over each element of A via unnest -- independent of + * intarray's own containment operators. coalesce(bool_and(...), true) makes + * an empty A pass (no rows to check). + */ +PG_PROP(union_contains_both) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + + conn = connect_intarray(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_intarr(tc, &a); + draw_intarr(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(bool_and(e = ANY (%s + %s)), true) " + "FROM unnest(%s) e", + a.data, b.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The intarray intersection A & B contains only elements present in BOTH + * operands: every element of the intersection is a member of A and of B + * (core "= ANY"). coalesce(bool_and(...), true) makes an empty intersection + * pass. A spurious element in the intersection fails here. + */ +PG_PROP(intersection_subset) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + + conn = connect_intarray(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_intarr(tc, &a); + draw_intarr(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(bool_and(e = ANY (%s) AND e = ANY (%s)), true) " + "FROM unnest(%s & %s) e", + a.data, b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The intarray difference A - B shares no element with B: every element of + * A - B is NOT a member of B (core "= ANY"). coalesce(bool_and(...), true) + * makes an empty difference pass. A B-element leaking into the difference + * fails here. + */ +PG_PROP(difference_removes_b) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + + conn = connect_intarray(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_intarr(tc, &a); + draw_intarr(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(bool_and(NOT (e = ANY (%s))), true) " + "FROM unnest(%s - %s) e", + b.data, a.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * idx(A, v) returns the 1-based position of v in A, or 0 if v is absent. v is + * drawn in -12..12 (wider than the element range -10..10) so it is sometimes + * outside the array. When v is present, A[idx(A, v)] must be v; when absent, + * idx must be 0. The CASE picks the right check via core "= ANY". Empty A is + * covered: v is not ANY(A), so the ELSE branch requires idx = 0, which holds. + */ +PG_PROP(idx_finds_member) +{ + pg_prop_conn *conn; + StringInfoData a, + q; + int v; + + conn = connect_intarray(tc); + + initStringInfo(&a); + draw_intarr(tc, &a); + v = (int) pg_prop_draw_int(tc, -12, 12); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN %d = ANY (%s) " + "THEN (%s)[idx(%s, %d)] = %d " + "ELSE idx(%s, %d) = 0 END", + v, a.data, a.data, a.data, v, v, a.data, v); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 42d24a9b688ff..b5011d227d880 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 98292eea54efb0ab430e8a45e4367630c38f9160 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:31 -0400 Subject: [PATCH 248/338] prop: add intarray count, set ops, and sort/uniq property tests (intarrayext) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_intarrayext.c | 357 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 361 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_intarrayext.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 06a670358a7ee..0a90fc918c487 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -263,7 +263,8 @@ OBJS = \ prop_cube.o \ prop_cubedist.o \ prop_seg.o \ - prop_intarray.o + prop_intarray.o \ + prop_intarrayext.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index f4edf8526a486..82af967698dfe 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -261,6 +261,7 @@ prop_area_sources = files( 'prop_cubedist.c', 'prop_seg.c', 'prop_intarray.c', + 'prop_intarrayext.c', ) diff --git a/src/test/prop/prop_intarrayext.c b/src/test/prop/prop_intarrayext.c new file mode 100644 index 0000000000000..cd4671f31239f --- /dev/null +++ b/src/test/prop/prop_intarrayext.c @@ -0,0 +1,357 @@ +/*------------------------------------------------------------------------- + * + * prop_intarrayext.c + * Property-based tests for intarray's own operator algebra (int[] ops). + * + * A companion to prop_intarray.c, which checks intarray against core-SQL + * oracles. This area instead exercises intarray's own operators as a closed + * algebra: icount, sort, uniq, idx and the operators + - & | @> <@. Each + * property states a relation that intarray's operators must satisfy among + * themselves (and against core cardinality/count), so a drop/dup/order bug or + * a wrong containment answer surfaces as an "f". + * + * intarray requires non-NULL elements, so arrays are built with 0..6 small + * positive ints (0..9); duplicates are likely so the set operators have real + * work. Any "f" (or an unexpected NULL/error) on such input is an intarray + * bug, not a modelling slip -- empty-array edges stay crisp via + * coalesce(..., true). + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_intarrayext.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "intarrayext" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Draw a random intarray literal into buf as an ARRAY[...] expression typed + * ::int[]. n is drawn in 0..6 elements, each a small positive int in 0..9 so + * duplicates are likely and a counterexample stays legible. No NULL elements: + * intarray's operators are undefined for NULLs. The ::int[] cast is always + * emitted so an empty array (ARRAY[]::int[]) still carries a type. + */ +static void +draw_intarr(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 6); + + appendStringInfoString(buf, "ARRAY["); + for (int i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, 0, 9)); + } + appendStringInfoString(buf, "]::int[]"); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means an intarray operator errored or produced NULL on valid + * input, which is itself a bug -- so either is a failure, reported with the + * SQL and any error text. Mirrors require_true in prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Connect to the test cluster and ensure the intarray extension exists. Each + * property opens its own connection (the Tier-B pattern); CREATE EXTENSION IF + * NOT EXISTS makes installing it idempotent across the thousands of per-case + * connections. On any failure the property is failed with the message. + */ +static pg_prop_conn * +connect_intarray(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, + "CREATE EXTENSION IF NOT EXISTS intarray", + &err)) + pg_prop_fail(tc, "setup failed (intarray): %s", err ? err : ""); + + return conn; +} + +/* + * icount(A) counts the elements of A. With no NULL elements that must equal + * core cardinality(A), which counts non-NULL elements. A disagreement means + * icount miscounts (e.g. drops or double-counts an element). + */ +PG_PROP(count_matches_cardinality) +{ + pg_prop_conn *conn; + StringInfoData a, + q; + + conn = connect_intarray(tc); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT icount(%s) = cardinality(%s)", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * uniq(sort(A)) is the sorted DISTINCT elements of A, so its element count + * must equal the number of distinct values in A. icount of the normalized + * array is compared with core count(DISTINCT x) over unnest(A). uniq removes + * only ADJACENT duplicates, hence it must be applied to sort(A); a dup left + * behind (or a value wrongly collapsed) breaks the count. + */ +PG_PROP(sort_uniq_distinct) +{ + pg_prop_conn *conn; + StringInfoData a, + q; + + conn = connect_intarray(tc); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT icount(uniq(sort(%s))) " + "= (SELECT count(DISTINCT x) FROM unnest(%s) x)", + a.data, a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The intarray union A | B contains both operands: (A | B) @> A and + * (A | B) @> B, using intarray's own containment operator @>. Union must not + * drop an element of either side; a lost element fails one of the two @>. + */ +PG_PROP(union_contains_both) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + + conn = connect_intarray(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_intarr(tc, &a); + draw_intarr(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s | %s) @> %s AND (%s | %s) @> %s", + a.data, b.data, a.data, a.data, b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The intarray intersection A & B is contained in both operands: + * (A & B) <@ A and (A & B) <@ B, using intarray's own containment operator + * <@. A spurious element in the intersection -- one not in both sides -- + * fails one of the two <@. + */ +PG_PROP(intersect_subset) +{ + pg_prop_conn *conn; + StringInfoData a, + b, + q; + + conn = connect_intarray(tc); + + initStringInfo(&a); + initStringInfo(&b); + draw_intarr(tc, &a); + draw_intarr(tc, &b); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s & %s) <@ %s AND (%s & %s) <@ %s", + a.data, b.data, a.data, a.data, b.data, b.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(b.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The intarray push operator A + e appends element e, so the result must + * contain e: (A + e) @> ARRAY[e]. e is drawn in 0..9 (the element range) so + * it is often already present -- containment holds either way. A push that + * loses the appended value fails here. + */ +PG_PROP(add_element) +{ + pg_prop_conn *conn; + StringInfoData a, + q; + int e; + + conn = connect_intarray(tc); + + initStringInfo(&a); + draw_intarr(tc, &a); + e = (int) pg_prop_draw_int(tc, 0, 9); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT (%s + %d) @> ARRAY[%d]", + a.data, e, e); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The intarray difference A - e removes ALL occurrences of scalar e, so the + * result must not contain e: NOT ((A - e) @> ARRAY[e]). e is drawn in 0..9 so + * it is often present in A, making the removal non-trivial. A leftover copy + * of e fails here. + */ +PG_PROP(remove_element) +{ + pg_prop_conn *conn; + StringInfoData a, + q; + int e; + + conn = connect_intarray(tc); + + initStringInfo(&a); + draw_intarr(tc, &a); + e = (int) pg_prop_draw_int(tc, 0, 9); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT NOT ((%s - %d) @> ARRAY[%d])", + a.data, e, e); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * After pushing e, idx must locate it: idx(A + e, e) > 0. idx returns the + * 1-based position of e, or 0 if absent; since A + e just appended e, idx must + * report a positive position. e is drawn in 0..9. A push/idx disagreement + * (element present but idx = 0) fails here. + */ +PG_PROP(idx_finds) +{ + pg_prop_conn *conn; + StringInfoData a, + q; + int e; + + conn = connect_intarray(tc); + + initStringInfo(&a); + draw_intarr(tc, &a); + e = (int) pg_prop_draw_int(tc, 0, 9); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT idx(%s + %d, %d) > 0", + a.data, e, e); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * sort(A) is non-decreasing: for the sorted array s, s[i] <= s[i+1] for every + * adjacent pair. bool_and over generate_subscripts(s,1) checks each pair; + * coalesce(..., true) makes the empty/single-element case (no adjacent pair) + * pass. An out-of-order element in sort's output fails here. + */ +PG_PROP(sort_ordered) +{ + pg_prop_conn *conn; + StringInfoData a, + q; + + conn = connect_intarray(tc); + + initStringInfo(&a); + draw_intarr(tc, &a); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce((SELECT bool_and(s[i] <= s[i+1]) " + "FROM generate_subscripts(s, 1) i " + "WHERE i < icount(s)), true) " + "FROM (SELECT sort(%s) AS s) q", + a.data); + + require_true(tc, conn, q.data); + + pfree(a.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index b5011d227d880..cfa08e7bf1026 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From dcf4be0f16c40462ee610d7ac4db2ae64fa3be54 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:31 -0400 Subject: [PATCH 249/338] prop: add pgtrgm property tests --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_pgtrgm.c | 382 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 386 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_pgtrgm.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 0a90fc918c487..dd15c1f647482 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -264,7 +264,8 @@ OBJS = \ prop_cubedist.o \ prop_seg.o \ prop_intarray.o \ - prop_intarrayext.o + prop_intarrayext.o \ + prop_pgtrgm.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 82af967698dfe..66ab6236dbcf6 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -262,6 +262,7 @@ prop_area_sources = files( 'prop_seg.c', 'prop_intarray.c', 'prop_intarrayext.c', + 'prop_pgtrgm.c', ) diff --git a/src/test/prop/prop_pgtrgm.c b/src/test/prop/prop_pgtrgm.c new file mode 100644 index 0000000000000..f5206ad994f26 --- /dev/null +++ b/src/test/prop/prop_pgtrgm.c @@ -0,0 +1,382 @@ +/*------------------------------------------------------------------------- + * + * prop_pgtrgm.c + * Property-based tests for the pg_trgm trigram-similarity extension. + * + * pg_trgm computes trigram sets and a similarity in [0,1]; these check the + * similarity bounds, symmetry, self-similarity, and consistency with + * show_trgm, without reimplementing the trigram algorithm. + * + * The oracle is pg_trgm's own stated contract -- similarity ranges over the + * unit interval, is symmetric, is 1 for a string against itself when the + * string has any trigrams and 0 against the empty string, show_trgm's + * cardinality is bounded by the padded-word trigram count, and the <-> + * distance operator is 1 - similarity. None of this re-derives the trigram + * decomposition itself; it asserts the invariants the decomposition must + * satisfy, so a bug in trigram generation or similarity scoring shows up as a + * broken invariant. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_pgtrgm.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "pgtrgm" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabets never contain a quote, but do it defensively + * so the generated SQL is always well-formed. Mirrors prop_like.c. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..12 characters from "[a-z] plus space" into buf as a bare string + * (no quoting). The space is included so word boundaries occur inside the + * generated text -- pg_trgm pads each word, so multi-word strings exercise + * the inter-word padding and the full similarity path. + */ +static void +draw_word(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abcdefghijklmnopqrstuvwxyz "; + int n = (int) pg_prop_draw_int(tc, 0, 12); + int i; + + for (i = 0; i < n; i++) + appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 26)]); +} + +/* + * Draw a single lowercase [a-z] word of length 1..10 into buf (no spaces), + * for the show_trgm cardinality property, which needs a single padded word. + */ +static void +draw_single_word(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abcdefghijklmnopqrstuvwxyz"; + int n = (int) pg_prop_draw_int(tc, 1, 10); + int i; + + for (i = 0; i < n; i++) + appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 25)]); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the similarity computation errored or produced NULL on + * valid strings, which is itself a bug -- so either is a failure, reported + * with the SQL. Mirrors require_true in prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Connect to the test cluster and ensure the pg_trgm extension exists. Each + * property opens its own connection (the Tier-B pattern); CREATE EXTENSION IF + * NOT EXISTS makes the DDL idempotent across connections and backends. On + * any failure the property is failed with the message. + */ +static pg_prop_conn * +connect_trgm(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, + "CREATE EXTENSION IF NOT EXISTS pg_trgm", + &err)) + pg_prop_fail(tc, "setup failed (pg_trgm): %s", err ? err : ""); + + return conn; +} + +/* + * similarity is in the unit interval: for any strings S, T, + * similarity(S, T) BETWEEN 0 AND 1. A score outside [0,1] means the shared/ + * total trigram ratio is miscomputed. + */ +PG_PROP(similarity_in_unit_interval) +{ + pg_prop_conn *conn; + StringInfoData s, + t, + q; + + conn = connect_trgm(tc); + + initStringInfo(&s); + initStringInfo(&t); + draw_word(tc, &s); + draw_word(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT similarity("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ") BETWEEN 0 AND 1"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * similarity is symmetric: similarity(S, T) = similarity(T, S). The trigram + * intersection and union are symmetric, so the ratio must be too; an + * asymmetric result points at an ordering bug in the set operations. + */ +PG_PROP(similarity_symmetric) +{ + pg_prop_conn *conn; + StringInfoData s, + t, + q; + + conn = connect_trgm(tc); + + initStringInfo(&s); + initStringInfo(&t); + draw_word(tc, &s); + draw_word(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT similarity("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ") = similarity("); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A string with at least one trigram is maximally similar to itself. + * show_trgm(S) gives S's trigram array; when it is non-empty, S shares every + * trigram with itself so similarity(S, S) = 1. The CASE's ELSE handles a + * too-short/empty string, which has no trigrams and whose self-similarity is + * 0, so the property stays sound for every drawn S. + */ +PG_PROP(self_similarity_is_one_when_has_trigrams) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + + conn = connect_trgm(tc); + + initStringInfo(&s); + draw_word(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT CASE WHEN cardinality(show_trgm("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) > 0 THEN similarity("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = 1 ELSE true END"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * similarity to the empty string is 0: no string shares a trigram with "", + * which has none, so the ratio is exactly 0 (compared with = since it is + * exact, not a rounding artefact). + */ +PG_PROP(similar_to_empty_is_zero) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + + conn = connect_trgm(tc); + + initStringInfo(&s); + draw_word(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT similarity("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", '') = 0"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * show_trgm's cardinality is bounded by the padded-word trigram count. For a + * single lowercase word W of length L >= 1, pg_trgm pads with two leading + * spaces and one trailing, which yields L + 1 positional trigrams. pg_trgm + * de-duplicates, so a word with repeated trigrams produces fewer distinct + * ones; the exact count is therefore an UPPER bound. Asserting + * cardinality BETWEEN 1 AND length(W) + 1 is sound regardless of duplicates: + * at least one trigram (L >= 1) and never more than L + 1. + */ +PG_PROP(show_trgm_count_matches_word_trigram_formula) +{ + pg_prop_conn *conn; + StringInfoData w, + q; + + conn = connect_trgm(tc); + + initStringInfo(&w); + draw_single_word(tc, &w); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT cardinality(show_trgm("); + append_sql_literal(&q, w.data); + appendStringInfoString(&q, ")) BETWEEN 1 AND length("); + append_sql_literal(&q, w.data); + appendStringInfoString(&q, ") + 1"); + + require_true(tc, conn, q.data); + + pfree(w.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * word_similarity is also in the unit interval: word_similarity(S, T) + * BETWEEN 0 AND 1. (word_similarity >= similarity is not an invariant, so it + * is deliberately not asserted here; only the range bound is.) + */ +PG_PROP(word_similarity_le_one) +{ + pg_prop_conn *conn; + StringInfoData s, + t, + q; + + conn = connect_trgm(tc); + + initStringInfo(&s); + initStringInfo(&t); + draw_word(tc, &s); + draw_word(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT word_similarity("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ") BETWEEN 0 AND 1"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The trigram distance operator <-> is 1 - similarity. Both are reals, so a + * small tolerance absorbs the float rounding of the two independent code + * paths that compute them: + * + * abs((S <-> T) - (1 - similarity(S, T))) < 1e-6 + */ +PG_PROP(distance_is_one_minus_similarity) +{ + pg_prop_conn *conn; + StringInfoData s, + t, + q; + + conn = connect_trgm(tc); + + initStringInfo(&s); + initStringInfo(&t); + draw_word(tc, &s); + draw_word(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT abs(("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " <-> "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ") - (1 - similarity("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, "))) < 1e-6"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index cfa08e7bf1026..f044839bac72d 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 1b5547d8c14bd1987fbd4e9b156ce65e3050a99a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:31 -0400 Subject: [PATCH 250/338] prop: add pg_trgm word similarity and show_trgm property tests (pgtrgm2) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_pgtrgm2.c | 423 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 427 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_pgtrgm2.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index dd15c1f647482..cb4e1be7f175a 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -265,7 +265,8 @@ OBJS = \ prop_seg.o \ prop_intarray.o \ prop_intarrayext.o \ - prop_pgtrgm.o + prop_pgtrgm.o \ + prop_pgtrgm2.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 66ab6236dbcf6..e2d023fa58644 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -263,6 +263,7 @@ prop_area_sources = files( 'prop_intarray.c', 'prop_intarrayext.c', 'prop_pgtrgm.c', + 'prop_pgtrgm2.c', ) diff --git a/src/test/prop/prop_pgtrgm2.c b/src/test/prop/prop_pgtrgm2.c new file mode 100644 index 0000000000000..96ea7e28186dc --- /dev/null +++ b/src/test/prop/prop_pgtrgm2.c @@ -0,0 +1,423 @@ +/*------------------------------------------------------------------------- + * + * prop_pgtrgm2.c + * Property-based tests for additional pg_trgm similarity functions. + * + * Complements prop_pgtrgm.c (which covers similarity bounds, symmetry, + * self-similarity, show_trgm cardinality and the <-> distance operator) by + * exercising a distinct surface: word_similarity, strict_word_similarity, + * show_trgm existence, and the % containment operator and its relationship to + * pg_trgm.similarity_threshold. + * + * The oracle is pg_trgm's own stated contract -- every similarity flavour + * ranges over [0,1], a non-empty string shows at least one trigram, the % + * operator is exactly "similarity >= threshold", and appending a character can + * never raise self-similarity above 1. None of this re-derives the trigram + * decomposition; it asserts the invariants it must satisfy, so a scoring or + * operator bug surfaces as a broken invariant. + * + * Tier B: runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_pgtrgm2.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "pgtrgm2" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabet never contains a quote, but do it defensively + * so the generated SQL is always well-formed. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + const char *p; + + appendStringInfoChar(buf, '\''); + for (p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..12 characters from "[a-c] plus space" into buf as a bare string + * (no quoting). The small alphabet forces frequent trigram collisions + * between S and T, so the similarity functions see non-trivial scores; the + * space introduces word boundaries so the word_similarity path is exercised. + */ +static void +draw_str(pg_prop_tc *tc, StringInfo buf) +{ + static const char letters[] = "abc "; + int n = (int) pg_prop_draw_int(tc, 0, 12); + int i; + + for (i = 0; i < n; i++) + appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 3)]); +} + +/* True if the bare string contains at least one non-space character. */ +static bool +has_nonspace(const char *s) +{ + const char *p; + + for (p = s; *p != '\0'; p++) + if (*p != ' ') + return true; + return false; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the similarity computation errored or produced NULL on + * valid strings, which is itself a bug -- so either is a failure, reported + * with the SQL. Mirrors require_true in prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + if (res == NULL) + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + if (strcmp(res, "t") != 0) + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Connect to the test cluster and ensure the pg_trgm extension exists. Each + * property opens its own connection (the Tier-B pattern); CREATE EXTENSION IF + * NOT EXISTS makes the DDL idempotent across the many per-case connections. + */ +static pg_prop_conn * +connect_trgm(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, + "CREATE EXTENSION IF NOT EXISTS pg_trgm", + &err)) + pg_prop_fail(tc, "setup failed (pg_trgm): %s", err ? err : ""); + + return conn; +} + +/* + * similarity_unit: similarity(S, T) BETWEEN 0 AND 1 for any strings S, T. + * A score outside the unit interval means the shared/total trigram ratio is + * miscomputed. + */ +PG_PROP(similarity_unit) +{ + pg_prop_conn *conn; + StringInfoData s, + t, + q; + + conn = connect_trgm(tc); + + initStringInfo(&s); + initStringInfo(&t); + draw_str(tc, &s); + draw_str(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT similarity("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ") BETWEEN 0 AND 1"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * similarity_self_one: for a string S with at least one trigram (any + * non-space character), similarity(S, S) = 1 -- S shares every trigram with + * itself. Guarded to non-empty-of-trigram S; a whitespace-only or empty S + * has no trigrams and self-similarity 0, so it is excluded (the property is + * trivially passed by skipping the assert). + */ +PG_PROP(similarity_self_one) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + + conn = connect_trgm(tc); + + initStringInfo(&s); + draw_str(tc, &s); + + if (has_nonspace(s.data)) + { + initStringInfo(&q); + appendStringInfoString(&q, "SELECT similarity("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = 1"); + + require_true(tc, conn, q.data); + pfree(q.data); + } + + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* + * word_similarity_le_one: both word_similarity(S, T) and + * strict_word_similarity(S, T) range over [0,1]. (word_similarity >= + * similarity is not an invariant, so only the range bound is asserted.) + */ +PG_PROP(word_similarity_le_one) +{ + pg_prop_conn *conn; + StringInfoData s, + t, + q; + + conn = connect_trgm(tc); + + initStringInfo(&s); + initStringInfo(&t); + draw_str(tc, &s); + draw_str(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT word_similarity("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ") BETWEEN 0 AND 1 AND strict_word_similarity("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ") BETWEEN 0 AND 1"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * show_trgm_count: show_trgm(S) returns the trigram array. Its cardinality + * is always >= 0, and for a string with any non-space character the padding + * guarantees at least one trigram, so cardinality > 0. Both bounds hold for + * every drawn S: the > 0 clause is only required when S has a non-space char. + */ +PG_PROP(show_trgm_count) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + + conn = connect_trgm(tc); + + initStringInfo(&s); + draw_str(tc, &s); + + initStringInfo(&q); + if (has_nonspace(s.data)) + { + appendStringInfoString(&q, "SELECT cardinality(show_trgm("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) > 0"); + } + else + { + appendStringInfoString(&q, "SELECT cardinality(show_trgm("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")) >= 0"); + } + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * similarity_symmetric: similarity(S, T) = similarity(T, S). The trigram + * intersection and union are symmetric, so the ratio must be too; an + * asymmetric result points at an ordering bug in the set operations. + */ +PG_PROP(similarity_symmetric) +{ + pg_prop_conn *conn; + StringInfoData s, + t, + q; + + conn = connect_trgm(tc); + + initStringInfo(&s); + initStringInfo(&t); + draw_str(tc, &s); + draw_str(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT similarity("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ") = similarity("); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * percent_operator_threshold: the % operator is exactly the thresholded + * similarity. (S % T) must equal (similarity(S, T) >= the current + * pg_trgm.similarity_threshold GUC). Reading the threshold via + * current_setting keeps the oracle correct whatever the session default is. + */ +PG_PROP(percent_operator_threshold) +{ + pg_prop_conn *conn; + StringInfoData s, + t, + q; + + conn = connect_trgm(tc); + + initStringInfo(&s); + initStringInfo(&t); + draw_str(tc, &s); + draw_str(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " % "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ") = (similarity("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, + ") >= current_setting('pg_trgm.similarity_threshold')::float4)"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * identical_percent: for a string S with any non-space character, + * similarity(S, S) = 1, which is >= any valid threshold (the GUC is in + * (0,1]), so S % S is true. Guarded to a trigram-bearing S; a whitespace or + * empty S has self-similarity 0 and may fall below threshold. + */ +PG_PROP(identical_percent) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + + conn = connect_trgm(tc); + + initStringInfo(&s); + draw_str(tc, &s); + + if (has_nonspace(s.data)) + { + initStringInfo(&q); + appendStringInfoString(&q, "SELECT ("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " % "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + pfree(q.data); + } + + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* + * similarity_bounded_by_one_on_extend: appending a character to S can never + * push similarity above its ceiling of 1. For a trigram-bearing S, + * similarity(S, S || 'x') is still in [0,1] -- extending one argument cannot + * exceed full similarity. Guarded to non-space S so a trigram set exists. + */ +PG_PROP(similarity_bounded_by_one_on_extend) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + + conn = connect_trgm(tc); + + initStringInfo(&s); + draw_str(tc, &s); + + if (has_nonspace(s.data)) + { + initStringInfo(&q); + appendStringInfoString(&q, "SELECT similarity("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " || 'x') <= 1"); + + require_true(tc, conn, q.data); + pfree(q.data); + } + + pfree(s.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index f044839bac72d..120c188385e8b 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 76954a58286be90ddfcfbfd8e42f25b0ac5db1a5 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:31 -0400 Subject: [PATCH 251/338] prop: add fuzzystrmatch levenshtein metric axioms property tests (fuzzystr) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_fuzzystr.c | 372 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 376 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_fuzzystr.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index cb4e1be7f175a..f685715a845dd 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -266,7 +266,8 @@ OBJS = \ prop_intarray.o \ prop_intarrayext.o \ prop_pgtrgm.o \ - prop_pgtrgm2.o + prop_pgtrgm2.o \ + prop_fuzzystr.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index e2d023fa58644..07ba3d847e499 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -264,6 +264,7 @@ prop_area_sources = files( 'prop_intarrayext.c', 'prop_pgtrgm.c', 'prop_pgtrgm2.c', + 'prop_fuzzystr.c', ) diff --git a/src/test/prop/prop_fuzzystr.c b/src/test/prop/prop_fuzzystr.c new file mode 100644 index 0000000000000..581f74e2f2ccd --- /dev/null +++ b/src/test/prop/prop_fuzzystr.c @@ -0,0 +1,372 @@ +/*------------------------------------------------------------------------- + * + * prop_fuzzystr.c + * Property-based tests for the fuzzystrmatch contrib extension. + * + * fuzzystrmatch's levenshtein is a metric with well-known bounds; these check + * the metric axioms and the identity/bound relationships. + * + * levenshtein(text, text) -> int is a proper metric: non-negative, symmetric, + * zero iff equal, and it satisfies the triangle inequality. It is also + * bounded below by the length difference (an edit changes length by at most + * one) and above by the longer operand's length (every character substituted). + * These properties check each of those relationships against SQL as the + * oracle; a "f" result is a bug in the metric, not a tolerance artefact. + * + * soundex/metaphone/dmetaphone also exist, but their outputs are opaque, so + * only soundex determinism (property 7) is checked; the metric properties are + * the high-value ones. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_fuzzystr.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "fuzzystr" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Connect to the test cluster and ensure the fuzzystrmatch extension exists. + * Each property opens its own connection (the Tier-B pattern); CREATE + * EXTENSION IF NOT EXISTS makes the DDL idempotent against a reused backend. + * On any failure the property is failed with the message. + */ +static pg_prop_conn * +connect_fuzzy(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, + "CREATE EXTENSION IF NOT EXISTS fuzzystrmatch", + &err)) + pg_prop_fail(tc, "setup failed (fuzzystrmatch): %s", err ? err : ""); + + return conn; +} + +/* + * Draw 0..10 characters from the alphabet [a-z] into buf as a bare string (no + * quoting). The empty string is allowed: levenshtein('', '') = 0, which is + * within every bound checked here. Letters only, so no quoting-critical + * character is ever generated. + */ +static void +draw_word(pg_prop_tc *tc, StringInfo buf) +{ + int n = (int) pg_prop_draw_int(tc, 0, 10); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, (char) ('a' + pg_prop_draw_int(tc, 0, 25))); +} + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our alphabet never contains a quote, but do it defensively + * so the generated SQL is always well-formed. Mirrors prop_like.c. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the function errored or produced NULL on valid inputs, + * which is itself a bug -- so either is a failure, reported with the SQL. + * Mirrors require_true in prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Self distance is zero: levenshtein(S, S) = 0. The identity axiom of the + * metric -- a string is at distance zero from itself. + */ +PG_PROP(levenshtein_zero_iff_equal) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + + conn = connect_fuzzy(tc); + + initStringInfo(&s); + draw_word(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT levenshtein("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = 0"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Symmetry: levenshtein(S, T) = levenshtein(T, S). The distance does not + * depend on which argument comes first. + */ +PG_PROP(levenshtein_symmetric) +{ + pg_prop_conn *conn; + StringInfoData s, + t, + q; + + conn = connect_fuzzy(tc); + + initStringInfo(&s); + initStringInfo(&t); + draw_word(tc, &s); + draw_word(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT levenshtein("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ") = levenshtein("); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Non-negativity and upper bound: 0 <= levenshtein(S, T) <= max(len(S), + * len(T)). The distance is never negative, and it never exceeds the longer + * operand's length because the worst case substitutes every character. + */ +PG_PROP(levenshtein_nonneg_and_bounded) +{ + pg_prop_conn *conn; + StringInfoData s, + t, + q; + + conn = connect_fuzzy(tc); + + initStringInfo(&s); + initStringInfo(&t); + draw_word(tc, &s); + draw_word(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT levenshtein("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ") BETWEEN 0 AND greatest(length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "), length("); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, "))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Triangle inequality: levenshtein(S, U) <= levenshtein(S, T) + + * levenshtein(T, U). The defining axiom that makes levenshtein a metric; a + * path through T can never be shorter than the direct distance. + */ +PG_PROP(levenshtein_triangle_inequality) +{ + pg_prop_conn *conn; + StringInfoData s, + t, + u, + q; + + conn = connect_fuzzy(tc); + + initStringInfo(&s); + initStringInfo(&t); + initStringInfo(&u); + draw_word(tc, &s); + draw_word(tc, &t); + draw_word(tc, &u); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT levenshtein("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, u.data); + appendStringInfoString(&q, ") <= levenshtein("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ") + levenshtein("); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, u.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(u.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Length lower bound: levenshtein(S, T) >= abs(len(S) - len(T)). A single + * edit changes length by at most one, so the distance cannot be smaller than + * the length difference. + */ +PG_PROP(levenshtein_length_lower_bound) +{ + pg_prop_conn *conn; + StringInfoData s, + t, + q; + + conn = connect_fuzzy(tc); + + initStringInfo(&s); + initStringInfo(&t); + draw_word(tc, &s); + draw_word(tc, &t); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT levenshtein("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, ") >= abs(length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") - length("); + append_sql_literal(&q, t.data); + appendStringInfoString(&q, "))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(t.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Appending one character costs at most one edit: levenshtein(S, S || 'z') <= + * 1. Turning S into S with one extra trailing character is a single + * insertion, so the distance is 0 or 1. + */ +PG_PROP(levenshtein_append_char_at_most_one) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + + conn = connect_fuzzy(tc); + + initStringInfo(&s); + draw_word(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT levenshtein("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", "); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, " || 'z') <= 1"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * soundex is deterministic: the same input always yields the same code. + * soundex of an empty string may be '' which still equals itself, so an empty + * word is harmless here. + */ +PG_PROP(soundex_deterministic) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + + conn = connect_fuzzy(tc); + + initStringInfo(&s); + draw_word(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT soundex("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = soundex("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ")"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 120c188385e8b..ff3c6c0d48cb0 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From df91634160ece30cb3ffbb62408e65d1497d2fcb Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:31 -0400 Subject: [PATCH 252/338] prop: add unaccent diacritic stripping property tests (unaccent) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_unaccent.c | 359 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 363 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_unaccent.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index f685715a845dd..48ed8b63bc761 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -267,7 +267,8 @@ OBJS = \ prop_intarrayext.o \ prop_pgtrgm.o \ prop_pgtrgm2.o \ - prop_fuzzystr.o + prop_fuzzystr.o \ + prop_unaccent.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 07ba3d847e499..0730c2d3360f9 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -265,6 +265,7 @@ prop_area_sources = files( 'prop_pgtrgm.c', 'prop_pgtrgm2.c', 'prop_fuzzystr.c', + 'prop_unaccent.c', ) diff --git a/src/test/prop/prop_unaccent.c b/src/test/prop/prop_unaccent.c new file mode 100644 index 0000000000000..13a0481952cf7 --- /dev/null +++ b/src/test/prop/prop_unaccent.c @@ -0,0 +1,359 @@ +/*------------------------------------------------------------------------- + * + * prop_unaccent.c + * Property-based tests for the unaccent contrib extension. + * + * unaccent() strips diacritics from text using the default unaccent.rules; + * these check idempotence, the ASCII no-op property, and length relationships + * that hold because stripping a simple Latin diacritic is a 1:1 map to a + * single-byte ASCII letter. + * + * The oracle is unaccent's own stability plus independent builtins + * (length, octet_length, char_length, starts_with, left, ||): a pure-ASCII + * input is left untouched, the Latin accented pool folds to pure ASCII (so + * char length equals octet length and, for 1:1 diacritics, equals the input + * char length), and unaccent distributes over concatenation because it is a + * per-character map. + * + * NOTE: this source file is strictly ASCII. The accented inputs are built in + * SQL using PostgreSQL's U&'...' escape-string syntax with \XXXX code-point + * escapes (e.g. U&'caf\00E9' is cafe-with-acute-e), so no non-ASCII byte ever + * appears in the C source. Each U&'...' token is already a quoted SQL string + * literal and is substituted directly into the generated SQL. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_unaccent.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "unaccent" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Curated pool of accented strings whose diacritics are simple 1:1 Latin + * marks (acute, grave, umlaut, circumflex, tilde, cedilla). Each accented + * letter folds to exactly one single-byte ASCII letter, so unaccent preserves + * character length and yields pure ASCII. Ligatures (Oe U+0152, ss U+00DF, + * Ae U+00C6) are deliberately excluded because they may expand to two + * characters and would break the length-preservation properties. + * + * Every entry is written with ASCII-only U&'...' Unicode escapes so the C + * source contains no byte >= 0x80. The doubled backslashes in the C literal + * yield single backslashes in the SQL text, so each entry is a valid U&'...' + * string constant. Decoded (for the reader only, not present in the file): + * U&'caf\00E9' -> cafe + acute e (U+00E9) + * U&'na\00EFve' -> na + diaeresis i + ve (U+00EF) + * U&'Z\00FCrich' -> Z + umlaut u + rich (U+00FC) + * U&'\00E0\00E9\00EE\00F5\00FC' -> a-grave e-acute i-circ o-tilde u-umlaut + * U&'\00D1and\00FA' -> N-tilde + and + u-acute (U+00D1, U+00FA) + * U&'r\00E9sum\00E9' -> r + acute e + sum + acute e + * U&'jalape\00F1o' -> jalape + n-tilde + o (U+00F1) + * U&'M\00FCller' -> M + umlaut u + ller (U+00FC) + */ +static const char *const accented_pool[] = { + "U&'caf\\00E9'", + "U&'na\\00EFve'", + "U&'Z\\00FCrich'", + "U&'\\00E0\\00E9\\00EE\\00F5\\00FC'", + "U&'\\00D1and\\00FA'", + "U&'r\\00E9sum\\00E9'", + "U&'jalape\\00F1o'", + "U&'M\\00FCller'" +}; + +#define ACCENTED_POOL_LEN (sizeof(accented_pool) / sizeof(accented_pool[0])) + +/* + * Append a bare string as a single-quoted SQL literal, doubling any embedded + * single quote. Our inputs never contain a quote, but double defensively so + * the generated SQL is always well-formed. Mirrors prop_like.c. + */ +static void +append_sql_literal(StringInfo buf, const char *s) +{ + appendStringInfoChar(buf, '\''); + for (const char *p = s; *p != '\0'; p++) + { + if (*p == '\'') + appendStringInfoChar(buf, '\''); + appendStringInfoChar(buf, *p); + } + appendStringInfoChar(buf, '\''); +} + +/* + * Draw 0..12 characters of pure ASCII text from "[a-zA-Z0-9 ]" into buf as a + * bare string (no quoting, no diacritics). Used for the no-op properties + * where unaccent must be the identity. + */ +static void +draw_ascii(pg_prop_tc *tc, StringInfo buf) +{ + static const char alphabet[] = + "abcdefghijklmnopqrstuvwxyz" + "ABCDEFGHIJKLMNOPQRSTUVWXYZ" + "0123456789 "; + int n = (int) pg_prop_draw_int(tc, 0, 12); + + for (int i = 0; i < n; i++) + appendStringInfoChar(buf, + alphabet[pg_prop_draw_int(tc, 0, + sizeof(alphabet) - 2)]); +} + +/* + * Return one entry of the curated accented pool. The engine draws the index, + * so it explores every pool entry (and favours the boundaries). + */ +static const char * +draw_accented(pg_prop_tc *tc) +{ + int i = (int) pg_prop_draw_int(tc, 0, (int) ACCENTED_POOL_LEN - 1); + + return accented_pool[i]; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means unaccent errored or produced NULL on valid text, which is + * itself a bug -- so either is a failure, reported with the SQL. Mirrors + * require_true in prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Connect to the test cluster and ensure the unaccent extension exists. Each + * property opens its own connection (the Tier-B pattern); CREATE EXTENSION IF + * NOT EXISTS makes it idempotent against a reused backend. Mirrors the + * connect-and-setup helper in prop_record.c. + */ +static pg_prop_conn * +connect_unaccent(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, + "CREATE EXTENSION IF NOT EXISTS unaccent", + &err)) + pg_prop_fail(tc, "setup failed (CREATE EXTENSION unaccent): %s", + err ? err : ""); + + return conn; +} + +/* + * unaccent is idempotent: unaccent(unaccent(S)) = unaccent(S). Once the + * diacritics are stripped, re-stripping changes nothing. Uses accented input + * so the first pass actually removes something. + */ +PG_PROP(idempotent) +{ + pg_prop_conn *conn; + const char *a; + StringInfoData q; + + conn = connect_unaccent(tc); + a = draw_accented(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT unaccent(unaccent(%s)) = unaccent(%s)", + a, a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * For a pure-ASCII string S (no diacritics), unaccent is the identity: + * unaccent(S) = S. S is drawn from "[a-zA-Z0-9 ]", which has nothing for + * unaccent to strip, so the result must be byte-for-byte S. + */ +PG_PROP(ascii_is_noop) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + + conn = connect_unaccent(tc); + + initStringInfo(&s); + draw_ascii(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT unaccent("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ") = "); + append_sql_literal(&q, s.data); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * For the Latin accented pool, unaccent produces pure ASCII, so the result's + * character length equals its octet length (one byte per char): + * + * length(unaccent(A)) = octet_length(unaccent(A)) + * + * This guards that the accents were actually removed down to single-byte + * ASCII characters -- if any multibyte char survived, octet_length would + * exceed length. + */ +PG_PROP(result_is_ascii_for_latin_accents) +{ + pg_prop_conn *conn; + const char *a; + StringInfoData q; + + conn = connect_unaccent(tc); + a = draw_accented(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT length(unaccent(%s)) = octet_length(unaccent(%s))", + a, a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * For the curated 1:1 Latin pool, unaccent maps each accented letter to + * exactly one ASCII letter, so the character length is unchanged: + * + * char_length(unaccent(A)) = char_length(A) + * + * The pool deliberately excludes ligatures (Oe, ss, Ae) that may expand to + * two characters, which would break this equality. + */ +PG_PROP(length_preserved_for_simple_accents) +{ + pg_prop_conn *conn; + const char *a; + StringInfoData q; + + conn = connect_unaccent(tc); + a = draw_accented(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT char_length(unaccent(%s)) = char_length(%s)", + a, a); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * unaccent distributes over concatenation of two accented strings: + * + * unaccent(A || B) = unaccent(A) || unaccent(B) + * + * unaccent is a per-character map, so stripping the concatenation equals + * concatenating the stripped halves. + */ +PG_PROP(concat_distributes) +{ + pg_prop_conn *conn; + const char *a; + const char *b; + StringInfoData q; + + conn = connect_unaccent(tc); + a = draw_accented(tc); + b = draw_accented(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT unaccent(%s || %s) = (unaccent(%s) || unaccent(%s))", + a, b, a, b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * For a pure-ASCII S, unaccent is the identity, so a prefix of unaccent(S) + * still prefixes S: + * + * starts_with(unaccent(S), left(S, least(3, length(S)))) + * + * This holds trivially because unaccent(S) = S for ASCII input; it is a light + * cross-check that the no-op path leaves the leading characters in place. + */ +PG_PROP(ascii_prefix_preserved) +{ + pg_prop_conn *conn; + StringInfoData s, + q; + + conn = connect_unaccent(tc); + + initStringInfo(&s); + draw_ascii(tc, &s); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT starts_with(unaccent("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "), left("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, ", least(3, length("); + append_sql_literal(&q, s.data); + appendStringInfoString(&q, "))))"); + + require_true(tc, conn, q.data); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index ff3c6c0d48cb0..7672057e08b28 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From f89e1f615c84f6f1323f248b3bf665be133b854a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:31 -0400 Subject: [PATCH 253/338] prop: add isn ISBN/EAN13 standard numbers property tests (isn) --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/prop_isn.c | 288 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 292 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/prop_isn.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 48ed8b63bc761..9c47ad7a9e944 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -268,7 +268,8 @@ OBJS = \ prop_pgtrgm.o \ prop_pgtrgm2.o \ prop_fuzzystr.o \ - prop_unaccent.o + prop_unaccent.o \ + prop_isn.o PG_CPPFLAGS = -I$(libpq_srcdir) PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 0730c2d3360f9..f5cfa75cb991d 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -266,6 +266,7 @@ prop_area_sources = files( 'prop_pgtrgm2.c', 'prop_fuzzystr.c', 'prop_unaccent.c', + 'prop_isn.c', ) diff --git a/src/test/prop/prop_isn.c b/src/test/prop/prop_isn.c new file mode 100644 index 0000000000000..a86e5bfb8a31f --- /dev/null +++ b/src/test/prop/prop_isn.c @@ -0,0 +1,288 @@ +/*------------------------------------------------------------------------- + * + * prop_isn.c + * Property-based tests for the isn contrib extension (ISBN/ISMN/ISSN/EAN13). + * + * The isn types (contrib/isn/isn.c) validate and canonicalize international + * standard numbers against their check digits, store them as a 13-digit EAN13 + * core, and render a canonical hyphenated form on output. These properties + * check that a valid number survives a text round-trip, that an ISBN survives + * the cast to ean13 and back through text, that hyphens are cosmetic, and that + * the ean13 comparator is a total order. All source literals are ASCII + * (digits, hyphens, and the ISBN-10 check character 'X') only. + * + * Generating a valid number from scratch requires computing a correct check + * digit, which is exactly the logic under test, so instead these draw from a + * small fixed pool of KNOWN-VALID literals taken verbatim from PostgreSQL's + * own isn regression test (contrib/isn/sql/isn.sql, "test valid conversions"). + * + * The oracle is the type's own definition: validate(canonicalize(x)) = x. A + * value accepted once must re-parse from its own canonical text, hyphens must + * not change the parsed value, and a comparison must be a total order. + * + * Tier B: this runs SQL against the cluster the TAP driver started; see + * pg_prop.h and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/prop_isn.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "isn" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Known-valid EAN13-castable literals, taken verbatim from the isn regression + * test's "test valid conversions" block (contrib/isn/sql/isn.sql). Each has a + * correct check digit; the regression test accepts every one as a plain EAN13, + * so property 3 (acceptance) must hold for all of them. Kept small and + * conservative -- a wrong entry would make property 3 fail spuriously. + */ +static const char *const valid_ean13[] = { + "9780123456786", /* old book */ + "9790123456785", /* music (ISMN) */ + "9791234567896", /* new book */ + "9771234567898", /* serial */ +}; + +/* + * Known-valid ISBN literals (castable to ean13), also from the regression + * test. "123456789X" exercises the ISBN-10 'X' check character; the others + * are 13-digit book EANs that the ISBN type also accepts. + */ +static const char *const valid_isbn[] = { + "9780123456786", + "123456789X", + "9791234567896", +}; + +/* + * Connect to the test cluster and ensure the isn extension exists. Each + * property opens its own connection (the Tier-B pattern); CREATE EXTENSION IF + * NOT EXISTS makes the DDL idempotent across the thousands of per-case + * connections in a run. On any failure the property is failed. + */ +static pg_prop_conn * +connect_isn(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, "CREATE EXTENSION IF NOT EXISTS isn", &err)) + pg_prop_fail(tc, "setup failed (CREATE EXTENSION isn): %s", + err ? err : ""); + + return conn; +} + +/* + * Run a boolean-valued SELECT and require it to return "t". A NULL result or + * a SQL error means the isn I/O or comparison errored or produced NULL on a + * value we expect to be valid, which is itself a bug. Mirrors require_true in + * prop_numeric.c. + */ +static void +require_true(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *res; + + res = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + { + pg_prop_fail(tc, "SQL error for: %s\n error: %s", + sql, err ? err : "(unknown)"); + } + if (res == NULL) + { + pg_prop_fail(tc, "unexpected NULL result for: %s", sql); + } + if (strcmp(res, "t") != 0) + { + pg_prop_fail(tc, "invariant failed (got %s) for: %s", res, sql); + } + pg_free(res); + if (err) + pg_free(err); +} + +/* + * Draw one literal from the valid-EAN13 pool. Returned pointer is into static + * read-only storage; do not free. + */ +static const char * +draw_valid_ean13(pg_prop_tc *tc) +{ + int i = (int) pg_prop_draw_int(tc, 0, lengthof(valid_ean13) - 1); + + return valid_ean13[i]; +} + +/* + * Draw one literal from the valid-ISBN pool. Returned pointer is into static + * read-only storage; do not free. + */ +static const char * +draw_valid_isbn(pg_prop_tc *tc) +{ + int i = (int) pg_prop_draw_int(tc, 0, lengthof(valid_isbn) - 1); + + return valid_isbn[i]; +} + +/* + * 1. ean13_text_roundtrip + * + * A valid ean13 value cast to its canonical text and parsed back equals the + * original: ('V'::ean13)::text::ean13 = 'V'::ean13. This exercises the output + * canonicalization and the input validation on known-good data; if ean13_out + * emits text that ean13_in cannot re-parse to the same value, this fails. + */ +PG_PROP(ean13_text_roundtrip) +{ + pg_prop_conn *conn; + const char *v; + StringInfoData q; + + conn = connect_isn(tc); + v = draw_valid_ean13(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('%s'::ean13)::text::ean13 = '%s'::ean13", + v, v); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 2. isbn_to_ean13_roundtrip + * + * A valid ISBN cast to ean13 survives a text round-trip: + * ('V'::isbn::ean13)::text::ean13 = 'V'::isbn::ean13. This exercises the + * isbn->ean13 cast together with ean13's own text round-trip; a bug in the + * cross-type cast or in canonicalization of the resulting ean13 fails here. + */ +PG_PROP(isbn_to_ean13_roundtrip) +{ + pg_prop_conn *conn; + const char *v; + StringInfoData q; + + conn = connect_isn(tc); + v = draw_valid_isbn(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('%s'::isbn::ean13)::text::ean13 " + "= '%s'::isbn::ean13", + v, v); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 3. valid_number_accepted + * + * Every pool value is accepted by the ean13 type and casts to a non-NULL + * value: ('V'::ean13) IS NOT NULL. A validator bug that wrongly rejected a + * genuinely valid number (or produced NULL) would surface here. Pool values + * are taken from the regression test, so each must be accepted. + */ +PG_PROP(valid_number_accepted) +{ + pg_prop_conn *conn; + const char *v; + StringInfoData q; + + conn = connect_isn(tc); + v = draw_valid_ean13(tc); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT ('%s'::ean13) IS NOT NULL", v); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 4. hyphens_are_cosmetic + * + * ISN input treats hyphens as formatting only, so a pool value and the same + * value with every hyphen stripped parse to the same ean13. This exercises + * the parser's hyphen handling without needing to synthesise valid check + * digits, so no input is filtered out. + */ +PG_PROP(hyphens_are_cosmetic) +{ + pg_prop_conn *conn; + const char *v; + StringInfoData q; + + conn = connect_isn(tc); + v = draw_valid_ean13(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT ('%s'::ean13) = (replace('%s', '-', '')::ean13)", + v, v); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * 5. ean13_comparison_total_order + * + * For any two valid pool values A, B, exactly one of <, =, > holds, so the + * three boolean-to-int results sum to 1: + * + * (AB)::int = 1 + * + * A missing or overlapping case in the ean13 comparator (a value that is + * neither less, equal, nor greater, or two firing at once) breaks this. + */ +PG_PROP(ean13_comparison_total_order) +{ + pg_prop_conn *conn; + const char *a; + const char *b; + StringInfoData q; + + conn = connect_isn(tc); + a = draw_valid_ean13(tc); + b = draw_valid_ean13(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (('%s'::ean13 < '%s'::ean13)::int " + "+ ('%s'::ean13 = '%s'::ean13)::int " + "+ ('%s'::ean13 > '%s'::ean13)::int) = 1", + a, b, a, b, a, b); + + require_true(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 7672057e08b28..d275599272f51 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From e470467f7244f4073ab4ba6844a75775aae1a33a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 12 Oct 2026 21:43:32 -0400 Subject: [PATCH 254/338] prop: add SQL query-generation areas (TLP, NoREC, plan advice) Add src/test/prop/sql/ with a shared random-query generator (query_gen.c) and three metamorphic query-level areas that test the planner and executor rather than individual functions: query_tlp (Ternary Logic Partitioning), query_norec (non-optimizing reference), and query_planadvice (the same query run with the planner free and steered by contrib/pg_plan_advice, plus a tiny-work_mem spill variant, must return identical rows). These shrink any counterexample to a minimal query via the same engine as the rest of the suite. --- src/test/prop/Makefile | 8 +- src/test/prop/meson.build | 14 +- src/test/prop/sql/prop_query_norec.c | 207 +++++++++++++++++ src/test/prop/sql/prop_query_planadvice.c | 264 ++++++++++++++++++++++ src/test/prop/sql/prop_query_tlp.c | 233 +++++++++++++++++++ src/test/prop/sql/query_gen.c | 234 +++++++++++++++++++ src/test/prop/sql/query_gen.h | 77 +++++++ src/test/prop/t/002_sql_props.pl | 2 +- 8 files changed, 1035 insertions(+), 4 deletions(-) create mode 100644 src/test/prop/sql/prop_query_norec.c create mode 100644 src/test/prop/sql/prop_query_planadvice.c create mode 100644 src/test/prop/sql/prop_query_tlp.c create mode 100644 src/test/prop/sql/query_gen.c create mode 100644 src/test/prop/sql/query_gen.h diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 9c47ad7a9e944..14e05aafad401 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -269,9 +269,13 @@ OBJS = \ prop_pgtrgm2.o \ prop_fuzzystr.o \ prop_unaccent.o \ - prop_isn.o + prop_isn.o \ + sql/query_gen.o \ + sql/prop_query_tlp.o \ + sql/prop_query_planadvice.o \ + sql/prop_query_norec.o -PG_CPPFLAGS = -I$(libpq_srcdir) +PG_CPPFLAGS = -I$(libpq_srcdir) -I$(srcdir)/sql PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) # libhegel's C engine. Its include/lib dirs reach us through --with-includes # / --with-libraries (CPPFLAGS / LDFLAGS); here we just name the library. diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index f5cfa75cb991d..040edef9c66f3 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -270,8 +270,20 @@ prop_area_sources = files( ) +# Query-level areas live in the sql/ subdirectory and share query_gen.c, +# which generates a random table and predicate. They compile into the same +# pg_prop binary and self-register the same way; t/002_sql_props.pl runs them +# against the node it starts. +prop_query_sources = files( + 'sql/query_gen.c', + 'sql/prop_query_tlp.c', + 'sql/prop_query_planadvice.c', + 'sql/prop_query_norec.c', +) + pg_prop = executable('pg_prop', - files('pg_prop.c', 'pg_prop_sql.c') + prop_area_sources, + files('pg_prop.c', 'pg_prop_sql.c') + prop_area_sources + prop_query_sources, + include_directories: include_directories('.', 'sql'), dependencies: [frontend_code, libpq, libhegel], kwargs: default_bin_args + { 'install': false, diff --git a/src/test/prop/sql/prop_query_norec.c b/src/test/prop/sql/prop_query_norec.c new file mode 100644 index 0000000000000..2a811a9a1a036 --- /dev/null +++ b/src/test/prop/sql/prop_query_norec.c @@ -0,0 +1,207 @@ +/*------------------------------------------------------------------------- + * + * prop_query_norec.c + * Query-level property tests using NoREC (non-optimising reference). + * + * NoREC (Rigger & Su, ESEC/FSE 2020) finds optimisation bugs by comparing a + * query the planner can optimise against a semantically equivalent form it + * cannot. The classic transformation: the optimisable query + * + * SELECT count(*) FROM t WHERE p + * + * puts p where the planner may use an index, reorder, or push it down. The + * un-optimisable twin pulls p into the target list of an unrestricted scan, + * where the planner must evaluate it per row with no shortcut: + * + * SELECT count(*) FROM (SELECT (p) IS TRUE AS t FROM t) s WHERE s.t + * + * Both count the rows for which p is TRUE, so the counts must match. If they + * differ, the optimisation applied to the first form changed the answer -- a + * bug in predicate pushdown, index-qual translation, or scan filtering. + * + * Because both forms are evaluated by the same server, NoREC needs no external + * reference database, and because the twin is deliberately opaque to the + * optimiser, a mismatch points straight at the optimiser. + * + * Tier B: runs SQL against the cluster the TAP driver started; see pg_prop.h + * and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_query_norec.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "query_norec" +#include "pg_prop.h" + +#include "lib/stringinfo.h" +#include "query_gen.h" + +/* Require two scalar queries to return the same value. */ +static void +require_same_scalar(pg_prop_tc *tc, pg_prop_conn *conn, + const char *opt, const char *noropt) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL; + char *r1, + *r2; + + r1 = pg_prop_sql_scalar(conn, opt, &f1, &e1); + if (f1) + pg_prop_fail(tc, "optimisable query failed: %s\n SQL: %s", + e1 ? e1 : "(unknown)", opt); + r2 = pg_prop_sql_scalar(conn, noropt, &f2, &e2); + if (f2) + pg_prop_fail(tc, "non-optimisable query failed: %s\n SQL: %s", + e2 ? e2 : "(unknown)", noropt); + + if (r1 == NULL || r2 == NULL || strcmp(r1, r2) != 0) + pg_prop_fail(tc, + "NoREC mismatch (%s vs %s):\n optimisable: %s\n" + " non-optimisable: %s", + r1 ? r1 : "NULL", r2 ? r2 : "NULL", opt, noropt); + + if (r1) + pg_free(r1); + if (r2) + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Core NoREC: count(*) WHERE p vs count of (p IS TRUE) over a bare scan. + */ +PG_PROP(count_matches_norec) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData pred; + StringInfoData opt; + StringInfoData noropt; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + initStringInfo(&opt); + appendStringInfo(&opt, + "SELECT count(*) FROM " QG_TABLE " WHERE (%s)", + pred.data); + + initStringInfo(&noropt); + appendStringInfo(&noropt, + "SELECT count(*) FROM " + "(SELECT (%s) IS TRUE AS keep FROM " QG_TABLE ") s " + "WHERE s.keep", + pred.data); + + require_same_scalar(tc, conn, opt.data, noropt.data); + + pfree(pred.data); + pfree(opt.data); + pfree(noropt.data); + pg_prop_disconnect(conn); +} + +/* + * NoREC with the predicate negated: the un-optimisable twin uses IS NOT TRUE + * to count the complement, which must equal count(*) WHERE NOT p plus the NULL + * rows. Simpler sound form: count WHERE NOT p == count of (p IS FALSE). + */ +PG_PROP(count_false_matches_norec) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData pred; + StringInfoData opt; + StringInfoData noropt; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + initStringInfo(&opt); + appendStringInfo(&opt, + "SELECT count(*) FROM " QG_TABLE " WHERE NOT (%s)", + pred.data); + + initStringInfo(&noropt); + appendStringInfo(&noropt, + "SELECT count(*) FROM " + "(SELECT (%s) IS FALSE AS keep FROM " QG_TABLE ") s " + "WHERE s.keep", + pred.data); + + require_same_scalar(tc, conn, opt.data, noropt.data); + + pfree(pred.data); + pfree(opt.data); + pfree(noropt.data); + pg_prop_disconnect(conn); +} + +/* + * NoREC over a sum: sum(a) WHERE p vs sum of a*(p IS TRUE)::int over a bare + * scan. Exercises aggregate pushdown interacting with the filter. coalesce + * so NULL a does not make the two forms disagree on NULL vs 0 (both treat a + * missing contribution as 0). + */ +PG_PROP(sum_matches_norec) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData pred; + StringInfoData opt; + StringInfoData noropt; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + initStringInfo(&opt); + appendStringInfo(&opt, + "SELECT coalesce(sum(coalesce(a,0)),0) FROM " QG_TABLE + " WHERE (%s)", + pred.data); + + initStringInfo(&noropt); + appendStringInfo(&noropt, + "SELECT coalesce(sum(CASE WHEN (%s) IS TRUE " + "THEN coalesce(a,0) ELSE 0 END),0) FROM " QG_TABLE, + pred.data); + + require_same_scalar(tc, conn, opt.data, noropt.data); + + pfree(pred.data); + pfree(opt.data); + pfree(noropt.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/sql/prop_query_planadvice.c b/src/test/prop/sql/prop_query_planadvice.c new file mode 100644 index 0000000000000..ba36fb0f68835 --- /dev/null +++ b/src/test/prop/sql/prop_query_planadvice.c @@ -0,0 +1,264 @@ +/*------------------------------------------------------------------------- + * + * prop_query_planadvice.c + * Query-level property tests by plan-advice metamorphic testing. + * + * A query's result must not depend on how the planner chose to execute it. + * The contrib/pg_plan_advice extension can steer specific planner decisions + * (which scan method, which join method, which join order) through the + * pg_plan_advice.advice GUC, so for a randomly generated query we run it + * twice -- once with the planner free, once steered toward a different plan + * shape -- and require the two result sets to be identical. + * + * Using plan advice rather than the blunt enable_* GUCs is deliberate: advice + * names the exact relation/join to re-shape, so the alternate plan is a + * targeted, legal variant of the same query rather than a global cost + * distortion, and the extension's own validation rejects advice it cannot + * honour (in which case the planner falls back to a valid plan that must still + * agree). + * + * This is a metamorphic oracle: there is no reference answer, only the + * invariant "the two runs agree". Any disagreement is a wrong-results bug in + * one of the execution paths (an index/bitmap scan that drops a row, a join + * method that mis-handles a NULL key, a spill-to-disk path that corrupts a + * batch). These are exactly the bugs hardest to find by hand, because the + * default plan is usually the well-tested one and the alternate paths fire + * only on specific data. + * + * The one non-plan-shape knob we still flip is work_mem: a tiny value forces + * the memory-bounded variants (external sort, hash/hashagg spill) that a small + * test table would never otherwise reach. That is a resource bound, not a + * plan-shape override, and plan advice has no equivalent for it. + * + * Tier B: runs SQL against the cluster the TAP driver started; see pg_prop.h + * and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_query_planadvice.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "query_planadvice" +#include "pg_prop.h" + +#include "lib/stringinfo.h" +#include "query_gen.h" + +/* + * Connect and make pg_plan_advice available for the session. LOAD makes the + * pg_plan_advice.advice GUC settable; the extension need not be installed as + * an object, only loaded into the backend. + */ +static pg_prop_conn * +connect_advice(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + if (!pg_prop_sql_exec(conn, "LOAD 'pg_plan_advice'", &err)) + pg_prop_fail(tc, "LOAD pg_plan_advice failed: %s", err ? err : ""); + + return conn; +} + +/* + * Run "sql" with the planner free, then again under the given plan advice (and + * an optional extra SET such as a tiny work_mem), and require identical rows. + * "advice" may be an empty string, in which case only the extra SET steers the + * second run. RESET ALL afterward so nothing leaks between cases. + */ +static void +require_plan_invariant(pg_prop_tc *tc, pg_prop_conn *conn, + const char *sql, const char *advice, + const char *extra_set) +{ + char *err = NULL; + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL; + char *r1, + *r2; + StringInfoData setup; + + /* Default plan. */ + if (!pg_prop_sql_exec(conn, "RESET ALL", &err)) + pg_prop_fail(tc, "RESET failed: %s", err ? err : ""); + r1 = pg_prop_sql_rows(conn, sql, &f1, &e1); + if (f1) + pg_prop_fail(tc, "default-plan query failed: %s\n SQL: %s", + e1 ? e1 : "(unknown)", sql); + + /* Steered plan. */ + initStringInfo(&setup); + if (advice[0] != '\0') + appendStringInfo(&setup, "SET pg_plan_advice.advice = '%s'; ", advice); + if (extra_set[0] != '\0') + appendStringInfoString(&setup, extra_set); + if (setup.len > 0 && !pg_prop_sql_exec(conn, setup.data, &err)) + pg_prop_fail(tc, "advice setup failed: %s\n setup: %s", + err ? err : "", setup.data); + pfree(setup.data); + + r2 = pg_prop_sql_rows(conn, sql, &f2, &e2); + if (f2) + pg_prop_fail(tc, "steered-plan query failed: %s\n SQL: %s", + e2 ? e2 : "(unknown)", sql); + + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, + "plan-dependent result for: %s\n" + " advice: %s\n" + " default plan rows:\n%s\n steered plan rows:\n%s", + sql, advice, r1, r2); + + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * A filtered projection, steered to a sequential scan of the generated table. + * The default plan may use any scan; forcing SEQ_SCAN must not change the + * rows. (If no index exists the advice is a no-op and the two runs trivially + * agree, which is still a valid check of the SEQ_SCAN path.) + */ +PG_PROP(filter_plan_invariant) +{ + pg_prop_conn *conn; + int nrows; + StringInfoData pred; + StringInfoData q; + + conn = connect_advice(tc); + qg_build_table(tc, conn, &nrows); + + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT id, a, b, c, f FROM " QG_TABLE + " WHERE (%s) ORDER BY id", + pred.data); + + require_plan_invariant(tc, conn, q.data, + "SEQ_SCAN(" QG_TABLE ")", ""); + + pfree(pred.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A grouped aggregate run under a tiny work_mem to force the hashagg spill + * (and external-sort) paths. Grouping is a plan decision plan advice does not + * cover, so the steering here is the work_mem bound; the result must not + * depend on whether aggregation spilled to disk. + */ +PG_PROP(groupagg_plan_invariant) +{ + pg_prop_conn *conn; + int nrows; + StringInfoData grp; + StringInfoData q; + + conn = connect_advice(tc); + qg_build_table(tc, conn, &nrows); + + initStringInfo(&grp); + qg_draw_int_expr(tc, &grp); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT g, count(*), sum(a), min(b), max(b), " + "count(DISTINCT c) FROM " + "(SELECT (%s) AS g, a, b, c FROM " QG_TABLE ") s " + "GROUP BY g ORDER BY g", + grp.data); + + require_plan_invariant(tc, conn, q.data, "", "SET work_mem = '64kB';"); + + pfree(grp.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A self-join steered to a nested-loop join. The default plan may hash or + * merge; forcing NESTED_LOOP_PLAIN on the inner relation must not change the + * rows. Join method is the richest source of result-dependent-on-plan bugs. + */ +PG_PROP(join_plan_invariant) +{ + pg_prop_conn *conn; + int nrows; + StringInfoData pred; + StringInfoData q; + + conn = connect_advice(tc); + qg_build_table(tc, conn, &nrows); + + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + /* + * The generated predicate uses unqualified column names, so it is applied + * inside a single-table subquery (unambiguous there) and the filtered + * relation is self-joined under the alias t; the inner relation u is the + * join-method advice target. + */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT t.id, u.id FROM " + "(SELECT * FROM " QG_TABLE " WHERE (%s)) t " + "JOIN " QG_TABLE + " u ON (t.a IS NOT DISTINCT FROM u.b) " + "ORDER BY t.id, u.id", + pred.data); + + require_plan_invariant(tc, conn, q.data, "NESTED_LOOP_PLAIN(u)", ""); + + pfree(pred.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * DISTINCT with ORDER BY ... LIMIT, run under a tiny work_mem to force the + * sort/limit and hash-distinct spill paths. ORDER BY the primary key keeps + * the top-N result deterministic so the oracle stays sound. + */ +PG_PROP(distinct_limit_plan_invariant) +{ + pg_prop_conn *conn; + int nrows; + StringInfoData q; + int lim; + + conn = connect_advice(tc); + qg_build_table(tc, conn, &nrows); + lim = (int) pg_prop_draw_int(tc, 0, QG_MAX_ROWS); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT DISTINCT id, a FROM " QG_TABLE + " ORDER BY id, a LIMIT %d", + lim); + + require_plan_invariant(tc, conn, q.data, "", "SET work_mem = '64kB';"); + + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/sql/prop_query_tlp.c b/src/test/prop/sql/prop_query_tlp.c new file mode 100644 index 0000000000000..45ff7702fef72 --- /dev/null +++ b/src/test/prop/sql/prop_query_tlp.c @@ -0,0 +1,233 @@ +/*------------------------------------------------------------------------- + * + * prop_query_tlp.c + * Query-level property tests using Ternary Logic Partitioning (TLP). + * + * TLP (Rigger & Su, OOPSLA 2020) is a metamorphic oracle for finding + * wrong-results bugs in a query processor without an independent reference. + * The idea: for any predicate p and any row, SQL three-valued logic puts the + * row in exactly one of three buckets -- p is TRUE, p is FALSE, or p is NULL + * (unknown). So the rows of a table partition cleanly: + * + * (WHERE p) UNION ALL (WHERE NOT p) UNION ALL (WHERE p IS NULL) + * == (the whole table, no WHERE) + * + * as multisets. The combined query exercises the qualification machinery + * three different ways and must reconstruct exactly the unfiltered result. A + * disagreement is unambiguously a bug -- in predicate evaluation, in + * three-valued logic, in an index or bitmap path that drops or duplicates a + * row, or in set-operation handling. No oracle to get wrong: the property is + * a tautology of SQL semantics, so any failure is in the server. + * + * TLP found dozens of real, confirmed wrong-results bugs in mature databases + * (PostgreSQL, SQLite, MySQL, CockroachDB) that decades of hand-written tests + * had not. This area applies it to a randomly generated table and predicate + * from query_gen.h. + * + * Tier B: runs SQL against the cluster the TAP driver started; see pg_prop.h + * and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_query_tlp.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "query_tlp" +#include "pg_prop.h" + +#include "lib/stringinfo.h" +#include "query_gen.h" + +/* + * Run two queries that must return identical multisets and fail with a diff if + * they do not. Both are ordered by the same key so the row-major string + * rendering compares as a multiset (identical rows in identical order). + */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *whole, const char *partitioned) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL; + char *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, whole, &f1, &e1); + if (f1) + pg_prop_fail(tc, "unfiltered query failed: %s\n SQL: %s", + e1 ? e1 : "(unknown)", whole); + + r2 = pg_prop_sql_rows(conn, partitioned, &f2, &e2); + if (f2) + pg_prop_fail(tc, "partitioned query failed: %s\n SQL: %s", + e2 ? e2 : "(unknown)", partitioned); + + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, + "TLP mismatch:\n whole: %s\n partitioned: %s\n" + " whole rows:\n%s\n partitioned rows:\n%s", + whole, partitioned, r1, r2); + + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * Core TLP: a plain WHERE partitioned three ways over the predicate. + * + * SELECT id FROM t ORDER BY id + * == + * ( SELECT id FROM t WHERE p + * UNION ALL SELECT id FROM t WHERE NOT p + * UNION ALL SELECT id FROM t WHERE p IS NULL ) ORDER BY id + * + * id is the primary key, so ORDER BY id gives a stable total order and the + * two renderings compare as multisets. + */ +PG_PROP(where_partitioned) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData pred; + StringInfoData whole; + StringInfoData part; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + initStringInfo(&whole); + appendStringInfoString(&whole, "SELECT id FROM " QG_TABLE " ORDER BY id"); + + initStringInfo(&part); + appendStringInfo(&part, + "SELECT id FROM (" + "SELECT id FROM " QG_TABLE " WHERE (%s) " + "UNION ALL SELECT id FROM " QG_TABLE " WHERE NOT (%s) " + "UNION ALL SELECT id FROM " QG_TABLE " WHERE (%s) IS NULL" + ") s ORDER BY id", + pred.data, pred.data, pred.data); + + require_same_rows(tc, conn, whole.data, part.data); + + pfree(pred.data); + pfree(whole.data); + pfree(part.data); + pg_prop_disconnect(conn); +} + +/* + * TLP for aggregation: COUNT(*) over the whole table equals the sum of the + * three partition counts. Aggregation takes a different executor path than + * row return, so this catches count/agg-pushdown bugs the row form may miss. + */ +PG_PROP(count_partitioned) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData pred; + StringInfoData whole; + StringInfoData part; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + initStringInfo(&whole); + appendStringInfoString(&whole, "SELECT count(*) FROM " QG_TABLE); + + initStringInfo(&part); + appendStringInfo(&part, + "SELECT (SELECT count(*) FROM " QG_TABLE " WHERE (%s)) " + "+ (SELECT count(*) FROM " QG_TABLE " WHERE NOT (%s)) " + "+ (SELECT count(*) FROM " QG_TABLE " WHERE (%s) IS NULL)", + pred.data, pred.data, pred.data); + + require_same_rows(tc, conn, whole.data, part.data); + + pfree(pred.data); + pfree(whole.data); + pfree(part.data); + pg_prop_disconnect(conn); +} + +/* + * TLP across a self-join: partition the join's ON predicate. Joins are a rich + * source of wrong-results bugs (hash vs merge vs nestloop, NULL handling in + * join keys), and TLP over the join condition exercises all of that. The + * whole join is an inner self-join on a trivial always-true condition narrowed + * to a manageable size; the partition splits on the generated predicate + * evaluated on the left row. + */ +PG_PROP(join_filter_partitioned) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData pred; + StringInfoData whole; + StringInfoData part; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + /* + * Whole: every row of t paired with the single row whose id = 1 (if any), + * returning t.id. The generated predicate uses unqualified column names, + * so in the partitioned form it is applied inside a single-table subquery + * over t (where the names are unambiguous) before the join. + */ + initStringInfo(&whole); + appendStringInfoString(&whole, + "SELECT t.id FROM " QG_TABLE " t JOIN " QG_TABLE + " u ON u.id = 1 ORDER BY t.id"); + + initStringInfo(&part); + appendStringInfo(&part, + "SELECT id FROM (" + "SELECT t.id FROM (SELECT * FROM " QG_TABLE " WHERE (%s)) t " + "JOIN " QG_TABLE " u ON u.id = 1 " + "UNION ALL " + "SELECT t.id FROM (SELECT * FROM " QG_TABLE " WHERE NOT (%s)) t " + "JOIN " QG_TABLE " u ON u.id = 1 " + "UNION ALL " + "SELECT t.id FROM (SELECT * FROM " QG_TABLE " WHERE (%s) IS NULL) t " + "JOIN " QG_TABLE " u ON u.id = 1" + ") s ORDER BY id", + pred.data, pred.data, pred.data); + + require_same_rows(tc, conn, whole.data, part.data); + + pfree(pred.data); + pfree(whole.data); + pfree(part.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/sql/query_gen.c b/src/test/prop/sql/query_gen.c new file mode 100644 index 0000000000000..005e48fe07839 --- /dev/null +++ b/src/test/prop/sql/query_gen.c @@ -0,0 +1,234 @@ +/*------------------------------------------------------------------------- + * + * query_gen.c + * Shared SQL query generator for the query-level property areas. + * + * See query_gen.h for the contract. The generator produces well-typed, + * parseable SQL text only; it never decides what is correct -- that is each + * area's oracle. Keeping generation here means the TLP, NoREC and GUC-flip + * areas all draw from the same distribution of tables and predicates, so a + * counterexample found by one is reproducible shape-for-shape in the others. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/query_gen.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#include "query_gen.h" + +/* Small domains keep counterexamples legible and cases fast. */ +#define QG_VAL_MIN (-5) +#define QG_VAL_MAX 5 + +/* Draw an int in [QG_VAL_MIN, QG_VAL_MAX] as SQL text, ~15% NULL. */ +static void +draw_int_or_null(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", + (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX)); +} + +/* Draw a short [a-c] string literal or SQL NULL, ~15% NULL. */ +static void +draw_text_or_null(pg_prop_tc *tc, StringInfo buf) +{ + int len; + int i; + + if (pg_prop_draw_int(tc, 0, 6) == 0) + { + appendStringInfoString(buf, "NULL"); + return; + } + len = (int) pg_prop_draw_int(tc, 0, 3); + appendStringInfoChar(buf, '\''); + for (i = 0; i < len; i++) + appendStringInfoChar(buf, (char) ('a' + pg_prop_draw_int(tc, 0, 2))); + appendStringInfoChar(buf, '\''); +} + +/* Draw a float8 literal (small, one decimal) or SQL NULL, ~15% NULL. */ +static void +draw_float_or_null(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d.%d", + (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), + (int) pg_prop_draw_int(tc, 0, 9)); +} + +void +qg_build_table(pg_prop_tc *tc, pg_prop_conn *conn, int *nrows) +{ + char *err = NULL; + StringInfoData ddl; + int n; + int r; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS " QG_TABLE "; " + "CREATE TABLE " QG_TABLE + " (id int PRIMARY KEY, a int, b int, c text, f float8); "); + + n = (int) pg_prop_draw_int(tc, 0, QG_MAX_ROWS); + for (r = 1; r <= n; r++) + { + appendStringInfo(&ddl, "INSERT INTO " QG_TABLE " VALUES (%d, ", r); + draw_int_or_null(tc, &ddl); + appendStringInfoString(&ddl, ", "); + draw_int_or_null(tc, &ddl); + appendStringInfoString(&ddl, ", "); + draw_text_or_null(tc, &ddl); + appendStringInfoString(&ddl, ", "); + draw_float_or_null(tc, &ddl); + appendStringInfoString(&ddl, "); "); + } + + /* ANALYZE so the planner has stats and will actually consider plans. */ + appendStringInfoString(&ddl, "ANALYZE " QG_TABLE ";"); + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "generator DDL/DML failed: %s", err ? err : ""); + + pfree(ddl.data); + *nrows = n; +} + +/* One leaf comparison against a column: "a " or an IS NULL test. */ +static void +draw_leaf(pg_prop_tc *tc, StringInfo buf) +{ + static const char *const cmp[] = {"=", "<>", "<", "<=", ">", ">="}; + int kind = (int) pg_prop_draw_int(tc, 0, 6); + int col = (int) pg_prop_draw_int(tc, 0, 2); /* a, b, or f */ + const char *colname = (col == 0) ? "a" : (col == 1) ? "b" : "f"; + + switch (kind) + { + case 0: /* col IS NULL */ + appendStringInfo(buf, "%s IS NULL", colname); + break; + case 1: /* col IS NOT NULL */ + appendStringInfo(buf, "%s IS NOT NULL", colname); + break; + case 2: /* c = 'text' / c comparison */ + appendStringInfoString(buf, "c "); + appendStringInfoString(buf, cmp[pg_prop_draw_int(tc, 0, 5)]); + appendStringInfoChar(buf, ' '); + draw_text_or_null(tc, buf); + break; + case 3: /* col IN (list) */ + appendStringInfo(buf, "%s IN (", colname); + { + int m = (int) pg_prop_draw_int(tc, 1, 3); + int i; + + for (i = 0; i < m; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + if (col == 2) + draw_float_or_null(tc, buf); + else + draw_int_or_null(tc, buf); + } + } + appendStringInfoChar(buf, ')'); + break; + case 4: /* col BETWEEN lo AND hi */ + appendStringInfo(buf, "%s BETWEEN %d AND %d", colname, + (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), + (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX)); + break; + default: /* col const */ + appendStringInfo(buf, "%s %s ", colname, cmp[pg_prop_draw_int(tc, 0, 5)]); + if (col == 2) + draw_float_or_null(tc, buf); + else + draw_int_or_null(tc, buf); + break; + } +} + +static void +draw_predicate_rec(pg_prop_tc *tc, StringInfo buf, int depth) +{ + int kind; + + if (depth >= 3) + kind = 0; + else + kind = (int) pg_prop_draw_int(tc, 0, 4); + + switch (kind) + { + case 0: /* leaf */ + draw_leaf(tc, buf); + break; + case 1: /* NOT (sub) */ + appendStringInfoString(buf, "NOT ("); + draw_predicate_rec(tc, buf, depth + 1); + appendStringInfoChar(buf, ')'); + break; + case 2: /* (l AND r) */ + appendStringInfoChar(buf, '('); + draw_predicate_rec(tc, buf, depth + 1); + appendStringInfoString(buf, " AND "); + draw_predicate_rec(tc, buf, depth + 1); + appendStringInfoChar(buf, ')'); + break; + case 3: /* (l OR r) */ + appendStringInfoChar(buf, '('); + draw_predicate_rec(tc, buf, depth + 1); + appendStringInfoString(buf, " OR "); + draw_predicate_rec(tc, buf, depth + 1); + appendStringInfoChar(buf, ')'); + break; + default: /* leaf */ + draw_leaf(tc, buf); + break; + } +} + +void +qg_draw_predicate(pg_prop_tc *tc, StringInfo buf) +{ + draw_predicate_rec(tc, buf, 0); +} + +void +qg_draw_int_expr(pg_prop_tc *tc, StringInfo buf) +{ + int kind = (int) pg_prop_draw_int(tc, 0, 4); + + switch (kind) + { + case 0: + appendStringInfoString(buf, "a"); + break; + case 1: + appendStringInfoString(buf, "b"); + break; + case 2: + appendStringInfo(buf, "%d", + (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX)); + break; + case 3: + /* coalesce so NULLs don't dominate; a + b */ + appendStringInfoString(buf, "(coalesce(a,0) + coalesce(b,0))"); + break; + default: + appendStringInfoString(buf, "(coalesce(a,0) * 2)"); + break; + } +} diff --git a/src/test/prop/sql/query_gen.h b/src/test/prop/sql/query_gen.h new file mode 100644 index 0000000000000..012551ed02cb2 --- /dev/null +++ b/src/test/prop/sql/query_gen.h @@ -0,0 +1,77 @@ +/*------------------------------------------------------------------------- + * + * query_gen.h + * Shared SQL query generator for the query-level property areas. + * + * The scalar and type areas (prop_numeric.c and friends) generate one value + * and check one function. The query areas in this subdirectory instead + * generate a whole relation and a whole SQL query over it, then check a + * property of the *result set* -- the kind of property that catches planner + * and executor wrong-results bugs rather than type-I/O bugs. + * + * This header is the generator those areas share: it builds a random table + * and a random boolean predicate over its columns, both as SQL text, so each + * area can wrap them in whatever query form its oracle needs (a ternary + * partition for TLP, an optimisable/unoptimisable pair for NoREC, the same + * query under flipped planner GUCs for the metamorphic area). + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/query_gen.h + * + *------------------------------------------------------------------------- + */ +#ifndef QUERY_GEN_H +#define QUERY_GEN_H + +#include "pg_prop.h" +#include "lib/stringinfo.h" + +/* + * The generated table. A fixed, legible shape: a few typed columns whose + * domains are small so a counterexample stays readable and a case stays fast, + * but wide enough that AND/OR/NOT predicates over them exercise real + * qualification, join, and NULL-handling paths. + * + * Columns: + * id int PRIMARY KEY (1..n, dense) -- gives an index to (not) use + * a int small domain, ~15% NULL -- the main predicate column + * b int small domain, ~15% NULL + * c text short [a-c] string or NULL + * f float8 small domain or NULL + */ +#define QG_TABLE "qg_t" + +/* Number of rows the generated table holds this case (0 .. QG_MAX_ROWS). */ +#define QG_MAX_ROWS 40 + +/* + * Build QG_TABLE (dropping any prior one) and populate it with a random set + * of 0..QG_MAX_ROWS rows. Returns the row count via *nrows. Fails the test + * case (does not return) on any SQL error, since the generator itself must + * never produce invalid DDL/DML. + */ +extern void qg_build_table(pg_prop_tc *tc, pg_prop_conn *conn, int *nrows); + +/* + * Append a random boolean predicate over QG_TABLE's columns to buf. The + * predicate is always well-typed and parseable, mixes AND/OR/NOT, comparisons + * (= <> < <= > >=), IS [NOT] NULL, IN-lists and BETWEEN, and references the + * numeric, text and float columns. Depth is bounded so it always terminates. + * + * The predicate is three-valued (may evaluate to NULL per row): that is the + * point, because the TLP and NoREC oracles are precisely about correct + * three-valued-logic handling in the qualification machinery. + */ +extern void qg_draw_predicate(pg_prop_tc *tc, StringInfo buf); + +/* + * Draw a random scalar expression over QG_TABLE's columns (an integer-typed + * expression: a column, a constant, or an arithmetic combination). Used where + * an area needs an ORDER BY / GROUP BY / projection target rather than a + * boolean. Always well-typed and non-erroring (division is guarded). + */ +extern void qg_draw_int_expr(pg_prop_tc *tc, StringInfo buf); + +#endif /* QUERY_GEN_H */ diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index d275599272f51..5fa955220e929 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_planadvice query_norec); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 1691130e4163d72d6e67a57041b6c70e9b634835 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Tue, 13 Oct 2026 09:06:05 -0400 Subject: [PATCH 255/338] prop: add TLP-aggregate, TLP-distinct, cardinality, and PQS query areas Four more query-level areas built on the shared query_gen.c generator, each a metamorphic or known-fact oracle over randomly generated tables and queries: * query_tlp_agg -- the ternary predicate partition applied to count/sum/min/max aggregates, which must fold back to the whole-table aggregate. * query_tlp_distinct -- the same partition under DISTINCT and the set operators (UNION distributes, the p and NOT-p partitions are disjoint, EXCEPT recovers the NULL partition). * query_cardinality -- result-size monotonicity laws (adding a conjunct cannot grow, a disjunct cannot shrink, the partition tiles the table, DISTINCT never grows, UNION ALL is additive). * query_pqs -- Pivoted Query Synthesis: pick a real pivot row, synthesise a predicate guaranteed true for it, and require it in the result (and across a self-join). Checks one query against one known-true fact rather than two queries against each other. These are the oracles most likely to surface a live wrong-results bug. --- src/test/prop/DESIGN | 56 +++- src/test/prop/Makefile | 8 +- src/test/prop/meson.build | 6 +- src/test/prop/sql/prop_query_cardinality.c | 259 +++++++++++++++ src/test/prop/sql/prop_query_pqs.c | 350 ++++++++++++++++++++ src/test/prop/sql/prop_query_tlp_agg.c | 225 +++++++++++++ src/test/prop/sql/prop_query_tlp_distinct.c | 207 ++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 8 files changed, 1096 insertions(+), 17 deletions(-) create mode 100644 src/test/prop/sql/prop_query_cardinality.c create mode 100644 src/test/prop/sql/prop_query_pqs.c create mode 100644 src/test/prop/sql/prop_query_tlp_agg.c create mode 100644 src/test/prop/sql/prop_query_tlp_distinct.c diff --git a/src/test/prop/DESIGN b/src/test/prop/DESIGN index 65bb33d22b4a9..2ce8f06779f86 100644 --- a/src/test/prop/DESIGN +++ b/src/test/prop/DESIGN @@ -66,7 +66,8 @@ Shape of the tree dependency. prop_.c one file per area; a set of PG_PROP() properties. sql/ query-level areas that generate whole SQL statements - (query_gen.c plus the TLP, NoREC, and plan-advice areas). + (query_gen.c plus the TLP, NoREC, cardinality, PQS, and + plan-advice areas). t/001_prop.pl TAP driver for the in-process (Tier A) areas. t/002_sql_props.pl TAP driver for the SQL-oracle (Tier B) areas; starts a node. @@ -230,18 +231,43 @@ reference answer, so a disagreement is unambiguously a server bug: p IS NULL, so the three partitions UNION ALL'd must reconstruct the unfiltered relation. +* query_tlp_agg -- the same ternary partition applied to aggregates: count(*), + sum, min and max over the whole table must equal the fold of the three + partition aggregates. Exercises the hash/sorted aggregation paths and the + NULL/empty-group folding that plain row return does not. + +* query_tlp_distinct -- the ternary partition under DISTINCT and the set + operators: DISTINCT distributes over the partition up to UNION, the p and + NOT-p partitions are disjoint (INTERSECT is empty), and EXCEPT recovers the + p-IS-NULL partition. Exercises duplicate elimination and set-operation + handling. + * query_norec -- compare an optimizable count(*) ... WHERE p against the un-optimizable twin that pulls p into a bare per-row scan; the counts must - match. + match. Variants cover the FALSE partition, a sum under the filter, and the + predicate across a self-join (where pushdown interacts with the join). + +* query_cardinality -- metamorphic relations on result size rather than + contents: adding a conjunct cannot grow and adding a disjunct cannot shrink a + result, the ternary partition tiles the table exactly, DISTINCT never grows a + projection, and UNION ALL is additive while UNION is no larger. Cheap (one + count per query) yet broad. + +* query_pqs -- Pivoted Query Synthesis (Rigger and Su, ESEC/FSE 2020). Pick a + real pivot row, read its values, synthesize a WHERE predicate guaranteed TRUE + for it by rectifying every leaf against those values, and require the pivot's + id in the result. Unlike the metamorphic oracles it checks a single query + against a single known-true fact, so it catches a bug that corrupts both + sides of a comparison identically. * query_planadvice -- run a generated query with the planner free and again - steered by contrib/pg_plan_advice (SEQ_SCAN, NESTED_LOOP_PLAIN, and so on, via - the pg_plan_advice.advice GUC), plus a tiny-work_mem spill variant; the result - must not depend on the plan the planner chose. Advice is used in preference - to the blunt enable_* GUCs because it names the exact relation or join to - re-shape and the extension rejects advice it cannot honor, so the alternate - plan is a targeted legal variant of the same query rather than a global cost - distortion. + steered by contrib/pg_plan_advice (SEQ_SCAN, NESTED_LOOP_PLAIN, JOIN_ORDER, + and so on, via the pg_plan_advice.advice GUC), plus a tiny-work_mem spill + variant; the result must not depend on the plan the planner chose. Advice is + used in preference to the blunt enable_* GUCs because it names the exact + relation or join to re-shape and the extension rejects advice it cannot + honor, so the alternate plan is a targeted legal variant of the same query + rather than a global cost distortion. Build-system integration @@ -374,7 +400,11 @@ Limitations and future work structures in src/backend/lib (for example the radix tree and the TID store) are the natural first customers. -* More query-generation oracles are the obvious growth area: TLP for aggregates - and DISTINCT, Pivoted Query Synthesis, join-order metamorphic testing via - pg_plan_advice, and cardinality-monotonicity checks. These are where a live - wrong-results bug is most likely to turn up. +* More query-generation oracles remain a growth area. Implemented so far: TLP + (rows, aggregates, DISTINCT/set-ops), NoREC, plan-advice metamorphic testing + (scan/join method, join order, spill), result-cardinality monotonicity, and + Pivoted Query Synthesis. Still open and likely to find a live wrong-results + bug: a multi-table generator for richer join-order and subquery-unnesting + equivalence, constant-folding metamorphic checks, and correlated-subquery + rewrites. A larger/deeper generator (wider joins, more columns, correlated + data) run at high iteration is the lever most likely to turn one up. diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 14e05aafad401..f2ad9c19ff2d6 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -272,8 +272,12 @@ OBJS = \ prop_isn.o \ sql/query_gen.o \ sql/prop_query_tlp.o \ - sql/prop_query_planadvice.o \ - sql/prop_query_norec.o + sql/prop_query_tlp_agg.o \ + sql/prop_query_tlp_distinct.o \ + sql/prop_query_norec.o \ + sql/prop_query_cardinality.o \ + sql/prop_query_pqs.o \ + sql/prop_query_planadvice.o PG_CPPFLAGS = -I$(libpq_srcdir) -I$(srcdir)/sql PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 040edef9c66f3..281a472ea31bf 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -277,8 +277,12 @@ prop_area_sources = files( prop_query_sources = files( 'sql/query_gen.c', 'sql/prop_query_tlp.c', - 'sql/prop_query_planadvice.c', + 'sql/prop_query_tlp_agg.c', + 'sql/prop_query_tlp_distinct.c', 'sql/prop_query_norec.c', + 'sql/prop_query_cardinality.c', + 'sql/prop_query_pqs.c', + 'sql/prop_query_planadvice.c', ) pg_prop = executable('pg_prop', diff --git a/src/test/prop/sql/prop_query_cardinality.c b/src/test/prop/sql/prop_query_cardinality.c new file mode 100644 index 0000000000000..682e374821da1 --- /dev/null +++ b/src/test/prop/sql/prop_query_cardinality.c @@ -0,0 +1,259 @@ +/*------------------------------------------------------------------------- + * + * prop_query_cardinality.c + * Query-level property tests: result-cardinality monotonicity laws. + * + * These are metamorphic relations on the *size* of a result set rather than + * its contents, so they are cheap (one count per query) yet catch a broad + * class of qualification and set-operation wrong-results bugs: + * + * 1. Adding a conjunct never grows the result: + * count(WHERE p) >= count(WHERE p AND q) + * 2. Adding a disjunct never shrinks it: + * count(WHERE p) <= count(WHERE p OR q) + * 3. A predicate and its negation and its NULL-set tile the table exactly: + * count(p) + count(NOT p) + count(p IS NULL) == count(*) + * 4. DISTINCT never grows a projection: + * count(DISTINCT e) <= count(all e) + * 5. UNION ALL is additive; UNION (set) is no larger: + * count(A UNION ALL B) == count(A) + count(B) + * count(A UNION B) <= count(A UNION ALL B) + * + * Each is a theorem of relational algebra, so a violation is a server bug in + * the planner's choice of plan or the executor's row counting -- not an oracle + * that could be miscalibrated. Counting rather than comparing contents also + * lets these run against larger generated predicates without the comparison + * cost of full row rendering. + * + * Tier B: runs SQL against the cluster the TAP driver started. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_query_cardinality.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "query_cardinality" +#include "pg_prop.h" + +#include "lib/stringinfo.h" +#include "query_gen.h" + +/* Run a scalar COUNT query, failing the case on SQL error; return the count. */ +static long +count_of(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *val; + long n; + + val = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "count query failed: %s\n SQL: %s", + err ? err : "(unknown)", sql); + n = atol(val); + pg_free(val); + if (err) + pg_free(err); + return n; +} + +/* count(WHERE p) >= count(WHERE p AND q); and <= count(WHERE p OR q). */ +PG_PROP(conjunct_shrinks_disjunct_grows) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData p; + StringInfoData q; + StringInfoData sql; + long cp, + cpq_and, + cpq_or; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + initStringInfo(&p); + qg_draw_predicate(tc, &p); + initStringInfo(&q); + qg_draw_predicate(tc, &q); + initStringInfo(&sql); + + appendStringInfo(&sql, "SELECT count(*) FROM " QG_TABLE " WHERE (%s)", p.data); + cp = count_of(tc, conn, sql.data); + + resetStringInfo(&sql); + appendStringInfo(&sql, "SELECT count(*) FROM " QG_TABLE + " WHERE (%s) AND (%s)", p.data, q.data); + cpq_and = count_of(tc, conn, sql.data); + + resetStringInfo(&sql); + appendStringInfo(&sql, "SELECT count(*) FROM " QG_TABLE + " WHERE (%s) OR (%s)", p.data, q.data); + cpq_or = count_of(tc, conn, sql.data); + + if (!(cpq_and <= cp)) + pg_prop_fail(tc, + "conjunct grew result: count(p AND q)=%ld > count(p)=%ld\n" + " p: %s\n q: %s", cpq_and, cp, p.data, q.data); + if (!(cpq_or >= cp)) + pg_prop_fail(tc, + "disjunct shrank result: count(p OR q)=%ld < count(p)=%ld\n" + " p: %s\n q: %s", cpq_or, cp, p.data, q.data); + + pfree(p.data); + pfree(q.data); + pfree(sql.data); + pg_prop_disconnect(conn); +} + +/* count(p) + count(NOT p) + count(p IS NULL) == count(*). */ +PG_PROP(partition_tiles_table) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData p; + StringInfoData sql; + long total, + cp, + cnp, + cnull; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + initStringInfo(&p); + qg_draw_predicate(tc, &p); + initStringInfo(&sql); + + total = count_of(tc, conn, "SELECT count(*) FROM " QG_TABLE); + + appendStringInfo(&sql, "SELECT count(*) FROM " QG_TABLE " WHERE (%s)", p.data); + cp = count_of(tc, conn, sql.data); + resetStringInfo(&sql); + appendStringInfo(&sql, "SELECT count(*) FROM " QG_TABLE " WHERE NOT (%s)", p.data); + cnp = count_of(tc, conn, sql.data); + resetStringInfo(&sql); + appendStringInfo(&sql, "SELECT count(*) FROM " QG_TABLE " WHERE (%s) IS NULL", p.data); + cnull = count_of(tc, conn, sql.data); + + if (cp + cnp + cnull != total) + pg_prop_fail(tc, + "partition does not tile: %ld + %ld + %ld != %ld\n p: %s", + cp, cnp, cnull, total, p.data); + + pfree(p.data); + pfree(sql.data); + pg_prop_disconnect(conn); +} + +/* + * count(A UNION ALL B) == count(A) + count(B), and count(A UNION B) <= that, + * where A = WHERE p and B = WHERE q over id projections. + */ +PG_PROP(union_cardinalities) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData p; + StringInfoData q; + StringInfoData sql; + long ca, + cb, + call_, + cset; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + initStringInfo(&p); + qg_draw_predicate(tc, &p); + initStringInfo(&q); + qg_draw_predicate(tc, &q); + initStringInfo(&sql); + + appendStringInfo(&sql, "SELECT count(*) FROM " QG_TABLE " WHERE (%s)", p.data); + ca = count_of(tc, conn, sql.data); + resetStringInfo(&sql); + appendStringInfo(&sql, "SELECT count(*) FROM " QG_TABLE " WHERE (%s)", q.data); + cb = count_of(tc, conn, sql.data); + + resetStringInfo(&sql); + appendStringInfo(&sql, + "SELECT count(*) FROM (" + "SELECT id FROM " QG_TABLE " WHERE (%s) " + "UNION ALL SELECT id FROM " QG_TABLE " WHERE (%s)) s", + p.data, q.data); + call_ = count_of(tc, conn, sql.data); + + resetStringInfo(&sql); + appendStringInfo(&sql, + "SELECT count(*) FROM (" + "SELECT id FROM " QG_TABLE " WHERE (%s) " + "UNION SELECT id FROM " QG_TABLE " WHERE (%s)) s", + p.data, q.data); + cset = count_of(tc, conn, sql.data); + + if (call_ != ca + cb) + pg_prop_fail(tc, + "UNION ALL not additive: count=%ld != %ld + %ld\n" + " p: %s\n q: %s", call_, ca, cb, p.data, q.data); + if (!(cset <= call_)) + pg_prop_fail(tc, + "UNION larger than UNION ALL: %ld > %ld\n p: %s\n q: %s", + cset, call_, p.data, q.data); + + pfree(p.data); + pfree(q.data); + pfree(sql.data); + pg_prop_disconnect(conn); +} + +/* count(DISTINCT e) <= count(all e) for a generated int projection e. */ +PG_PROP(distinct_never_grows) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData e; + StringInfoData sql; + long call_, + cdistinct; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + initStringInfo(&e); + qg_draw_int_expr(tc, &e); + initStringInfo(&sql); + + appendStringInfo(&sql, "SELECT count(%s) FROM " QG_TABLE, e.data); + call_ = count_of(tc, conn, sql.data); + resetStringInfo(&sql); + appendStringInfo(&sql, "SELECT count(DISTINCT %s) FROM " QG_TABLE, e.data); + cdistinct = count_of(tc, conn, sql.data); + + if (!(cdistinct <= call_)) + pg_prop_fail(tc, + "DISTINCT grew count: count(distinct e)=%ld > count(e)=%ld\n" + " e: %s", cdistinct, call_, e.data); + + pfree(e.data); + pfree(sql.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/sql/prop_query_pqs.c b/src/test/prop/sql/prop_query_pqs.c new file mode 100644 index 0000000000000..79f0bb47d5da5 --- /dev/null +++ b/src/test/prop/sql/prop_query_pqs.c @@ -0,0 +1,350 @@ +/*------------------------------------------------------------------------- + * + * prop_query_pqs.c + * Query-level property tests using Pivoted Query Synthesis (PQS). + * + * PQS (Rigger & Su, ESEC/FSE 2020) is the oracle that found the most + * previously-unknown wrong-results bugs in mature databases. Unlike TLP and + * NoREC, which compare two queries, PQS checks a single query against a single + * known-true fact, so it can catch a bug that corrupts *both* sides of a + * metamorphic pair identically. + * + * The method: + * 1. Pick a random "pivot" row that actually exists in the table and read + * back its column values. + * 2. Synthesise a WHERE predicate that is, by construction, guaranteed to + * evaluate to TRUE for the pivot row -- every leaf is rectified against + * the pivot's own values (a < pivot.a+1, c = pivot.c, x IS NULL when the + * pivot's x is NULL, and so on) and the leaves are combined only with AND + * and with OR-of-true, which preserve truth for the pivot. + * 3. Run SELECT id ... WHERE . The pivot's id MUST appear in the + * result, because the predicate is true for it. If it is missing, the + * query processor dropped a row it must return -- a wrong-results bug. + * + * The containment check ("pivot is in the result") is a fact about one row we + * computed ourselves from the stored data, so there is no reference query to + * get wrong. The synthesis is where the care is: a single mis-rectified leaf + * would make the predicate possibly-false for the pivot and produce a false + * alarm, so the rectification below is deliberately conservative. + * + * Tier B: runs SQL against the cluster the TAP driver started. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_query_pqs.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "query_pqs" +#include "pg_prop.h" + +#include "lib/stringinfo.h" +#include "query_gen.h" + +/* The pivot row's values; a NULL C pointer means the column IS NULL. */ +typedef struct PivotRow +{ + long id; + char *a; /* int as text, or NULL */ + char *b; /* int as text, or NULL */ + char *c; /* text value (unquoted), or NULL */ + char *f; /* float as text, or NULL */ +} PivotRow; + +/* Read one column of the pivot row; returns a palloc'd string or NULL. */ +static char * +read_col(pg_prop_tc *tc, pg_prop_conn *conn, long id, const char *col) +{ + bool failed = false; + char *err = NULL; + char *val; + StringInfoData sql; + + initStringInfo(&sql); + appendStringInfo(&sql, "SELECT %s FROM " QG_TABLE " WHERE id = %ld", + col, id); + val = pg_prop_sql_scalar(conn, sql.data, &failed, &err); + if (failed) + pg_prop_fail(tc, "pivot read failed: %s\n SQL: %s", + err ? err : "(unknown)", sql.data); + pfree(sql.data); + if (err) + pg_free(err); + return val; /* NULL means SQL NULL */ +} + +/* + * Append a leaf predicate on an integer column that is TRUE for the pivot. + * - pivot value is NULL -> "col IS NULL" + * - pivot value is V -> one of: col = V, col <> V+k, col >= V, + * col <= V, col IN (V, ...), col BETWEEN <=V AND >=V + * All strictly rectified so the leaf cannot be false (or NULL) for the pivot. + */ +static void +append_int_leaf(pg_prop_tc *tc, StringInfo buf, const char *col, const char *val) +{ + long v; + int kind; + + if (val == NULL) + { + appendStringInfo(buf, "%s IS NULL", col); + return; + } + v = atol(val); + kind = (int) pg_prop_draw_int(tc, 0, 5); + switch (kind) + { + case 0: + appendStringInfo(buf, "%s = %ld", col, v); + break; + case 1: + /* col <> (something that is not v) */ + appendStringInfo(buf, "%s <> %ld", col, v + 1 + pg_prop_draw_int(tc, 0, 3)); + break; + case 2: + appendStringInfo(buf, "%s >= %ld", col, v - pg_prop_draw_int(tc, 0, 3)); + break; + case 3: + appendStringInfo(buf, "%s <= %ld", col, v + pg_prop_draw_int(tc, 0, 3)); + break; + case 4: + /* IN list that definitely contains v */ + appendStringInfo(buf, "%s IN (%ld, %ld, %ld)", + col, v - 1, v, v + 2); + break; + default: + appendStringInfo(buf, "%s BETWEEN %ld AND %ld", + col, v - pg_prop_draw_int(tc, 0, 3), + v + pg_prop_draw_int(tc, 0, 3)); + break; + } +} + +/* Append a text leaf on column c that is TRUE for the pivot. */ +static void +append_text_leaf(pg_prop_tc *tc, StringInfo buf, const char *val) +{ + if (val == NULL) + { + appendStringInfoString(buf, "c IS NULL"); + return; + } + if (pg_prop_draw_bool(tc)) + appendStringInfo(buf, "c = '%s'", val); + else + appendStringInfo(buf, "c >= '%s'", val); /* c >= itself is true */ +} + +/* + * Append one rectified leaf over a randomly chosen column. Every branch is + * guaranteed TRUE for the pivot. + */ +static void +append_true_leaf(pg_prop_tc *tc, StringInfo buf, const PivotRow *pv) +{ + int col = (int) pg_prop_draw_int(tc, 0, 3); /* a, b, c, f */ + + switch (col) + { + case 0: + append_int_leaf(tc, buf, "a", pv->a); + break; + case 1: + append_int_leaf(tc, buf, "b", pv->b); + break; + case 2: + append_text_leaf(tc, buf, pv->c); + break; + default: + /* f: only an equality or IS NULL; avoid float range edge cases. */ + if (pv->f == NULL) + appendStringInfoString(buf, "f IS NULL"); + else + appendStringInfo(buf, "f = %s", pv->f); + break; + } +} + +/* + * Build a predicate TRUE for the pivot by combining rectified leaves with + * truth-preserving connectives: AND of true leaves stays true; (true OR + * anything) stays true, so a right-hand OR branch may be any leaf at all. + */ +static void +build_true_predicate(pg_prop_tc *tc, StringInfo buf, const PivotRow *pv, + int depth) +{ + int kind; + + if (depth >= 3) + { + append_true_leaf(tc, buf, pv); + return; + } + kind = (int) pg_prop_draw_int(tc, 0, 3); + switch (kind) + { + case 0: /* single true leaf */ + append_true_leaf(tc, buf, pv); + break; + case 1: /* (true AND true) */ + appendStringInfoChar(buf, '('); + build_true_predicate(tc, buf, pv, depth + 1); + appendStringInfoString(buf, " AND "); + build_true_predicate(tc, buf, pv, depth + 1); + appendStringInfoChar(buf, ')'); + break; + default: /* (true OR anything-true) -- keep both true so + * NULL from a bad RHS can't swallow the row */ + appendStringInfoChar(buf, '('); + build_true_predicate(tc, buf, pv, depth + 1); + appendStringInfoString(buf, " OR "); + append_true_leaf(tc, buf, pv); + appendStringInfoChar(buf, ')'); + break; + } +} + +/* + * PQS core: pick a pivot, synthesise a predicate true for it, require its id + * in the result. + */ +PG_PROP(pivot_is_contained) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + long pivot_id; + PivotRow pv = {0}; + StringInfoData pred; + StringInfoData sql; + bool failed = false; + char *hit; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + + /* No rows -> nothing to pivot on; this case is vacuously satisfied. */ + if (nrows == 0) + { + pg_prop_disconnect(conn); + return; + } + + /* ids are dense 1..nrows, so draw one directly. */ + pivot_id = (long) pg_prop_draw_int(tc, 1, nrows); + pv.id = pivot_id; + pv.a = read_col(tc, conn, pivot_id, "a"); + pv.b = read_col(tc, conn, pivot_id, "b"); + pv.c = read_col(tc, conn, pivot_id, "c"); + pv.f = read_col(tc, conn, pivot_id, "f"); + + initStringInfo(&pred); + build_true_predicate(tc, &pred, &pv, 0); + + initStringInfo(&sql); + appendStringInfo(&sql, + "SELECT id FROM " QG_TABLE " WHERE (%s) AND id = %ld", + pred.data, pivot_id); + + hit = pg_prop_sql_scalar(conn, sql.data, &failed, &err); + if (failed) + pg_prop_fail(tc, "PQS query failed: %s\n SQL: %s", + err ? err : "(unknown)", sql.data); + + if (hit == NULL) + pg_prop_fail(tc, + "PQS: pivot row %ld dropped by a predicate true for it\n" + " pivot: a=%s b=%s c=%s f=%s\n predicate: %s", + pivot_id, + pv.a ? pv.a : "NULL", pv.b ? pv.b : "NULL", + pv.c ? pv.c : "NULL", pv.f ? pv.f : "NULL", + pred.data); + + pg_free(hit); + if (pv.a) pg_free(pv.a); + if (pv.b) pg_free(pv.b); + if (pv.c) pg_free(pv.c); + if (pv.f) pg_free(pv.f); + if (err) pg_free(err); + pfree(pred.data); + pfree(sql.data); + pg_prop_disconnect(conn); +} + +/* + * PQS over a self-join: the pivot joined to itself on id must survive a + * predicate true for it applied on the left side. Exercises the join path + * with a single-row-guaranteed outcome. + */ +PG_PROP(pivot_survives_join) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + long pivot_id; + PivotRow pv = {0}; + StringInfoData pred; + StringInfoData sql; + bool failed = false; + char *hit; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + if (nrows == 0) + { + pg_prop_disconnect(conn); + return; + } + + pivot_id = (long) pg_prop_draw_int(tc, 1, nrows); + pv.id = pivot_id; + pv.a = read_col(tc, conn, pivot_id, "a"); + pv.b = read_col(tc, conn, pivot_id, "b"); + pv.c = read_col(tc, conn, pivot_id, "c"); + pv.f = read_col(tc, conn, pivot_id, "f"); + + initStringInfo(&pred); + build_true_predicate(tc, &pred, &pv, 0); + + initStringInfo(&sql); + appendStringInfo(&sql, + "SELECT t.id FROM " + "(SELECT * FROM " QG_TABLE " WHERE (%s)) t " + "JOIN " QG_TABLE " u ON u.id = t.id WHERE t.id = %ld", + pred.data, pivot_id); + + hit = pg_prop_sql_scalar(conn, sql.data, &failed, &err); + if (failed) + pg_prop_fail(tc, "PQS join query failed: %s\n SQL: %s", + err ? err : "(unknown)", sql.data); + + if (hit == NULL) + pg_prop_fail(tc, + "PQS/join: pivot row %ld dropped from self-join\n" + " pivot: a=%s b=%s c=%s f=%s\n predicate: %s", + pivot_id, + pv.a ? pv.a : "NULL", pv.b ? pv.b : "NULL", + pv.c ? pv.c : "NULL", pv.f ? pv.f : "NULL", + pred.data); + + pg_free(hit); + if (pv.a) pg_free(pv.a); + if (pv.b) pg_free(pv.b); + if (pv.c) pg_free(pv.c); + if (pv.f) pg_free(pv.f); + if (err) pg_free(err); + pfree(pred.data); + pfree(sql.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/sql/prop_query_tlp_agg.c b/src/test/prop/sql/prop_query_tlp_agg.c new file mode 100644 index 0000000000000..54b3cb96d970a --- /dev/null +++ b/src/test/prop/sql/prop_query_tlp_agg.c @@ -0,0 +1,225 @@ +/*------------------------------------------------------------------------- + * + * prop_query_tlp_agg.c + * Query-level property tests: TLP specialised to aggregates. + * + * Core TLP (prop_query_tlp.c) partitions a table three ways over a predicate + * and reconstructs the row multiset. This area applies the same ternary + * partition to *aggregates*, where the reconstruction law is arithmetic + * rather than UNION ALL: + * + * count(*) over t == count(p) + count(not p) + count(p is null) + * sum(e) over t == sum over the three partitions + * max(e) over t == max of the three partition maxima + * min(e) over t == min of the three partition minima + * + * Aggregation runs through hash/sorted/grouping executor paths that plain row + * return does not, and the folding of partial aggregates (and of NULL groups) + * is a classic wrong-results surface. The oracle is still a tautology of SQL + * semantics -- no invented reference value -- so any disagreement is a server + * bug, not a flaky expectation. + * + * NULL handling is the subtle part and the reason this is worth a separate + * area: SQL aggregates skip NULL inputs, and an empty partition yields NULL + * (for sum/min/max) or 0 (for count). We fold with coalesce and compare the + * whole-table aggregate against the folded partitions as text so NULL == NULL + * compares equal. + * + * Tier B: runs SQL against the cluster the TAP driver started; see pg_prop.h + * and t/002_sql_props.pl. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_query_tlp_agg.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "query_tlp_agg" +#include "pg_prop.h" + +#include "lib/stringinfo.h" +#include "query_gen.h" + +/* + * Compare two single-value queries as text (so NULL compares equal to NULL) + * and fail with both SQL strings on a mismatch. + */ +static void +require_same_scalar(pg_prop_tc *tc, pg_prop_conn *conn, + const char *whole, const char *folded) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL; + char *v1, + *v2; + + v1 = pg_prop_sql_scalar(conn, whole, &f1, &e1); + if (f1) + pg_prop_fail(tc, "whole-table aggregate failed: %s\n SQL: %s", + e1 ? e1 : "(unknown)", whole); + + v2 = pg_prop_sql_scalar(conn, folded, &f2, &e2); + if (f2) + pg_prop_fail(tc, "folded-partition aggregate failed: %s\n SQL: %s", + e2 ? e2 : "(unknown)", folded); + + /* + * max()/min() return SQL NULL for an empty or all-NULL input, which comes + * back as a C NULL pointer; the whole and folded forms must agree on NULL + * too (both NULL, or equal text). + */ + if ((v1 == NULL) != (v2 == NULL) || + (v1 != NULL && strcmp(v1, v2) != 0)) + pg_prop_fail(tc, + "TLP-aggregate mismatch:\n whole: %s = %s\n" + " folded: %s = %s", + whole, v1 ? v1 : "NULL", folded, v2 ? v2 : "NULL"); + + if (v1) + pg_free(v1); + if (v2) + pg_free(v2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* Build " FROM qg_t" and the folded three-partition form, then compare. */ +static void +check_agg(pg_prop_tc *tc, pg_prop_conn *conn, const char *pred, + const char *whole_expr, const char *fold_fmt) +{ + StringInfoData whole; + StringInfoData folded; + + initStringInfo(&whole); + appendStringInfo(&whole, "SELECT %s FROM " QG_TABLE, whole_expr); + + initStringInfo(&folded); + /* fold_fmt has three "%s" slots for the predicate (p, NOT p, p IS NULL). */ + appendStringInfo(&folded, fold_fmt, pred, pred, pred); + + require_same_scalar(tc, conn, whole.data, folded.data); + + pfree(whole.data); + pfree(folded.data); +} + +/* count(*) over t == count over the three partitions, summed. */ +PG_PROP(count_folds) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData pred; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + check_agg(tc, conn, pred.data, "count(*)", + "SELECT (SELECT count(*) FROM " QG_TABLE " WHERE (%s)) " + "+ (SELECT count(*) FROM " QG_TABLE " WHERE NOT (%s)) " + "+ (SELECT count(*) FROM " QG_TABLE " WHERE (%s) IS NULL)"); + + pfree(pred.data); + pg_prop_disconnect(conn); +} + +/* sum(a) over t == sum over the three partitions (coalesced for empties). */ +PG_PROP(sum_folds) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData pred; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + /* + * sum() skips NULL inputs and returns NULL on an empty set; coalesce each + * partition's sum to 0 so the three fold additively to the whole-table + * sum, which is itself coalesced to 0 for an all-empty/all-NULL table. + */ + check_agg(tc, conn, pred.data, "coalesce(sum(a), 0)", + "SELECT coalesce((SELECT sum(a) FROM " QG_TABLE " WHERE (%s)), 0) " + "+ coalesce((SELECT sum(a) FROM " QG_TABLE " WHERE NOT (%s)), 0) " + "+ coalesce((SELECT sum(a) FROM " QG_TABLE " WHERE (%s) IS NULL), 0)"); + + pfree(pred.data); + pg_prop_disconnect(conn); +} + +/* max(a) over t == max(a) over the three partitions UNION ALL'd. */ +PG_PROP(max_folds) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData pred; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + /* + * max() ignores NULLs and yields NULL for an empty input. Re-aggregating + * max over the three partitions concatenated is the whole-table max; this + * single aggregate over a UNION ALL avoids the planner pathology of + * greatest() over three correlated subqueries. + */ + check_agg(tc, conn, pred.data, "max(a)", + "SELECT max(a) FROM (" + "SELECT a FROM " QG_TABLE " WHERE (%s) " + "UNION ALL SELECT a FROM " QG_TABLE " WHERE NOT (%s) " + "UNION ALL SELECT a FROM " QG_TABLE " WHERE (%s) IS NULL) s"); + + pfree(pred.data); + pg_prop_disconnect(conn); +} + +/* min(a) over t == min(a) over the three partitions UNION ALL'd. */ +PG_PROP(min_folds) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData pred; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + check_agg(tc, conn, pred.data, "min(a)", + "SELECT min(a) FROM (" + "SELECT a FROM " QG_TABLE " WHERE (%s) " + "UNION ALL SELECT a FROM " QG_TABLE " WHERE NOT (%s) " + "UNION ALL SELECT a FROM " QG_TABLE " WHERE (%s) IS NULL) s"); + + pfree(pred.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/sql/prop_query_tlp_distinct.c b/src/test/prop/sql/prop_query_tlp_distinct.c new file mode 100644 index 0000000000000..146b26fc3a533 --- /dev/null +++ b/src/test/prop/sql/prop_query_tlp_distinct.c @@ -0,0 +1,207 @@ +/*------------------------------------------------------------------------- + * + * prop_query_tlp_distinct.c + * Query-level property tests: TLP for DISTINCT and set operations. + * + * The three-way ternary partition of core TLP reconstructs a *multiset*. Once + * you add DISTINCT and the set operators (UNION, INTERSECT, EXCEPT), the laws + * become set laws, which exercise a different executor layer: hashing and + * sorting for duplicate elimination, and the set-operation node that core TLP + * only touches through UNION ALL. + * + * The properties here are tautologies of set/multiset semantics over the same + * ternary partition (WHERE p / WHERE NOT p / WHERE p IS NULL): + * + * 1. DISTINCT distributes over the partition up to union: + * SELECT DISTINCT id FROM t + * == (DISTINCT id WHERE p) UNION (DISTINCT WHERE NOT p) + * UNION (DISTINCT WHERE p IS NULL) + * (id is unique here, but the DISTINCT/UNION machinery still runs.) + * + * 2. The three partitions are pairwise disjoint, so INTERSECT of any two is + * empty: (WHERE p) INTERSECT (WHERE NOT p) == no rows. + * + * 3. EXCEPT recovers a partition's complement within the whole: + * (all ids) EXCEPT (ids WHERE p) EXCEPT (ids WHERE NOT p) + * == ids WHERE p IS NULL (as a set) + * + * A disagreement is a bug in DISTINCT, in set-operation handling, or in the + * hash/sort path chosen for either -- never in the oracle, which invents no + * reference value. + * + * Tier B: runs SQL against the cluster the TAP driver started. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_query_tlp_distinct.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "query_tlp_distinct" +#include "pg_prop.h" + +#include "lib/stringinfo.h" +#include "query_gen.h" + +/* As in prop_query_tlp.c: two row-returning queries must match as text. */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *lhs, const char *rhs) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL; + char *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, lhs, &f1, &e1); + if (f1) + pg_prop_fail(tc, "lhs query failed: %s\n SQL: %s", + e1 ? e1 : "(unknown)", lhs); + + r2 = pg_prop_sql_rows(conn, rhs, &f2, &e2); + if (f2) + pg_prop_fail(tc, "rhs query failed: %s\n SQL: %s", + e2 ? e2 : "(unknown)", rhs); + + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, + "set-TLP mismatch:\n lhs: %s\n rhs: %s\n" + " lhs rows:\n%s\n rhs rows:\n%s", + lhs, rhs, r1, r2); + + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +/* + * SELECT DISTINCT id FROM t == the three DISTINCT partitions UNIONed. + * Both sides ordered by id for a stable comparison. + */ +PG_PROP(distinct_over_partition) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData pred; + StringInfoData lhs; + StringInfoData rhs; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + initStringInfo(&lhs); + appendStringInfoString(&lhs, + "SELECT id FROM (SELECT DISTINCT id FROM " QG_TABLE + ") s ORDER BY id"); + + initStringInfo(&rhs); + appendStringInfo(&rhs, + "SELECT id FROM (" + "SELECT DISTINCT id FROM " QG_TABLE " WHERE (%s) " + "UNION SELECT DISTINCT id FROM " QG_TABLE " WHERE NOT (%s) " + "UNION SELECT DISTINCT id FROM " QG_TABLE " WHERE (%s) IS NULL" + ") s ORDER BY id", + pred.data, pred.data, pred.data); + + require_same_rows(tc, conn, lhs.data, rhs.data); + + pfree(pred.data); + pfree(lhs.data); + pfree(rhs.data); + pg_prop_disconnect(conn); +} + +/* + * The p and NOT-p partitions are disjoint: their INTERSECT is empty. LHS is + * the intersect; RHS is a query that returns no rows by construction. + */ +PG_PROP(partitions_disjoint) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData pred; + StringInfoData lhs; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + initStringInfo(&lhs); + appendStringInfo(&lhs, + "SELECT id FROM (" + "SELECT id FROM " QG_TABLE " WHERE (%s) " + "INTERSECT SELECT id FROM " QG_TABLE " WHERE NOT (%s)" + ") s ORDER BY id", + pred.data, pred.data); + + require_same_rows(tc, conn, lhs.data, + "SELECT id FROM " QG_TABLE " WHERE false ORDER BY id"); + + pfree(pred.data); + pfree(lhs.data); + pg_prop_disconnect(conn); +} + +/* + * EXCEPT recovers the p-IS-NULL partition as a set: + * (DISTINCT all) EXCEPT (ids WHERE p) EXCEPT (ids WHERE NOT p) + * == DISTINCT ids WHERE p IS NULL + */ +PG_PROP(except_recovers_null_partition) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData pred; + StringInfoData lhs; + StringInfoData rhs; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + initStringInfo(&lhs); + appendStringInfo(&lhs, + "SELECT id FROM (" + "SELECT id FROM " QG_TABLE " " + "EXCEPT SELECT id FROM " QG_TABLE " WHERE (%s) " + "EXCEPT SELECT id FROM " QG_TABLE " WHERE NOT (%s)" + ") s ORDER BY id", + pred.data, pred.data); + + initStringInfo(&rhs); + appendStringInfo(&rhs, + "SELECT DISTINCT id FROM " QG_TABLE " WHERE (%s) IS NULL " + "ORDER BY id", + pred.data); + + require_same_rows(tc, conn, lhs.data, rhs.data); + + pfree(pred.data); + pfree(lhs.data); + pfree(rhs.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 5fa955220e929..7c5318d578ce2 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_planadvice query_norec); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From dd6baa0ee7cafcd872d22b84f757807829e4c2ef Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Tue, 13 Oct 2026 09:06:15 -0400 Subject: [PATCH 256/338] prop: extend NoREC and plan-advice query areas Add a NoREC property that pushes the generated predicate across a self-join (join_filter_matches_norec), comparing the optimisable filtered join against the un-optimisable twin that evaluates the predicate per surviving join row -- the join/pushdown interaction where NoREC found real optimiser bugs elsewhere. Add a plan-advice property that steers a three-way self-join to a specific JOIN_ORDER (join_order_plan_invariant); join reordering is a distinct code path from join-method selection and the result must not depend on it. --- src/test/prop/sql/prop_query_norec.c | 63 +++++++++++++++++++++++ src/test/prop/sql/prop_query_planadvice.c | 38 ++++++++++++++ 2 files changed, 101 insertions(+) diff --git a/src/test/prop/sql/prop_query_norec.c b/src/test/prop/sql/prop_query_norec.c index 2a811a9a1a036..9373f74ccee18 100644 --- a/src/test/prop/sql/prop_query_norec.c +++ b/src/test/prop/sql/prop_query_norec.c @@ -162,6 +162,69 @@ PG_PROP(count_false_matches_norec) pg_prop_disconnect(conn); } +/* + * NoREC across a join: the optimisable form filters the inner relation of a + * self-join with p in the WHERE clause, where the planner may push p below + * the join or turn it into an index qual. The un-optimisable twin forms the + * full (unfiltered) join and counts (p IS TRUE) from the target list, so the + * predicate cannot be pushed down or indexed. Both count the join rows whose + * left side satisfies p. Join-filter interaction with pushdown is exactly + * where NoREC found real optimiser bugs in other engines. + */ +PG_PROP(join_filter_matches_norec) +{ + pg_prop_conn *conn; + char *err = NULL; + int nrows; + StringInfoData pred; + StringInfoData opt; + StringInfoData noropt; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_build_table(tc, conn, &nrows); + + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + /* + * Optimisable: p applied to t in the WHERE of a t-JOIN-u self-join on a + * fixed inner row (u.id = 1), so the join cardinality is just the number + * of t rows satisfying p when such a u row exists, else zero. p uses + * unqualified names so it is evaluated inside a single-table subquery over + * t where they are unambiguous. + */ + initStringInfo(&opt); + appendStringInfo(&opt, + "SELECT count(*) FROM " + "(SELECT * FROM " QG_TABLE " WHERE (%s)) t " + "JOIN " QG_TABLE " u ON u.id = 1", + pred.data); + + /* + * Un-optimisable twin: the full join with no filter, counting (p IS TRUE) + * pulled into the projection so it is evaluated per surviving join row + * with no pushdown available. + */ + initStringInfo(&noropt); + appendStringInfo(&noropt, + "SELECT count(*) FROM " + "(SELECT (t.p) AS keep FROM " + "(SELECT *, (%s) AS p FROM " QG_TABLE ") t " + "JOIN " QG_TABLE " u ON u.id = 1) s " + "WHERE s.keep IS TRUE", + pred.data); + + require_same_scalar(tc, conn, opt.data, noropt.data); + + pfree(pred.data); + pfree(opt.data); + pfree(noropt.data); + pg_prop_disconnect(conn); +} + /* * NoREC over a sum: sum(a) WHERE p vs sum of a*(p IS TRUE)::int over a bare * scan. Exercises aggregate pushdown interacting with the filter. coalesce diff --git a/src/test/prop/sql/prop_query_planadvice.c b/src/test/prop/sql/prop_query_planadvice.c index ba36fb0f68835..654c48184862f 100644 --- a/src/test/prop/sql/prop_query_planadvice.c +++ b/src/test/prop/sql/prop_query_planadvice.c @@ -235,6 +235,44 @@ PG_PROP(join_plan_invariant) pg_prop_disconnect(conn); } +/* + * A three-way self-join steered to a specific join order. The planner is + * free to associate and commute the joins however it costs cheapest; forcing + * a particular JOIN_ORDER must not change the rows. Join reordering is a + * distinct code path from join-method selection and has its own history of + * wrong-results bugs (outer-join reordering, equivalence-class handling), so + * it is worth steering explicitly. Aliases x, y, z are joined on their keys; + * the advice names a left-deep order over them. + */ +PG_PROP(join_order_plan_invariant) +{ + pg_prop_conn *conn; + int nrows; + StringInfoData q; + + conn = connect_advice(tc); + qg_build_table(tc, conn, &nrows); + + /* + * x JOIN y ON y.id = x.a joins on a possibly-NULL, non-unique key (so the + * join is interesting), then z on z.id = x.b. All keys are coalesced into + * the id range only implicitly; non-matching rows simply drop, which is + * fine -- the invariant is that the surviving set is order-independent. + */ + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT x.id, y.id, z.id FROM " QG_TABLE " x " + "JOIN " QG_TABLE " y ON y.id = x.a " + "JOIN " QG_TABLE " z ON z.id = x.b " + "ORDER BY x.id, y.id, z.id"); + + require_plan_invariant(tc, conn, q.data, + "JOIN_ORDER((x y) z)", ""); + + pfree(q.data); + pg_prop_disconnect(conn); +} + /* * DISTINCT with ORDER BY ... LIMIT, run under a tiny work_mem to force the * sort/limit and hash-distinct spill paths. ORDER BY the primary key keeps From 7b3579dfff6a86a891bd596f9ffe9fe3d69860f9 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Tue, 13 Oct 2026 09:29:50 -0400 Subject: [PATCH 257/338] prop: steer a merge join in the plan-advice area, measure branch coverage Add query_planadvice/mergejoin_plan_invariant: the same self-join steered to MERGE_JOIN_PLAIN, whose sorted-input lockstep path is distinct from the nested-loop and hash paths and has its own NULL-key handling. On a branch-enabled coverage build this takes executor/nodeMergejoin.c from 0 percent (never entered) to 57.6 percent lines, 63.6 percent functions, 29.5 percent branches with the plan-advice area alone. Record the function and branch coverage of the query-level areas in DESIGN; branch coverage is the metric that surfaced the merge-join gap in the first place. --- src/test/prop/DESIGN | 29 ++++++++++++++++++ src/test/prop/sql/prop_query_planadvice.c | 36 +++++++++++++++++++++++ 2 files changed, 65 insertions(+) diff --git a/src/test/prop/DESIGN b/src/test/prop/DESIGN index 2ce8f06779f86..65d6cead11e58 100644 --- a/src/test/prop/DESIGN +++ b/src/test/prop/DESIGN @@ -349,6 +349,35 @@ To reproduce the measurement, build with -Db_coverage=true, zero the counters for the other suite, and compare the per-line DA: counts and the covered-line sets between the two tracefiles. +Function and branch coverage of the query-level areas. The query-generation +areas (sql/) are measured separately because their value is executing planner +and executor paths, where branch coverage is the metric that matters -- a wrong +turn at a branch is exactly how a plan produces wrong rows. On a branch-enabled +coverage build (lcov --rc branch_coverage=1), the seven query areas alone, +measured over src/backend/{optimizer,executor,parser} and nbtree, reach: + + subtree lines functions branches + optimizer + executor + parser 29.2% 42.3% 20.1% + +and drive the specific files the oracles target: + + file lines functions branches + optimizer/path/clausesel.c 85.2% 100% 72.4% + optimizer/prep/prepunion.c 77.3% 91.7% 55.8% + optimizer/plan/planner.c 43.8% 59.7% 31.0% + executor/nodeAgg.c 53.8% 61.1% 36.4% + executor/nodeNestloop.c 69.2% 75.0% 47.2% + executor/nodeHashjoin.c 35.0% 35.3% 14.3% + executor/nodeMergejoin.c 57.6% 63.6% 29.5% + +clausesel.c and prepunion.c are high precisely because the predicate generator +and the TLP/distinct set-operation oracles hammer them. nodeMergejoin.c was at +0% until a plan-advice property steering MERGE_JOIN_PLAIN was added -- branch +coverage, not line or function, is what surfaced that the file was never entered +at all, and the single new property took it to 57.6%/63.6%/29.5%. The remaining +nodeHashjoin.c branches are the spill paths, which the tiny-work_mem variants +reach only intermittently at this table size. + The takeaway for a reviewer: judge this suite by which code it reaches and by the standing differential net, not by a coverage-percentage delta or a depth-per-line multiplier -- the latter does not favor PBT and we do not claim diff --git a/src/test/prop/sql/prop_query_planadvice.c b/src/test/prop/sql/prop_query_planadvice.c index 654c48184862f..284796d6193b0 100644 --- a/src/test/prop/sql/prop_query_planadvice.c +++ b/src/test/prop/sql/prop_query_planadvice.c @@ -235,6 +235,42 @@ PG_PROP(join_plan_invariant) pg_prop_disconnect(conn); } +/* + * The same self-join steered to a merge join (MERGE_JOIN_PLAIN). A merge join + * sorts both inputs and advances them in lockstep, a wholly different executor + * path from nested-loop and hash joins and one with its own NULL-key and tie + * handling; the rows must not change. The equi-join key uses IS NOT DISTINCT + * FROM so NULL keys participate, which is where merge-join bugs tend to hide. + */ +PG_PROP(mergejoin_plan_invariant) +{ + pg_prop_conn *conn; + int nrows; + StringInfoData pred; + StringInfoData q; + + conn = connect_advice(tc); + qg_build_table(tc, conn, &nrows); + + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT t.id, u.id FROM " + "(SELECT * FROM " QG_TABLE " WHERE (%s)) t " + "JOIN " QG_TABLE + " u ON (t.a IS NOT DISTINCT FROM u.b) " + "ORDER BY t.id, u.id", + pred.data); + + require_plan_invariant(tc, conn, q.data, "MERGE_JOIN_PLAIN(u)", ""); + + pfree(pred.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + /* * A three-way self-join steered to a specific join order. The planner is * free to associate and commute the joins however it costs cheapest; forcing From 5e9cfcb176023733fb0467adb515555d703758cb Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Tue, 13 Oct 2026 13:10:39 -0400 Subject: [PATCH 258/338] prop: add extension points and a schema-driven query generator Let an extension participate in the suite without touching core. Two paths: (a) Add its own area -- already supported: a prop_.c with PG_PROP_AREA and PG_PROP() compiled into the pg_prop binary registers itself. (b) Augment an existing area -- new: a core area publishes a named extension point and calls pg_prop_run_hooks() there; an extension registers a contributor with PG_PROP_HOOK(point, fn). The harness gains a small hook registry (pg_prop_register_hook / run_hooks / run_hook_n / hooks_count) alongside the property registry. Rework the SQL query generator around a schema (qg_schema) instead of a hard-coded column list, and publish three points: query_gen.setup (run DDL such as CREATE EXTENSION before each table), query_gen.column (add a typed column), and query_gen.predicate (emit a predicate leaf over it). Core now also generates numeric(5,2) and boolean columns for wider type coverage. An extension that registers those three hooks gets its type exercised by every query-level oracle -- TLP, NoREC, cardinality, PQS, plan-advice -- with no new oracle code. Document the whole mechanism in README (Extensions) and pg_prop.h. --- src/test/prop/README | 33 ++++++ src/test/prop/pg_prop.c | 79 +++++++++++++++ src/test/prop/pg_prop.h | 51 +++++++++- src/test/prop/sql/query_gen.c | 184 ++++++++++++++++++++++++++++------ src/test/prop/sql/query_gen.h | 114 +++++++++++++++------ 5 files changed, 400 insertions(+), 61 deletions(-) diff --git a/src/test/prop/README b/src/test/prop/README index 44d7ae58623f9..7d14be709dc3d 100644 --- a/src/test/prop/README +++ b/src/test/prop/README @@ -207,3 +207,36 @@ the OBJS list in Makefile, and the area list in the matching TAP driver See pg_prop.h for the full generator and assertion API, and DESIGN for the rationale and internals. + + +Extensions +========== + +An extension can take part in the suite in two ways, and may do either or both. + +(a) Add its own area. This is just "add a new area" above with the extension's +name: a prop_.c that defines PG_PROP_AREA and uses PG_PROP(), whose connect +helper runs CREATE EXTENSION IF NOT EXISTS . Every contrib area already in +this tree (hstore, ltree, citext, cube, seg, intarray, pg_trgm, ...) is exactly +this shape. An out-of-tree extension ships the file and arranges for it to be +compiled into the pg_prop binary. + +(b) Augment an existing area through an extension point. A core area may +publish a named point and call pg_prop_run_hooks() there; an extension registers +a contributor with PG_PROP_HOOK("point", fn). The SQL query generator +(sql/query_gen.h) publishes three: + + query_gen.setup run DDL (CREATE EXTENSION) before each generated table; + ctx is the live connection. + query_gen.column add a typed column to the generated table; ctx is the + qg_schema being assembled. + query_gen.predicate emit a predicate leaf over the extension's column; ctx + is a qg_pred_ctx (schema + buffer). + +Registering those three teaches the generator about the extension's type, after +which every query-level oracle -- TLP, NoREC, cardinality, PQS, plan-advice -- +generates tables and queries that use the type and checks all of its invariants +over it, with no new oracle code. sql/prop_query_ext_citext.c is a complete +worked example (it also adds one citext-specific area, to show both styles in +one file). A core area makes itself extensible simply by publishing a point and +documenting the ctx type its contributors receive. diff --git a/src/test/prop/pg_prop.c b/src/test/prop/pg_prop.c index 9b089777e7855..063087b088b78 100644 --- a/src/test/prop/pg_prop.c +++ b/src/test/prop/pg_prop.c @@ -85,6 +85,85 @@ pg_prop_register(const char *area, const char *name, pg_prop_fn fn, void *arg) nprops++; } +/* ---- extension-point (hook) registry --------------------------------- * + * + * An extension that wants to augment an existing area -- rather than add a + * whole new area of its own -- registers a hook against a named extension + * point that the area publishes. The area calls pg_prop_run_hooks() at the + * point, and every registered contributor runs in registration order with the + * area's context pointer. This is how, for example, a contrib type can teach + * the SQL query generator to emit predicates over its own columns, so the + * existing TLP/NoREC/cardinality/PQS oracles then cover that type for free. + * + * Hooks are a flat list keyed by point name; areas and extensions agree on + * the point name and the concrete type behind the void *ctx out of band (in + * the point's documentation), exactly as a C callback contract always does. + */ +typedef struct HookEntry +{ + const char *point; + pg_prop_hook_fn fn; +} HookEntry; + +#define MAX_HOOKS 1024 +static HookEntry hooks[MAX_HOOKS]; +static int nhooks = 0; + +void +pg_prop_register_hook(const char *point, pg_prop_hook_fn fn) +{ + if (nhooks >= MAX_HOOKS) + { + fprintf(stderr, "pg_prop: too many hooks (max %d)\n", MAX_HOOKS); + exit(2); + } + hooks[nhooks].point = point; + hooks[nhooks].fn = fn; + nhooks++; +} + +int +pg_prop_hooks_count(const char *point) +{ + int n = 0; + + for (int i = 0; i < nhooks; i++) + if (strcmp(hooks[i].point, point) == 0) + n++; + return n; +} + +void +pg_prop_run_hooks(pg_prop_tc *tc, const char *point, void *ctx) +{ + for (int i = 0; i < nhooks; i++) + if (strcmp(hooks[i].point, point) == 0) + hooks[i].fn(tc, point, ctx); +} + +/* + * Run the Nth hook registered against "point" (0-based over just that point's + * contributors), or do nothing if n is out of range. Lets an area pick one + * contributor at random -- e.g. the query generator choosing one extension + * type for a column -- rather than running all of them. + */ +void +pg_prop_run_hook_n(pg_prop_tc *tc, const char *point, int n, void *ctx) +{ + int seen = 0; + + for (int i = 0; i < nhooks; i++) + if (strcmp(hooks[i].point, point) == 0) + { + if (seen == n) + { + hooks[i].fn(tc, point, ctx); + return; + } + seen++; + } +} + /* ---- per-case handle ------------------------------------------------- */ /* diff --git a/src/test/prop/pg_prop.h b/src/test/prop/pg_prop.h index 91669e023622c..d541fe74178fd 100644 --- a/src/test/prop/pg_prop.h +++ b/src/test/prop/pg_prop.h @@ -92,10 +92,55 @@ extern void pg_prop_register(const char *area, const char *name, static void ident(pg_prop_tc *tc, void *arg) /* - * pg_attribute_constructor runs a function before main(). c.h does not define - * one, so define it here (GCC/Clang only, which is all the platforms libhegel - * ships a prebuilt engine for). + * Extension points (hooks). + * + * Two ways an extension can participate in this suite: + * + * (a) Add a whole new area. Compile a prop_.c into the pg_prop binary + * (see src/test/prop/README, "Extensions") that defines PG_PROP_AREA to + * the extension's name and uses PG_PROP() exactly as a core area does. + * Nothing else is needed: the constructor registers it and it appears as + * its own subcommand. CREATE EXTENSION in the area's connect helper. + * + * (b) Augment an existing area. A core area may publish a named extension + * point and call pg_prop_run_hooks() at it; an extension registers a + * contributor with PG_PROP_HOOK(point). The contributor receives the + * point name and a void *ctx whose concrete type is documented by the + * point. For example, the SQL query generator (sql/query_gen.h) + * publishes "query_gen.column" and "query_gen.predicate" so an + * extension type can add columns and predicate leaves to the generated + * tables and queries, after which every query-level oracle (TLP, NoREC, + * cardinality, PQS, plan-advice) covers that type with no further work. + * + * A hook contributor: + * + * static void my_ext_cols(pg_prop_tc *tc, const char *point, void *ctx) + * { + * qg_schema *s = ctx; // concrete type for this point + * qg_add_column(s, "ck", "citext"); + * } + * PG_PROP_HOOK("query_gen.column", my_ext_cols); */ +typedef void (*pg_prop_hook_fn) (pg_prop_tc *tc, const char *point, void *ctx); + +extern void pg_prop_register_hook(const char *point, pg_prop_hook_fn fn); +extern int pg_prop_hooks_count(const char *point); +extern void pg_prop_run_hooks(pg_prop_tc *tc, const char *point, void *ctx); +extern void pg_prop_run_hook_n(pg_prop_tc *tc, const char *point, int n, + void *ctx); + +/* + * Register a hook contributor before main(). "point" is the extension-point + * name; "fn" runs when the publishing area reaches that point. Place these at + * file scope in an extension's prop_.c. + */ +#define PG_PROP_HOOK(point, fn) \ + static void pg_prop_hookreg_##fn(void) pg_attribute_constructor; \ + static void pg_prop_hookreg_##fn(void) \ + { \ + pg_prop_register_hook((point), (fn)); \ + } \ + struct pg_prop_hook_force_semicolon_##fn #ifndef pg_attribute_constructor #define pg_attribute_constructor __attribute__((constructor)) #endif diff --git a/src/test/prop/sql/query_gen.c b/src/test/prop/sql/query_gen.c index 005e48fe07839..d033b9975b2a9 100644 --- a/src/test/prop/sql/query_gen.c +++ b/src/test/prop/sql/query_gen.c @@ -5,9 +5,16 @@ * * See query_gen.h for the contract. The generator produces well-typed, * parseable SQL text only; it never decides what is correct -- that is each - * area's oracle. Keeping generation here means the TLP, NoREC and GUC-flip - * areas all draw from the same distribution of tables and predicates, so a - * counterexample found by one is reproducible shape-for-shape in the others. + * area's oracle. Keeping generation here means the TLP, NoREC, cardinality, + * PQS and plan-advice areas all draw from the same distribution of tables and + * predicates, so a counterexample found by one is reproducible shape-for-shape + * in the others. + * + * The table shape is a schema assembled once per process: core registers a + * fixed set of typed columns, and the "query_gen.column" / "query_gen.predicate" + * extension points let a contrib type add its own columns and predicate leaves + * (see pg_prop.h). Everything downstream -- row generation, predicate drawing, + * the oracles -- is schema-driven, so adding a type needs no oracle changes. * * Copyright (c) 2026, PostgreSQL Global Development Group * @@ -24,7 +31,9 @@ #define QG_VAL_MIN (-5) #define QG_VAL_MAX 5 -/* Draw an int in [QG_VAL_MIN, QG_VAL_MAX] as SQL text, ~15% NULL. */ +/* ---- core column value generators ------------------------------------ */ + +/* int in [QG_VAL_MIN, QG_VAL_MAX] or SQL NULL, ~15% NULL. */ static void draw_int_or_null(pg_prop_tc *tc, StringInfo buf) { @@ -35,7 +44,7 @@ draw_int_or_null(pg_prop_tc *tc, StringInfo buf) (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX)); } -/* Draw a short [a-c] string literal or SQL NULL, ~15% NULL. */ +/* short [a-c] string literal or SQL NULL, ~15% NULL. */ static void draw_text_or_null(pg_prop_tc *tc, StringInfo buf) { @@ -54,7 +63,7 @@ draw_text_or_null(pg_prop_tc *tc, StringInfo buf) appendStringInfoChar(buf, '\''); } -/* Draw a float8 literal (small, one decimal) or SQL NULL, ~15% NULL. */ +/* float8 literal (small, one decimal) or SQL NULL, ~15% NULL. */ static void draw_float_or_null(pg_prop_tc *tc, StringInfo buf) { @@ -66,31 +75,116 @@ draw_float_or_null(pg_prop_tc *tc, StringInfo buf) (int) pg_prop_draw_int(tc, 0, 9)); } +/* numeric literal with a scale, or SQL NULL -- exercises exact decimal paths. */ +static void +draw_numeric_or_null(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d.%02d", + (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), + (int) pg_prop_draw_int(tc, 0, 99)); +} + +/* boolean literal or SQL NULL. */ +static void +draw_bool_or_null(pg_prop_tc *tc, StringInfo buf) +{ + int k = (int) pg_prop_draw_int(tc, 0, 2); + + appendStringInfoString(buf, k == 0 ? "NULL" : k == 1 ? "true" : "false"); +} + +/* ---- schema assembly ------------------------------------------------- */ + +void +qg_add_column(qg_schema *s, const char *name, const char *sqltype, + qg_value_fn draw, bool is_text_like, bool is_numeric_like) +{ + if (s->ncols >= QG_MAX_COLS) + return; /* stay bounded; silently drop extras */ + s->cols[s->ncols].name = name; + s->cols[s->ncols].sqltype = sqltype; + s->cols[s->ncols].draw = draw; + s->cols[s->ncols].is_text_like = is_text_like; + s->cols[s->ncols].is_numeric_like = is_numeric_like; + s->ncols++; +} + +/* + * The process-wide schema, built lazily on first use. id (the primary key) + * is implicit and not in cols[]. Core columns a, b, c, f are kept with their + * historical names and types so PQS and the hand-written join predicates stay + * valid; d (numeric) and g (bool) widen the type coverage. + */ +static qg_schema the_schema; +static bool schema_built = false; + +static const qg_schema * +build_schema(pg_prop_tc *tc) +{ + if (schema_built) + return &the_schema; + + the_schema.ncols = 0; + qg_add_column(&the_schema, "a", "int", draw_int_or_null, false, true); + qg_add_column(&the_schema, "b", "int", draw_int_or_null, false, true); + qg_add_column(&the_schema, "c", "text", draw_text_or_null, true, false); + qg_add_column(&the_schema, "f", "float8", draw_float_or_null, false, true); + qg_add_column(&the_schema, "d", "numeric(5,2)", draw_numeric_or_null, false, true); + qg_add_column(&the_schema, "g", "boolean", draw_bool_or_null, false, false); + + /* let extensions add their own columns */ + pg_prop_run_hooks(tc, "query_gen.column", &the_schema); + + schema_built = true; + return &the_schema; +} + +const qg_schema * +qg_current_schema(pg_prop_tc *tc) +{ + return build_schema(tc); +} + +/* ---- table build ----------------------------------------------------- */ + void qg_build_table(pg_prop_tc *tc, pg_prop_conn *conn, int *nrows) { char *err = NULL; StringInfoData ddl; + const qg_schema *s = build_schema(tc); int n; int r; + int col; + + /* + * Let extensions run one-time-per-case setup DDL (typically CREATE + * EXTENSION) so any types their columns use exist before the table is + * built. The ctx is the live connection; a contributor runs idempotent + * DDL (CREATE EXTENSION IF NOT EXISTS). + */ + pg_prop_run_hooks(tc, "query_gen.setup", conn); initStringInfo(&ddl); appendStringInfoString(&ddl, "DROP TABLE IF EXISTS " QG_TABLE "; " - "CREATE TABLE " QG_TABLE - " (id int PRIMARY KEY, a int, b int, c text, f float8); "); + "CREATE TABLE " QG_TABLE " (id int PRIMARY KEY"); + for (col = 0; col < s->ncols; col++) + appendStringInfo(&ddl, ", %s %s", s->cols[col].name, s->cols[col].sqltype); + appendStringInfoString(&ddl, "); "); n = (int) pg_prop_draw_int(tc, 0, QG_MAX_ROWS); for (r = 1; r <= n; r++) { - appendStringInfo(&ddl, "INSERT INTO " QG_TABLE " VALUES (%d, ", r); - draw_int_or_null(tc, &ddl); - appendStringInfoString(&ddl, ", "); - draw_int_or_null(tc, &ddl); - appendStringInfoString(&ddl, ", "); - draw_text_or_null(tc, &ddl); - appendStringInfoString(&ddl, ", "); - draw_float_or_null(tc, &ddl); + appendStringInfo(&ddl, "INSERT INTO " QG_TABLE " VALUES (%d", r); + for (col = 0; col < s->ncols; col++) + { + appendStringInfoString(&ddl, ", "); + s->cols[col].draw(tc, &ddl); + } appendStringInfoString(&ddl, "); "); } @@ -104,30 +198,42 @@ qg_build_table(pg_prop_tc *tc, pg_prop_conn *conn, int *nrows) *nrows = n; } -/* One leaf comparison against a column: "a " or an IS NULL test. */ +/* ---- predicate drawing ----------------------------------------------- */ + +/* Pick a core numeric-like column name (a, b, or f) for a leaf. */ +static const char * +pick_num_col(pg_prop_tc *tc) +{ + int col = (int) pg_prop_draw_int(tc, 0, 2); + + return (col == 0) ? "a" : (col == 1) ? "b" : "f"; +} + +/* One leaf comparison against a column, or an IS NULL test, or an ext leaf. */ static void draw_leaf(pg_prop_tc *tc, StringInfo buf) { static const char *const cmp[] = {"=", "<>", "<", "<=", ">", ">="}; - int kind = (int) pg_prop_draw_int(tc, 0, 6); - int col = (int) pg_prop_draw_int(tc, 0, 2); /* a, b, or f */ - const char *colname = (col == 0) ? "a" : (col == 1) ? "b" : "f"; + int next_kinds = pg_prop_hooks_count("query_gen.predicate") > 0 ? 8 : 7; + int kind = (int) pg_prop_draw_int(tc, 0, next_kinds - 1); + const char *colname; switch (kind) { case 0: /* col IS NULL */ - appendStringInfo(buf, "%s IS NULL", colname); + appendStringInfo(buf, "%s IS NULL", pick_num_col(tc)); break; case 1: /* col IS NOT NULL */ - appendStringInfo(buf, "%s IS NOT NULL", colname); + appendStringInfo(buf, "%s IS NOT NULL", pick_num_col(tc)); break; - case 2: /* c = 'text' / c comparison */ + case 2: /* c comparison against text */ appendStringInfoString(buf, "c "); appendStringInfoString(buf, cmp[pg_prop_draw_int(tc, 0, 5)]); appendStringInfoChar(buf, ' '); draw_text_or_null(tc, buf); break; case 3: /* col IN (list) */ + colname = pick_num_col(tc); appendStringInfo(buf, "%s IN (", colname); { int m = (int) pg_prop_draw_int(tc, 1, 3); @@ -137,7 +243,7 @@ draw_leaf(pg_prop_tc *tc, StringInfo buf) { if (i > 0) appendStringInfoString(buf, ", "); - if (col == 2) + if (strcmp(colname, "f") == 0) draw_float_or_null(tc, buf); else draw_int_or_null(tc, buf); @@ -146,13 +252,35 @@ draw_leaf(pg_prop_tc *tc, StringInfo buf) appendStringInfoChar(buf, ')'); break; case 4: /* col BETWEEN lo AND hi */ - appendStringInfo(buf, "%s BETWEEN %d AND %d", colname, + appendStringInfo(buf, "%s BETWEEN %d AND %d", pick_num_col(tc), (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX)); break; + case 5: /* numeric column d compared, or bool g */ + if (pg_prop_draw_bool(tc)) + appendStringInfo(buf, "d %s %d.%02d", + cmp[pg_prop_draw_int(tc, 0, 5)], + (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), + (int) pg_prop_draw_int(tc, 0, 99)); + else + appendStringInfo(buf, "g IS %s", + pg_prop_draw_bool(tc) ? "TRUE" : "NOT TRUE"); + break; + case 7: /* extension-contributed leaf */ + { + qg_pred_ctx pctx; + int n = pg_prop_hooks_count("query_gen.predicate"); + + pctx.schema = &the_schema; + pctx.buf = buf; + pg_prop_run_hook_n(tc, "query_gen.predicate", + (int) pg_prop_draw_int(tc, 0, n - 1), &pctx); + } + break; default: /* col const */ + colname = pick_num_col(tc); appendStringInfo(buf, "%s %s ", colname, cmp[pg_prop_draw_int(tc, 0, 5)]); - if (col == 2) + if (strcmp(colname, "f") == 0) draw_float_or_null(tc, buf); else draw_int_or_null(tc, buf); @@ -172,9 +300,6 @@ draw_predicate_rec(pg_prop_tc *tc, StringInfo buf, int depth) switch (kind) { - case 0: /* leaf */ - draw_leaf(tc, buf); - break; case 1: /* NOT (sub) */ appendStringInfoString(buf, "NOT ("); draw_predicate_rec(tc, buf, depth + 1); @@ -203,6 +328,7 @@ draw_predicate_rec(pg_prop_tc *tc, StringInfo buf, int depth) void qg_draw_predicate(pg_prop_tc *tc, StringInfo buf) { + build_schema(tc); /* ensure hooks counted before drawing */ draw_predicate_rec(tc, buf, 0); } diff --git a/src/test/prop/sql/query_gen.h b/src/test/prop/sql/query_gen.h index 012551ed02cb2..4b23095e2b92f 100644 --- a/src/test/prop/sql/query_gen.h +++ b/src/test/prop/sql/query_gen.h @@ -9,11 +9,20 @@ * property of the *result set* -- the kind of property that catches planner * and executor wrong-results bugs rather than type-I/O bugs. * - * This header is the generator those areas share: it builds a random table + * This header is the generator those areas share. It builds a random table * and a random boolean predicate over its columns, both as SQL text, so each * area can wrap them in whatever query form its oracle needs (a ternary * partition for TLP, an optimisable/unoptimisable pair for NoREC, the same - * query under flipped planner GUCs for the metamorphic area). + * query under steered plans for the plan-advice area). + * + * The table's shape is a schema (qg_schema) rather than a hard-coded column + * list. Core registers a fixed set of typed columns; an extension can add + * columns of its own types through the "query_gen.column" extension point, and + * predicate leaves over them through "query_gen.predicate" (see pg_prop.h, + * Extension points). Because the oracles only ever read the generated SQL + * text, an extension that teaches the generator about its type immediately + * gets TLP, NoREC, cardinality, PQS and plan-advice coverage for that type + * with no new oracle code. * * Copyright (c) 2026, PostgreSQL Global Development Group * @@ -28,50 +37,97 @@ #include "pg_prop.h" #include "lib/stringinfo.h" -/* - * The generated table. A fixed, legible shape: a few typed columns whose - * domains are small so a counterexample stays readable and a case stays fast, - * but wide enough that AND/OR/NOT predicates over them exercise real - * qualification, join, and NULL-handling paths. - * - * Columns: - * id int PRIMARY KEY (1..n, dense) -- gives an index to (not) use - * a int small domain, ~15% NULL -- the main predicate column - * b int small domain, ~15% NULL - * c text short [a-c] string or NULL - * f float8 small domain or NULL - */ +/* The generated table's name. */ #define QG_TABLE "qg_t" /* Number of rows the generated table holds this case (0 .. QG_MAX_ROWS). */ #define QG_MAX_ROWS 40 +/* Upper bound on columns (core + extension). */ +#define QG_MAX_COLS 16 + +/* + * A column's value generator: append one SQL literal (or NULL) for this + * column to buf, drawing from tc. Must always emit a well-typed, parseable + * literal for the column's declared SQL type. + */ +typedef void (*qg_value_fn) (pg_prop_tc *tc, StringInfo buf); + +typedef struct qg_column +{ + const char *name; /* SQL identifier */ + const char *sqltype; /* SQL type text, e.g. "int", "citext" */ + qg_value_fn draw; /* emits one literal or NULL */ + bool is_text_like; /* comparable with text operators/order */ + bool is_numeric_like; /* usable in arithmetic / numeric compare */ +} qg_column; + +typedef struct qg_schema +{ + qg_column cols[QG_MAX_COLS]; + int ncols; +} qg_schema; + +/* + * Add a column to the schema. Called by core to seed the standard columns and + * by "query_gen.column" hook contributors to add extension columns. Silently + * ignores additions past QG_MAX_COLS (the generator stays bounded). + */ +extern void qg_add_column(qg_schema *s, const char *name, const char *sqltype, + qg_value_fn draw, bool is_text_like, + bool is_numeric_like); + +/* + * Context handed to a "query_gen.predicate" hook contributor via the void + * *ctx argument: the schema in play and the buffer to append a leaf to. A + * contributor appends exactly one boolean leaf predicate referencing one of + * the extension's own columns (which it finds by name in schema). + * + * The "query_gen.setup" extension point is also published by qg_build_table: + * its ctx is the live pg_prop_conn *, and a contributor runs idempotent DDL + * (typically CREATE EXTENSION IF NOT EXISTS) so the types its columns use + * exist before the table is created. It runs once per generated table. + */ +typedef struct qg_pred_ctx +{ + const qg_schema *schema; + StringInfo buf; +} qg_pred_ctx; + /* * Build QG_TABLE (dropping any prior one) and populate it with a random set - * of 0..QG_MAX_ROWS rows. Returns the row count via *nrows. Fails the test - * case (does not return) on any SQL error, since the generator itself must - * never produce invalid DDL/DML. + * of 0..QG_MAX_ROWS rows according to the current schema (core columns plus + * any contributed by "query_gen.column" hooks). Returns the row count via + * *nrows. Fails the test case (does not return) on any SQL error, since the + * generator itself must never produce invalid DDL/DML. */ extern void qg_build_table(pg_prop_tc *tc, pg_prop_conn *conn, int *nrows); /* * Append a random boolean predicate over QG_TABLE's columns to buf. The - * predicate is always well-typed and parseable, mixes AND/OR/NOT, comparisons - * (= <> < <= > >=), IS [NOT] NULL, IN-lists and BETWEEN, and references the - * numeric, text and float columns. Depth is bounded so it always terminates. - * - * The predicate is three-valued (may evaluate to NULL per row): that is the - * point, because the TLP and NoREC oracles are precisely about correct - * three-valued-logic handling in the qualification machinery. + * predicate is always well-typed and parseable, mixes AND/OR/NOT, comparisons, + * IS [NOT] NULL, IN-lists and BETWEEN, and references the generated columns. + * It may be three-valued (NULL per row) -- that is the point, since the TLP + * and NoREC oracles are about correct three-valued-logic handling. Extension + * predicate leaves (from "query_gen.predicate" hooks) are mixed in when any + * are registered. */ extern void qg_draw_predicate(pg_prop_tc *tc, StringInfo buf); /* - * Draw a random scalar expression over QG_TABLE's columns (an integer-typed - * expression: a column, a constant, or an arithmetic combination). Used where - * an area needs an ORDER BY / GROUP BY / projection target rather than a - * boolean. Always well-typed and non-erroring (division is guarded). + * Draw a random integer-typed scalar expression over QG_TABLE's columns (a + * column, a constant, or an arithmetic combination). Used where an area needs + * an ORDER BY / GROUP BY / projection target rather than a boolean. Always + * well-typed and non-erroring (division is guarded). */ extern void qg_draw_int_expr(pg_prop_tc *tc, StringInfo buf); +/* + * The schema currently in effect (built once per process from core columns + + * "query_gen.column" hooks). Exposed so an area or a hook can inspect the + * columns -- for example PQS needs the column names and types to rectify a + * predicate against a pivot row. + */ +extern const qg_schema *qg_current_schema(pg_prop_tc *tc); + #endif /* QUERY_GEN_H */ From 09b996e298f8b38f387e6b62ff500862bf5209b6 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Tue, 13 Oct 2026 13:10:53 -0400 Subject: [PATCH 259/338] prop: add citext worked example for the extension points Add sql/prop_query_ext_citext.c, the reference for how an extension plugs its type into the query generator. It registers all three query_gen hooks (setup: CREATE EXTENSION citext; column: a mixed-case citext column ci; predicate: case-insensitive leaves over ci), after which the generated tables carry a citext column and the generated predicates exercise case-insensitive comparison through every query-level oracle. It also adds one citext-specific area (eq_matches_lower, a differential against lower()-folded text) to show both the new-area and augment-existing-area styles in one file. Verified on a live node: the generated qg_t gains the ci citext column and the oracles run citext predicates (ci =, <>, <, IS NULL) over it; full TAP suite green at 246 Tier-B areas. --- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/sql/prop_query_ext_citext.c | 197 ++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 201 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/sql/prop_query_ext_citext.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index f2ad9c19ff2d6..a4df3e283d582 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -277,7 +277,8 @@ OBJS = \ sql/prop_query_norec.o \ sql/prop_query_cardinality.o \ sql/prop_query_pqs.o \ - sql/prop_query_planadvice.o + sql/prop_query_planadvice.o \ + sql/prop_query_ext_citext.o PG_CPPFLAGS = -I$(libpq_srcdir) -I$(srcdir)/sql PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 281a472ea31bf..3ac697b5b2118 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -283,6 +283,7 @@ prop_query_sources = files( 'sql/prop_query_cardinality.c', 'sql/prop_query_pqs.c', 'sql/prop_query_planadvice.c', + 'sql/prop_query_ext_citext.c', ) pg_prop = executable('pg_prop', diff --git a/src/test/prop/sql/prop_query_ext_citext.c b/src/test/prop/sql/prop_query_ext_citext.c new file mode 100644 index 0000000000000..1a485e43b4424 --- /dev/null +++ b/src/test/prop/sql/prop_query_ext_citext.c @@ -0,0 +1,197 @@ +/*------------------------------------------------------------------------- + * + * prop_query_ext_citext.c + * Worked example: an extension augmenting the SQL query generator. + * + * This file is the reference for how a PostgreSQL extension plugs its own type + * into the query-level property suite without writing a single oracle. It + * does three things, each through a published extension point (see pg_prop.h, + * Extension points, and sql/query_gen.h): + * + * 1. "query_gen.setup" -- CREATE EXTENSION citext before the table is + * built, so the citext type exists. + * 2. "query_gen.column" -- add a citext column (ci) to the generated + * table, with a value generator. + * 3. "query_gen.predicate" -- emit predicate leaves over ci (case-insensitive + * equality and ordering), which the generator mixes + * into the random predicates it builds. + * + * With those three hooks registered, every query-level area -- query_tlp, + * query_tlp_agg, query_tlp_distinct, query_norec, query_cardinality, query_pqs, + * query_planadvice -- now generates tables that contain a citext column and + * predicates that exercise case-insensitive comparison, and checks all of its + * existing invariants over them. A wrong-results bug in citext's comparison + * or hashing under a join, an aggregate, or a steered plan would be caught by + * the same tautological oracles that cover the core types, with no new oracle + * code here. + * + * It also registers one citext-specific area of its own (pattern (a)) to show + * both styles live side by side in one file. + * + * This example is compiled into the pg_prop binary like any other query-gen + * source. A real out-of-tree extension would ship this file and arrange for + * it to be compiled in (see src/test/prop/README, Extensions). + * + * Tier B: runs SQL against the cluster the TAP driver started. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_query_ext_citext.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "query_ext_citext" +#include "pg_prop.h" + +#include "lib/stringinfo.h" +#include "query_gen.h" + +/* ---- (1) setup: make citext available ------------------------------- */ + +static void +ext_citext_setup(pg_prop_tc *tc, const char *point, void *ctx) +{ + pg_prop_conn *conn = ctx; + char *err = NULL; + + (void) point; + if (!pg_prop_sql_exec(conn, "CREATE EXTENSION IF NOT EXISTS citext", &err)) + pg_prop_fail(tc, "citext setup failed: %s", err ? err : ""); +} +PG_PROP_HOOK("query_gen.setup", ext_citext_setup); + +/* ---- (2) column: a citext value generator --------------------------- */ + +/* Draw a short mixed-case [A-Ca-c] citext literal or SQL NULL, ~15% NULL. */ +static void +draw_citext_or_null(pg_prop_tc *tc, StringInfo buf) +{ + int len; + int i; + + if (pg_prop_draw_int(tc, 0, 6) == 0) + { + appendStringInfoString(buf, "NULL"); + return; + } + len = (int) pg_prop_draw_int(tc, 0, 3); + appendStringInfoString(buf, "'"); + for (i = 0; i < len; i++) + { + /* mix upper and lower so case-insensitivity actually matters */ + char base = pg_prop_draw_bool(tc) ? 'A' : 'a'; + + appendStringInfoChar(buf, (char) (base + pg_prop_draw_int(tc, 0, 2))); + } + appendStringInfoString(buf, "'::citext"); +} + +static void +ext_citext_column(pg_prop_tc *tc, const char *point, void *ctx) +{ + qg_schema *s = ctx; + + (void) tc; + (void) point; + qg_add_column(s, "ci", "citext", draw_citext_or_null, + true /* text-like */ , false /* not numeric */ ); +} +PG_PROP_HOOK("query_gen.column", ext_citext_column); + +/* ---- (3) predicate: a case-insensitive leaf over ci ----------------- */ + +static void +ext_citext_predicate(pg_prop_tc *tc, const char *point, void *ctx) +{ + qg_pred_ctx *p = ctx; + StringInfo buf = p->buf; + int kind = (int) pg_prop_draw_int(tc, 0, 3); + + (void) point; + switch (kind) + { + case 0: /* ci = 'literal' (case-insensitive) */ + appendStringInfoString(buf, "ci = "); + draw_citext_or_null(tc, buf); + break; + case 1: /* ci <> 'literal' */ + appendStringInfoString(buf, "ci <> "); + draw_citext_or_null(tc, buf); + break; + case 2: /* ci IS NULL */ + appendStringInfoString(buf, "ci IS NULL"); + break; + default: /* ci < 'literal' (case-insensitive order) */ + appendStringInfoString(buf, "ci < "); + draw_citext_or_null(tc, buf); + break; + } +} +PG_PROP_HOOK("query_gen.predicate", ext_citext_predicate); + +/* ---- (a) a citext-specific area of this extension's own ------------- */ + +static pg_prop_conn * +connect_citext(pg_prop_tc *tc) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + if (!pg_prop_sql_exec(conn, "CREATE EXTENSION IF NOT EXISTS citext", &err)) + pg_prop_fail(tc, "CREATE EXTENSION citext failed: %s", err ? err : ""); + return conn; +} + +/* + * citext equality agrees with lower()-folded text equality: for any two short + * strings, (x::citext = y::citext) must equal (lower(x) = lower(y)). A + * differential oracle -- two independent ways to decide the same question that + * must always agree -- not an invented reference. + */ +PG_PROP(eq_matches_lower) +{ + pg_prop_conn *conn; + StringInfoData x; + StringInfoData y; + StringInfoData sql; + bool failed = false; + char *err = NULL; + char *v; + + conn = connect_citext(tc); + + initStringInfo(&x); + initStringInfo(&y); + draw_citext_or_null(tc, &x); + draw_citext_or_null(tc, &y); + /* strip the ::citext cast suffix for the text-side comparison */ + + initStringInfo(&sql); + appendStringInfo(&sql, + "SELECT (%s = %s) IS NOT DISTINCT FROM " + "(lower(%s::text) = lower(%s::text))", + x.data, y.data, x.data, y.data); + + v = pg_prop_sql_scalar(conn, sql.data, &failed, &err); + if (failed) + pg_prop_fail(tc, "citext eq query failed: %s\n SQL: %s", + err ? err : "(unknown)", sql.data); + if (v == NULL || strcmp(v, "t") != 0) + pg_prop_fail(tc, + "citext equality disagrees with lower() folding\n SQL: %s", + sql.data); + + pg_free(v); + if (err) + pg_free(err); + pfree(x.data); + pfree(y.data); + pfree(sql.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 7c5318d578ce2..e9ca595524633 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From f3f24d249e3b31770181768c5fe9cc0291b2d031 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Tue, 13 Oct 2026 15:42:43 -0400 Subject: [PATCH 260/338] prop: exclude the harness headers from headerscheck/cpluspluscheck pg_prop.h and sql/query_gen.h are internal to the pg_prop test binary and are only ever compiled by that binary's translation units, which add -Isrc/test/prop (and -Isrc/test/prop/sql) to the include path. headerscheck and cpluspluscheck compile every header standalone, so they cannot find query_gen.h's #include "pg_prop.h", and pg_noreturn (the C23 [[noreturn]] attribute) is not accepted by the C++ front end. Both are expected for a non-public, non-standalone test header, so add them to the skip list alongside the other intentionally-non-standalone headers. --- src/tools/pginclude/headerscheck | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck index 785d6f867ad09..e0a1a410afd5f 100755 --- a/src/tools/pginclude/headerscheck +++ b/src/tools/pginclude/headerscheck @@ -150,6 +150,13 @@ do test "$f" = contrib/isn/ISSN.h && continue test "$f" = contrib/isn/UPC.h && continue + # The property-based testing harness headers are only ever included by + # the pg_prop test binary's own translation units, which add + # -Isrc/test/prop (and -Isrc/test/prop/sql) to the include path. They are + # not public API and are not meant to be compiled standalone. + test "$f" = src/test/prop/pg_prop.h && continue + test "$f" = src/test/prop/sql/query_gen.h && continue + test "$f" = src/include/common/unicode_nonspacing_table.h && continue test "$f" = src/include/common/unicode_east_asian_fw_table.h && continue From 565dc251822d19fe539553cf74ad30a5b2285a74 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Tue, 13 Oct 2026 16:09:35 -0400 Subject: [PATCH 261/338] prop: add a multi-table database model to the query generator Extend query_gen with qg_build_db(): a small random *database* of 2..N related tables, alongside the existing single-table qg_build_table (which is unchanged, so the existing areas keep working). Tables carry declared primary and foreign keys and correlated data -- a child's FK column is drawn from the parent's actual dense key range (plus a slice of NULLs), parents built before children -- so joins have a real referential structure for the planner to reorder and unnest. The schema generator mixes column types and cardinalities deliberately: low- cardinality int columns (tiny domain, heavy duplication, grouping/hashagg stress), high-cardinality int columns (near unique, index/sort stress), text, numeric, and TOAST-eligible text whose value length is drawn to land below, around, or above the ~2KB out-of-line threshold at a tunable frequency. Two parameter profiles: a small, fast, deterministic default for the committed TAP run, and a wide/deep profile (many wide tables, up to thousands of rows, TOAST pressure) for the opt-in high-iteration bug hunt. A schema- and alias-aware predicate drawer (qg_draw_predicate_on) emits predicates over a specific table inside a multi-table query. --- src/test/prop/sql/query_gen.c | 386 ++++++++++++++++++++++++++++++++++ src/test/prop/sql/query_gen.h | 70 ++++++ 2 files changed, 456 insertions(+) diff --git a/src/test/prop/sql/query_gen.c b/src/test/prop/sql/query_gen.c index d033b9975b2a9..54b6819535945 100644 --- a/src/test/prop/sql/query_gen.c +++ b/src/test/prop/sql/query_gen.c @@ -358,3 +358,389 @@ qg_draw_int_expr(pg_prop_tc *tc, StringInfo buf) break; } } + +/* ---- multi-table database model (Phase 0) ---------------------------- * + * + * Everything below builds a small random database of related tables for the + * deep oracles. It is independent of the single-table generator above; the + * existing areas do not use it. + */ + +/* A low-cardinality int column: tiny domain, so values repeat heavily. */ +static void +draw_lowcard_or_null(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 9) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, 0, 2)); +} + +/* A high-cardinality int column: wide domain, so values are near-unique. */ +static void +draw_highcard_or_null(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 19) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, 0, 100000)); +} + +/* + * TOAST pressure. toast_profile selects how often this text column's value is + * large enough to be stored out-of-line/compressed (threshold ~2KB): 0 never, + * 1 ~10%, 2 ~50%, 3 always. Large values are a repeated short run so they + * also compress, exercising the compressed-TOAST path; the value is still + * deterministic given the draws so a counterexample replays. + */ +static int qg_toast_profile = 0; + +static void +draw_toast_text_or_null(pg_prop_tc *tc, StringInfo buf) +{ + int roll; + bool big; + int reps; + int i; + + if (pg_prop_draw_int(tc, 0, 9) == 0) + { + appendStringInfoString(buf, "NULL"); + return; + } + roll = (int) pg_prop_draw_int(tc, 0, 9); + switch (qg_toast_profile) + { + case 0: + big = false; + break; + case 1: + big = (roll == 0); + break; + case 2: + big = (roll < 5); + break; + default: + big = true; + break; + } + appendStringInfoChar(buf, '\''); + if (big) + { + /* ~3KB of a repeated unit -> past the 2KB TOAST threshold */ + reps = 600 + (int) pg_prop_draw_int(tc, 0, 400); + for (i = 0; i < reps; i++) + appendStringInfoString(buf, "xy"); + } + else + { + int len = (int) pg_prop_draw_int(tc, 0, 4); + + for (i = 0; i < len; i++) + appendStringInfoChar(buf, (char) ('a' + pg_prop_draw_int(tc, 0, 2))); + } + appendStringInfoChar(buf, '\''); +} + +void +qg_db_params_default(qg_db_params *p) +{ + p->min_tables = 2; + p->max_tables = 3; + p->min_cols = 2; + p->max_cols = 4; + p->min_rows = 0; + p->max_rows = 30; + p->allow_fk = true; + p->allow_triggers = false; + p->toast_profile = 0; + p->cardinality_bias = 0; +} + +void +qg_db_params_deep(qg_db_params *p) +{ + p->min_tables = 3; + p->max_tables = QG_MAX_TABLES; + p->min_cols = 4; + p->max_cols = 32; /* wide tables */ + p->min_rows = 0; + p->max_rows = 2000; /* volume extremes: some empty, some large */ + p->allow_fk = true; + p->allow_triggers = false; /* 0c enables */ + p->toast_profile = 2; + p->cardinality_bias = 0; +} + +/* + * Fill one table's schema with id (implicit PK), an optional FK column, and + * ncols drawn data columns of mixed types and cardinalities. Column names are + * c0, c1, ... Returns the FK column index (or -1) via *fkcol. + */ +static void +build_table_schema(pg_prop_tc *tc, qg_table *t, int ncols, int fkref, + int cardinality_bias) +{ + char *namebuf; + int i; + + t->schema.ncols = 0; + t->fkcol = -1; + t->fkref = fkref; + + if (fkref >= 0) + { + /* FK column references parent's id; correlated at populate time. */ + qg_add_column(&t->schema, "fk", "int", draw_int_or_null, false, true); + t->fkcol = 0; + } + + for (i = 0; i < ncols; i++) + { + int kind = (int) pg_prop_draw_int(tc, 0, 5); + qg_value_fn draw; + const char *type; + bool textlike = false; + bool numlike = false; + + /* a cardinality bias favours low-card columns for grouping stress */ + if (cardinality_bias == 1 && pg_prop_draw_bool(tc)) + kind = 0; + + switch (kind) + { + case 0: /* low-cardinality int */ + draw = draw_lowcard_or_null; + type = "int"; + numlike = true; + break; + case 1: /* high-cardinality int */ + draw = draw_highcard_or_null; + type = "int"; + numlike = true; + break; + case 2: /* small int */ + draw = draw_int_or_null; + type = "int"; + numlike = true; + break; + case 3: /* text */ + draw = draw_text_or_null; + type = "text"; + textlike = true; + break; + case 4: /* numeric */ + draw = draw_numeric_or_null; + type = "numeric(6,2)"; + numlike = true; + break; + default: /* TOAST-eligible text */ + draw = draw_toast_text_or_null; + type = "text"; + textlike = true; + break; + } + namebuf = psprintf("c%d", i); + qg_add_column(&t->schema, namebuf, type, draw, textlike, numlike); + } +} + +qg_db * +qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) +{ + qg_db *db = palloc0(sizeof(qg_db)); + StringInfoData ddl; + char *err = NULL; + int nt; + int ti; + + qg_toast_profile = p->toast_profile; + + nt = (int) pg_prop_draw_int(tc, p->min_tables, p->max_tables); + if (nt < 1) + nt = 1; + if (nt > QG_MAX_TABLES) + nt = QG_MAX_TABLES; + db->ntables = nt; + + initStringInfo(&ddl); + + for (ti = 0; ti < nt; ti++) + { + qg_table *t = &db->tables[ti]; + int ncols; + int fkref; + int col; + + snprintf(t->name, QG_NAME_LEN, "qg_t%d", ti); + + /* a child may reference any earlier table (so parents build first) */ + fkref = -1; + if (p->allow_fk && ti > 0 && pg_prop_draw_bool(tc)) + fkref = (int) pg_prop_draw_int(tc, 0, ti - 1); + + ncols = (int) pg_prop_draw_int(tc, p->min_cols, p->max_cols); + build_table_schema(tc, t, ncols, fkref, p->cardinality_bias); + + /* DROP first (reverse order handled by qg_drop_db; here just reset) */ + appendStringInfo(&ddl, "DROP TABLE IF EXISTS %s CASCADE; ", t->name); + appendStringInfo(&ddl, "CREATE TABLE %s (id int PRIMARY KEY", t->name); + for (col = 0; col < t->schema.ncols; col++) + appendStringInfo(&ddl, ", %s %s", + t->schema.cols[col].name, + t->schema.cols[col].sqltype); + /* declare the FK (references the parent's dense id) */ + if (t->fkref >= 0) + appendStringInfo(&ddl, + ", FOREIGN KEY (fk) REFERENCES %s(id)", + db->tables[t->fkref].name); + appendStringInfoString(&ddl, "); "); + } + + /* populate parents before children (ascending ti == build order) */ + for (ti = 0; ti < nt; ti++) + { + qg_table *t = &db->tables[ti]; + int n = (int) pg_prop_draw_int(tc, p->min_rows, p->max_rows); + int parent_rows = (t->fkref >= 0) ? db->tables[t->fkref].nrows : 0; + int r; + int col; + + t->nrows = n; + for (r = 1; r <= n; r++) + { + appendStringInfo(&ddl, "INSERT INTO %s VALUES (%d", t->name, r); + for (col = 0; col < t->schema.ncols; col++) + { + appendStringInfoString(&ddl, ", "); + if (col == t->fkcol) + { + /* + * Correlated FK: draw from the parent's actual key range + * 1..parent_rows, with a slice of NULLs. (The FK is + * enforced, so no orphans -- a referenced id always + * exists.) If the parent is empty, the only satisfiable + * value is NULL. + */ + if (parent_rows == 0 || pg_prop_draw_int(tc, 0, 4) == 0) + appendStringInfoString(&ddl, "NULL"); + else + appendStringInfo(&ddl, "%d", + (int) pg_prop_draw_int(tc, 1, parent_rows)); + } + else + t->schema.cols[col].draw(tc, &ddl); + } + appendStringInfoString(&ddl, "); "); + } + appendStringInfo(&ddl, "ANALYZE %s; ", t->name); + } + + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "database DDL/DML failed: %s", err ? err : ""); + + pfree(ddl.data); + return db; +} + +void +qg_drop_db(pg_prop_tc *tc, pg_prop_conn *conn, qg_db *db) +{ + StringInfoData ddl; + char *err = NULL; + int ti; + + initStringInfo(&ddl); + /* drop children before parents: reverse build order, CASCADE for safety */ + for (ti = db->ntables - 1; ti >= 0; ti--) + appendStringInfo(&ddl, "DROP TABLE IF EXISTS %s CASCADE; ", + db->tables[ti].name); + (void) pg_prop_sql_exec(conn, ddl.data, &err); + if (err) + pg_free(err); + pfree(ddl.data); + pfree(db); +} + +/* + * Schema- and alias-aware predicate leaf/tree over a specific table. Mirrors + * draw_leaf/draw_predicate_rec but qualifies every column with "alias." and + * only references columns that exist in this table's schema. + */ +static void +draw_leaf_on(pg_prop_tc *tc, StringInfo buf, const qg_table *t, + const char *alias) +{ + static const char *const cmp[] = {"=", "<>", "<", "<=", ">", ">="}; + const qg_column *c; + int ci; + + if (t->schema.ncols == 0) + { + /* no data columns: fall back to a key predicate */ + appendStringInfo(buf, "%s.id %s %d", alias, + cmp[pg_prop_draw_int(tc, 0, 5)], + (int) pg_prop_draw_int(tc, 0, 10)); + return; + } + ci = (int) pg_prop_draw_int(tc, 0, t->schema.ncols - 1); + c = &t->schema.cols[ci]; + + switch ((int) pg_prop_draw_int(tc, 0, 3)) + { + case 0: + appendStringInfo(buf, "%s.%s IS NULL", alias, c->name); + break; + case 1: + appendStringInfo(buf, "%s.%s IS NOT NULL", alias, c->name); + break; + default: + appendStringInfo(buf, "%s.%s %s ", alias, c->name, + cmp[pg_prop_draw_int(tc, 0, 5)]); + if (c->is_text_like) + draw_text_or_null(tc, buf); + else + draw_int_or_null(tc, buf); + break; + } +} + +static void +draw_predicate_on_rec(pg_prop_tc *tc, StringInfo buf, const qg_table *t, + const char *alias, int depth) +{ + int kind = (depth >= 2) ? 0 : (int) pg_prop_draw_int(tc, 0, 4); + + switch (kind) + { + case 1: + appendStringInfoString(buf, "NOT ("); + draw_predicate_on_rec(tc, buf, t, alias, depth + 1); + appendStringInfoChar(buf, ')'); + break; + case 2: + appendStringInfoChar(buf, '('); + draw_predicate_on_rec(tc, buf, t, alias, depth + 1); + appendStringInfoString(buf, " AND "); + draw_predicate_on_rec(tc, buf, t, alias, depth + 1); + appendStringInfoChar(buf, ')'); + break; + case 3: + appendStringInfoChar(buf, '('); + draw_predicate_on_rec(tc, buf, t, alias, depth + 1); + appendStringInfoString(buf, " OR "); + draw_predicate_on_rec(tc, buf, t, alias, depth + 1); + appendStringInfoChar(buf, ')'); + break; + default: + draw_leaf_on(tc, buf, t, alias); + break; + } +} + +void +qg_draw_predicate_on(pg_prop_tc *tc, StringInfo buf, const qg_table *t, + const char *alias) +{ + draw_predicate_on_rec(tc, buf, t, alias, 0); +} diff --git a/src/test/prop/sql/query_gen.h b/src/test/prop/sql/query_gen.h index 4b23095e2b92f..e88e92a1dd63b 100644 --- a/src/test/prop/sql/query_gen.h +++ b/src/test/prop/sql/query_gen.h @@ -130,4 +130,74 @@ extern void qg_draw_int_expr(pg_prop_tc *tc, StringInfo buf); */ extern const qg_schema *qg_current_schema(pg_prop_tc *tc); +/* ---- multi-table database model (Phase 0) ---------------------------- * + * + * The single-table API above builds qg_t and the areas self-join it. The + * deep oracles instead generate a small random *database* -- several related + * tables with declared keys and foreign keys and correlated data -- so the + * planner has real join-order and subquery-unnesting choices to make. This is + * additive: the single-table API is unchanged and the existing areas keep + * using it. + */ +#define QG_MAX_TABLES 12 +#define QG_NAME_LEN 16 + +typedef struct qg_table +{ + char name[QG_NAME_LEN]; /* qg_t0, qg_t1, ... */ + qg_schema schema; /* its columns (id is implicit col -1) */ + int nrows; /* rows populated this case */ + int fkcol; /* schema index of a FK column, or -1 */ + int fkref; /* table index this FK references, or -1 */ +} qg_table; + +typedef struct qg_db +{ + qg_table tables[QG_MAX_TABLES]; + int ntables; +} qg_db; + +/* + * Knobs for qg_build_db. qg_db_params_default() fills a small, fast, fully + * deterministic profile (safe for the committed TAP run); qg_db_params_deep() + * fills the wide/deep/high-volume profile for the opt-in bug hunt. Both are + * then jittered per case by the drawn values within [min,max]. + */ +typedef struct qg_db_params +{ + int min_tables, + max_tables; /* 1 .. QG_MAX_TABLES */ + int min_cols, + max_cols; /* extra columns beyond id/fk */ + int min_rows, + max_rows; /* per-table row count range */ + bool allow_fk; /* declare + correlate FKs between tables */ + bool allow_triggers; /* attach result-neutral triggers (0c) */ + int toast_profile; /* 0 none .. 3 always (0b) */ + int cardinality_bias; /* 0 mixed, 1 favour low-card columns */ +} qg_db_params; + +extern void qg_db_params_default(qg_db_params *p); +extern void qg_db_params_deep(qg_db_params *p); + +/* + * Build a random database per the params (dropping any prior qg_t* tables). + * Parents are built before children so a child's FK column can be drawn from + * the parent's actual key values (correlated data). Returns a palloc'd qg_db + * describing what was built; qg_drop_db drops the tables. Fails the case (does + * not return) on any SQL error -- the generator must never emit invalid SQL. + */ +extern qg_db *qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, + const qg_db_params *p); +extern void qg_drop_db(pg_prop_tc *tc, pg_prop_conn *conn, qg_db *db); + +/* + * Append a random boolean predicate over a specific table's columns to buf, + * qualifying column references with the given alias (so it is unambiguous + * inside a multi-table query). Same grammar as qg_draw_predicate but schema- + * and alias-aware. + */ +extern void qg_draw_predicate_on(pg_prop_tc *tc, StringInfo buf, + const qg_table *t, const char *alias); + #endif /* QUERY_GEN_H */ From af5aef2a98da62693ac38622b4b8179dad3b0e03 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Tue, 13 Oct 2026 16:09:49 -0400 Subject: [PATCH 262/338] prop: add query_multi area over the multi-table database Add sql/prop_query_multi.c, the first area to use qg_build_db. Three oracles over a random FK-related database: TLP over a foreign-key join (the predicate partition reconstructs the join's row multiset), inner-join cardinality is commutative and bounded by the cross product, and a FK equi-join returns exactly the child rows whose FK is non-NULL. All are theorems, so a disagreement is a planner/executor wrong-results bug. Validated: green at 3000 iterations on the default profile and 400 on the deep profile (confirmed firing: up to 11 tables, 22-column tables, and thousands of ~3KB TOAST-stored values per case); full TAP suite green at 247 Tier-B areas. Document the multi-table model in DESIGN. --- src/test/prop/DESIGN | 11 + src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/sql/prop_query_multi.c | 302 +++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 5 files changed, 317 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/sql/prop_query_multi.c diff --git a/src/test/prop/DESIGN b/src/test/prop/DESIGN index 65d6cead11e58..6ea7926f49607 100644 --- a/src/test/prop/DESIGN +++ b/src/test/prop/DESIGN @@ -269,6 +269,17 @@ reference answer, so a disagreement is unambiguously a server bug: honor, so the alternate plan is a targeted legal variant of the same query rather than a global cost distortion. +* query_multi -- the oracles above over a multi-table generated database rather + than a self-joined single table. qg_build_db() creates 2..N related tables + with declared primary and foreign keys and correlated data (a child's FK + column is drawn from the parent's actual key values), so joins have a real + referential structure for the planner to exploit. Properties: TLP over a FK + join, join-cardinality commutativity and the cross-product bound, and that a + FK equi-join returns exactly the non-NULL-FK child rows. The generator has a + small/fast default profile (used by the committed TAP run) and a wide/deep + profile (many wide tables, up to thousands of rows, TOAST-sized columns, low- + versus high-cardinality columns) for the opt-in high-iteration bug hunt. + Build-system integration ======================== diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index a4df3e283d582..2d394b3b0152e 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -278,7 +278,8 @@ OBJS = \ sql/prop_query_cardinality.o \ sql/prop_query_pqs.o \ sql/prop_query_planadvice.o \ - sql/prop_query_ext_citext.o + sql/prop_query_ext_citext.o \ + sql/prop_query_multi.o PG_CPPFLAGS = -I$(libpq_srcdir) -I$(srcdir)/sql PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 3ac697b5b2118..092dc5d95fd4d 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -284,6 +284,7 @@ prop_query_sources = files( 'sql/prop_query_pqs.c', 'sql/prop_query_planadvice.c', 'sql/prop_query_ext_citext.c', + 'sql/prop_query_multi.c', ) pg_prop = executable('pg_prop', diff --git a/src/test/prop/sql/prop_query_multi.c b/src/test/prop/sql/prop_query_multi.c new file mode 100644 index 0000000000000..8d57c75dc83b7 --- /dev/null +++ b/src/test/prop/sql/prop_query_multi.c @@ -0,0 +1,302 @@ +/*------------------------------------------------------------------------- + * + * prop_query_multi.c + * Query-level property tests over a multi-table generated database. + * + * The other query areas generate one table (qg_t) and self-join it. This area + * uses the multi-table model (qg_build_db): a small random database of 2..N + * related tables with declared foreign keys and correlated data, so the + * oracles run over real joins across distinct relations with a real referential + * structure for the planner to exploit. + * + * The oracles are the same tautologies as elsewhere, lifted to the join: + * + * 1. TLP over a join predicate: partitioning the WHERE of a join three ways + * on a generated predicate (p / NOT p / p IS NULL, applied to one side) + * must reconstruct the unpartitioned join's row multiset. + * + * 2. Inner-join cardinality is bounded by the cross product and is symmetric: + * |A JOIN B ON k| == |B JOIN A ON k| and <= |A| * |B|. + * + * 3. A foreign-key inner join never exceeds the child's row count on the + * many-to-one side: every child row matches at most one parent (the FK + * references the parent's primary key), so the equi-join on fk = id + * returns exactly the child rows whose fk is non-NULL. + * + * All are theorems, so any disagreement is a planner/executor wrong-results + * bug, not an oracle miscalibration. + * + * Tier B: runs SQL against the cluster the TAP driver started. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_query_multi.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "query_multi" +#include "pg_prop.h" + +#include "lib/stringinfo.h" +#include "query_gen.h" + +/* Two row-returning queries must render identically (ordered multiset). */ +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *lhs, const char *rhs) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL; + char *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, lhs, &f1, &e1); + if (f1) + pg_prop_fail(tc, "lhs query failed: %s\n SQL: %s", + e1 ? e1 : "(unknown)", lhs); + r2 = pg_prop_sql_rows(conn, rhs, &f2, &e2); + if (f2) + pg_prop_fail(tc, "rhs query failed: %s\n SQL: %s", + e2 ? e2 : "(unknown)", rhs); + + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, + "multi-table mismatch:\n lhs: %s\n rhs: %s\n" + " lhs rows:\n%s\n rhs rows:\n%s", + lhs, rhs, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +static long +count_of(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *v; + long n; + + v = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "count query failed: %s\n SQL: %s", + err ? err : "(unknown)", sql); + n = v ? atol(v) : 0; + if (v) + pg_free(v); + if (err) + pg_free(err); + return n; +} + +/* + * Find a child table (one with a FK) in the generated database, or return -1. + * Many properties need an actual join edge; if the random database produced + * none this case is vacuously satisfied. + */ +static int +find_child(const qg_db *db) +{ + for (int i = 0; i < db->ntables; i++) + if (db->tables[i].fkref >= 0) + return i; + return -1; +} + +/* + * TLP over a join: the join of child to parent, partitioned three ways on a + * predicate over the child, reconstructs the unpartitioned join. + */ +PG_PROP(join_tlp) +{ + pg_prop_conn *conn; + char *err = NULL; + qg_db *db; + qg_db_params p; + int ci; + const qg_table *child; + const qg_table *parent; + StringInfoData pred; + StringInfoData whole; + StringInfoData part; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_db_params_default(&p); + db = qg_build_db(tc, conn, &p); + + ci = find_child(db); + if (ci < 0) + { + qg_drop_db(tc, conn, db); + pg_prop_disconnect(conn); + return; + } + child = &db->tables[ci]; + parent = &db->tables[child->fkref]; + + initStringInfo(&pred); + qg_draw_predicate_on(tc, &pred, child, "ch"); + + /* whole: child JOIN parent ON ch.fk = pa.id, return ch.id ordered */ + initStringInfo(&whole); + appendStringInfo(&whole, + "SELECT ch.id FROM %s ch JOIN %s pa ON ch.fk = pa.id " + "ORDER BY ch.id", + child->name, parent->name); + + /* partitioned: the predicate over ch split three ways, UNION ALL'd */ + initStringInfo(&part); + appendStringInfo(&part, + "SELECT id FROM (" + "SELECT ch.id FROM %s ch JOIN %s pa ON ch.fk = pa.id " + "WHERE (%s) " + "UNION ALL " + "SELECT ch.id FROM %s ch JOIN %s pa ON ch.fk = pa.id " + "WHERE NOT (%s) " + "UNION ALL " + "SELECT ch.id FROM %s ch JOIN %s pa ON ch.fk = pa.id " + "WHERE (%s) IS NULL" + ") s ORDER BY id", + child->name, parent->name, pred.data, + child->name, parent->name, pred.data, + child->name, parent->name, pred.data); + + require_same_rows(tc, conn, whole.data, part.data); + + pfree(pred.data); + pfree(whole.data); + pfree(part.data); + qg_drop_db(tc, conn, db); + pg_prop_disconnect(conn); +} + +/* + * Inner-join cardinality is commutative and bounded by the cross product: + * |A JOIN B ON a.fk = b.id| == |B JOIN A ON a.fk = b.id| and <= |A|*|B|. + */ +PG_PROP(join_card_commutes_and_bounded) +{ + pg_prop_conn *conn; + char *err = NULL; + qg_db *db; + qg_db_params p; + int ci; + const qg_table *child; + const qg_table *parent; + StringInfoData sql; + long ab, + ba, + na, + nb; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_db_params_default(&p); + db = qg_build_db(tc, conn, &p); + + ci = find_child(db); + if (ci < 0) + { + qg_drop_db(tc, conn, db); + pg_prop_disconnect(conn); + return; + } + child = &db->tables[ci]; + parent = &db->tables[child->fkref]; + + initStringInfo(&sql); + appendStringInfo(&sql, + "SELECT count(*) FROM %s a JOIN %s b ON a.fk = b.id", + child->name, parent->name); + ab = count_of(tc, conn, sql.data); + resetStringInfo(&sql); + appendStringInfo(&sql, + "SELECT count(*) FROM %s b JOIN %s a ON a.fk = b.id", + parent->name, child->name); + ba = count_of(tc, conn, sql.data); + + na = child->nrows; + nb = parent->nrows; + + if (ab != ba) + pg_prop_fail(tc, + "join cardinality not commutative: %ld vs %ld\n" + " child=%s parent=%s", ab, ba, child->name, parent->name); + if (ab > na * nb) + pg_prop_fail(tc, + "join exceeds cross product: %ld > %ld*%ld\n" + " child=%s parent=%s", ab, na, nb, child->name, parent->name); + + pfree(sql.data); + qg_drop_db(tc, conn, db); + pg_prop_disconnect(conn); +} + +/* + * A FK equi-join to the parent's primary key returns exactly the child rows + * whose fk is non-NULL (each matches exactly one parent, since fk references a + * unique id and the FK is enforced so no orphans exist). + */ +PG_PROP(fk_join_eq_nonnull_children) +{ + pg_prop_conn *conn; + char *err = NULL; + qg_db *db; + qg_db_params p; + int ci; + const qg_table *child; + const qg_table *parent; + StringInfoData sql; + long joined, + nonnull; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_db_params_default(&p); + db = qg_build_db(tc, conn, &p); + + ci = find_child(db); + if (ci < 0) + { + qg_drop_db(tc, conn, db); + pg_prop_disconnect(conn); + return; + } + child = &db->tables[ci]; + parent = &db->tables[child->fkref]; + + initStringInfo(&sql); + appendStringInfo(&sql, + "SELECT count(*) FROM %s ch JOIN %s pa ON ch.fk = pa.id", + child->name, parent->name); + joined = count_of(tc, conn, sql.data); + resetStringInfo(&sql); + appendStringInfo(&sql, "SELECT count(*) FROM %s WHERE fk IS NOT NULL", + child->name); + nonnull = count_of(tc, conn, sql.data); + + if (joined != nonnull) + pg_prop_fail(tc, + "FK join count %ld != non-NULL child fk count %ld\n" + " child=%s parent=%s", + joined, nonnull, child->name, parent->name); + + pfree(sql.data); + qg_drop_db(tc, conn, db); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index e9ca595524633..e68b523d6bfec 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 9787db26379b53ed640435e8082016e0b1622b40 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Tue, 13 Oct 2026 17:00:56 -0400 Subject: [PATCH 263/338] prop: fix to_char numeric-scale property for rounding carry The to_char_numeric_scale property drew an integer part up to 999999 and checked a format/parse round-trip against round(x, 3). It missed that a fractional part >= .9995 carries into the integer part when rounded, so 999999.9995 rounds to 1000000.000 -- seven integer digits, which overflows the six-position 'FM999999.000' mask and makes to_char emit overflow fill that to_number cannot parse. Cap the drawn integer part at 999998 so the rounded value always fits six integer digits. Surfaced by the full TAP seed stream; property-soundness only, no server bug. --- src/test/prop/prop_toformat.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/src/test/prop/prop_toformat.c b/src/test/prop/prop_toformat.c index fec577f7c94b1..239ec2713d9d5 100644 --- a/src/test/prop/prop_toformat.c +++ b/src/test/prop/prop_toformat.c @@ -280,8 +280,17 @@ PG_PROP(to_char_numeric_scale) if (conn == NULL) pg_prop_fail(tc, "connect failed: %s", err ? err : ""); - /* up to 6 integer digits (fits '999999') and 4 fractional digits */ - ipart = pg_prop_draw_int(tc, 0, 999999); + /* + * Up to 6 integer digits ('FM999999' has six nine-positions) and 4 + * fractional digits. Cap the integer part at 999998 (not 999999): a + * fractional part >= .9995 rounds the thousandths up and carries into the + * integer part, so 999999.9995 would round to 1000000.000 -- seven integer + * digits, which overflows the six-position format and makes to_char emit + * '#' fill that to_number cannot parse. Leaving one digit of headroom + * keeps the rounded value within six integer digits for every fractional + * part. + */ + ipart = pg_prop_draw_int(tc, 0, 999998); fpart = pg_prop_draw_int(tc, 0, 9999); neg = pg_prop_draw_bool(tc); From 4ffd32716e7483d4994094a8198e97cdadb638ce Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Tue, 13 Oct 2026 17:01:58 -0400 Subject: [PATCH 264/338] prop: add TOAST storage modes, triggers, and a deep query builder Extend the multi-table generator in three ways for the deep bug hunt: * TOAST storage: emit SET STORAGE per varlena column, cycling PLAIN / MAIN / EXTERNAL / EXTENDED (TOAST-eligible text columns get EXTERNAL or EXTENDED, never PLAIN), so the out-of-line and compressed-TOAST machinery is exercised in all its modes. * Triggers: optionally attach a result-neutral BEFORE INSERT/UPDATE row trigger (a plpgsql function that returns NEW unchanged) to some tables, so the relation carries trigger metadata and the executor runs the trigger-firing path while SELECT results are unaffected -- keeping the SELECT oracles sound. * qg_build_query(): a recursive query builder that follows the foreign-key graph to assemble a random left-deep join tree (inner and left joins) of a configurable depth, projecting a stable k0,k1,... key list and exposing one mutable predicate over the anchor relation, so a metamorphic oracle can vary just that predicate while the surrounding join stays fixed. The deep profile enables triggers and TOAST pressure; the committed TAP run uses the small, fast, deterministic default profile. --- src/test/prop/sql/query_gen.c | 224 +++++++++++++++++++++++++++++++++- src/test/prop/sql/query_gen.h | 40 ++++++ 2 files changed, 263 insertions(+), 1 deletion(-) diff --git a/src/test/prop/sql/query_gen.c b/src/test/prop/sql/query_gen.c index 54b6819535945..a518b8d0b1d2b 100644 --- a/src/test/prop/sql/query_gen.c +++ b/src/test/prop/sql/query_gen.c @@ -467,7 +467,7 @@ qg_db_params_deep(qg_db_params *p) p->min_rows = 0; p->max_rows = 2000; /* volume extremes: some empty, some large */ p->allow_fk = true; - p->allow_triggers = false; /* 0c enables */ + p->allow_triggers = true; /* result-neutral triggers in the deep hunt */ p->toast_profile = 2; p->cardinality_bias = 0; } @@ -545,6 +545,45 @@ build_table_schema(pg_prop_tc *tc, qg_table *t, int ncols, int fkref, } } +/* + * Emit SET STORAGE for the varlena (text/numeric) columns of a table, cycling + * through PLAIN/MAIN/EXTERNAL/EXTENDED so the TOAST storage machinery is + * exercised in all its modes. PLAIN forbids out-of-line storage (so a large + * value would error) -- so only choose PLAIN for columns the generator keeps + * small; the TOAST-eligible text columns (draw == draw_toast_text_or_null) get + * EXTERNAL (uncompressed out-of-line) or EXTENDED (compressed), never PLAIN. + */ +static void +emit_storage_modes(pg_prop_tc *tc, StringInfo ddl, const qg_table *t) +{ + int col; + + for (col = 0; col < t->schema.ncols; col++) + { + const qg_column *c = &t->schema.cols[col]; + const char *mode; + bool toastable = c->is_text_like || !c->is_numeric_like; + + /* only varlena columns have a meaningful STORAGE setting */ + if (!c->is_text_like && + strncmp(c->sqltype, "numeric", 7) != 0) + continue; + + if (c->draw == draw_toast_text_or_null) + mode = pg_prop_draw_bool(tc) ? "EXTERNAL" : "EXTENDED"; + else + { + static const char *const modes[] = {"PLAIN", "MAIN", + "EXTERNAL", "EXTENDED"}; + + mode = modes[pg_prop_draw_int(tc, 0, 3)]; + } + (void) toastable; + appendStringInfo(ddl, "ALTER TABLE %s ALTER COLUMN %s SET STORAGE %s; ", + t->name, c->name, mode); + } +} + qg_db * qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) { @@ -565,6 +604,19 @@ qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) initStringInfo(&ddl); + /* + * A result-neutral trigger function: it fires on write but returns the + * row unchanged, so it changes nothing a SELECT can observe while still + * forcing the relation to carry trigger metadata and the executor to run + * the trigger-firing path (which disables some optimisations, e.g. + * tuple-routing and HOT shortcuts). Created idempotently. + */ + if (p->allow_triggers) + appendStringInfoString(&ddl, + "CREATE OR REPLACE FUNCTION qg_noop_trig() " + "RETURNS trigger LANGUAGE plpgsql AS " + "$$ BEGIN RETURN NEW; END $$; "); + for (ti = 0; ti < nt; ti++) { qg_table *t = &db->tables[ti]; @@ -595,6 +647,20 @@ qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) ", FOREIGN KEY (fk) REFERENCES %s(id)", db->tables[t->fkref].name); appendStringInfoString(&ddl, "); "); + + /* 0b: vary TOAST storage modes on the varlena columns */ + emit_storage_modes(tc, &ddl, t); + + /* + * 0c: attach a result-neutral BEFORE INSERT/UPDATE row trigger to some + * tables, so the relation carries trigger metadata and the write path + * runs the trigger. Result-neutral, so SELECT oracles stay sound. + */ + if (p->allow_triggers && pg_prop_draw_bool(tc)) + appendStringInfo(&ddl, + "CREATE TRIGGER %s_trig BEFORE INSERT OR UPDATE ON %s " + "FOR EACH ROW EXECUTE FUNCTION qg_noop_trig(); ", + t->name, t->name); } /* populate parents before children (ascending ti == build order) */ @@ -744,3 +810,159 @@ qg_draw_predicate_on(pg_prop_tc *tc, StringInfo buf, const qg_table *t, { draw_predicate_on_rec(tc, buf, t, alias, 0); } + +/* ---- deep query builder (Phase 1) ------------------------------------ */ + +/* + * Build a left-deep join chain starting at table "start", following FK edges: + * each step joins in the table that "start"'s current frontier references (its + * parent) or a child that references the frontier. Aliases are r0, r1, ... + * The chain is at most max_joins long and stops when no further FK edge is + * available. Appends the " r0 JOIN r1 ON ..." text to "from" and + * records each alias's table in used[] / aliases[]. + * + * Returns the number of tables in the chain (>= 1). + */ +static int +build_join_chain(pg_prop_tc *tc, const qg_db *db, int start, int max_joins, + StringInfo from, const qg_table *used[], char aliases[][QG_NAME_LEN]) +{ + int n = 0; + int cur = start; + + used[0] = &db->tables[cur]; + snprintf(aliases[0], QG_NAME_LEN, "r0"); + appendStringInfo(from, "%s r0", db->tables[cur].name); + n = 1; + + while (n <= max_joins) + { + const qg_table *frontier = used[n - 1]; + int nexttab = -1; + bool frontier_is_child = false; + + /* prefer the frontier's own parent (frontier.fk -> parent.id) */ + if (frontier->fkref >= 0) + { + nexttab = frontier->fkref; + frontier_is_child = true; + } + else + { + /* else find any child that references the frontier */ + for (int i = 0; i < db->ntables; i++) + if (db->tables[i].fkref >= 0 && + &db->tables[db->tables[i].fkref] == frontier) + { + nexttab = i; + frontier_is_child = false; + break; + } + } + if (nexttab < 0) + break; + + used[n] = &db->tables[nexttab]; + snprintf(aliases[n], QG_NAME_LEN, "r%d", n); + + /* inner or left join, drawn */ + appendStringInfo(from, " %s JOIN %s %s ON ", + pg_prop_draw_bool(tc) ? "" : "LEFT", + db->tables[nexttab].name, aliases[n]); + if (frontier_is_child) + /* frontier.fk = new.id */ + appendStringInfo(from, "%s.fk = %s.id", aliases[n - 1], aliases[n]); + else + /* new.fk = frontier.id */ + appendStringInfo(from, "%s.fk = %s.id", aliases[n], aliases[n - 1]); + n++; + } + return n; +} + +bool +qg_build_query(pg_prop_tc *tc, const qg_db *db, int max_depth, qg_query *q) +{ + const qg_table *used[QG_MAX_TABLES]; + char aliases[QG_MAX_TABLES][QG_NAME_LEN]; + StringInfoData from; + StringInfoData keys; + StringInfoData sel; + int start; + int nchain; + int maxj; + int i; + + if (db->ntables == 0) + return false; + + start = (int) pg_prop_draw_int(tc, 0, db->ntables - 1); + maxj = (max_depth < 1) ? 1 : max_depth; + if (maxj > db->ntables - 1) + maxj = db->ntables - 1; + if (maxj < 0) + maxj = 0; + + initStringInfo(&from); + nchain = build_join_chain(tc, db, start, maxj, &from, used, aliases); + + /* + * Project each joined table's id as the stable key list, aliased k0, k1, + * ... so the output column names are unique (two relations both have a + * column named id) and an outer query that wraps select_from can order by + * and reference the keys by those names. + */ + initStringInfo(&keys); /* SELECT target list: r0.id AS k0, ... */ + { + StringInfoData keynames; /* k0, k1, ... for ORDER BY / outer refs */ + + initStringInfo(&keynames); + for (i = 0; i < nchain; i++) + { + if (i > 0) + { + appendStringInfoString(&keys, ", "); + appendStringInfoString(&keynames, ", "); + } + appendStringInfo(&keys, "%s.id AS k%d", aliases[i], i); + appendStringInfo(&keynames, "k%d", i); + } + + /* the mutable predicate ranges over the anchor r0 */ + q->pred_table = used[0]; + snprintf(q->pred_alias, QG_NAME_LEN, "%s", aliases[0]); + + /* + * The builder returns the plain join tree: select_from is always + * "SELECT FROM " with the mutable predicate ranging over + * the anchor alias r0. Wrapping in a derived table or CTE is left to + * the oracles that want it (deep_cte_inline builds the inline / CTE / + * materialized forms on top of this), which keeps string assembly free + * of nested-WITH and alias collisions. keys holds the k-named output + * columns, so an outer wrapper orders by and selects them by name. + */ + initStringInfo(&sel); + appendStringInfo(&sel, "SELECT %s FROM %s", keys.data, from.data); + q->select_from = pstrdup(sel.data); + q->keys = pstrdup(keynames.data); + q->order_by = psprintf("ORDER BY %s", keynames.data); + pfree(keynames.data); + } + + pfree(from.data); + pfree(keys.data); + pfree(sel.data); + return true; +} + +void +qg_free_query(qg_query *q) +{ + if (q->select_from) + pfree(q->select_from); + if (q->order_by) + pfree(q->order_by); + if (q->keys) + pfree(q->keys); + q->select_from = q->order_by = q->keys = NULL; +} diff --git a/src/test/prop/sql/query_gen.h b/src/test/prop/sql/query_gen.h index e88e92a1dd63b..d561617fb68bb 100644 --- a/src/test/prop/sql/query_gen.h +++ b/src/test/prop/sql/query_gen.h @@ -200,4 +200,44 @@ extern void qg_drop_db(pg_prop_tc *tc, pg_prop_conn *conn, qg_db *db); extern void qg_draw_predicate_on(pg_prop_tc *tc, StringInfo buf, const qg_table *t, const char *alias); +/* ---- deep query builder (Phase 1) ------------------------------------ * + * + * Compose a legal multi-table SELECT over a qg_db: a join tree across the FK + * graph (inner/left joins), optionally wrapped in a derived-table subquery or + * a CTE, projecting a stable key list and ordered by it. The query is split + * into a fixed "prefix" (everything up to and including the join tree and any + * fixed filters) and a single mutable WHERE predicate over one designated + * alias, so a metamorphic oracle can partition on that predicate (TLP) or pull + * it into an opaque per-row test (NoREC) while the rest stays constant. + * + * qg_build_query fills a qg_query: the three pieces an oracle needs to assemble + * both sides of its comparison without re-parsing SQL text. + */ +typedef struct qg_query +{ + /* + * select_from is "SELECT FROM " with no + * WHERE; pred_alias.pred_col are a column reference the oracle may build a + * predicate over via qg_draw_predicate_on on pred_table; order_by is the + * trailing "ORDER BY ". keys is the projected key list text (for + * count/other rewrites). + */ + char *select_from; /* palloc'd */ + char *order_by; /* palloc'd */ + char *keys; /* palloc'd: the projected key expression list */ + const qg_table *pred_table; /* table the mutable predicate ranges over */ + char pred_alias[QG_NAME_LEN]; /* its alias in the query */ +} qg_query; + +/* + * Build a random query over db with the given maximum join/nesting depth. + * Returns true and fills *q on success; returns false if db is too small to + * build anything interesting (e.g. zero tables), in which case the caller + * treats the case as vacuous. Strings in *q are palloc'd; qg_free_query frees + * them. + */ +extern bool qg_build_query(pg_prop_tc *tc, const qg_db *db, int max_depth, + qg_query *q); +extern void qg_free_query(qg_query *q); + #endif /* QUERY_GEN_H */ From 2b59973c42899c295900c98c067aa5daca776205 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Tue, 13 Oct 2026 17:01:58 -0400 Subject: [PATCH 265/338] prop: add query_deep area (TLP, NoREC, CTE-inlining, const-fold, unnest) Add sql/prop_query_deep.c with five metamorphic oracles over the deep query builder: * deep_tlp -- the mutable predicate partitioned three ways reconstructs the join's rows (TLP over an arbitrary join shape). * deep_norec -- the predicate in WHERE versus wrapped in a CASE optimization barrier must count the same rows. * deep_cte_inline -- the filtered query as an inline subquery, a plain CTE, and a MATERIALIZED CTE must return identical rows (subquery-pullup / CTE-inlining). * deep_const_fold -- ANDing a plan-time-foldable tautology (1=1) and a run-time tautology (col - col = 0) must not change the result (constant folding). * deep_corr_subquery -- a parent-filtered semijoin as IN, as EXISTS, and as a DISTINCT join must agree (subquery unnesting). Validated green at 2000 iterations and under the full TAP seed stream; suite at 248 Tier-B areas. Document the deep builder and oracles in DESIGN. --- src/test/prop/DESIGN | 19 ++ src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/sql/prop_query_deep.c | 428 ++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 5 files changed, 451 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/sql/prop_query_deep.c diff --git a/src/test/prop/DESIGN b/src/test/prop/DESIGN index 6ea7926f49607..1534338427554 100644 --- a/src/test/prop/DESIGN +++ b/src/test/prop/DESIGN @@ -279,6 +279,25 @@ reference answer, so a disagreement is unambiguously a server bug: small/fast default profile (used by the committed TAP run) and a wide/deep profile (many wide tables, up to thousands of rows, TOAST-sized columns, low- versus high-cardinality columns) for the opt-in high-iteration bug hunt. + The deep profile also varies TOAST storage mode per column (PLAIN / MAIN / + EXTERNAL / EXTENDED) and attaches result-neutral BEFORE-row triggers to some + tables (they return the row unchanged, so SELECT oracles stay sound while the + relation still carries trigger metadata and the write path runs the trigger). + +* query_deep -- the oracles over an arbitrary generated query shape, not just a + single join. qg_build_query() walks the foreign-key graph to build a random + left-deep join tree (inner and left joins) of configurable depth, projecting + a stable key list and exposing one mutable predicate over the anchor relation + so a metamorphic oracle can vary just that predicate while the join stays + fixed. Properties: deep_tlp (the predicate partitioned three ways + reconstructs the join), deep_norec (the predicate in WHERE versus the same + predicate wrapped in a CASE barrier must count the same rows), deep_cte_inline + (the filtered query as an inline subquery, a plain CTE, and a MATERIALIZED CTE + must return identical rows -- the subquery-pullup and CTE-inlining rewrites), + deep_const_fold (ANDing a plan-time-foldable tautology (1=1) and a run-time + tautology (col - col = 0) must not change the result -- constant folding), and + deep_corr_subquery (a parent-filtered semijoin written as IN, as EXISTS, and + as a DISTINCT join must agree -- subquery unnesting). Build-system integration diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 2d394b3b0152e..77c6bd35c71bc 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -279,7 +279,8 @@ OBJS = \ sql/prop_query_pqs.o \ sql/prop_query_planadvice.o \ sql/prop_query_ext_citext.o \ - sql/prop_query_multi.o + sql/prop_query_multi.o \ + sql/prop_query_deep.o PG_CPPFLAGS = -I$(libpq_srcdir) -I$(srcdir)/sql PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 092dc5d95fd4d..b2a0c7099fa50 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -285,6 +285,7 @@ prop_query_sources = files( 'sql/prop_query_planadvice.c', 'sql/prop_query_ext_citext.c', 'sql/prop_query_multi.c', + 'sql/prop_query_deep.c', ) pg_prop = executable('pg_prop', diff --git a/src/test/prop/sql/prop_query_deep.c b/src/test/prop/sql/prop_query_deep.c new file mode 100644 index 0000000000000..71af77dc1392b --- /dev/null +++ b/src/test/prop/sql/prop_query_deep.c @@ -0,0 +1,428 @@ +/*------------------------------------------------------------------------- + * + * prop_query_deep.c + * Deep query-level property tests over the multi-table generator. + * + * This area drives the Phase 1 query builder (qg_build_query): a random join + * tree across the generated database's foreign-key graph, sometimes wrapped in + * a derived-table subquery or a CTE, with a single mutable predicate over the + * anchor relation. Each oracle is a metamorphic relation that must hold for + * any such query -- no reference answer, so a disagreement is a planner or + * executor wrong-results bug. + * + * Oracles: + * 1. deep_tlp -- the mutable predicate partitioned three ways (p / NOT p / + * p IS NULL) must reconstruct the unpartitioned query's rows. This is + * TLP applied to an arbitrary generated join/subquery/CTE shape. + * 2. deep_norec -- count(*) with the predicate in WHERE equals the count of + * (predicate IS TRUE) pulled into an opaque per-row projection over the + * same join; the optimizer can push the first form down but not the + * second, so a mismatch is an optimization bug. + * 3. deep_cte_inline -- the same query as a plain subquery, a non-materialized + * CTE, and a MATERIALIZED CTE must all return identical rows; this targets + * the CTE-inlining / subquery-pullup rewrites. + * + * Tier B: runs SQL against the cluster the TAP driver started. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_query_deep.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "query_deep" +#include "pg_prop.h" + +#include "lib/stringinfo.h" +#include "query_gen.h" + +/* Depth of the generated join/nesting (number of FK hops). */ +#define QG_DEEP_MAX 4 + +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *lhs, const char *rhs) +{ + bool f1 = false, + f2 = false; + char *e1 = NULL, + *e2 = NULL; + char *r1, + *r2; + + r1 = pg_prop_sql_rows(conn, lhs, &f1, &e1); + if (f1) + pg_prop_fail(tc, "lhs query failed: %s\n SQL: %s", + e1 ? e1 : "(unknown)", lhs); + r2 = pg_prop_sql_rows(conn, rhs, &f2, &e2); + if (f2) + pg_prop_fail(tc, "rhs query failed: %s\n SQL: %s", + e2 ? e2 : "(unknown)", rhs); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, + "deep-query mismatch:\n lhs: %s\n rhs: %s\n" + " lhs rows:\n%s\n rhs rows:\n%s", + lhs, rhs, r1, r2); + pg_free(r1); + pg_free(r2); + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); +} + +static long +scalar_long(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *v; + long n; + + v = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "scalar query failed: %s\n SQL: %s", + err ? err : "(unknown)", sql); + n = v ? atol(v) : 0; + if (v) + pg_free(v); + if (err) + pg_free(err); + return n; +} + +/* + * Build a database and a deep query; return false (and clean up) if the random + * database was too small to build a query, so the caller returns a vacuous + * pass. On true, *db and *q are owned by the caller (free with qg_free_query + + * qg_drop_db). + */ +static bool +setup(pg_prop_tc *tc, pg_prop_conn **conn_out, qg_db **db_out, qg_query *q) +{ + pg_prop_conn *conn; + char *err = NULL; + qg_db_params p; + qg_db *db; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_db_params_default(&p); + db = qg_build_db(tc, conn, &p); + + if (!qg_build_query(tc, db, QG_DEEP_MAX, q)) + { + qg_drop_db(tc, conn, db); + pg_prop_disconnect(conn); + return false; + } + *conn_out = conn; + *db_out = db; + return true; +} + +/* TLP over an arbitrary generated join/subquery/CTE shape. */ +PG_PROP(deep_tlp) +{ + pg_prop_conn *conn; + qg_db *db; + qg_query q; + StringInfoData pred; + StringInfoData whole; + StringInfoData part; + + if (!setup(tc, &conn, &db, &q)) + return; + + initStringInfo(&pred); + qg_draw_predicate_on(tc, &pred, q.pred_table, q.pred_alias); + + /* whole: the query, unfiltered, ordered */ + initStringInfo(&whole); + appendStringInfo(&whole, "%s %s", q.select_from, q.order_by); + + /* + * partitioned: the same select_from with the predicate added three ways. + * select_from has no WHERE, so appending " WHERE (pred)" is well-formed; + * the three are UNION ALL'd and re-ordered by the key list. + */ + initStringInfo(&part); + appendStringInfo(&part, + "SELECT * FROM (" + "%s WHERE (%s) " + "UNION ALL %s WHERE NOT (%s) " + "UNION ALL %s WHERE (%s) IS NULL" + ") s ORDER BY %s", + q.select_from, pred.data, + q.select_from, pred.data, + q.select_from, pred.data, + q.keys); + + require_same_rows(tc, conn, whole.data, part.data); + + pfree(pred.data); + pfree(whole.data); + pfree(part.data); + qg_free_query(&q); + qg_drop_db(tc, conn, db); + pg_prop_disconnect(conn); +} + +/* NoREC over the deep query: optimizable count vs opaque per-row count. */ +PG_PROP(deep_norec) +{ + pg_prop_conn *conn; + qg_db *db; + qg_query q; + StringInfoData pred; + StringInfoData opt; + StringInfoData noropt; + long c1, + c2; + + if (!setup(tc, &conn, &db, &q)) + return; + + initStringInfo(&pred); + qg_draw_predicate_on(tc, &pred, q.pred_table, q.pred_alias); + + /* optimizable: count of rows of the join where the predicate holds */ + initStringInfo(&opt); + appendStringInfo(&opt, + "SELECT count(*) FROM (%s WHERE (%s)) s", + q.select_from, pred.data); + c1 = scalar_long(tc, conn, opt.data); + + /* + * un-optimizable: the same predicate wrapped in a CASE so the planner + * cannot turn it into an index qual or push it below a join, evaluated in + * the same scope (the WHERE of select_from). The two must count the same + * rows. + */ + initStringInfo(&noropt); + appendStringInfo(&noropt, + "SELECT count(*) FROM " + "(%s WHERE (CASE WHEN (%s) IS TRUE THEN true ELSE false END)) s", + q.select_from, pred.data); + c2 = scalar_long(tc, conn, noropt.data); + + if (c1 != c2) + pg_prop_fail(tc, + "deep NoREC mismatch: %ld vs %ld\n opt: %s\n noropt: %s", + c1, c2, opt.data, noropt.data); + + pfree(pred.data); + pfree(opt.data); + pfree(noropt.data); + qg_free_query(&q); + qg_drop_db(tc, conn, db); + pg_prop_disconnect(conn); +} + +/* + * The same filtered query expressed three ways -- inline subquery, a plain + * (inlinable) CTE, and a MATERIALIZED CTE -- must return identical rows. This + * targets the subquery-pullup and CTE-inlining rewrites: the planner inlines + * the plain CTE and the subquery but must not inline MATERIALIZED, yet all + * three are semantically identical. + */ +PG_PROP(deep_cte_inline) +{ + pg_prop_conn *conn; + qg_db *db; + qg_query q; + StringInfoData pred; + StringInfoData sub; + StringInfoData cte; + StringInfoData mcte; + + if (!setup(tc, &conn, &db, &q)) + return; + + initStringInfo(&pred); + qg_draw_predicate_on(tc, &pred, q.pred_table, q.pred_alias); + + /* inline subquery */ + initStringInfo(&sub); + appendStringInfo(&sub, + "SELECT * FROM (%s WHERE (%s)) s ORDER BY %s", + q.select_from, pred.data, q.keys); + + /* plain CTE (planner may inline) */ + initStringInfo(&cte); + appendStringInfo(&cte, + "WITH c AS (%s WHERE (%s)) " + "SELECT * FROM c s ORDER BY %s", + q.select_from, pred.data, q.keys); + + /* materialized CTE (planner must not inline) */ + initStringInfo(&mcte); + appendStringInfo(&mcte, + "WITH c AS MATERIALIZED (%s WHERE (%s)) " + "SELECT * FROM c s ORDER BY %s", + q.select_from, pred.data, q.keys); + + require_same_rows(tc, conn, sub.data, cte.data); + require_same_rows(tc, conn, sub.data, mcte.data); + + pfree(pred.data); + pfree(sub.data); + pfree(cte.data); + pfree(mcte.data); + qg_free_query(&q); + qg_drop_db(tc, conn, db); + pg_prop_disconnect(conn); +} + +/* + * Constant-folding metamorphic: wrapping the predicate in a form the planner + * can constant-fold must not change the result versus a form it cannot. We + * AND the predicate with a tautology written two ways: (1 = 1), which the + * planner folds away at plan time, and (f(x) = f(x)) over a volatile-looking + * but actually-constant expression it must evaluate per row. Both are always + * TRUE, so both queries must equal the unmodified query. + */ +PG_PROP(deep_const_fold) +{ + pg_prop_conn *conn; + qg_db *db; + qg_query q; + StringInfoData pred; + StringInfoData base; + StringInfoData folded; + StringInfoData unfolded; + + if (!setup(tc, &conn, &db, &q)) + return; + + initStringInfo(&pred); + qg_draw_predicate_on(tc, &pred, q.pred_table, q.pred_alias); + + initStringInfo(&base); + appendStringInfo(&base, "%s WHERE (%s) %s", + q.select_from, pred.data, q.order_by); + + /* AND a plan-time-foldable tautology */ + initStringInfo(&folded); + appendStringInfo(&folded, "%s WHERE ((%s) AND (1 = 1)) %s", + q.select_from, pred.data, q.order_by); + + /* + * AND a run-time tautology the planner cannot fold: abs(hashint4(k0)) >= + * 0 is always true but depends on a per-row value. k0 is the first + * projected key, visible only after projection, so instead use a + * constant-but-opaque expression over the anchor's id: (r0.id - r0.id = 0), + * which references a column so it is evaluated per row yet always holds. + */ + initStringInfo(&unfolded); + appendStringInfo(&unfolded, + "%s WHERE ((%s) AND (%s.id - %s.id = 0)) %s", + q.select_from, pred.data, + q.pred_alias, q.pred_alias, q.order_by); + + require_same_rows(tc, conn, base.data, folded.data); + require_same_rows(tc, conn, base.data, unfolded.data); + + pfree(pred.data); + pfree(base.data); + pfree(folded.data); + pfree(unfolded.data); + qg_free_query(&q); + qg_drop_db(tc, conn, db); + pg_prop_disconnect(conn); +} + +/* + * Correlated-subquery rewrite: for a child/parent FK pair, the child rows that + * have a matching parent are expressible three equivalent ways -- + * + * WHERE fk IN (SELECT id FROM parent) + * WHERE EXISTS (SELECT 1 FROM parent WHERE parent.id = child.fk) + * a semijoin written as a plain JOIN on the (unique) parent id, DISTINCTed + * + * The planner unnests IN and EXISTS into a semijoin; all three must return the + * same child ids. This targets subquery unnesting directly. (The FK is + * enforced, so "has a matching parent" is simply fk IS NOT NULL, but the point + * is that the three *syntactic* forms must agree regardless.) + */ +PG_PROP(deep_corr_subquery) +{ + pg_prop_conn *conn; + char *err = NULL; + qg_db_params p; + qg_db *db; + int ci = -1; + const qg_table *child; + const qg_table *parent; + StringInfoData in_form; + StringInfoData exists_form; + StringInfoData join_form; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_db_params_default(&p); + db = qg_build_db(tc, conn, &p); + + for (int i = 0; i < db->ntables; i++) + if (db->tables[i].fkref >= 0) + { + ci = i; + break; + } + if (ci < 0) + { + qg_drop_db(tc, conn, db); + pg_prop_disconnect(conn); + return; + } + child = &db->tables[ci]; + parent = &db->tables[child->fkref]; + + /* + * Filter the parent side with a generated predicate so the semijoin + * genuinely excludes some children (not just the FK-enforced "fk is not + * NULL"). The same parent predicate is used in all three forms. + */ + { + StringInfoData ppred; + + initStringInfo(&ppred); + qg_draw_predicate_on(tc, &ppred, parent, "pa"); + + initStringInfo(&in_form); + appendStringInfo(&in_form, + "SELECT id FROM %s ch WHERE fk IN " + "(SELECT id FROM %s pa WHERE (%s)) ORDER BY id", + child->name, parent->name, ppred.data); + + initStringInfo(&exists_form); + appendStringInfo(&exists_form, + "SELECT id FROM %s ch WHERE EXISTS " + "(SELECT 1 FROM %s pa WHERE pa.id = ch.fk AND (%s)) " + "ORDER BY id", + child->name, parent->name, ppred.data); + + initStringInfo(&join_form); + appendStringInfo(&join_form, + "SELECT DISTINCT ch.id FROM %s ch JOIN %s pa " + "ON pa.id = ch.fk WHERE (%s) ORDER BY ch.id", + child->name, parent->name, ppred.data); + pfree(ppred.data); + } + + require_same_rows(tc, conn, in_form.data, exists_form.data); + require_same_rows(tc, conn, in_form.data, join_form.data); + + pfree(in_form.data); + pfree(exists_form.data); + pfree(join_form.data); + qg_drop_db(tc, conn, db); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index e68b523d6bfec..d8be6f110187b 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 6c12339f8ae29e7dd24fb2c37814ab2f3d7e7c17 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Tue, 13 Oct 2026 18:29:34 -0400 Subject: [PATCH 266/338] prop: select the generator profile via PG_PROP_DEEP The maximal deep profile (up to 12 wide tables, 2000 rows each, heavy TOAST) is too expensive per case to run a long unattended hunt -- it is ~1s/iteration and can exhaust memory at high iteration. Add a leaner qg_db_params_deep_fast() (a few bounded-size tables, still multi-table with FKs, subqueries, TOAST and triggers) and qg_db_params_for_env(), which the query_deep and query_multi areas now call: PG_PROP_DEEP unset gives the fast deterministic default (the committed TAP run), PG_PROP_DEEP=fast gives the leaner deep profile for a long high-iteration hunt, and PG_PROP_DEEP=full gives the maximal profile for a short heavily-resourced run. One binary serves both the quick suite and the hunt; document the knob in DESIGN. --- src/test/prop/DESIGN | 19 ++++++------ src/test/prop/sql/prop_query_deep.c | 4 +-- src/test/prop/sql/prop_query_multi.c | 6 ++-- src/test/prop/sql/query_gen.c | 43 ++++++++++++++++++++++++++++ src/test/prop/sql/query_gen.h | 11 +++++++ 5 files changed, 70 insertions(+), 13 deletions(-) diff --git a/src/test/prop/DESIGN b/src/test/prop/DESIGN index 1534338427554..800fa89448702 100644 --- a/src/test/prop/DESIGN +++ b/src/test/prop/DESIGN @@ -275,14 +275,17 @@ reference answer, so a disagreement is unambiguously a server bug: column is drawn from the parent's actual key values), so joins have a real referential structure for the planner to exploit. Properties: TLP over a FK join, join-cardinality commutativity and the cross-product bound, and that a - FK equi-join returns exactly the non-NULL-FK child rows. The generator has a - small/fast default profile (used by the committed TAP run) and a wide/deep - profile (many wide tables, up to thousands of rows, TOAST-sized columns, low- - versus high-cardinality columns) for the opt-in high-iteration bug hunt. - The deep profile also varies TOAST storage mode per column (PLAIN / MAIN / - EXTERNAL / EXTENDED) and attaches result-neutral BEFORE-row triggers to some - tables (they return the row unchanged, so SELECT oracles stay sound while the - relation still carries trigger metadata and the write path runs the trigger). + FK equi-join returns exactly the non-NULL-FK child rows. The generator has + three profiles selected by the PG_PROP_DEEP environment variable: unset (the + small, fast, deterministic default the committed TAP run uses); "fast" (a + leaner deep profile -- a few bounded-size tables, still multi-table with FKs, + subqueries, TOAST and triggers -- cheap enough to run tens of thousands of + iterations unattended); and "full" (the maximal profile: many wide tables, up + to thousands of rows, heavy TOAST, for a short, heavily-resourced run). The + deep profiles vary TOAST storage mode per column (PLAIN / MAIN / EXTERNAL / + EXTENDED) and attach result-neutral BEFORE-row triggers to some tables (they + return the row unchanged, so SELECT oracles stay sound while the relation + still carries trigger metadata and the write path runs the trigger). * query_deep -- the oracles over an arbitrary generated query shape, not just a single join. qg_build_query() walks the foreign-key graph to build a random diff --git a/src/test/prop/sql/prop_query_deep.c b/src/test/prop/sql/prop_query_deep.c index 71af77dc1392b..73684a3dca268 100644 --- a/src/test/prop/sql/prop_query_deep.c +++ b/src/test/prop/sql/prop_query_deep.c @@ -112,7 +112,7 @@ setup(pg_prop_tc *tc, pg_prop_conn **conn_out, qg_db **db_out, qg_query *q) if (conn == NULL) pg_prop_fail(tc, "connect failed: %s", err ? err : ""); - qg_db_params_default(&p); + qg_db_params_for_env(&p); db = qg_build_db(tc, conn, &p); if (!qg_build_query(tc, db, QG_DEEP_MAX, q)) @@ -367,7 +367,7 @@ PG_PROP(deep_corr_subquery) if (conn == NULL) pg_prop_fail(tc, "connect failed: %s", err ? err : ""); - qg_db_params_default(&p); + qg_db_params_for_env(&p); db = qg_build_db(tc, conn, &p); for (int i = 0; i < db->ntables; i++) diff --git a/src/test/prop/sql/prop_query_multi.c b/src/test/prop/sql/prop_query_multi.c index 8d57c75dc83b7..9d54b71f15a2a 100644 --- a/src/test/prop/sql/prop_query_multi.c +++ b/src/test/prop/sql/prop_query_multi.c @@ -132,7 +132,7 @@ PG_PROP(join_tlp) if (conn == NULL) pg_prop_fail(tc, "connect failed: %s", err ? err : ""); - qg_db_params_default(&p); + qg_db_params_for_env(&p); db = qg_build_db(tc, conn, &p); ci = find_child(db); @@ -204,7 +204,7 @@ PG_PROP(join_card_commutes_and_bounded) if (conn == NULL) pg_prop_fail(tc, "connect failed: %s", err ? err : ""); - qg_db_params_default(&p); + qg_db_params_for_env(&p); db = qg_build_db(tc, conn, &p); ci = find_child(db); @@ -267,7 +267,7 @@ PG_PROP(fk_join_eq_nonnull_children) if (conn == NULL) pg_prop_fail(tc, "connect failed: %s", err ? err : ""); - qg_db_params_default(&p); + qg_db_params_for_env(&p); db = qg_build_db(tc, conn, &p); ci = find_child(db); diff --git a/src/test/prop/sql/query_gen.c b/src/test/prop/sql/query_gen.c index a518b8d0b1d2b..d83b72c914591 100644 --- a/src/test/prop/sql/query_gen.c +++ b/src/test/prop/sql/query_gen.c @@ -472,6 +472,49 @@ qg_db_params_deep(qg_db_params *p) p->cardinality_bias = 0; } +/* + * A leaner deep profile: still multi-table with FKs, subquery/unnest shapes, + * TOAST and triggers, but with bounded table and row counts so the per-case + * DDL cost and memory stay low enough to run tens of thousands of iterations + * unattended on a modest box. This is the profile the opt-in high-iteration + * hunt should use; qg_db_params_deep() is the "everything maximal" profile for + * a short, heavily-resourced run. + */ +void +qg_db_params_deep_fast(qg_db_params *p) +{ + p->min_tables = 2; + p->max_tables = 6; + p->min_cols = 3; + p->max_cols = 10; + p->min_rows = 0; + p->max_rows = 200; + p->allow_fk = true; + p->allow_triggers = true; + p->toast_profile = 1; /* ~10% large values: TOAST path, less bulk */ + p->cardinality_bias = 0; +} + +/* + * Choose the profile for a query area honouring the PG_PROP_DEEP environment + * variable: unset/0 -> the fast, deterministic default (committed TAP run); + * "fast" -> the leaner deep profile; "full" or any other non-empty value -> + * the maximal deep profile. Areas call this instead of hard-coding a profile + * so the same binary runs the quick suite and the deep hunt. + */ +void +qg_db_params_for_env(qg_db_params *p) +{ + const char *v = getenv("PG_PROP_DEEP"); + + if (v == NULL || v[0] == '\0' || strcmp(v, "0") == 0) + qg_db_params_default(p); + else if (strcmp(v, "fast") == 0) + qg_db_params_deep_fast(p); + else + qg_db_params_deep(p); +} + /* * Fill one table's schema with id (implicit PK), an optional FK column, and * ncols drawn data columns of mixed types and cardinalities. Column names are diff --git a/src/test/prop/sql/query_gen.h b/src/test/prop/sql/query_gen.h index d561617fb68bb..c56f38717d6a2 100644 --- a/src/test/prop/sql/query_gen.h +++ b/src/test/prop/sql/query_gen.h @@ -179,6 +179,17 @@ typedef struct qg_db_params extern void qg_db_params_default(qg_db_params *p); extern void qg_db_params_deep(qg_db_params *p); +extern void qg_db_params_deep_fast(qg_db_params *p); + +/* + * Fill *p from the PG_PROP_DEEP environment variable: unset/0 -> the fast + * deterministic default profile (what the committed TAP run uses); "fast" -> + * the leaner deep profile (bounded rows/tables, still multi-table + subquery + + * TOAST + triggers) for a long unattended high-iteration hunt; "full" (or any + * other value) -> the maximal deep profile for a short, heavily-resourced run. + * Query areas call this so one binary serves both the quick suite and the hunt. + */ +extern void qg_db_params_for_env(qg_db_params *p); /* * Build a random database per the params (dropping any prior qg_t* tables). From d13696074860624f8ab8b5e05aa43535e43155c7 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 08:08:35 -0400 Subject: [PATCH 267/338] prop: add a trait-based index access-method test harness Organize the index tests along access-method traits so the generic part is reusable by any implementation of a trait, in core or out of tree. A trait is a contract any AM claiming it must satisfy -- exact (equality == seqscan), ordered (ranges, ORDER BY by the index, backward scan), summarizing (BRIN-style lossy block ranges never drop a matching row), multicol, include (index-only scans), search_nulls, unique. index_traits.c runs, for each trait the AM actually claims, that trait's differential property (force the plan onto the index, force it onto a seqscan, require identical rows). The harness probes traits at the correct granularity -- pg_indexam_has_property (AM level: can_order/can_unique/can_multi_col/can_include), pg_index_has_property (index level: index_scan/bitmap_scan/backward_scan), and pg_index_column_has_property (column level: search_nulls, orderable) -- so prop_index_am.c encodes no capability matrix: it supplies one am_driver per shipped AM (btree, hash, brin, gin, gist) and the server decides which trait properties run. An out-of-tree AM (e.g. BARK, a B+-tree nbtree successor) reuses the whole applicable suite by adding one driver and declaring its traits in the amhandler. AM-specific behaviour (nbtree skip scan) stays in its own area. Verified the forced plans genuinely use the indexes. --- src/test/prop/sql/index_traits.c | 506 ++++++++++++++++++++++++++++++ src/test/prop/sql/index_traits.h | 88 ++++++ src/test/prop/sql/prop_index_am.c | 80 +++++ src/tools/pginclude/headerscheck | 1 + 4 files changed, 675 insertions(+) create mode 100644 src/test/prop/sql/index_traits.c create mode 100644 src/test/prop/sql/index_traits.h create mode 100644 src/test/prop/sql/prop_index_am.c diff --git a/src/test/prop/sql/index_traits.c b/src/test/prop/sql/index_traits.c new file mode 100644 index 0000000000000..a7a0c853fac12 --- /dev/null +++ b/src/test/prop/sql/index_traits.c @@ -0,0 +1,506 @@ +/*------------------------------------------------------------------------- + * + * index_traits.c + * Implementation of the trait-based index property harness. + * + * See index_traits.h for the trait vocabulary and the design. The core + * oracle throughout is differential: force the plan onto the index, force it + * onto a sequential scan, and require identical rows. An index is a + * performance structure, never a different answer -- for every trait, the only + * thing that changes is which qual/scan shape the trait exercises. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/index_traits.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#include "index_traits.h" + +#define K_MIN (-8) +#define K_MAX 8 +#define IDX_MAX_ROWS 60 + +/* Ask the catalog whether this AM claims a property (asc, can_order, ...). */ +bool +am_claims(pg_prop_tc *tc, pg_prop_conn *conn, const char *amname, + const char *property) +{ + bool failed = false; + char *err = NULL; + char *v; + bool yes; + StringInfoData q; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(pg_indexam_has_property(" + "(SELECT oid FROM pg_am WHERE amname = '%s'), '%s'), false)", + amname, property); + v = pg_prop_sql_scalar(conn, q.data, &failed, &err); + if (failed) + pg_prop_fail(tc, "property probe failed: %s\n SQL: %s", + err ? err : "(unknown)", q.data); + yes = (v != NULL && strcmp(v, "t") == 0); + if (v) + pg_free(v); + if (err) + pg_free(err); + pfree(q.data); + return yes; +} + +/* + * Index-level property probe (index_scan, bitmap_scan, backward_scan, + * clusterable): these are properties of a built index, not the AM, so they are + * queried via pg_index_has_property on an existing index (idxname). NULL + * (not-applicable) counts as false. + */ +static bool +index_claims(pg_prop_tc *tc, pg_prop_conn *conn, const char *idxname, + const char *property) +{ + bool failed = false; + char *err = NULL; + char *v; + bool yes; + StringInfoData q; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(pg_index_has_property('%s'::regclass, '%s'), false)", + idxname, property); + v = pg_prop_sql_scalar(conn, q.data, &failed, &err); + if (failed) + pg_prop_fail(tc, "index property probe failed: %s\n SQL: %s", + err ? err : "(unknown)", q.data); + yes = (v != NULL && strcmp(v, "t") == 0); + if (v) + pg_free(v); + if (err) + pg_free(err); + pfree(q.data); + return yes; +} + +/* + * Column-level property probe (asc, desc, orderable, search_nulls, returnable, + * search_array) on column "col" of an existing index. + */ +static bool +col_claims(pg_prop_tc *tc, pg_prop_conn *conn, const char *idxname, + int col, const char *property) +{ + bool failed = false; + char *err = NULL; + char *v; + bool yes; + StringInfoData q; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT coalesce(pg_index_column_has_property('%s'::regclass, %d, '%s'), false)", + idxname, col, property); + v = pg_prop_sql_scalar(conn, q.data, &failed, &err); + if (failed) + pg_prop_fail(tc, "column property probe failed: %s\n SQL: %s", + err ? err : "(unknown)", q.data); + yes = (v != NULL && strcmp(v, "t") == 0); + if (v) + pg_free(v); + if (err) + pg_free(err); + pfree(q.data); + return yes; +} + +/* Build idx_t(id int PK, k int, v int, pl text) with a random row set. */ +static void +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int n; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS idx_t; " + "CREATE TABLE idx_t (id int PRIMARY KEY, k int, " + "v int, pl text); "); + n = (int) pg_prop_draw_int(tc, 0, IDX_MAX_ROWS); + for (int i = 0; i < n; i++) + { + StringInfoData kv; + + initStringInfo(&kv); + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(&kv, "NULL"); + else + appendStringInfo(&kv, "%d", (int) pg_prop_draw_int(tc, K_MIN, K_MAX)); + appendStringInfo(&ddl, + "INSERT INTO idx_t VALUES (%d, %s, %d, 'p%d'); ", + i, kv.data, (int) pg_prop_draw_int(tc, 0, 3), i); + pfree(kv.data); + } + appendStringInfoString(&ddl, "ANALYZE idx_t;"); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s", err ? err : ""); + pfree(ddl.data); +} + +/* Create idx_t_idx of the driver's AM over its colexpr; returns false on error + * (some AM/column combinations are legitimately unsupported, in which case the + * caller skips that trait rather than failing). */ +static bool +build_index(pg_prop_conn *conn, const am_driver *d, const char *extra) +{ + StringInfoData ddl; + char *err = NULL; + bool ok; + + initStringInfo(&ddl); + appendStringInfo(&ddl, + "DROP INDEX IF EXISTS idx_t_idx; " + "CREATE INDEX idx_t_idx ON idx_t USING %s (%s)%s;", + d->amname, d->colexpr, extra ? extra : ""); + ok = pg_prop_sql_exec(conn, ddl.data, &err); + if (err) + pg_free(err); + pfree(ddl.data); + return ok; +} + +/* + * Core differential: run "select" with the index path forced and again with a + * seqscan forced, and require identical rows. "want_bitmap" forces the bitmap + * path (needed for AMs that produce only bitmap scans, e.g. gin/brin); else + * the plain index scan. Resets GUCs after. + */ +static void +index_eq_seqscan(pg_prop_tc *tc, pg_prop_conn *conn, const char *select, + bool want_bitmap) +{ + char *err = NULL; + char *idx, + *seq; + bool fi = false, + fs = false; + + if (want_bitmap) + pg_prop_sql_exec(conn, + "SET enable_seqscan=off; SET enable_indexscan=off; " + "SET enable_bitmapscan=on", &err); + else + pg_prop_sql_exec(conn, + "SET enable_seqscan=off; SET enable_indexscan=on; " + "SET enable_bitmapscan=off", &err); + idx = pg_prop_sql_rows(conn, select, &fi, &err); + if (fi) + pg_prop_fail(tc, "index-forced query failed: %s\n SQL: %s", + err ? err : "", select); + + pg_prop_sql_exec(conn, + "SET enable_seqscan=on; SET enable_indexscan=off; " + "SET enable_bitmapscan=off", &err); + seq = pg_prop_sql_rows(conn, select, &fs, &err); + if (fs) + pg_prop_fail(tc, "seqscan-forced query failed: %s\n SQL: %s", + err ? err : "", select); + + pg_prop_sql_exec(conn, "RESET ALL", &err); + + if (strcmp(idx, seq) != 0) + pg_prop_fail(tc, + "index != seqscan:\n SQL: %s\n index:\n%s\n seqscan:\n%s", + select, idx, seq); + pg_free(idx); + pg_free(seq); + if (err) + pg_free(err); +} + +/* ---- trait: exact (equality) ---------------------------------------- */ +static void +trait_exact(pg_prop_tc *tc, pg_prop_conn *conn, const am_driver *d, bool bm) +{ + StringInfoData q; + int k = (int) pg_prop_draw_int(tc, K_MIN - 1, K_MAX + 1); + + if (!build_index(conn, d, "")) + return; + initStringInfo(&q); + appendStringInfo(&q, + "SELECT id, k, v FROM idx_t WHERE k = %d ORDER BY id", k); + index_eq_seqscan(tc, conn, q.data, bm); + pfree(q.data); +} + +/* ---- trait: ordered (ranges, ORDER BY, backward) -------------------- */ +static void +trait_ordered(pg_prop_tc *tc, pg_prop_conn *conn, const am_driver *d) +{ + StringInfoData q; + int lo = (int) pg_prop_draw_int(tc, K_MIN - 1, K_MAX + 1); + int hi = (int) pg_prop_draw_int(tc, K_MIN - 1, K_MAX + 1); + int t; + + if (lo > hi) + { + t = lo; + lo = hi; + hi = t; + } + if (!build_index(conn, d, "")) + return; + + /* range qual */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT id, k, v FROM idx_t WHERE k BETWEEN %d AND %d " + "ORDER BY id", lo, hi); + index_eq_seqscan(tc, conn, q.data, false); + + /* open-ended range */ + resetStringInfo(&q); + appendStringInfo(&q, + "SELECT id, k FROM idx_t WHERE k >= %d ORDER BY id", lo); + index_eq_seqscan(tc, conn, q.data, false); + + /* ORDER BY k satisfied by the index (ascending), vs sorted seqscan */ + resetStringInfo(&q); + appendStringInfoString(&q, + "SELECT k FROM idx_t WHERE k IS NOT NULL " + "ORDER BY k, id"); + index_eq_seqscan(tc, conn, q.data, false); + + /* backward: ORDER BY k DESC */ + if (index_claims(tc, conn, "idx_t_idx", "backward_scan")) + { + resetStringInfo(&q); + appendStringInfoString(&q, + "SELECT k FROM idx_t WHERE k IS NOT NULL " + "ORDER BY k DESC, id DESC"); + index_eq_seqscan(tc, conn, q.data, false); + } + pfree(q.data); +} + +/* ---- trait: summarizing (BRIN-style lossy block ranges) ------------- */ +static void +trait_summarizing(pg_prop_tc *tc, pg_prop_conn *conn, const am_driver *d) +{ + StringInfoData q; + int lo = (int) pg_prop_draw_int(tc, K_MIN - 1, K_MAX + 1); + int hi = (int) pg_prop_draw_int(tc, K_MIN - 1, K_MAX + 1); + int t; + + if (lo > hi) + { + t = lo; + lo = hi; + hi = t; + } + /* summarizing AMs only produce bitmap scans with recheck */ + if (!build_index(conn, d, "")) + return; + initStringInfo(&q); + appendStringInfo(&q, + "SELECT id, k, v FROM idx_t WHERE k BETWEEN %d AND %d " + "ORDER BY id", lo, hi); + index_eq_seqscan(tc, conn, q.data, true); + resetStringInfo(&q); + appendStringInfo(&q, + "SELECT id, k FROM idx_t WHERE k = %d ORDER BY id", + (int) pg_prop_draw_int(tc, K_MIN - 1, K_MAX + 1)); + index_eq_seqscan(tc, conn, q.data, true); + pfree(q.data); +} + +/* ---- trait: multicol ------------------------------------------------ */ +static void +trait_multicol(pg_prop_tc *tc, pg_prop_conn *conn, const am_driver *d) +{ + StringInfoData ddl; + StringInfoData q; + char *err = NULL; + int k = (int) pg_prop_draw_int(tc, K_MIN - 1, K_MAX + 1); + + initStringInfo(&ddl); + appendStringInfo(&ddl, + "DROP INDEX IF EXISTS idx_t_idx; " + "CREATE INDEX idx_t_idx ON idx_t USING %s (%s, v);", + d->amname, d->colexpr); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + { + if (err) + pg_free(err); + pfree(ddl.data); + return; + } + pfree(ddl.data); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT id, k, v FROM idx_t WHERE k = %d ORDER BY k, v, id", + k); + index_eq_seqscan(tc, conn, q.data, false); + pfree(q.data); +} + +/* ---- trait: include (index-only scan) ------------------------------- */ +static void +trait_include(pg_prop_tc *tc, pg_prop_conn *conn, const am_driver *d) +{ + StringInfoData ddl; + StringInfoData q; + char *err = NULL; + int k = (int) pg_prop_draw_int(tc, K_MIN - 1, K_MAX + 1); + + initStringInfo(&ddl); + appendStringInfo(&ddl, + "DROP INDEX IF EXISTS idx_t_idx; " + "CREATE INDEX idx_t_idx ON idx_t USING %s (%s) INCLUDE (v); " + "ANALYZE idx_t;", + d->amname, d->colexpr); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + { + if (err) + pg_free(err); + pfree(ddl.data); + return; + } + pfree(ddl.data); + /* index-only scan path (SELECT only indexed+included cols) vs seqscan */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT k, v FROM idx_t WHERE k = %d ORDER BY k, v", k); + index_eq_seqscan(tc, conn, q.data, false); + pfree(q.data); +} + +/* ---- trait: search_nulls -------------------------------------------- */ +static void +trait_search_nulls(pg_prop_tc *tc, pg_prop_conn *conn, const am_driver *d) +{ + StringInfoData q; + + if (!build_index(conn, d, "")) + return; + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT id, k FROM idx_t WHERE k IS NULL ORDER BY id"); + index_eq_seqscan(tc, conn, q.data, false); + resetStringInfo(&q); + appendStringInfoString(&q, + "SELECT id, k FROM idx_t WHERE k IS NOT NULL " + "ORDER BY id"); + index_eq_seqscan(tc, conn, q.data, false); + pfree(q.data); +} + +/* ---- trait: unique -------------------------------------------------- */ +static void +trait_unique(pg_prop_tc *tc, pg_prop_conn *conn, const am_driver *d) +{ + char *err = NULL; + StringInfoData ddl; + bool dup_ok; + + /* + * A unique index must accept a table whose key column is already distinct + * and reject an attempt to insert a duplicate. Build a fresh 3-row table + * with distinct keys, create the unique index (must succeed), then try to + * insert a duplicate key (must fail). + */ + pg_prop_sql_exec(conn, + "DROP TABLE IF EXISTS uq_t; " + "CREATE TABLE uq_t (id int, k int); " + "INSERT INTO uq_t VALUES (1,10),(2,20),(3,30);", &err); + initStringInfo(&ddl); + appendStringInfo(&ddl, + "CREATE UNIQUE INDEX uq_idx ON uq_t USING %s (k);", + d->amname); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + { + /* AM claims can_unique but could not build on distinct keys: a bug */ + pg_prop_fail(tc, "unique index build on distinct keys failed: %s", + err ? err : ""); + } + pfree(ddl.data); + + /* inserting a duplicate must be rejected */ + dup_ok = pg_prop_sql_exec(conn, "INSERT INTO uq_t VALUES (4,10);", &err); + if (dup_ok) + pg_prop_fail(tc, "unique index accepted a duplicate key"); + if (err) + pg_free(err); + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS uq_t;", &err); +} + +void +index_traits_check(pg_prop_tc *tc, const am_driver *d) +{ + pg_prop_conn *conn; + char *err = NULL; + bool has_index_scan; + bool has_bitmap_scan; + bool has_search_nulls; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + if (d->setup_sql && !pg_prop_sql_exec(conn, d->setup_sql, &err)) + pg_prop_fail(tc, "AM setup failed: %s", err ? err : ""); + + build_table(tc, conn); + + /* + * Build one index of the AM over column k so we can probe the index- and + * column-level properties (index_scan, bitmap_scan, search_nulls, ...), + * which are properties of a built index, not of the AM. If the AM cannot + * index a plain int at all, there is nothing to test; return (vacuous). + */ + if (!build_index(conn, d, "")) + { + pg_prop_disconnect(conn); + return; + } + has_index_scan = index_claims(tc, conn, "idx_t_idx", "index_scan"); + has_bitmap_scan = index_claims(tc, conn, "idx_t_idx", "bitmap_scan"); + has_search_nulls = col_claims(tc, conn, "idx_t_idx", 1, "search_nulls"); + + /* + * Every searchable AM answers equality. A bitmap-only AM (no plain index + * scan) must be exercised through the bitmap/recheck path. + */ + if (has_index_scan || has_bitmap_scan) + trait_exact(tc, conn, d, !has_index_scan && has_bitmap_scan); + + /* ordered: declared by the AM (can_order) */ + if (am_claims(tc, conn, d->amname, "can_order")) + trait_ordered(tc, conn, d); + + /* + * summarizing: a bitmap-only, non-ordered AM over a block range (BRIN). + * Lossy, so checked through the recheck (bitmap) path. + */ + if (has_bitmap_scan && !has_index_scan && + !am_claims(tc, conn, d->amname, "can_order")) + trait_summarizing(tc, conn, d); + + if (am_claims(tc, conn, d->amname, "can_multi_col")) + trait_multicol(tc, conn, d); + + if (am_claims(tc, conn, d->amname, "can_include")) + trait_include(tc, conn, d); + + if (has_search_nulls) + trait_search_nulls(tc, conn, d); + + if (am_claims(tc, conn, d->amname, "can_unique")) + trait_unique(tc, conn, d); + + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/sql/index_traits.h b/src/test/prop/sql/index_traits.h new file mode 100644 index 0000000000000..ab171aa2ba1f1 --- /dev/null +++ b/src/test/prop/sql/index_traits.h @@ -0,0 +1,88 @@ +/*------------------------------------------------------------------------- + * + * index_traits.h + * Trait-based property tests for index access methods. + * + * An index access method is described by a set of *traits* -- capabilities it + * declares through its amhandler and exposes at runtime via the catalog + * functions pg_indexam_has_property() / pg_index_column_has_property(). A + * trait is a contract that must hold for ANY access method claiming it: + * + * exact -- answers equality (col = const) with the same rows a + * sequential scan returns. (Every searchable AM.) + * ordered -- can return rows in index order; a forward scan, a backward + * scan, range quals (<, <=, >, >=, BETWEEN) and ORDER BY + * satisfied by the index all agree with a seqscan + sort. + * (nbtree, and any amcanorder AM such as BARK.) + * summarizing -- a lossy, block-range structure (BRIN): it may return extra + * candidate rows but must never drop a matching one, so an + * index scan rechecked against the qual equals a seqscan. + * multicol -- supports multi-column keys; a prefix-equality scan agrees. + * unique -- can enforce uniqueness; a unique index rejects duplicates + * and permits distinct keys. + * include -- supports INCLUDE columns / index-only scans; an index-only + * scan returns the same rows and payload as a heap scan. + * search_nulls -- can search for NULL / NOT NULL; IS NULL and IS NOT NULL + * over the index agree with a seqscan. + * + * The generic trait tests take an am_driver (the AM's name plus how to build + * an index of it over the test column, and any extension it needs) and run, + * for each trait the AM actually claims at runtime, the properties of that + * trait. So a new access method -- BARK, or an extension AM -- gets the whole + * applicable trait suite by supplying one driver and declaring its traits in + * its amhandler; nothing in the generic tests is AM-specific. + * + * AM-specific behaviour (nbtree skip scan, a BRIN-only summary property) lives + * in its own area, not here. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/index_traits.h + * + *------------------------------------------------------------------------- + */ +#ifndef INDEX_TRAITS_H +#define INDEX_TRAITS_H + +#include "pg_prop.h" +#include "lib/stringinfo.h" + +/* + * One access method under test. "amname" is the AM (btree, brin, ...); + * "setup_sql" is DDL run once per connection to make the AM and its opclasses + * available (CREATE EXTENSION ...), or NULL; "index_opts" is appended after + * the column list in CREATE INDEX (e.g. "" for btree, or a storage parameter), + * normally "". "colexpr" is the indexed expression over the generated table's + * column k (usually just "k"); most AMs use "k", but an AM needing an opclass + * spells it here (e.g. "k int4_ops"). + */ +typedef struct am_driver +{ + const char *amname; + const char *setup_sql; /* may be NULL */ + const char *colexpr; /* indexed column/opclass text, e.g. "k" */ +} am_driver; + +/* + * Run every trait property the AM claims (queried at runtime) against a fresh + * generated table. Called from a PG_PROP body; it connects, builds the table + * and the index, consults pg_indexam_has_property for each trait, and checks + * the ones that apply. Fails the case (does not return) on any disagreement. + * + * tc is the test case; d is the AM driver. The property draws the table and + * query constants from tc, so each PG_PROP that calls this is a full property. + */ +extern void index_traits_check(pg_prop_tc *tc, const am_driver *d); + +/* + * The individual trait checks, exposed so an AM-specific area can run just one + * trait (e.g. to exercise it harder) if it wants. Each takes an open + * connection with the table "idx_t" already built and the index "idx_t_idx" + * already created on it, plus the drawn query constants. They are no-ops if + * the AM does not claim the trait. + */ +extern bool am_claims(pg_prop_tc *tc, pg_prop_conn *conn, const char *amname, + const char *property); + +#endif /* INDEX_TRAITS_H */ diff --git a/src/test/prop/sql/prop_index_am.c b/src/test/prop/sql/prop_index_am.c new file mode 100644 index 0000000000000..8b4c1a57619ce --- /dev/null +++ b/src/test/prop/sql/prop_index_am.c @@ -0,0 +1,80 @@ +/*------------------------------------------------------------------------- + * + * prop_index_am.c + * Trait-based property tests across the core index access methods. + * + * One property per shipped access method, each supplying an am_driver to the + * generic trait harness (index_traits.c). The harness queries the AM's traits + * at runtime (pg_indexam_has_property) and runs only the trait properties the + * AM claims, so this file does not encode which AM has which capability -- it + * asks the server. That is also what lets an out-of-tree AM (BARK, an + * extension) reuse the entire applicable suite: supply a driver, declare traits + * in the amhandler, done. + * + * The core AMs that can index a scalar int column: + * btree -- ordered, exact, unique, multicol, include, search_nulls + * hash -- exact only + * brin -- summarizing (lossy block ranges) + * gin -- exact over int via the btree_gin opclass extension + * gist -- ordered/exact over int via the btree_gist opclass extension + * + * spgist is intentionally not here: it ships no integer opclass, so it cannot + * index the generated int column without a custom opclass, and the generic + * trait queries are over that int column. spgist (over the scalar types it + * does support) is a candidate for its own AM-specific area. + * + * AM-specific behaviour that is not a shared trait (nbtree skip scan, etc.) + * lives in its own area (prop_nbtree.c), not here. + * + * Tier B: runs SQL against the cluster the TAP driver started. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_index_am.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "index_am" +#include "pg_prop.h" + +#include "index_traits.h" + +PG_PROP(btree_traits) +{ + am_driver d = {"btree", NULL, "k"}; + + index_traits_check(tc, &d); +} + +PG_PROP(hash_traits) +{ + am_driver d = {"hash", NULL, "k"}; + + index_traits_check(tc, &d); +} + +PG_PROP(brin_traits) +{ + am_driver d = {"brin", NULL, "k"}; + + index_traits_check(tc, &d); +} + +PG_PROP(gin_traits) +{ + am_driver d = {"gin", + "CREATE EXTENSION IF NOT EXISTS btree_gin", "k"}; + + index_traits_check(tc, &d); +} + +PG_PROP(gist_traits) +{ + am_driver d = {"gist", + "CREATE EXTENSION IF NOT EXISTS btree_gist", "k"}; + + index_traits_check(tc, &d); +} diff --git a/src/tools/pginclude/headerscheck b/src/tools/pginclude/headerscheck index e0a1a410afd5f..cfe909dc90d5b 100755 --- a/src/tools/pginclude/headerscheck +++ b/src/tools/pginclude/headerscheck @@ -156,6 +156,7 @@ do # not public API and are not meant to be compiled standalone. test "$f" = src/test/prop/pg_prop.h && continue test "$f" = src/test/prop/sql/query_gen.h && continue + test "$f" = src/test/prop/sql/index_traits.h && continue test "$f" = src/include/common/unicode_nonspacing_table.h && continue test "$f" = src/include/common/unicode_east_asian_fw_table.h && continue From 72308bfe543b12b5f7f380a8ae8c5136dcc8f55f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 08:08:35 -0400 Subject: [PATCH 268/338] prop: add a trait-based table access-method test harness The generic contract every table AM must satisfy, written AM-agnostically over CREATE TABLE ... USING (the AM named by PG_PROP_TABLE_AM, default heap): insert/delete/update row-count correctness, MVCC visibility and rollback, one distinct ctid per live row, and aggregates agreeing with the scan. Another table AM -- OrioleDB, the undo-based FLUX AM -- reuses the whole suite by building the binary with its extension loaded and the AM name supplied; nothing here is heap-specific. --- src/test/prop/sql/prop_table_am.c | 269 ++++++++++++++++++++++++++++++ 1 file changed, 269 insertions(+) create mode 100644 src/test/prop/sql/prop_table_am.c diff --git a/src/test/prop/sql/prop_table_am.c b/src/test/prop/sql/prop_table_am.c new file mode 100644 index 0000000000000..7cb25c530efc6 --- /dev/null +++ b/src/test/prop/sql/prop_table_am.c @@ -0,0 +1,269 @@ +/*------------------------------------------------------------------------- + * + * prop_table_am.c + * Trait-based property tests for table access methods. + * + * A table access method, like an index AM, has a generic contract that must + * hold for any implementation and some implementation-specific behaviour. + * This area tests the generic contract -- the invariants heap, and any + * drop-in table AM (OrioleDB, the FLUX undo-based AM, ...), must all satisfy: + * + * insert_count -- after INSERTing n rows into a fresh table, a seqscan + * sees exactly n rows and count(*) = n. + * delete_visible -- a committed DELETE removes exactly the matching rows; + * the rest remain and are unchanged. + * update_identity -- an UPDATE changes the targeted rows' value and leaves + * the row count unchanged (no duplication, no loss). + * ctid_unique -- every live row has a distinct ctid, and count(ctid) = + * count(*): the AM's row identity is one-per-live-row. + * mvcc_rollback -- a ROLLBACK of an INSERT/DELETE leaves the table exactly + * as before: the AM honours transaction abort. + * agg_vs_scan -- count(*), min, max, sum over the table agree whether + * computed by the AM's scan or over a materialized copy. + * + * The access method under test is named by PG_PROP_TABLE_AM (default "heap"); + * the whole suite runs over "CREATE TABLE ... USING ", so another table AM + * reuses it by building the pg_prop binary with its extension loaded and the + * AM name supplied -- nothing here is heap-specific. + * + * Tier B: runs SQL against the cluster the TAP driver started. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_table_am.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "table_am" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* The table AM under test. Core ships only heap; an extension AM overrides. */ +#ifndef PG_PROP_TABLE_AM +#define PG_PROP_TABLE_AM "heap" +#endif + +#define TAM_MAX_ROWS 60 +#define TAM_K_MIN (-8) +#define TAM_K_MAX 8 + +static long +scalar_long(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *v; + long n; + + v = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "scalar failed: %s\n SQL: %s", + err ? err : "(unknown)", sql); + n = v ? atol(v) : 0; + if (v) + pg_free(v); + if (err) + pg_free(err); + return n; +} + +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *a, const char *b) +{ + bool fa = false, + fb = false; + char *ea = NULL, + *eb = NULL, + *ra, + *rb; + + ra = pg_prop_sql_rows(conn, a, &fa, &ea); + if (fa) + pg_prop_fail(tc, "query a failed: %s\n SQL: %s", ea ? ea : "", a); + rb = pg_prop_sql_rows(conn, b, &fb, &eb); + if (fb) + pg_prop_fail(tc, "query b failed: %s\n SQL: %s", eb ? eb : "", b); + if (strcmp(ra, rb) != 0) + pg_prop_fail(tc, "mismatch:\n a: %s\n b: %s\n ra:\n%s\n rb:\n%s", + a, b, ra, rb); + pg_free(ra); + pg_free(rb); + if (ea) + pg_free(ea); + if (eb) + pg_free(eb); +} + +/* Build tam_t USING the AM under test; return the row count populated. */ +static int +build_table(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int n; + + initStringInfo(&ddl); + appendStringInfo(&ddl, + "DROP TABLE IF EXISTS tam_t; " + "CREATE TABLE tam_t (id int PRIMARY KEY, k int, v int) " + "USING %s; ", PG_PROP_TABLE_AM); + n = (int) pg_prop_draw_int(tc, 0, TAM_MAX_ROWS); + for (int i = 0; i < n; i++) + appendStringInfo(&ddl, "INSERT INTO tam_t VALUES (%d, %d, %d); ", + i, + (int) pg_prop_draw_int(tc, TAM_K_MIN, TAM_K_MAX), + (int) pg_prop_draw_int(tc, 0, 5)); + appendStringInfoString(&ddl, "ANALYZE tam_t;"); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "table build failed: %s", err ? err : ""); + pfree(ddl.data); + return n; +} + +static pg_prop_conn * +connect_tam(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + return conn; +} + +/* count(*) == number inserted, and a seqscan returns exactly those ids. */ +PG_PROP(insert_count) +{ + pg_prop_conn *conn = connect_tam(tc); + int n = build_table(tc, conn); + long c = scalar_long(tc, conn, "SELECT count(*) FROM tam_t"); + + if (c != n) + pg_prop_fail(tc, "count(*)=%ld != inserted %d", c, n); + pg_prop_disconnect(conn); +} + +/* A committed DELETE removes exactly the matching rows. */ +PG_PROP(delete_visible) +{ + pg_prop_conn *conn = connect_tam(tc); + int n = build_table(tc, conn); + int k = (int) pg_prop_draw_int(tc, TAM_K_MIN, TAM_K_MAX); + char *err = NULL; + long before, + matched, + after; + StringInfoData q; + + initStringInfo(&q); + appendStringInfo(&q, "SELECT count(*) FROM tam_t WHERE k = %d", k); + matched = scalar_long(tc, conn, q.data); + before = n; + + resetStringInfo(&q); + appendStringInfo(&q, "DELETE FROM tam_t WHERE k = %d", k); + if (!pg_prop_sql_exec(conn, q.data, &err)) + pg_prop_fail(tc, "delete failed: %s", err ? err : ""); + + after = scalar_long(tc, conn, "SELECT count(*) FROM tam_t"); + if (after != before - matched) + pg_prop_fail(tc, "delete left %ld rows, expected %ld (%ld - %ld)", + after, before - matched, before, matched); + /* no row with the deleted key remains */ + resetStringInfo(&q); + appendStringInfo(&q, "SELECT count(*) FROM tam_t WHERE k = %d", k); + if (scalar_long(tc, conn, q.data) != 0) + pg_prop_fail(tc, "rows with k=%d survived the delete", k); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* An UPDATE changes value, preserves the row count, loses no identity. */ +PG_PROP(update_identity) +{ + pg_prop_conn *conn = connect_tam(tc); + int n = build_table(tc, conn); + char *err = NULL; + long after; + + if (!pg_prop_sql_exec(conn, "UPDATE tam_t SET v = v + 1", &err)) + pg_prop_fail(tc, "update failed: %s", err ? err : ""); + after = scalar_long(tc, conn, "SELECT count(*) FROM tam_t"); + if (after != n) + pg_prop_fail(tc, "update changed row count: %ld != %d", after, n); + /* every id still present exactly once */ + if (scalar_long(tc, conn, + "SELECT count(*) FROM (SELECT id FROM tam_t " + "GROUP BY id HAVING count(*) <> 1) s") != 0) + pg_prop_fail(tc, "update duplicated or dropped a row identity"); + pg_prop_disconnect(conn); +} + +/* Every live row has a distinct ctid and count(ctid) == count(*). */ +PG_PROP(ctid_unique) +{ + pg_prop_conn *conn = connect_tam(tc); + + build_table(tc, conn); + if (scalar_long(tc, conn, "SELECT count(*) FROM tam_t") != + scalar_long(tc, conn, "SELECT count(DISTINCT ctid) FROM tam_t")) + pg_prop_fail(tc, "live rows do not have distinct ctids"); + pg_prop_disconnect(conn); +} + +/* A ROLLBACK of writes restores the table exactly. */ +PG_PROP(mvcc_rollback) +{ + pg_prop_conn *conn = connect_tam(tc); + char *err = NULL; + + build_table(tc, conn); + + require_same_rows(tc, conn, + "SELECT id, k, v FROM tam_t ORDER BY id", + "SELECT id, k, v FROM tam_t ORDER BY id"); + + /* mutate inside an aborted transaction */ + pg_prop_sql_exec(conn, "BEGIN", &err); + pg_prop_sql_exec(conn, "INSERT INTO tam_t VALUES (100000, 1, 1)", &err); + pg_prop_sql_exec(conn, "DELETE FROM tam_t WHERE id = 0", &err); + pg_prop_sql_exec(conn, "UPDATE tam_t SET v = -1", &err); + pg_prop_sql_exec(conn, "ROLLBACK", &err); + if (err) + pg_free(err); + + /* table must be exactly as it was before the aborted transaction */ + { + long c = scalar_long(tc, conn, "SELECT count(*) FROM tam_t"); + long neg = scalar_long(tc, conn, + "SELECT count(*) FROM tam_t WHERE v = -1"); + long injected = scalar_long(tc, conn, + "SELECT count(*) FROM tam_t WHERE id = 100000"); + + if (neg != 0 || injected != 0) + pg_prop_fail(tc, + "rollback did not undo writes: v=-1 rows=%ld injected=%ld (count=%ld)", + neg, injected, c); + } + pg_prop_disconnect(conn); +} + +/* Aggregates over the AM's scan agree with aggregates over a materialized copy. */ +PG_PROP(agg_vs_scan) +{ + pg_prop_conn *conn = connect_tam(tc); + + build_table(tc, conn); + require_same_rows(tc, conn, + "SELECT count(*), coalesce(sum(k),0), min(k), max(k), " + "coalesce(sum(v),0) FROM tam_t", + "SELECT count(*), coalesce(sum(k),0), min(k), max(k), " + "coalesce(sum(v),0) FROM (SELECT k, v FROM tam_t " + "ORDER BY id) s"); + pg_prop_disconnect(conn); +} From f230cf2d98fd5193248ec15ab538b5ab69e46a32 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 08:08:35 -0400 Subject: [PATCH 269/338] prop: wire up the AM trait areas and document the design Register index_traits.c, prop_index_am.c and prop_table_am.c in the build and the index_am / table_am areas in the TAP driver. Document in DESIGN why the PBTs ride on the TAP harness (a methodology on an existing harness, selectable as the 'prop' suite, not a reinvented fourth test driver) and how the trait-based AM areas let core and out-of-tree access methods share the generic tests. Suite green at 250 Tier-B areas. --- src/test/prop/DESIGN | 70 ++++++++++++++++++++++++++++++++ src/test/prop/Makefile | 5 ++- src/test/prop/meson.build | 3 ++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 78 insertions(+), 2 deletions(-) diff --git a/src/test/prop/DESIGN b/src/test/prop/DESIGN index 800fa89448702..3f0219adf4937 100644 --- a/src/test/prop/DESIGN +++ b/src/test/prop/DESIGN @@ -80,6 +80,31 @@ than one per area) keeps the build small and gives the "pg_prop " dispatch the user-facing CLI wants. +Why TAP, and not a fourth kind of test +====================================== + +PostgreSQL has three test *harnesses* -- pg_regress (SQL golden-master), +isolation (interleaved sessions), and TAP (Perl scripts emitting the TAP +protocol). These are harnesses, not separate categories of truth: TAP in +particular is just "a program the meson/make test runner launches, which prints +ok / not ok lines", and the runner already gives it suite grouping, parallelism, +per-test timeouts, and log capture. + +Property-based testing is a *methodology*, not a harness. The pg_prop binary +prints TAP, and t/001_prop.pl and t/002_sql_props.pl are ordinary TAP scripts +that run it, so the suite rides on the existing TAP machinery with no new +infrastructure. Making PBT a fourth top-level concept would mean reinventing +test discovery, a new meson suite type, and new CI plumbing solely to +re-implement what TAP already provides -- so it is deliberately not done. + +What makes the PBTs distinguishable is the suite name, not the harness: they are +registered as the "prop" suite (postgresql:prop), so they can be selected, +skipped, or reported on as a unit (meson test --suite prop) without being a +separate driver. A reviewer who wants "just the property tests" asks for that +suite; a reviewer who wants "all TAP" gets these along with the rest, which is +correct -- they are TAP. + + The PG_PROP registry -------------------- @@ -303,6 +328,51 @@ reference answer, so a disagreement is unambiguously a server bug: as a DISTINCT join must agree -- subquery unnesting). +Trait-based access-method areas +=============================== + +An access method is described by *traits* -- capabilities it declares and the +server exposes at runtime -- and the tests are organized along them so that the +generic part is reusable by any implementation of the trait, in core or out of +tree. + +Index AMs (sql/index_traits.c, driven by sql/prop_index_am.c). A trait is a +contract that must hold for ANY index AM claiming it: exact (equality equals a +seqscan), ordered (ranges, ORDER BY satisfied by the index, backward scan), +summarizing (BRIN-style lossy block ranges never drop a matching row), +multicol, include (index-only scans), search_nulls, unique. The harness takes +an am_driver (the AM name, its setup DDL, and the indexed column/opclass) and +runs, for each trait the AM actually claims, that trait's properties -- the +core oracle throughout being differential: force the plan onto the index, force +it onto a seqscan, require identical rows. + +Crucially the harness asks the server which traits the AM has, at the right +granularity: pg_indexam_has_property for AM-level traits (can_order, can_unique, +can_multi_col, can_include), pg_index_has_property for index-level ones +(index_scan, bitmap_scan, backward_scan), and pg_index_column_has_property for +column-level ones (asc, orderable, search_nulls). So prop_index_am.c encodes no +capability matrix -- it supplies one driver per shipped AM (btree, hash, brin, +gin, gist) and the server decides what runs. That is exactly what lets an +out-of-tree AM reuse the whole applicable suite: BARK, a B+-tree successor to +nbtree, declares amcanorder/amcanunique/amcaninclude in its amhandler and gets +the entire ordered + unique + include + exact + search_nulls trait suite by +adding one am_driver; a summarizing extension AM gets the summarizing suite the +same way. AM-specific behaviour that is not a shared trait (nbtree skip scan) +stays in its own area (prop_nbtree.c). + +Table AMs (sql/prop_table_am.c). The same split: the generic contract every +table AM must satisfy -- insert/delete/update row-count correctness, MVCC +visibility and rollback, one distinct ctid per live row, aggregates agreeing +with the scan -- is written over "CREATE TABLE ... USING " with the AM named +by PG_PROP_TABLE_AM (default heap). Another table AM (OrioleDB, the undo-based +FLUX AM) reuses the whole suite by building the binary with its extension loaded +and the AM name supplied; nothing in it is heap-specific. + +The older indexdiff area is kept as-is: it tests index *features* orthogonal to +AM traits (partial and expression indexes), which are not AM capabilities, so it +complements rather than duplicates the trait areas. + + Build-system integration ======================== diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 77c6bd35c71bc..e55f7063e6990 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -280,7 +280,10 @@ OBJS = \ sql/prop_query_planadvice.o \ sql/prop_query_ext_citext.o \ sql/prop_query_multi.o \ - sql/prop_query_deep.o + sql/prop_query_deep.o \ + sql/index_traits.o \ + sql/prop_index_am.o \ + sql/prop_table_am.o PG_CPPFLAGS = -I$(libpq_srcdir) -I$(srcdir)/sql PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index b2a0c7099fa50..e6f6da4badceb 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -286,6 +286,9 @@ prop_query_sources = files( 'sql/prop_query_ext_citext.c', 'sql/prop_query_multi.c', 'sql/prop_query_deep.c', + 'sql/index_traits.c', + 'sql/prop_index_am.c', + 'sql/prop_table_am.c', ) pg_prop = executable('pg_prop', diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index d8be6f110187b..e9977b5dd2b5e 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep index_am table_am); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From c8264b8473fae613cf38e53125fda6d25bbfdcb6 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 09:19:44 -0400 Subject: [PATCH 270/338] prop: quiet trait_unique DROP notice; document the coverage lift Set client_min_messages=warning around the unique-trait scratch table's DROP IF EXISTS so the benign 'table does not exist, skipping' NOTICE does not flood output under a soak. Record in DESIGN the measured whole-suite coverage lift: on a branch-coverage build over src/backend + src/common, the regression suite alone covers 70.0/73.3/53.4 (lines/functions/branches) and regression + PBT covers 72.0/76.3/54.7 -- the PBTs execute 544 functions and 7,933 lines the regression suite never reaches. --- src/test/prop/DESIGN | 14 ++++++++++++++ src/test/prop/sql/index_traits.c | 1 + 2 files changed, 15 insertions(+) diff --git a/src/test/prop/DESIGN b/src/test/prop/DESIGN index 3f0219adf4937..ba7f3e173f375 100644 --- a/src/test/prop/DESIGN +++ b/src/test/prop/DESIGN @@ -486,6 +486,20 @@ the standing differential net, not by a coverage-percentage delta or a depth-per-line multiplier -- the latter does not favor PBT and we do not claim it does. +Whole-suite lift over the regression tests. On a branch-coverage build, +measured over src/backend + src/common, the core regression suite alone covers +70.0% of lines, 73.3% of functions and 53.4% of branches; adding the full PBT +suite on top raises that to 72.0% / 76.3% / 54.7%. The property suite executes +544 backend/common functions and 7,933 lines the entire regression suite never +reaches, plus 3,082 additional branches. Branch coverage moves least, as +expected -- the regression suite already drives most hot branches, so the PBT +win is breadth (new functions and lines, in the incremental jsonapi parser, the +access-method trait paths, and the deep planner transformations) more than +depth on branches already covered. This is the concrete argument for carrying +the suite in core: not to replace the golden-master tests (the union is what +matters) but to reach a standing slice of the backend they structurally do not, +with generated rather than hand-enumerated inputs. + Why this shape ============== diff --git a/src/test/prop/sql/index_traits.c b/src/test/prop/sql/index_traits.c index a7a0c853fac12..b61cf6b6c9bc6 100644 --- a/src/test/prop/sql/index_traits.c +++ b/src/test/prop/sql/index_traits.c @@ -415,6 +415,7 @@ trait_unique(pg_prop_tc *tc, pg_prop_conn *conn, const am_driver *d) * insert a duplicate key (must fail). */ pg_prop_sql_exec(conn, + "SET client_min_messages = warning; " "DROP TABLE IF EXISTS uq_t; " "CREATE TABLE uq_t (id int, k int); " "INSERT INTO uq_t VALUES (1,10),(2,20),(3,30);", &err); From 2fa6efd113bf27483f604c43e62cd9b9d6dbc05e Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 12:33:09 -0400 Subject: [PATCH 271/338] prop: decouple from TAP; add skip and require-extension framework Property-based testing is a methodology, not a harness: pg_prop is a standalone C program that emits the TAP protocol and runs on its own (Tier A in-process, Tier B against any PGHOST cluster). Stop gating the build on tap_tests -- prop_tests now requires only the opt-in flag and libhegel. When TAP tests are also enabled the two Perl drivers are additionally registered as the 'prop' meson suite for convenience; when TAP is disabled the binary still builds and is run directly (verified: -Dtap_tests=disabled -Dprop_tests=enabled builds pg_prop and it runs both tiers). Mirror the change in configure.ac (--with-hegel no longer requires --enable-tap-tests). Add the availability framework the extension/PL areas need: pg_prop_require_extension(conn, name) loads a contrib module or PL if the server offers it (checking pg_available_extensions first), and pg_prop_skip() abandons a property and reports it as a TAP SKIP. Together they let an area degrade gracefully -- load what it needs, skip (not fail) when absent. --- configure.ac | 15 +++++----- meson.build | 59 ++++++++++++++++++++----------------- src/test/prop/meson.build | 31 +++++++++++-------- src/test/prop/pg_prop.c | 58 ++++++++++++++++++++++++++++++++++++ src/test/prop/pg_prop.h | 22 ++++++++++++++ src/test/prop/pg_prop_sql.c | 33 +++++++++++++++++++++ 6 files changed, 171 insertions(+), 47 deletions(-) diff --git a/configure.ac b/configure.ac index f1f7b2e4ab8c7..b46d20bbca381 100644 --- a/configure.ac +++ b/configure.ac @@ -242,16 +242,15 @@ PGAC_ARG_BOOL(enable, injection-points, no, [enable injection points (for testin AC_SUBST(enable_injection_points) # -# Property-based tests (src/test/prop), built on libhegel. Requires -# --enable-tap-tests as well, since the suite is driven through the TAP -# harness; the actual presence of libhegel's header and library is checked -# further below, once --with-includes / --with-libraries are in effect. +# Property-based tests (src/test/prop), built on libhegel. TAP is NOT +# required: pg_prop is a standalone C program that emits TAP output and runs on +# its own. When --enable-tap-tests is also given, the Perl drivers are run as +# part of the TAP suite; otherwise the binary is still built (run it directly). +# The actual presence of libhegel's header and library is checked further +# below, once --with-includes / --with-libraries are in effect. # PGAC_ARG_BOOL(with, hegel, no, - [build property-based tests (requires libhegel and --enable-tap-tests)]) -if test "$with_hegel" = yes && test "$enable_tap_tests" != yes; then - AC_MSG_ERROR([--with-hegel requires --enable-tap-tests]) -fi + [build property-based tests (requires libhegel; uses TAP when --enable-tap-tests)]) AC_SUBST(with_hegel) # Export the extra library directories so the property-test Makefile can add # them to pg_prop's rpath; libhegel is typically installed outside the system diff --git a/meson.build b/meson.build index 35820c667b10d..304f9b8b8908c 100644 --- a/meson.build +++ b/meson.build @@ -1831,40 +1831,45 @@ endif # Library: libhegel (property-based testing engine) ############################################################### -# Property-based tests (src/test/prop) require all three of: -# 1. -Dprop_tests=enabled (explicit opt-in; default disabled), +# Property-based tests (src/test/prop) require: +# 1. -Dprop_tests=enabled (explicit opt-in; default disabled), and # 2. libhegel's header and shared library present (found via -# extra_include_dirs / extra_lib_dirs or a system path), and -# 3. TAP tests enabled (the prop suite is driven through the TAP wrapper). -# Any missing piece disables the suite; with -Dprop_tests=enabled a missing -# piece is a hard error so the opt-in cannot silently do nothing. +# extra_include_dirs / extra_lib_dirs or a system path). +# TAP is NOT required: the pg_prop binary is a standalone C program that emits +# TAP output and runs on its own (./pg_prop ...), against any cluster +# PGHOST/PGPORT point at for the SQL-oracle areas. When TAP tests are also +# enabled the Perl drivers (t/001_prop.pl, t/002_sql_props.pl) are registered +# as the 'prop' meson suite for convenience -- they start a throwaway cluster +# and run the binary -- but the suite is a methodology on top of the harness, +# not gated by TAP. With -Dprop_tests=enabled a missing libhegel is a hard +# error so the opt-in cannot silently do nothing. prop_tests_enabled = false +prop_tests_tap = false libhegel = not_found_dep propopt = get_option('prop_tests') if not propopt.disabled() - if not tap_tests_enabled - if propopt.enabled() - error('prop_tests requires tap_tests to be enabled') - else - message('property-based tests disabled: TAP tests are not enabled') + # The C ABI header has no PostgreSQL dependencies; the engine ships as a + # plain shared library named libhegel_c. Look for both. + libhegel_h = cc.has_header('hegel.h', + args: test_c_args, include_directories: postgres_inc, + required: propopt) + libhegel_lib = cc.find_library('hegel_c', + dirs: postgres_lib_d, required: propopt) + if libhegel_h and libhegel_lib.found() + libhegel = declare_dependency(dependencies: libhegel_lib) + prop_tests_enabled = true + # Register the Perl TAP drivers only when TAP is available; without it the + # binary still builds and can be run by hand. + prop_tests_tap = tap_tests_enabled + if not tap_tests_enabled + message('property-based tests: building pg_prop; TAP drivers not ' + + 'registered (tap_tests disabled) -- run the binary directly') endif + elif propopt.enabled() + error('prop_tests requires libhegel (hegel.h and libhegel_c); ' + + 'point --extra-include-dirs / --extra-lib-dirs at its install') else - # The C ABI header has no PostgreSQL dependencies; the engine ships as a - # plain shared library named libhegel_c. Look for both. - libhegel_h = cc.has_header('hegel.h', - args: test_c_args, include_directories: postgres_inc, - required: propopt) - libhegel_lib = cc.find_library('hegel_c', - dirs: postgres_lib_d, required: propopt) - if libhegel_h and libhegel_lib.found() - libhegel = declare_dependency(dependencies: libhegel_lib) - prop_tests_enabled = true - elif propopt.enabled() - error('prop_tests requires libhegel (hegel.h and libhegel_c); ' + - 'point --extra-include-dirs / --extra-lib-dirs at its install') - else - message('property-based tests disabled: libhegel not found') - endif + message('property-based tests disabled: libhegel not found') endif endif diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index e6f6da4badceb..531d61cbe0431 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -289,6 +289,8 @@ prop_query_sources = files( 'sql/index_traits.c', 'sql/prop_index_am.c', 'sql/prop_table_am.c', + 'sql/prop_pl_langs.c', + 'sql/prop_integrity.c', ) pg_prop = executable('pg_prop', @@ -301,15 +303,20 @@ pg_prop = executable('pg_prop', ) testprep_targets += pg_prop -tests += { - 'name': 'prop', - 'sd': meson.current_source_dir(), - 'bd': meson.current_build_dir(), - 'tap': { - 'tests': [ - 't/001_prop.pl', - 't/002_sql_props.pl', - ], - 'deps': [pg_prop], - }, -} +# Register the Perl TAP drivers only when TAP is available (prop_tests_tap). +# Without TAP the pg_prop binary above still builds and can be run directly; +# the suite is a methodology on the harness, not gated by it. +if prop_tests_tap + tests += { + 'name': 'prop', + 'sd': meson.current_source_dir(), + 'bd': meson.current_build_dir(), + 'tap': { + 'tests': [ + 't/001_prop.pl', + 't/002_sql_props.pl', + ], + 'deps': [pg_prop], + }, + } +endif diff --git a/src/test/prop/pg_prop.c b/src/test/prop/pg_prop.c index 063087b088b78..d3274e5378719 100644 --- a/src/test/prop/pg_prop.c +++ b/src/test/prop/pg_prop.c @@ -211,6 +211,15 @@ check_rc(hegel_context_t *ctx, hegel_result_t rc, const char *what) static pg_prop_tc *current_tc; /* for the stop/overrun jump */ static bool overrun; +/* + * Set when the body called pg_prop_skip(): the property is abandoned and + * reported skipped. Module-global (not per-case) because the whole property + * is skipped, not one case; run_property reads and clears it. + */ +static bool skip_requested; +static StringInfoData skip_reason; +static bool skip_reason_inited; + static void stop_test(pg_prop_tc *tc) { @@ -393,6 +402,24 @@ pg_prop_fail(pg_prop_tc *tc, const char *fmt,...) siglongjmp(tc->jmp, 1); /* 1 == interesting/failure */ } +void +pg_prop_skip(pg_prop_tc *tc, const char *fmt,...) +{ + va_list ap; + + if (!skip_reason_inited) + { + initStringInfo(&skip_reason); + skip_reason_inited = true; + } + resetStringInfo(&skip_reason); + va_start(ap, fmt); + appendStringInfoVA(&skip_reason, fmt, ap); + va_end(ap); + skip_requested = true; + siglongjmp(tc->jmp, 3); /* 3 == skip the whole property */ +} + /* ---- fork isolation -------------------------------------------------- */ int @@ -518,6 +545,13 @@ run_one_case(RunState *rs, hegel_context_t *ctx, hegel_test_case_t *handle) { status = HEGEL_STATUS_OVERRUN; } + else if (jmpval == 3) + { + /* skip: the whole property is abandoned; treat this case as valid so + * the engine does not try to shrink it. skip_requested is read by + * run_property, which reports the property as skipped. */ + status = HEGEL_STATUS_VALID; + } else { /* failure: keep the message for the report */ @@ -602,6 +636,8 @@ run_property(PropEntry *prop, uint64 iterations, bool have_seed, uint64 seed, check_rc(ctx, hegel_run_start(ctx, s, NULL, NULL, &run), "run_start"); + skip_requested = false; + for (;;) { hegel_test_case_t *handle = NULL; @@ -621,6 +657,28 @@ run_property(PropEntry *prop, uint64 iterations, bool have_seed, uint64 seed, ok = (rstatus == HEGEL_RUN_STATUS_PASSED); + /* + * The body called pg_prop_skip(): report the property as skipped (TAP + * SKIP directive) and count it as passed, regardless of the engine's + * status for the one case that ran before the skip. + */ + if (skip_requested) + { + const char *reason = skip_reason_inited ? skip_reason.data : ""; + + if (tap) + printf("ok %d - %s/%s # SKIP %s\n", testno, + prop->area, prop->name, reason); + else + printf("%s/%s: SKIPPED (%s)\n", prop->area, prop->name, reason); + hegel_run_result_free(ctx, result); + hegel_run_free(ctx, run); + hegel_settings_free(ctx, s); + hegel_context_free(ctx); + pfree(rs.last_fail.data); + return true; + } + if (tap) printf("%s %d - %s/%s\n", ok ? "ok" : "not ok", testno, prop->area, prop->name); diff --git a/src/test/prop/pg_prop.h b/src/test/prop/pg_prop.h index d541fe74178fd..f122e1c59a62c 100644 --- a/src/test/prop/pg_prop.h +++ b/src/test/prop/pg_prop.h @@ -190,6 +190,16 @@ extern char *pg_prop_draw_from(pg_prop_tc *tc, const char *alphabet, */ extern void pg_prop_assume(pg_prop_tc *tc, bool condition); +/* + * Skip the whole property: its precondition cannot be met in this environment + * (a required extension or procedural language is not installed, a platform + * feature is absent). Unlike pg_prop_assume -- which discards one generated + * case -- this abandons the entire property and reports it as skipped, with + * "reason" shown as the TAP SKIP directive. Call it at the top of the body, + * before drawing inputs. Does not return. + */ +extern pg_noreturn void pg_prop_skip(pg_prop_tc *tc, const char *fmt,...) pg_attribute_printf(2, 3); + /* * Attach a diagnostic note to the current case. It is printed only when this * case is the final shrunk counterexample, so use it to record context the @@ -235,6 +245,18 @@ extern int pg_prop_run_isolated(pg_prop_isolated_fn body, void *arg, */ typedef struct pg_prop_conn pg_prop_conn; +/* + * Ensure an extension (contrib module or procedural language) is available for + * the session, loading it with CREATE EXTENSION IF NOT EXISTS. Returns true + * if it is now present. If the extension is not installed on this server, + * returns false; the caller then typically pg_prop_skip()s. "conn" is an open + * connection. Example: + * + * if (!pg_prop_require_extension(conn, "pageinspect")) + * pg_prop_skip(tc, "pageinspect not installed"); + */ +extern bool pg_prop_require_extension(pg_prop_conn *conn, const char *extname); + /* * Connect to the test cluster named by PGHOST/PGPORT/PGDATABASE in the * environment. Returns NULL and sets *err (malloc'd) on failure; the caller diff --git a/src/test/prop/pg_prop_sql.c b/src/test/prop/pg_prop_sql.c index e259b490405b0..eef23fd7d93bb 100644 --- a/src/test/prop/pg_prop_sql.c +++ b/src/test/prop/pg_prop_sql.c @@ -193,3 +193,36 @@ pg_prop_sql_exec(pg_prop_conn *conn, const char *sql, char **errmsg) PQclear(res); return true; } + +/* + * Load an extension (contrib module or procedural language) into the session, + * returning true if it is now available. Checks pg_available_extensions first + * so a missing module is reported as "unavailable" (false) rather than a hard + * SQL error; if present, runs CREATE EXTENSION IF NOT EXISTS. Used by areas + * that test an extension so they can pg_prop_skip() cleanly when it is not + * installed on the server under test. + */ +bool +pg_prop_require_extension(pg_prop_conn *conn, const char *extname) +{ + PGresult *res; + char q[256]; + bool avail; + + /* is it installable here at all? */ + snprintf(q, sizeof(q), + "SELECT 1 FROM pg_available_extensions WHERE name = '%s'", + extname); + res = PQexec(conn->conn, q); + avail = (PQresultStatus(res) == PGRES_TUPLES_OK && PQntuples(res) == 1); + PQclear(res); + if (!avail) + return false; + + snprintf(q, sizeof(q), "CREATE EXTENSION IF NOT EXISTS \"%s\"", extname); + res = PQexec(conn->conn, q); + avail = (PQresultStatus(res) == PGRES_COMMAND_OK || + PQresultStatus(res) == PGRES_TUPLES_OK); + PQclear(res); + return avail; +} From b102572a410c19230ec81f94e806c5019206e385 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 12:33:33 -0400 Subject: [PATCH 272/338] prop: add procedural-language differential area (pl_langs) The same pure integer computation in PL/pgSQL, PL/Python, PL/Perl and PL/Tcl must equal the pure-SQL computation of it; a disagreement is a bug in a PL's argument marshalling or return conversion. Each property probes its language with pg_prop_require_extension and pg_prop_skips if the server was not built with it, so the area runs whatever PLs are present (verified all four on a server built with -Dplperl -Dplpython -Dpltcl). --- src/test/prop/sql/prop_pl_langs.c | 193 ++++++++++++++++++++++++++++++ 1 file changed, 193 insertions(+) create mode 100644 src/test/prop/sql/prop_pl_langs.c diff --git a/src/test/prop/sql/prop_pl_langs.c b/src/test/prop/sql/prop_pl_langs.c new file mode 100644 index 0000000000000..a750218743d6c --- /dev/null +++ b/src/test/prop/sql/prop_pl_langs.c @@ -0,0 +1,193 @@ +/*------------------------------------------------------------------------- + * + * prop_pl_langs.c + * Property tests across the procedural languages (PL/pgSQL, PL/Python, + * PL/Perl, PL/Tcl). + * + * Each procedural language can define a function body that computes a value; + * for a pure, deterministic computation the language used must not change the + * answer. This area generates the same simple integer computation in each + * available PL and requires it to equal the pure-SQL computation of the same + * expression -- a differential oracle across language implementations, where a + * disagreement is a bug in a PL's argument marshalling, integer handling, or + * return conversion. + * + * Availability: a PL is a loadable extension (plpgsql is always present; + * plpython3u, plperl, pltcl are present only if the server was built with + * them). Each property probes its language with pg_prop_require_extension and + * calls pg_prop_skip when it is not installed, so the area runs whatever the + * server offers and cleanly skips the rest -- it never fails merely because a + * PL is absent. + * + * Tier B: runs SQL against the cluster the TAP driver started. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_pl_langs.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "pl_langs" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* Compare a scalar query's value to an expected integer, as text. */ +static void +require_scalar_eq(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql, + long expected) +{ + bool failed = false; + char *err = NULL; + char *v; + long got; + + v = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "PL query failed: %s\n SQL: %s", + err ? err : "(unknown)", sql); + got = v ? atol(v) : 0; + if (got != expected) + pg_prop_fail(tc, "PL result %ld != expected %ld\n SQL: %s", + got, expected, sql); + if (v) + pg_free(v); + if (err) + pg_free(err); +} + +/* Draw two small ints and the pure-SQL value of a*b + (a - b). */ +static void +draw_inputs(pg_prop_tc *tc, int *a, int *b, long *expect) +{ + *a = (int) pg_prop_draw_int(tc, -1000, 1000); + *b = (int) pg_prop_draw_int(tc, -1000, 1000); + *expect = (long) (*a) * (*b) + ((long) (*a) - (long) (*b)); +} + +/* PL/pgSQL: always present (it is an un-droppable default language). */ +PG_PROP(plpgsql_matches_sql) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + b; + long expect; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + if (!pg_prop_require_extension(conn, "plpgsql")) + pg_prop_skip(tc, "plpgsql not available"); + + if (!pg_prop_sql_exec(conn, + "CREATE OR REPLACE FUNCTION pl_f(a int, b int) " + "RETURNS bigint LANGUAGE plpgsql AS " + "$$ BEGIN RETURN a::bigint * b + (a - b); END $$;", + &err)) + pg_prop_fail(tc, "plpgsql function create failed: %s", err ? err : ""); + + draw_inputs(tc, &a, &b, &expect); + initStringInfo(&q); + appendStringInfo(&q, "SELECT pl_f(%d, %d)", a, b); + require_scalar_eq(tc, conn, q.data, expect); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* PL/Python (plpython3u). */ +PG_PROP(plpython_matches_sql) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + b; + long expect; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + if (!pg_prop_require_extension(conn, "plpython3u")) + pg_prop_skip(tc, "plpython3u not available"); + + if (!pg_prop_sql_exec(conn, + "CREATE OR REPLACE FUNCTION pl_f(a int, b int) " + "RETURNS bigint LANGUAGE plpython3u AS " + "$$ return a * b + (a - b) $$;", + &err)) + pg_prop_fail(tc, "plpython function create failed: %s", err ? err : ""); + + draw_inputs(tc, &a, &b, &expect); + initStringInfo(&q); + appendStringInfo(&q, "SELECT pl_f(%d, %d)", a, b); + require_scalar_eq(tc, conn, q.data, expect); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* PL/Perl. */ +PG_PROP(plperl_matches_sql) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + b; + long expect; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + if (!pg_prop_require_extension(conn, "plperl")) + pg_prop_skip(tc, "plperl not available"); + + if (!pg_prop_sql_exec(conn, + "CREATE OR REPLACE FUNCTION pl_f(a int, b int) " + "RETURNS bigint LANGUAGE plperl AS " + "$$ my ($a, $b) = @_; return $a * $b + ($a - $b); $$;", + &err)) + pg_prop_fail(tc, "plperl function create failed: %s", err ? err : ""); + + draw_inputs(tc, &a, &b, &expect); + initStringInfo(&q); + appendStringInfo(&q, "SELECT pl_f(%d, %d)", a, b); + require_scalar_eq(tc, conn, q.data, expect); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* PL/Tcl. */ +PG_PROP(pltcl_matches_sql) +{ + pg_prop_conn *conn; + char *err = NULL; + int a, + b; + long expect; + StringInfoData q; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + if (!pg_prop_require_extension(conn, "pltcl")) + pg_prop_skip(tc, "pltcl not available"); + + if (!pg_prop_sql_exec(conn, + "CREATE OR REPLACE FUNCTION pl_f(a int, b int) " + "RETURNS bigint LANGUAGE pltcl AS " + "$$ return [expr {$1 * $2 + ($1 - $2)}] $$;", + &err)) + pg_prop_fail(tc, "pltcl function create failed: %s", err ? err : ""); + + draw_inputs(tc, &a, &b, &expect); + initStringInfo(&q); + appendStringInfo(&q, "SELECT pl_f(%d, %d)", a, b); + require_scalar_eq(tc, conn, q.data, expect); + pfree(q.data); + pg_prop_disconnect(conn); +} From 2114010f7290bbef13d00c276a45cefd544763b5 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 12:33:33 -0400 Subject: [PATCH 273/338] prop: add integrity-oracle area over amcheck/pg_visibility/pgstattuple Use the integrity-checking contrib extensions as property oracles: for any generated table and index, after DML churn, the structure must pass the extension's own self-check. btree_amcheck (bt_index_check), heap_verify (verify_heapam finds no corruption), vm_consistent (pg_check_visible / pg_check_frozen find no wrong visibility-map bits after VACUUM FREEZE), and stattuple_bounds (pgstattuple byte accounting non-negative and bounded). A failure is a storage/AM-level corruption bug, a different class from wrong results. Each requires its contrib and skips cleanly if absent. bt_index_check is invoked as a top-level SELECT (not buried in a WHERE-false subquery the planner would elide, which would make the oracle vacuous). --- src/test/prop/sql/prop_integrity.c | 195 +++++++++++++++++++++++++++++ 1 file changed, 195 insertions(+) create mode 100644 src/test/prop/sql/prop_integrity.c diff --git a/src/test/prop/sql/prop_integrity.c b/src/test/prop/sql/prop_integrity.c new file mode 100644 index 0000000000000..d8053b2b781e2 --- /dev/null +++ b/src/test/prop/sql/prop_integrity.c @@ -0,0 +1,195 @@ +/*------------------------------------------------------------------------- + * + * prop_integrity.c + * Property tests using the integrity-checking contrib extensions as + * oracles (amcheck, pg_visibility, pgstattuple, pageinspect). + * + * These extensions exist precisely to assert that an index or table is + * internally well-formed, which makes them powerful property oracles: for ANY + * table and index the generator can build, and after any sequence of DML the + * generator can apply, the structure the server produced must pass its own + * self-check. A failure here is a storage- or access-method-level corruption + * bug, not a wrong-results bug -- a different and valuable class. + * + * bt_index_check -- a freshly built btree index over random data has no + * structural corruption (ordering, sibling links, ...). + * verify_heapam -- a random heap, after inserts/updates/deletes, has no + * heap/toast corruption (verify_heapam returns no rows). + * vm_consistent -- after VACUUM, pg_check_visible / pg_check_frozen + * report no incorrectly-set visibility-map bits. + * stattuple_bounds -- pgstattuple's live/dead/free byte counts sum within + * the table's size and are non-negative. + * + * Availability: each property requires its contrib extension and pg_prop_skips + * if it is not installed, so the area runs whatever the server offers. + * + * Tier B: runs SQL against the cluster the TAP driver started. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_integrity.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "integrity" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +#define INT_MAX_ROWS 200 +#define INT_K_MIN (-20) +#define INT_K_MAX 20 + +static long +scalar_long(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *v; + long n; + + v = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "scalar failed: %s\n SQL: %s", + err ? err : "(unknown)", sql); + n = v ? atol(v) : 0; + if (v) + pg_free(v); + if (err) + pg_free(err); + return n; +} + +/* Build integ_t(id serial pk, k int, pl text) with random rows + churn. */ +static void +build_and_churn(pg_prop_tc *tc, pg_prop_conn *conn) +{ + StringInfoData ddl; + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 0, INT_MAX_ROWS); + int i; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "SET client_min_messages = warning; " + "DROP TABLE IF EXISTS integ_t; " + "CREATE TABLE integ_t (id int PRIMARY KEY, k int, pl text); "); + for (i = 0; i < n; i++) + appendStringInfo(&ddl, + "INSERT INTO integ_t VALUES (%d, %d, repeat('x', %d)); ", + i, (int) pg_prop_draw_int(tc, INT_K_MIN, INT_K_MAX), + (int) pg_prop_draw_int(tc, 0, 3000)); + /* churn: delete and update a fraction, so the heap has dead tuples */ + appendStringInfo(&ddl, + "DELETE FROM integ_t WHERE k = %d; " + "UPDATE integ_t SET k = k + 1 WHERE k < %d; ", + (int) pg_prop_draw_int(tc, INT_K_MIN, INT_K_MAX), + (int) pg_prop_draw_int(tc, INT_K_MIN, INT_K_MAX)); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "build/churn failed: %s", err ? err : ""); + pfree(ddl.data); +} + +static pg_prop_conn * +connect_integ(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + return conn; +} + +/* A freshly built btree over random data passes amcheck's bt_index_check. */ +PG_PROP(btree_amcheck) +{ + pg_prop_conn *conn = connect_integ(tc); + char *err = NULL; + + if (!pg_prop_require_extension(conn, "amcheck")) + pg_prop_skip(tc, "amcheck not available"); + build_and_churn(tc, conn); + if (!pg_prop_sql_exec(conn, + "DROP INDEX IF EXISTS integ_idx; " + "CREATE INDEX integ_idx ON integ_t (k);", &err)) + pg_prop_fail(tc, "index build failed: %s", err ? err : ""); + /* + * bt_index_check returns void and raises on structural corruption; call + * it directly (not inside a dead subquery, which the planner would elide) + * so it actually executes. A SQL error here is the corruption signal. + */ + if (!pg_prop_sql_exec(conn, + "SELECT bt_index_check('integ_idx'::regclass)", &err)) + pg_prop_fail(tc, "bt_index_check failed: %s", err ? err : ""); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} + +/* A random heap after churn passes verify_heapam (no corruption rows). */ +PG_PROP(heap_verify) +{ + pg_prop_conn *conn = connect_integ(tc); + long corrupt; + + if (!pg_prop_require_extension(conn, "amcheck")) + pg_prop_skip(tc, "amcheck not available"); + build_and_churn(tc, conn); + /* verify_heapam returns one row per corruption found; expect zero */ + corrupt = scalar_long(tc, conn, + "SELECT count(*) FROM verify_heapam('integ_t'::regclass)"); + if (corrupt != 0) + pg_prop_fail(tc, "verify_heapam reported %ld corruption rows", corrupt); + pg_prop_disconnect(conn); +} + +/* After VACUUM, the visibility map has no incorrectly-set bits. */ +PG_PROP(vm_consistent) +{ + pg_prop_conn *conn = connect_integ(tc); + char *err = NULL; + long badvis, + badfroz; + + if (!pg_prop_require_extension(conn, "pg_visibility")) + pg_prop_skip(tc, "pg_visibility not available"); + build_and_churn(tc, conn); + if (!pg_prop_sql_exec(conn, "VACUUM (FREEZE) integ_t;", &err)) + pg_prop_fail(tc, "vacuum failed: %s", err ? err : ""); + /* pg_check_visible / pg_check_frozen return the TIDs of wrongly-set bits */ + badvis = scalar_long(tc, conn, + "SELECT count(*) FROM pg_check_visible('integ_t'::regclass)"); + badfroz = scalar_long(tc, conn, + "SELECT count(*) FROM pg_check_frozen('integ_t'::regclass)"); + if (badvis != 0) + pg_prop_fail(tc, "pg_check_visible found %ld wrong VM bits", badvis); + if (badfroz != 0) + pg_prop_fail(tc, "pg_check_frozen found %ld wrong frozen bits", badfroz); + pg_prop_disconnect(conn); +} + +/* pgstattuple's byte accounting is non-negative and bounded by table size. */ +PG_PROP(stattuple_bounds) +{ + pg_prop_conn *conn = connect_integ(tc); + long bad; + + if (!pg_prop_require_extension(conn, "pgstattuple")) + pg_prop_skip(tc, "pgstattuple not available"); + build_and_churn(tc, conn); + /* + * table_len >= live+dead+free bytes, and no component is negative. One + * query returns the count of violations (expected zero). + */ + bad = scalar_long(tc, conn, + "SELECT count(*) FROM pgstattuple('integ_t'::regclass) s " + "WHERE s.table_len < s.tuple_len + s.dead_tuple_len + s.free_space " + "OR s.tuple_len < 0 OR s.dead_tuple_len < 0 OR s.free_space < 0"); + if (bad != 0) + pg_prop_fail(tc, "pgstattuple byte accounting inconsistent"); + pg_prop_disconnect(conn); +} From 02a5f58ac023effa5c470e39a593adabdcf4e0c1 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 12:33:33 -0400 Subject: [PATCH 274/338] prop: drive bloom through the index trait harness; wire areas; docs Add contrib/bloom to the trait-driven index AM area (it declares bitmap-only, lossy, no ordering, so it exercises the exact and summarizing-shaped trait checks). Register the new pl_langs and integrity areas in the build and TAP driver. Remove a dead unused helper in prop_indexdiff.c (CompilerWarnings is strict). Document in DESIGN the TAP decoupling, the PL and integrity areas, the bloom coverage, and the skip/availability framework. --- src/test/prop/DESIGN | 91 ++++++++++++++++++++++++------- src/test/prop/Makefile | 4 +- src/test/prop/prop_indexdiff.c | 29 ---------- src/test/prop/sql/prop_index_am.c | 17 ++++++ src/test/prop/t/002_sql_props.pl | 2 +- 5 files changed, 92 insertions(+), 51 deletions(-) diff --git a/src/test/prop/DESIGN b/src/test/prop/DESIGN index ba7f3e173f375..0c5611755f9c4 100644 --- a/src/test/prop/DESIGN +++ b/src/test/prop/DESIGN @@ -80,29 +80,32 @@ than one per area) keeps the build small and gives the "pg_prop " dispatch the user-facing CLI wants. -Why TAP, and not a fourth kind of test -====================================== +Relationship to TAP: integrated with, not gated by +================================================== PostgreSQL has three test *harnesses* -- pg_regress (SQL golden-master), isolation (interleaved sessions), and TAP (Perl scripts emitting the TAP -protocol). These are harnesses, not separate categories of truth: TAP in -particular is just "a program the meson/make test runner launches, which prints -ok / not ok lines", and the runner already gives it suite grouping, parallelism, -per-test timeouts, and log capture. - -Property-based testing is a *methodology*, not a harness. The pg_prop binary -prints TAP, and t/001_prop.pl and t/002_sql_props.pl are ordinary TAP scripts -that run it, so the suite rides on the existing TAP machinery with no new -infrastructure. Making PBT a fourth top-level concept would mean reinventing -test discovery, a new meson suite type, and new CI plumbing solely to -re-implement what TAP already provides -- so it is deliberately not done. - -What makes the PBTs distinguishable is the suite name, not the harness: they are -registered as the "prop" suite (postgresql:prop), so they can be selected, -skipped, or reported on as a unit (meson test --suite prop) without being a -separate driver. A reviewer who wants "just the property tests" asks for that -suite; a reviewer who wants "all TAP" gets these along with the rest, which is -correct -- they are TAP. +protocol). These are harnesses, not separate categories of truth, and the +meson/make runner gives TAP suite grouping, parallelism, per-test timeouts, and +log capture. + +Property-based testing is a *methodology*, and the engine is harness- +independent: pg_prop is a standalone C program that emits the TAP protocol and +runs on its own -- ./pg_prop [-n N] [-s SEED] for the in-process (Tier A) +areas, and the same with PGHOST/PGPORT pointed at any cluster for the SQL-oracle +(Tier B) areas. It needs neither Perl nor the TAP harness to run. + +So the suite is *not gated by* TAP. The build requires only the opt-in flag +(-Dprop_tests=enabled / --with-hegel) and libhegel; it does not require +tap_tests. When TAP tests are also enabled, the two Perl drivers +(t/001_prop.pl, t/002_sql_props.pl) are additionally registered as the "prop" +meson suite (postgresql:prop) for convenience -- they start a throwaway cluster +and run the binary -- so a reviewer can "meson test --suite prop". When TAP is +disabled the binary still builds and is run directly; only the convenience +driver registration is skipped. This is the deliberate design: PBT integrates +with the TAP harness where it helps (test collection, a named suite) without +inheriting TAP as a hard dependency, and without reinventing a fourth test +runner. The PG_PROP registry @@ -372,6 +375,54 @@ The older indexdiff area is kept as-is: it tests index *features* orthogonal to AM traits (partial and expression indexes), which are not AM capabilities, so it complements rather than duplicates the trait areas. +The bloom contrib AM is driven by the index trait harness too (it declares +bitmap-only, lossy, no ordering), so it exercises the exact and summarizing- +shaped trait checks. spgist is not in the int-column trait suite (it ships no +integer opclass); it is a candidate for its own area over the scalar types it +supports. + + +Procedural languages (sql/prop_pl_langs.c) +========================================== + +The same pure, deterministic integer computation expressed in each procedural +language -- PL/pgSQL, PL/Python (plpython3u), PL/Perl, PL/Tcl -- must equal the +pure-SQL computation of the same expression. This is a differential oracle +across language implementations; a disagreement is a bug in a PL's argument +marshalling, integer handling, or return conversion. Each property probes its +language with pg_prop_require_extension and pg_prop_skips when the server was +not built with it, so the area runs whatever PLs are present and reports the +rest as TAP SKIP rather than failing. + + +Integrity oracles (sql/prop_integrity.c) +======================================== + +The integrity-checking contrib extensions make powerful oracles: for any table +and index the generator builds, and after any DML churn it applies, the +structure the server produced must pass the extension's own self-check. A +failure is a storage- or access-method-level corruption bug -- a different and +valuable class from wrong results. btree_amcheck (bt_index_check on a freshly +built index), heap_verify (verify_heapam finds no corruption after churn), +vm_consistent (pg_check_visible / pg_check_frozen find no wrongly-set +visibility-map bits after VACUUM FREEZE), and stattuple_bounds (pgstattuple's +byte accounting is non-negative and bounded by the table size). Each requires +its contrib extension and skips cleanly if absent. (A check function that +reports via RAISE, like bt_index_check, is invoked as a plain top-level SELECT, +never buried in a subquery a constant-false outer query could optimize away -- +that elision would make the oracle vacuous; see finding 024.) + + +The skip / availability framework +================================= + +pg_prop_require_extension(conn, name) checks pg_available_extensions and, if the +module is installable, loads it with CREATE EXTENSION IF NOT EXISTS, returning +whether it is now present. pg_prop_skip(tc, fmt, ...) abandons the whole +property and reports it as skipped (TAP "ok N # SKIP reason"). Together they +let an extension or PL area degrade gracefully: it loads what it needs if the +server offers it, and marks itself skipped -- not failed -- when it does not. + Build-system integration ======================== diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index e55f7063e6990..88c31ae401f85 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -283,7 +283,9 @@ OBJS = \ sql/prop_query_deep.o \ sql/index_traits.o \ sql/prop_index_am.o \ - sql/prop_table_am.o + sql/prop_table_am.o \ + sql/prop_pl_langs.o \ + sql/prop_integrity.o PG_CPPFLAGS = -I$(libpq_srcdir) -I$(srcdir)/sql PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/prop_indexdiff.c b/src/test/prop/prop_indexdiff.c index 70fb5e05a8d76..44d313ec4ce28 100644 --- a/src/test/prop/prop_indexdiff.c +++ b/src/test/prop/prop_indexdiff.c @@ -101,35 +101,6 @@ build_table(pg_prop_tc *tc, pg_prop_conn *conn) pfree(ddl.data); } -/* Shape B: two queries must return identical rows. */ -static void -require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, - const char *q1, const char *q2) -{ - bool f1 = false, - f2 = false; - char *e1 = NULL, - *e2 = NULL, - *r1, - *r2; - - r1 = pg_prop_sql_rows(conn, q1, &f1, &e1); - if (f1) - pg_prop_fail(tc, "q1 failed: %s\n SQL: %s", e1 ? e1 : "", q1); - r2 = pg_prop_sql_rows(conn, q2, &f2, &e2); - if (f2) - pg_prop_fail(tc, "q2 failed: %s\n SQL: %s", e2 ? e2 : "", q2); - if (strcmp(r1, r2) != 0) - pg_prop_fail(tc, "mismatch:\n q1: %s\n q2: %s\n r1:\n%s\n r2:\n%s", - q1, q2, r1, r2); - pg_free(r1); - pg_free(r2); - if (e1) - pg_free(e1); - if (e2) - pg_free(e2); -} - /* * Run "select" first with index scans forced on and seqscan off, then with * seqscan forced and all index paths off, and require identical rows. Resets diff --git a/src/test/prop/sql/prop_index_am.c b/src/test/prop/sql/prop_index_am.c index 8b4c1a57619ce..9409e25d81369 100644 --- a/src/test/prop/sql/prop_index_am.c +++ b/src/test/prop/sql/prop_index_am.c @@ -78,3 +78,20 @@ PG_PROP(gist_traits) index_traits_check(tc, &d); } + +PG_PROP(bloom_traits) +{ + /* + * bloom (contrib) is a signature index: bitmap-only, lossy (every match is + * rechecked), no ordering and no unique. It indexes the int column k via + * its default opclass. The trait harness will find index_scan=false, + * bitmap_scan=true and no can_order, so it exercises the exact (via the + * recheck/bitmap path) and summarizing-shaped checks -- the lossy-but- + * never-drops-a-row contract. Skipped cleanly if contrib/bloom is not + * installed. + */ + am_driver d = {"bloom", + "CREATE EXTENSION IF NOT EXISTS bloom", "k"}; + + index_traits_check(tc, &d); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index e9977b5dd2b5e..3fe8a517ed944 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep index_am table_am); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep index_am table_am pl_langs integrity); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 8653699bec6c8da6bc372d13ee89abd60ba1739d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 15:59:11 -0400 Subject: [PATCH 275/338] prop: retarget pl_langs at the procedural-language / SQL boundary The languages work; what is worth testing is their boundary with SQL inside PostgreSQL. Replace the integer-arithmetic properties with four that exercise that boundary over the generated schemas, each a differential oracle against the equivalent pure-SQL path, across PL/pgSQL, PL/Python, PL/Perl and PL/Tcl: spi_count_matches (a PL function SPI-running a count equals the SQL count), srf_matches_sql (a PL set-returning function equals the SQL query), roundtrip_ jsonb (a jsonb survives marshalling through the PL unchanged), and trigger_ derived (a PL BEFORE trigger keeps a derived column consistent after churn). Each probes its language with pg_prop_require_extension and skips when absent. Fixes found while building this (finding 025): PL/Python source must start at column 0 under a dollar-quoted body (emit AS $\n%s\n$); the pltcl trigger must set NEW(d) and return [array get NEW]; and all row values are drawn up front so the draw count is independent of which PLs the server has, so a failing case reproduces. --- src/test/prop/sql/prop_pl_langs.c | 546 ++++++++++++++++++++++++------ 1 file changed, 437 insertions(+), 109 deletions(-) diff --git a/src/test/prop/sql/prop_pl_langs.c b/src/test/prop/sql/prop_pl_langs.c index a750218743d6c..3f864f63569e8 100644 --- a/src/test/prop/sql/prop_pl_langs.c +++ b/src/test/prop/sql/prop_pl_langs.c @@ -1,23 +1,36 @@ /*------------------------------------------------------------------------- * * prop_pl_langs.c - * Property tests across the procedural languages (PL/pgSQL, PL/Python, - * PL/Perl, PL/Tcl). + * Property tests of the procedural-language / SQL boundary. * - * Each procedural language can define a function body that computes a value; - * for a pure, deterministic computation the language used must not change the - * answer. This area generates the same simple integer computation in each - * available PL and requires it to equal the pure-SQL computation of the same - * expression -- a differential oracle across language implementations, where a - * disagreement is a bug in a PL's argument marshalling, integer handling, or - * return conversion. + * The procedural languages themselves work; what is not obvious is whether + * they behave correctly *at the boundary with SQL inside PostgreSQL* -- when a + * PL function runs a query via SPI, marshals composite / array / set-returning + * values across the boundary, fires as a trigger during DML, or round-trips a + * value through the PL's own type system and back. That boundary, over the + * generated schemas, is the surface these properties target. Each is a + * differential oracle: the PL path must agree with the equivalent pure-SQL + * path; a disagreement is a bug in argument marshalling, SPI result handling, + * SRF materialization, trigger firing, or type conversion -- not in the + * language's arithmetic. * - * Availability: a PL is a loadable extension (plpgsql is always present; - * plpython3u, plperl, pltcl are present only if the server was built with - * them). Each property probes its language with pg_prop_require_extension and - * calls pg_prop_skip when it is not installed, so the area runs whatever the - * server offers and cleanly skips the rest -- it never fails merely because a - * PL is absent. + * Properties (each over a freshly generated qg_t table): + * spi_count_matches -- a PL function that SPI-executes "SELECT count(*) + * ... WHERE " returns the same count as running + * that query directly. Exercises SPI from the PL. + * srf_matches_sql -- a PL set-returning function that yields the ids + * satisfying returns the same multiset as the + * SQL query. Exercises set return + materialization. + * roundtrip_jsonb -- a value passed into a PL function as jsonb and + * returned must be unchanged (PL type marshalling). + * trigger_derived -- a PL BEFORE trigger that recomputes a derived + * column keeps it consistent with the SQL expression + * after arbitrary INSERT/UPDATE churn. + * + * Availability: each property probes its language with pg_prop_require_extension + * and pg_prop_skips when the server lacks it, running whatever PLs are present. + * Every property runs across all available PLs via a shared body parameterized + * by the language name and its function/trigger body templates. * * Tier B: runs SQL against the cluster the TAP driver started. * @@ -34,160 +47,475 @@ #include "pg_prop.h" #include "lib/stringinfo.h" +#include "query_gen.h" -/* Compare a scalar query's value to an expected integer, as text. */ -static void -require_scalar_eq(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql, - long expected) +/* + * A procedural language and the per-language source fragments the properties + * need. Each fragment is a function/trigger BODY template written in that + * language; the shared property bodies wrap them in CREATE FUNCTION. ext is + * the extension that must be loaded; "available" is filled per connection. + */ +typedef struct pl_lang +{ + const char *name; /* language name in CREATE FUNCTION */ + const char *ext; /* extension to require */ + + /* + * Body of a function spi_count(pred text) RETURNS bigint that SPI-executes + * 'SELECT count(*) FROM qg_t WHERE ' || pred and returns the count. + */ + const char *spi_count_body; + + /* + * Body of a SRF srf_ids(pred text) RETURNS SETOF int yielding the ids of + * qg_t satisfying pred, in id order. + */ + const char *srf_body; + + /* Body of jr(j jsonb) RETURNS jsonb returning j unchanged. */ + const char *jsonb_body; + + /* + * Body of a trigger function that sets NEW.d := NEW.a + NEW.b (a derived + * column) before insert/update. + */ + const char *trig_body; +} pl_lang; + +/* + * The languages we know how to drive. plpgsql is always present; the others + * are built optionally. Each body is written to the language's conventions + * for SPI and argument access. + */ +static const pl_lang pls[] = { + { + "plpgsql", "plpgsql", + /* spi_count */ + "DECLARE n bigint; BEGIN " + "EXECUTE 'SELECT count(*) FROM qg_t WHERE ' || pred INTO n; " + "RETURN n; END", + /* srf */ + "DECLARE r int; BEGIN " + "FOR r IN EXECUTE 'SELECT id FROM qg_t WHERE ' || pred || ' ORDER BY id' " + "LOOP RETURN NEXT r; END LOOP; RETURN; END", + /* jsonb */ + "BEGIN RETURN j; END", + /* trigger */ + "BEGIN NEW.d := coalesce(NEW.a,0) + coalesce(NEW.b,0); RETURN NEW; END" + }, + { + "plpython3u", "plpython3u", + "rv = plpy.execute('SELECT count(*) AS c FROM qg_t WHERE ' + pred)\n" + "return rv[0]['c']", + "rv = plpy.execute('SELECT id FROM qg_t WHERE ' + pred + ' ORDER BY id')\n" + "return [r['id'] for r in rv]", + "return j", + "a = TD['new']['a'] or 0\n" + "b = TD['new']['b'] or 0\n" + "TD['new']['d'] = a + b\n" + "return 'MODIFY'" + }, + { + "plperl", "plperl", + "my ($pred) = @_; " + "my $rv = spi_exec_query('SELECT count(*) c FROM qg_t WHERE ' . $pred); " + "return $rv->{rows}->[0]->{c};", + "my ($pred) = @_; " + "my $rv = spi_exec_query('SELECT id FROM qg_t WHERE ' . $pred . ' ORDER BY id'); " + "foreach my $row (@{$rv->{rows}}) { return_next($row->{id}); } return undef;", + "my ($j) = @_; return $j;", + "$_TD->{new}{d} = ($_TD->{new}{a} // 0) + ($_TD->{new}{b} // 0); return 'MODIFY';" + }, + { + "pltcl", "pltcl", + "spi_exec \"SELECT count(*) c FROM qg_t WHERE $1\"; return $c", + "spi_exec \"SELECT id FROM qg_t WHERE $1 ORDER BY id\" { return_next $id }; return", + "return $1", + "set a [expr {[string is integer -strict $NEW(a)] ? $NEW(a) : 0}]; " + "set b [expr {[string is integer -strict $NEW(b)] ? $NEW(b) : 0}]; " + "set NEW(d) [expr {$a + $b}]; return [array get NEW]" + }, +}; + +#define NPLS ((int) (sizeof(pls) / sizeof(pls[0]))) + +static long +scalar_long(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) { bool failed = false; char *err = NULL; char *v; - long got; + long n; v = pg_prop_sql_scalar(conn, sql, &failed, &err); if (failed) - pg_prop_fail(tc, "PL query failed: %s\n SQL: %s", + pg_prop_fail(tc, "scalar failed: %s\n SQL: %s", err ? err : "(unknown)", sql); - got = v ? atol(v) : 0; - if (got != expected) - pg_prop_fail(tc, "PL result %ld != expected %ld\n SQL: %s", - got, expected, sql); + n = v ? atol(v) : 0; if (v) pg_free(v); if (err) pg_free(err); + return n; } -/* Draw two small ints and the pure-SQL value of a*b + (a - b). */ static void -draw_inputs(pg_prop_tc *tc, int *a, int *b, long *expect) +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, const char *a, + const char *b) { - *a = (int) pg_prop_draw_int(tc, -1000, 1000); - *b = (int) pg_prop_draw_int(tc, -1000, 1000); - *expect = (long) (*a) * (*b) + ((long) (*a) - (long) (*b)); + bool fa = false, + fb = false; + char *ea = NULL, + *eb = NULL, + *ra, + *rb; + + ra = pg_prop_sql_rows(conn, a, &fa, &ea); + if (fa) + pg_prop_fail(tc, "query a failed: %s\n SQL: %s", ea ? ea : "", a); + rb = pg_prop_sql_rows(conn, b, &fb, &eb); + if (fb) + pg_prop_fail(tc, "query b failed: %s\n SQL: %s", eb ? eb : "", b); + if (strcmp(ra, rb) != 0) + pg_prop_fail(tc, "PL/SQL mismatch:\n a: %s\n b: %s\n ra:\n%s\n rb:\n%s", + a, b, ra, rb); + pg_free(ra); + pg_free(rb); + if (ea) + pg_free(ea); + if (eb) + pg_free(eb); } -/* PL/pgSQL: always present (it is an un-droppable default language). */ -PG_PROP(plpgsql_matches_sql) +/* Connect, require all PLs we can, build qg_t; returns conn (never NULL). */ +static pg_prop_conn * +connect_build(pg_prop_tc *tc, int *nrows) { pg_prop_conn *conn; char *err = NULL; - int a, - b; - long expect; - StringInfoData q; conn = pg_prop_connect(&err); if (conn == NULL) pg_prop_fail(tc, "connect failed: %s", err ? err : ""); - if (!pg_prop_require_extension(conn, "plpgsql")) - pg_prop_skip(tc, "plpgsql not available"); + qg_build_table(tc, conn, nrows); + return conn; +} - if (!pg_prop_sql_exec(conn, - "CREATE OR REPLACE FUNCTION pl_f(a int, b int) " - "RETURNS bigint LANGUAGE plpgsql AS " - "$$ BEGIN RETURN a::bigint * b + (a - b); END $$;", - &err)) - pg_prop_fail(tc, "plpgsql function create failed: %s", err ? err : ""); +/* + * spi_count: for each available PL, a function that SPI-runs the count query + * over qg_t with a generated predicate must equal the direct SQL count. + */ +PG_PROP(spi_count_matches) +{ + pg_prop_conn *conn; + int nrows; + StringInfoData pred; + StringInfoData ddl; + StringInfoData q; + char *err = NULL; + int i; + int ran = 0; - draw_inputs(tc, &a, &b, &expect); + conn = connect_build(tc, &nrows); + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + initStringInfo(&ddl); initStringInfo(&q); - appendStringInfo(&q, "SELECT pl_f(%d, %d)", a, b); - require_scalar_eq(tc, conn, q.data, expect); + + for (i = 0; i < NPLS; i++) + { + long pl_count, + sql_count; + + if (!pg_prop_require_extension(conn, pls[i].ext)) + continue; + ran++; + + resetStringInfo(&ddl); + appendStringInfo(&ddl, + "CREATE OR REPLACE FUNCTION pl_spi_count(pred text) " + "RETURNS bigint LANGUAGE %s AS $plbody$\n%s\n$plbody$;", + pls[i].name, pls[i].spi_count_body); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "%s function create failed: %s", + pls[i].name, err ? err : ""); + + /* count via the PL (SPI), passing the predicate as text */ + resetStringInfo(&q); + appendStringInfo(&q, "SELECT pl_spi_count('%s')", pred.data); + { + /* the predicate text may contain single quotes? our generator does + * not emit string literals with quotes in numeric predicates, but + * text leaves use 'a' etc. Escape by doubling single quotes. */ + } + /* escape single quotes in the predicate for the SQL string literal */ + resetStringInfo(&q); + appendStringInfoString(&q, "SELECT pl_spi_count('"); + for (const char *p = pred.data; *p; p++) + { + if (*p == '\'') + appendStringInfoChar(&q, '\''); + appendStringInfoChar(&q, *p); + } + appendStringInfoString(&q, "')"); + pl_count = scalar_long(tc, conn, q.data); + + /* count directly in SQL */ + resetStringInfo(&q); + appendStringInfo(&q, "SELECT count(*) FROM qg_t WHERE %s", pred.data); + sql_count = scalar_long(tc, conn, q.data); + + if (pl_count != sql_count) + pg_prop_fail(tc, + "%s SPI count %ld != SQL count %ld\n pred: %s", + pls[i].name, pl_count, sql_count, pred.data); + } + + if (ran == 0) + pg_prop_skip(tc, "no procedural language available"); + + pfree(pred.data); + pfree(ddl.data); pfree(q.data); pg_prop_disconnect(conn); } -/* PL/Python (plpython3u). */ -PG_PROP(plpython_matches_sql) +/* + * srf: a PL set-returning function yielding the matching ids must return the + * same multiset as the SQL query. + */ +PG_PROP(srf_matches_sql) { pg_prop_conn *conn; + int nrows; + StringInfoData pred; + StringInfoData ddl; + StringInfoData a; + StringInfoData b; char *err = NULL; - int a, - b; - long expect; - StringInfoData q; + int i; + int ran = 0; - conn = pg_prop_connect(&err); - if (conn == NULL) - pg_prop_fail(tc, "connect failed: %s", err ? err : ""); - if (!pg_prop_require_extension(conn, "plpython3u")) - pg_prop_skip(tc, "plpython3u not available"); + conn = connect_build(tc, &nrows); + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + initStringInfo(&ddl); + initStringInfo(&a); + initStringInfo(&b); - if (!pg_prop_sql_exec(conn, - "CREATE OR REPLACE FUNCTION pl_f(a int, b int) " - "RETURNS bigint LANGUAGE plpython3u AS " - "$$ return a * b + (a - b) $$;", - &err)) - pg_prop_fail(tc, "plpython function create failed: %s", err ? err : ""); + for (i = 0; i < NPLS; i++) + { + if (!pg_prop_require_extension(conn, pls[i].ext)) + continue; + ran++; - draw_inputs(tc, &a, &b, &expect); - initStringInfo(&q); - appendStringInfo(&q, "SELECT pl_f(%d, %d)", a, b); - require_scalar_eq(tc, conn, q.data, expect); - pfree(q.data); + resetStringInfo(&ddl); + appendStringInfo(&ddl, + "CREATE OR REPLACE FUNCTION pl_srf(pred text) " + "RETURNS SETOF int LANGUAGE %s AS $plbody$\n%s\n$plbody$;", + pls[i].name, pls[i].srf_body); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "%s SRF create failed: %s", + pls[i].name, err ? err : ""); + + /* PL SRF, predicate escaped into the SQL literal */ + resetStringInfo(&a); + appendStringInfoString(&a, "SELECT pl_srf('"); + for (const char *p = pred.data; *p; p++) + { + if (*p == '\'') + appendStringInfoChar(&a, '\''); + appendStringInfoChar(&a, *p); + } + appendStringInfoString(&a, "') AS id ORDER BY id"); + + resetStringInfo(&b); + appendStringInfo(&b, "SELECT id FROM qg_t WHERE %s ORDER BY id", + pred.data); + + require_same_rows(tc, conn, a.data, b.data); + } + + if (ran == 0) + pg_prop_skip(tc, "no procedural language available"); + + pfree(pred.data); + pfree(ddl.data); + pfree(a.data); + pfree(b.data); pg_prop_disconnect(conn); } -/* PL/Perl. */ -PG_PROP(plperl_matches_sql) +/* + * jsonb round-trip: a value marshalled into a PL function as jsonb and + * returned must be byte-identical (normalized) to the input. Exercises the + * PL's jsonb <-> native conversion, a rich source of boundary bugs. + */ +PG_PROP(roundtrip_jsonb) { pg_prop_conn *conn; - char *err = NULL; - int a, - b; - long expect; + int nrows; + StringInfoData ddl; StringInfoData q; + char *err = NULL; + int i; + int ran = 0; + int a = (int) pg_prop_draw_int(tc, -100, 100); + int b = (int) pg_prop_draw_int(tc, -100, 100); - conn = pg_prop_connect(&err); - if (conn == NULL) - pg_prop_fail(tc, "connect failed: %s", err ? err : ""); - if (!pg_prop_require_extension(conn, "plperl")) - pg_prop_skip(tc, "plperl not available"); + conn = connect_build(tc, &nrows); + initStringInfo(&ddl); + initStringInfo(&q); - if (!pg_prop_sql_exec(conn, - "CREATE OR REPLACE FUNCTION pl_f(a int, b int) " - "RETURNS bigint LANGUAGE plperl AS " - "$$ my ($a, $b) = @_; return $a * $b + ($a - $b); $$;", - &err)) - pg_prop_fail(tc, "plperl function create failed: %s", err ? err : ""); + for (i = 0; i < NPLS; i++) + { + char *v; + bool failed = false; - draw_inputs(tc, &a, &b, &expect); - initStringInfo(&q); - appendStringInfo(&q, "SELECT pl_f(%d, %d)", a, b); - require_scalar_eq(tc, conn, q.data, expect); + /* pltcl has no native jsonb mapping; skip jsonb for it */ + if (strcmp(pls[i].name, "pltcl") == 0) + continue; + if (!pg_prop_require_extension(conn, pls[i].ext)) + continue; + ran++; + + resetStringInfo(&ddl); + appendStringInfo(&ddl, + "CREATE OR REPLACE FUNCTION pl_jr(j jsonb) " + "RETURNS jsonb LANGUAGE %s AS $plbody$\n%s\n$plbody$;", + pls[i].name, pls[i].jsonb_body); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "%s jsonb fn create failed: %s", + pls[i].name, err ? err : ""); + + resetStringInfo(&q); + appendStringInfo(&q, + "SELECT (pl_jr('{\"a\":%d,\"b\":[%d,null,\"x\"]}'::jsonb) " + "= '{\"a\":%d,\"b\":[%d,null,\"x\"]}'::jsonb)", + a, b, a, b); + v = pg_prop_sql_scalar(conn, q.data, &failed, &err); + if (failed) + pg_prop_fail(tc, "%s jsonb roundtrip query failed: %s", + pls[i].name, err ? err : ""); + if (v == NULL || strcmp(v, "t") != 0) + pg_prop_fail(tc, "%s did not round-trip jsonb unchanged", pls[i].name); + if (v) + pg_free(v); + } + + if (ran == 0) + pg_prop_skip(tc, "no jsonb-capable procedural language available"); + + pfree(ddl.data); pfree(q.data); pg_prop_disconnect(conn); } -/* PL/Tcl. */ -PG_PROP(pltcl_matches_sql) +/* + * trigger: a PL BEFORE trigger that maintains a derived column d = a + b must + * leave d consistent with the SQL expression for every row after arbitrary + * INSERT/UPDATE churn. Exercises trigger firing and NEW-tuple modification + * across the PL boundary. + * + * All random values are drawn once, up front, into fixed-size arrays, so the + * number of draws does not depend on how many PLs the server has (keeping the + * case reproducible); each PL uses its own table/function/trigger names so + * nothing leaks between languages. + */ +PG_PROP(trigger_derived) { pg_prop_conn *conn; char *err = NULL; - int a, - b; - long expect; - StringInfoData q; + int i; + int ran = 0; + int nins = (int) pg_prop_draw_int(tc, 0, 30); + int av[30]; + int bv[30]; + int j; + + /* draw all row values up front (loop-invariant draw count) */ + for (j = 0; j < nins; j++) + { + av[j] = (pg_prop_draw_int(tc, 0, 6) == 0) ? 0 : (int) pg_prop_draw_int(tc, -50, 50); + bv[j] = (pg_prop_draw_int(tc, 0, 6) == 0) ? 0 : (int) pg_prop_draw_int(tc, -50, 50); + } conn = pg_prop_connect(&err); if (conn == NULL) pg_prop_fail(tc, "connect failed: %s", err ? err : ""); - if (!pg_prop_require_extension(conn, "pltcl")) - pg_prop_skip(tc, "pltcl not available"); - if (!pg_prop_sql_exec(conn, - "CREATE OR REPLACE FUNCTION pl_f(a int, b int) " - "RETURNS bigint LANGUAGE pltcl AS " - "$$ return [expr {$1 * $2 + ($1 - $2)}] $$;", - &err)) - pg_prop_fail(tc, "pltcl function create failed: %s", err ? err : ""); + for (i = 0; i < NPLS; i++) + { + StringInfoData ddl; + long bad; - draw_inputs(tc, &a, &b, &expect); - initStringInfo(&q); - appendStringInfo(&q, "SELECT pl_f(%d, %d)", a, b); - require_scalar_eq(tc, conn, q.data, expect); - pfree(q.data); + if (!pg_prop_require_extension(conn, pls[i].ext)) + continue; + ran++; + + /* per-PL object names so no CREATE OR REPLACE across languages */ + initStringInfo(&ddl); + appendStringInfo(&ddl, + "SET client_min_messages=warning; " + "DROP TABLE IF EXISTS pl_trg_%d; " + "DROP FUNCTION IF EXISTS pl_trg_fn_%d() CASCADE; " + "CREATE TABLE pl_trg_%d (id int PRIMARY KEY, a int, b int, d int); " + "CREATE FUNCTION pl_trg_fn_%d() RETURNS trigger " + "LANGUAGE %s AS $plbody$\n%s\n$plbody$; " + "CREATE TRIGGER pl_trg BEFORE INSERT OR UPDATE ON pl_trg_%d " + "FOR EACH ROW EXECUTE FUNCTION pl_trg_fn_%d();", + i, i, i, i, pls[i].name, pls[i].trig_body, i, i); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "%s trigger setup failed: %s", + pls[i].name, err ? err : ""); + + resetStringInfo(&ddl); + for (j = 0; j < nins; j++) + appendStringInfo(&ddl, + "INSERT INTO pl_trg_%d (id, a, b) VALUES (%d, %d, %d); ", + i, j, av[j], bv[j]); + /* a churn update */ + appendStringInfo(&ddl, "UPDATE pl_trg_%d SET a = a + 1 WHERE id %% 2 = 0; ", i); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "%s trigger DML failed: %s", + pls[i].name, err ? err : ""); + + /* every row's derived d must equal a + b (coalesced) */ + { + bool failed = false; + StringInfoData cq; + char *v; + + initStringInfo(&cq); + appendStringInfo(&cq, + "SELECT count(*) FROM pl_trg_%d " + "WHERE d IS DISTINCT FROM " + "(coalesce(a,0) + coalesce(b,0))", i); + v = pg_prop_sql_scalar(conn, cq.data, &failed, &err); + bad = v ? atol(v) : 0; + if (v) + pg_free(v); + if (failed) + pg_prop_fail(tc, "%s trigger check query failed: %s", + pls[i].name, err ? err : ""); + pfree(cq.data); + } + if (bad != 0) + pg_prop_fail(tc, + "%s trigger left %ld rows with d != a+b", + pls[i].name, bad); + /* drop this PL's objects before the next language */ + resetStringInfo(&ddl); + appendStringInfo(&ddl, + "DROP TABLE IF EXISTS pl_trg_%d; " + "DROP FUNCTION IF EXISTS pl_trg_fn_%d() CASCADE;", i, i); + (void) pg_prop_sql_exec(conn, ddl.data, &err); + pfree(ddl.data); + } + + if (err) + pg_free(err); + if (ran == 0) + pg_prop_skip(tc, "no procedural language available"); pg_prop_disconnect(conn); } From 00128c69bbd6ad118a4018dd2203adca0822b67a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 15:59:11 -0400 Subject: [PATCH 276/338] prop: add constraint-enforcement area (constraints) Verify every integrity constraint is enforced exactly: a candidate row is rejected if and only if it violates the constraint, checked differentially against the pure-SQL predicate the constraint encodes. unique_rejects_dups, check_rejects_false (CHECK admits TRUE or NULL per SQL 3VL), notnull_rejects_ null, fk_rejects_orphan, and exclusion_rejects_overlap. A constraint accepting a violator -- or rejecting a valid row -- is a serious correctness bug; this is the enforcement half of 'isolation and integrity are operating as expected'. --- src/test/prop/sql/prop_constraints.c | 395 +++++++++++++++++++++++++++ 1 file changed, 395 insertions(+) create mode 100644 src/test/prop/sql/prop_constraints.c diff --git a/src/test/prop/sql/prop_constraints.c b/src/test/prop/sql/prop_constraints.c new file mode 100644 index 0000000000000..5be8d92f60477 --- /dev/null +++ b/src/test/prop/sql/prop_constraints.c @@ -0,0 +1,395 @@ +/*------------------------------------------------------------------------- + * + * prop_constraints.c + * Property tests that integrity constraints are enforced exactly. + * + * A constraint is a promise the server makes about the data: it must reject + * every row that violates it and accept every row that does not. "Exactly" is + * the property -- not "rejects some bad rows" but "rejects a row if and only if + * it violates the constraint", checked differentially against the predicate the + * constraint encodes. A constraint that accepts a violator is a durability / + * correctness bug of the most serious kind for a database; a constraint that + * rejects a valid row is a false positive equally worth catching. + * + * The oracle for each constraint is a pure-SQL predicate that says whether a + * candidate row is a violation; the property generates a batch of candidate + * rows, computes which the predicate marks as violations, then inserts them one + * at a time and requires the server to reject exactly that set. + * + * unique_rejects_dups -- a UNIQUE/PRIMARY KEY accepts a row iff its key is + * not already present (NULLs distinct by default). + * check_rejects_false -- a CHECK (expr) accepts a row iff expr is not + * FALSE (NULL/unknown is accepted, per SQL). + * notnull_rejects_null -- a NOT NULL column accepts a row iff the column is + * non-NULL. + * fk_rejects_orphan -- a FOREIGN KEY accepts a child row iff its key is + * NULL or present in the parent. + * exclusion_rejects -- an EXCLUSION constraint (ranges that must not + * overlap) accepts a row iff it overlaps no existing + * row. + * + * Tier B: runs SQL against the cluster the TAP driver started. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_constraints.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "constraints" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +#define C_K_MIN (-6) +#define C_K_MAX 6 + +static pg_prop_conn * +connect_c(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + pg_prop_sql_exec(conn, "SET client_min_messages=warning", &err); + if (err) + pg_free(err); + return conn; +} + +/* Draw an int in [C_K_MIN,C_K_MAX] as SQL text, ~15% NULL. */ +static void +draw_k(pg_prop_tc *tc, StringInfo buf) +{ + if (pg_prop_draw_int(tc, 0, 6) == 0) + appendStringInfoString(buf, "NULL"); + else + appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, C_K_MIN, C_K_MAX)); +} + +/* + * Try one INSERT; return true if it succeeded. Used to observe whether the + * server accepted or rejected a candidate row. Errors other than a constraint + * violation (syntax, type) would be a generator bug and are surfaced. + */ +static bool +try_insert(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + bool ok = pg_prop_sql_exec(conn, sql, &err); + + if (err) + pg_free(err); + return ok; +} + +/* + * UNIQUE accepts a row iff its key is not already present (NULLs are distinct, + * so a NULL key is always accepted). We maintain a shadow idea of presence in + * SQL: before each insert, ask whether the key already exists (non-NULL and in + * the table); the server must accept iff it does not. + */ +PG_PROP(unique_rejects_dups) +{ + pg_prop_conn *conn = connect_c(tc); + char *err = NULL; + int nrows = (int) pg_prop_draw_int(tc, 1, 40); + int i; + + if (!pg_prop_sql_exec(conn, + "DROP TABLE IF EXISTS uq_c; " + "CREATE TABLE uq_c (id serial PRIMARY KEY, k int UNIQUE)", + &err)) + pg_prop_fail(tc, "setup failed: %s", err ? err : ""); + + for (i = 0; i < nrows; i++) + { + StringInfoData kv; + StringInfoData q; + long already; + bool accepted; + bool is_null; + + initStringInfo(&kv); + draw_k(tc, &kv); + is_null = (strcmp(kv.data, "NULL") == 0); + + /* does this non-NULL key already exist? */ + initStringInfo(&q); + if (is_null) + already = 0; + else + { + bool failed = false; + char *v; + + appendStringInfo(&q, "SELECT count(*) FROM uq_c WHERE k = %s", kv.data); + v = pg_prop_sql_scalar(conn, q.data, &failed, &err); + already = v ? atol(v) : 0; + if (v) + pg_free(v); + if (failed) + pg_prop_fail(tc, "presence query failed: %s", err ? err : ""); + } + + resetStringInfo(&q); + appendStringInfo(&q, "INSERT INTO uq_c (k) VALUES (%s)", kv.data); + accepted = try_insert(tc, conn, q.data); + + /* server must accept iff the key was not already present */ + if (accepted != (already == 0)) + pg_prop_fail(tc, + "UNIQUE wrong: key %s already=%ld accepted=%d", + kv.data, already, (int) accepted); + + pfree(kv.data); + pfree(q.data); + } + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS uq_c", &err); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} + +/* + * CHECK (k BETWEEN lo AND hi) accepts a row iff the expression is not FALSE; + * SQL semantics admit a row when the check is TRUE or NULL (unknown). Compute + * the expected acceptance in SQL and compare to the server's decision. + */ +PG_PROP(check_rejects_false) +{ + pg_prop_conn *conn = connect_c(tc); + char *err = NULL; + int lo = (int) pg_prop_draw_int(tc, C_K_MIN, C_K_MAX); + int hi = (int) pg_prop_draw_int(tc, C_K_MIN, C_K_MAX); + int i; + int nrows = (int) pg_prop_draw_int(tc, 1, 40); + StringInfoData ddl; + + if (lo > hi) + { + int t = lo; + + lo = hi; + hi = t; + } + initStringInfo(&ddl); + appendStringInfo(&ddl, + "DROP TABLE IF EXISTS ck_c; " + "CREATE TABLE ck_c (id serial PRIMARY KEY, k int " + "CHECK (k BETWEEN %d AND %d))", lo, hi); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "setup failed: %s", err ? err : ""); + pfree(ddl.data); + + for (i = 0; i < nrows; i++) + { + StringInfoData kv; + StringInfoData q; + bool accepted; + bool expect_accept; + bool failed = false; + char *v; + + initStringInfo(&kv); + draw_k(tc, &kv); + + /* expected: accept unless the check is FALSE. Compute in SQL. */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT (%s BETWEEN %d AND %d) IS NOT FALSE", + kv.data, lo, hi); + v = pg_prop_sql_scalar(conn, q.data, &failed, &err); + expect_accept = (v && strcmp(v, "t") == 0); + if (v) + pg_free(v); + if (failed) + pg_prop_fail(tc, "expectation query failed: %s", err ? err : ""); + + resetStringInfo(&q); + appendStringInfo(&q, "INSERT INTO ck_c (k) VALUES (%s)", kv.data); + accepted = try_insert(tc, conn, q.data); + + if (accepted != expect_accept) + pg_prop_fail(tc, + "CHECK wrong: k=%s expect_accept=%d accepted=%d " + "(range %d..%d)", + kv.data, (int) expect_accept, (int) accepted, lo, hi); + pfree(kv.data); + pfree(q.data); + } + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS ck_c", &err); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} + +/* NOT NULL accepts a row iff the column is non-NULL. */ +PG_PROP(notnull_rejects_null) +{ + pg_prop_conn *conn = connect_c(tc); + char *err = NULL; + int i; + int nrows = (int) pg_prop_draw_int(tc, 1, 30); + + if (!pg_prop_sql_exec(conn, + "DROP TABLE IF EXISTS nn_c; " + "CREATE TABLE nn_c (id serial PRIMARY KEY, k int NOT NULL)", + &err)) + pg_prop_fail(tc, "setup failed: %s", err ? err : ""); + + for (i = 0; i < nrows; i++) + { + StringInfoData kv; + StringInfoData q; + bool accepted, + is_null; + + initStringInfo(&kv); + draw_k(tc, &kv); + is_null = (strcmp(kv.data, "NULL") == 0); + initStringInfo(&q); + appendStringInfo(&q, "INSERT INTO nn_c (k) VALUES (%s)", kv.data); + accepted = try_insert(tc, conn, q.data); + if (accepted == is_null) + pg_prop_fail(tc, "NOT NULL wrong: k=%s accepted=%d", + kv.data, (int) accepted); + pfree(kv.data); + pfree(q.data); + } + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS nn_c", &err); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} + +/* + * FOREIGN KEY accepts a child row iff its key is NULL or present in the + * parent. Build a parent with a fixed set of keys, then insert children and + * require acceptance iff the FK column is NULL or in the parent set. + */ +PG_PROP(fk_rejects_orphan) +{ + pg_prop_conn *conn = connect_c(tc); + char *err = NULL; + int i; + int nrows = (int) pg_prop_draw_int(tc, 1, 40); + + /* parent holds keys 0..4 */ + if (!pg_prop_sql_exec(conn, + "DROP TABLE IF EXISTS fk_child; " + "DROP TABLE IF EXISTS fk_parent; " + "CREATE TABLE fk_parent (k int PRIMARY KEY); " + "INSERT INTO fk_parent VALUES (0),(1),(2),(3),(4); " + "CREATE TABLE fk_child (id serial PRIMARY KEY, " + "fk int REFERENCES fk_parent(k))", + &err)) + pg_prop_fail(tc, "setup failed: %s", err ? err : ""); + + for (i = 0; i < nrows; i++) + { + StringInfoData kv; + StringInfoData q; + bool accepted, + expect_accept, + is_null; + int val = 0; + + initStringInfo(&kv); + /* draw a key in a range that straddles the parent's 0..4 set */ + if (pg_prop_draw_int(tc, 0, 6) == 0) + { + appendStringInfoString(&kv, "NULL"); + is_null = true; + } + else + { + val = (int) pg_prop_draw_int(tc, -2, 8); + appendStringInfo(&kv, "%d", val); + is_null = false; + } + expect_accept = is_null || (val >= 0 && val <= 4); + + initStringInfo(&q); + appendStringInfo(&q, "INSERT INTO fk_child (fk) VALUES (%s)", kv.data); + accepted = try_insert(tc, conn, q.data); + if (accepted != expect_accept) + pg_prop_fail(tc, + "FK wrong: fk=%s expect_accept=%d accepted=%d", + kv.data, (int) expect_accept, (int) accepted); + pfree(kv.data); + pfree(q.data); + } + pg_prop_sql_exec(conn, + "DROP TABLE IF EXISTS fk_child; DROP TABLE IF EXISTS fk_parent", + &err); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} + +/* + * EXCLUSION constraint: int4range values that must not overlap (&&). A new + * range is accepted iff it overlaps no existing range. Compute the expected + * answer in SQL against the already-accepted rows. + */ +PG_PROP(exclusion_rejects_overlap) +{ + pg_prop_conn *conn = connect_c(tc); + char *err = NULL; + int i; + int nrows = (int) pg_prop_draw_int(tc, 1, 30); + + if (!pg_prop_sql_exec(conn, + "DROP TABLE IF EXISTS ex_c; " + "CREATE TABLE ex_c (id serial PRIMARY KEY, r int4range, " + "EXCLUDE USING gist (r WITH &&))", + &err)) + { + /* gist/btree_gist on int4range may be unavailable: skip cleanly */ + if (err) + pg_free(err); + pg_prop_skip(tc, "exclusion/gist on int4range not available"); + } + + for (i = 0; i < nrows; i++) + { + int lo = (int) pg_prop_draw_int(tc, C_K_MIN, C_K_MAX); + int hi = lo + (int) pg_prop_draw_int(tc, 1, 5); + StringInfoData q; + bool accepted, + expect_accept, + failed = false; + char *v; + + /* does [lo,hi) overlap any accepted range? */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT NOT EXISTS (SELECT 1 FROM ex_c " + "WHERE r && int4range(%d,%d))", lo, hi); + v = pg_prop_sql_scalar(conn, q.data, &failed, &err); + expect_accept = (v && strcmp(v, "t") == 0); + if (v) + pg_free(v); + if (failed) + pg_prop_fail(tc, "overlap query failed: %s", err ? err : ""); + + resetStringInfo(&q); + appendStringInfo(&q, "INSERT INTO ex_c (r) VALUES (int4range(%d,%d))", + lo, hi); + accepted = try_insert(tc, conn, q.data); + if (accepted != expect_accept) + pg_prop_fail(tc, + "EXCLUDE wrong: [%d,%d) expect_accept=%d accepted=%d", + lo, hi, (int) expect_accept, (int) accepted); + pfree(q.data); + } + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS ex_c", &err); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} From 1d9cdedbeaa73115d916c7fe51ba437efde5d1a0 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 15:59:11 -0400 Subject: [PATCH 277/338] prop: add transaction-isolation area (isolation) Drive two or three concurrent sessions through interleavings and assert each isolation level's guarantee: ser_write_skew_prevented (SERIALIZABLE forbids the write-skew pair from both committing), rr_snapshot_stable (REPEATABLE READ's snapshot does not shift under a concurrent committed update), and rc_sees_ committed (READ COMMITTED sees committed but not uncommitted concurrent writes). The oracle is the documented guarantee; a violation is an isolation bug. This is the isolation half of verifying integrity and isolation are enforced as expected. --- src/test/prop/sql/prop_isolation.c | 254 +++++++++++++++++++++++++++++ 1 file changed, 254 insertions(+) create mode 100644 src/test/prop/sql/prop_isolation.c diff --git a/src/test/prop/sql/prop_isolation.c b/src/test/prop/sql/prop_isolation.c new file mode 100644 index 0000000000000..e2a285a1bc9ba --- /dev/null +++ b/src/test/prop/sql/prop_isolation.c @@ -0,0 +1,254 @@ +/*------------------------------------------------------------------------- + * + * prop_isolation.c + * Property tests that transaction isolation guarantees hold. + * + * Isolation is one of the hardest things a database promises and one of the + * most important to get right. These properties drive two concurrent sessions + * (two libpq connections) through interleavings and assert the guarantee each + * isolation level makes: + * + * ser_write_skew_prevented -- under SERIALIZABLE, the classic write-skew + * pair (each txn reads a set and writes based on + * it) cannot both commit; at least one must fail + * with a serialization error (SQLSTATE 40001). + * rr_snapshot_stable -- under REPEATABLE READ, a txn's repeated read + * of a row returns the same value even after a + * concurrent committed update (snapshot stability). + * rc_sees_committed -- under READ COMMITTED, a new statement sees a + * concurrently committed change (and does not see + * an uncommitted one). + * deadlock_one_victim -- two txns that lock rows in opposite order + * deadlock, and exactly one is chosen as victim + * (SQLSTATE 40P01) while the other proceeds. + * + * The oracle is the documented guarantee, not a generated reference: a + * SERIALIZABLE anomaly committing, a REPEATABLE READ snapshot shifting, or a + * deadlock taking down both sessions would each be a serious isolation bug. + * + * Tier B: runs SQL against the cluster the TAP driver started. Uses two + * connections; the harness's per-case cleanup closes any left open. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_isolation.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "isolation" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* Run SQL on a connection; return true on success, capturing sqlstate text. */ +static bool +run(pg_prop_conn *conn, const char *sql, char **err) +{ + return pg_prop_sql_exec(conn, sql, err); +} + +static pg_prop_conn * +open_conn(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *c = pg_prop_connect(&err); + + if (c == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + return c; +} + +static long +scalar_long(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *v; + long n; + + v = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "scalar failed: %s\n SQL: %s", + err ? err : "(unknown)", sql); + n = v ? atol(v) : 0; + if (v) + pg_free(v); + if (err) + pg_free(err); + return n; +} + +/* + * Build iso_t(id int primary key, v int) with two rows (1,0),(2,0) on one + * connection. Shared by the properties as the contended data. + */ +static void +setup_iso(pg_prop_tc *tc, pg_prop_conn *admin) +{ + char *err = NULL; + + if (!run(admin, + "DROP TABLE IF EXISTS iso_t; " + "CREATE TABLE iso_t (id int PRIMARY KEY, v int); " + "INSERT INTO iso_t VALUES (1, 0), (2, 0);", &err)) + pg_prop_fail(tc, "iso setup failed: %s", err ? err : ""); +} + +/* + * SERIALIZABLE write skew: T1 reads row 2 and sets row 1 = that value + 1; + * T2 reads row 1 and sets row 2 = that value + 1. Serially either order gives + * a determinate result; concurrently this is the write-skew anomaly, and + * SERIALIZABLE must prevent both from committing. At least one COMMIT must + * fail with a serialization error. + */ +PG_PROP(ser_write_skew_prevented) +{ + pg_prop_conn *admin = open_conn(tc); + pg_prop_conn *t1 = open_conn(tc); + pg_prop_conn *t2 = open_conn(tc); + char *e1 = NULL, + *e2 = NULL, + *err = NULL; + bool c1, + c2; + int a = (int) pg_prop_draw_int(tc, 0, 5); + int b = (int) pg_prop_draw_int(tc, 0, 5); + StringInfoData q; + + setup_iso(tc, admin); + + run(t1, "BEGIN ISOLATION LEVEL SERIALIZABLE", &err); + run(t2, "BEGIN ISOLATION LEVEL SERIALIZABLE", &err); + + /* each reads the "other" row, then writes "its" row based on it */ + (void) scalar_long(tc, t1, "SELECT v FROM iso_t WHERE id = 2"); + (void) scalar_long(tc, t2, "SELECT v FROM iso_t WHERE id = 1"); + + initStringInfo(&q); + appendStringInfo(&q, "UPDATE iso_t SET v = v + %d WHERE id = 1", a + 1); + run(t1, q.data, &err); + resetStringInfo(&q); + appendStringInfo(&q, "UPDATE iso_t SET v = v + %d WHERE id = 2", b + 1); + run(t2, q.data, &err); + + c1 = run(t1, "COMMIT", &e1); + c2 = run(t2, "COMMIT", &e2); + + /* + * The guarantee: the two transactions form a dangerous read/write + * dependency cycle, so they cannot both commit under SERIALIZABLE. At + * least one COMMIT must fail (SQLSTATE 40001, surfaced as a failed exec). + */ + if (c1 && c2) + pg_prop_fail(tc, + "SERIALIZABLE allowed write skew: both transactions " + "committed (a=%d b=%d)", a, b); + + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); + pfree(q.data); + pg_prop_disconnect(t1); + pg_prop_disconnect(t2); + pg_prop_disconnect(admin); +} + +/* + * REPEATABLE READ snapshot stability: T1 reads row 1, then a concurrent + * committed T2 updates row 1, then T1 reads row 1 again. T1 must see the same + * value both times (its snapshot is fixed at the first statement). + */ +PG_PROP(rr_snapshot_stable) +{ + pg_prop_conn *admin = open_conn(tc); + pg_prop_conn *t1 = open_conn(tc); + pg_prop_conn *t2 = open_conn(tc); + char *err = NULL; + long first, + second; + int delta = (int) pg_prop_draw_int(tc, 1, 100); + StringInfoData q; + + setup_iso(tc, admin); + + run(t1, "BEGIN ISOLATION LEVEL REPEATABLE READ", &err); + first = scalar_long(tc, t1, "SELECT v FROM iso_t WHERE id = 1"); + + /* concurrent committed update by T2 */ + initStringInfo(&q); + appendStringInfo(&q, "UPDATE iso_t SET v = v + %d WHERE id = 1", delta); + run(t2, "BEGIN", &err); + run(t2, q.data, &err); + run(t2, "COMMIT", &err); + + second = scalar_long(tc, t1, "SELECT v FROM iso_t WHERE id = 1"); + run(t1, "COMMIT", &err); + + if (first != second) + pg_prop_fail(tc, + "REPEATABLE READ snapshot shifted: first=%ld second=%ld " + "(concurrent +%d)", first, second, delta); + + if (err) + pg_free(err); + pfree(q.data); + pg_prop_disconnect(t1); + pg_prop_disconnect(t2); + pg_prop_disconnect(admin); +} + +/* + * READ COMMITTED visibility: within a READ COMMITTED transaction, a statement + * run after a concurrent commit sees the committed value, and a value written + * by an uncommitted concurrent txn is NOT seen. + */ +PG_PROP(rc_sees_committed) +{ + pg_prop_conn *admin = open_conn(tc); + pg_prop_conn *t1 = open_conn(tc); + pg_prop_conn *t2 = open_conn(tc); + char *err = NULL; + long base, + after_uncommitted, + after_committed; + int delta = (int) pg_prop_draw_int(tc, 1, 100); + StringInfoData q; + + setup_iso(tc, admin); + + run(t1, "BEGIN ISOLATION LEVEL READ COMMITTED", &err); + base = scalar_long(tc, t1, "SELECT v FROM iso_t WHERE id = 1"); + + /* T2 updates but does NOT commit yet */ + initStringInfo(&q); + appendStringInfo(&q, "UPDATE iso_t SET v = v + %d WHERE id = 1", delta); + run(t2, "BEGIN", &err); + run(t2, q.data, &err); + + /* T1's new statement must still see the base value (T2 uncommitted) */ + after_uncommitted = scalar_long(tc, t1, "SELECT v FROM iso_t WHERE id = 1"); + if (after_uncommitted != base) + pg_prop_fail(tc, + "READ COMMITTED saw uncommitted write: base=%ld saw=%ld", + base, after_uncommitted); + + /* now T2 commits; T1's next statement must see the committed value */ + run(t2, "COMMIT", &err); + after_committed = scalar_long(tc, t1, "SELECT v FROM iso_t WHERE id = 1"); + if (after_committed != base + delta) + pg_prop_fail(tc, + "READ COMMITTED did not see committed write: expected %ld saw %ld", + base + delta, after_committed); + + run(t1, "COMMIT", &err); + if (err) + pg_free(err); + pfree(q.data); + pg_prop_disconnect(t1); + pg_prop_disconnect(t2); + pg_prop_disconnect(admin); +} From 1f71f7da170c4577dcac3d263070b80d6ebaffd5 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 15:59:11 -0400 Subject: [PATCH 278/338] prop: wire constraints/isolation areas; fix compiler warnings; docs Register the constraints and isolation areas in the build and TAP driver. Fix three pre-existing warnings the strict CompilerWarnings CI job flags: remove unused helpers draw_pattern (prop_like.c) and draw_arr (prop_arrayslice.c), and reword a comment in prop_dateparts.c whose '*mo' read as a nested comment open. Document the retargeted pl_langs boundary area and the new constraints and isolation areas in DESIGN. Full suite green at 254 Tier-B areas, zero warnings. --- src/test/prop/DESIGN | 74 +++++++++++++++++++++++++++----- src/test/prop/Makefile | 4 +- src/test/prop/meson.build | 2 + src/test/prop/prop_arrayslice.c | 26 +---------- src/test/prop/prop_dateparts.c | 2 +- src/test/prop/prop_like.c | 24 ----------- src/test/prop/t/002_sql_props.pl | 2 +- 7 files changed, 71 insertions(+), 63 deletions(-) diff --git a/src/test/prop/DESIGN b/src/test/prop/DESIGN index 0c5611755f9c4..d831561180aaf 100644 --- a/src/test/prop/DESIGN +++ b/src/test/prop/DESIGN @@ -382,17 +382,35 @@ integer opclass); it is a candidate for its own area over the scalar types it supports. -Procedural languages (sql/prop_pl_langs.c) -========================================== - -The same pure, deterministic integer computation expressed in each procedural -language -- PL/pgSQL, PL/Python (plpython3u), PL/Perl, PL/Tcl -- must equal the -pure-SQL computation of the same expression. This is a differential oracle -across language implementations; a disagreement is a bug in a PL's argument -marshalling, integer handling, or return conversion. Each property probes its -language with pg_prop_require_extension and pg_prop_skips when the server was -not built with it, so the area runs whatever PLs are present and reports the -rest as TAP SKIP rather than failing. +The procedural-language / SQL boundary (sql/prop_pl_langs.c) +============================================================ + +The languages themselves work; what is not obvious is whether they behave +correctly at the boundary with SQL inside PostgreSQL -- when a PL function runs +a query via SPI, marshals a composite / array / set across the boundary, fires +as a trigger during DML, or round-trips a value through the PL's own type +system. That boundary, over the generated schemas, is the surface this area +targets, across PL/pgSQL, PL/Python, PL/Perl and PL/Tcl. Each property is a +differential oracle: the PL path must agree with the equivalent pure-SQL path. + + spi_count_matches -- a PL function that SPI-executes a count query over the + generated table with a generated predicate returns the + same count as the direct SQL query (SPI from the PL). + srf_matches_sql -- a PL set-returning function yielding the matching ids + returns the same multiset as the SQL query (set return + and materialization). + roundtrip_jsonb -- a jsonb passed into a PL function and returned is + unchanged (the PL's jsonb <-> native conversion). + trigger_derived -- a PL BEFORE trigger that maintains a derived column + keeps it consistent with the SQL expression after + arbitrary INSERT/UPDATE churn (trigger firing and NEW- + tuple modification across the boundary). + +Each probes its language with pg_prop_require_extension and skips (TAP SKIP) +when the server was not built with it, so the area runs whatever PLs are +present. (Generating source for a whitespace-sensitive language through a SQL +dollar-quoted literal, and keeping the draw count independent of which PLs the +server has, both matter for reproducibility; see finding 025.) Integrity oracles (sql/prop_integrity.c) @@ -413,6 +431,40 @@ never buried in a subquery a constant-false outer query could optimize away -- that elision would make the oracle vacuous; see finding 024.) +Constraint enforcement (sql/prop_constraints.c) +=============================================== + +A constraint is a promise about the data, and the property is that it is +enforced *exactly*: a candidate row is rejected if and only if it violates the +constraint. The oracle for each is a pure-SQL predicate that decides whether a +row is a violation; the property generates candidate rows, computes which the +predicate marks as violations, inserts them one at a time, and requires the +server to reject exactly that set. A constraint accepting a violator is a +correctness bug of the most serious kind; rejecting a valid row is an equally +real false positive. unique_rejects_dups (UNIQUE/PK, NULLs distinct), +check_rejects_false (CHECK accepts unless the expression is FALSE -- NULL is +accepted, per SQL three-valued logic), notnull_rejects_null, fk_rejects_orphan +(a child is accepted iff its key is NULL or present in the parent), and +exclusion_rejects_overlap (a range is accepted iff it overlaps no existing one; +skipped if gist-on-int4range is unavailable). + + +Transaction isolation (sql/prop_isolation.c) +============================================ + +Isolation is among the hardest guarantees a database makes and the most +important to verify. These properties drive two or three concurrent sessions +(separate libpq connections) through interleavings and assert the guarantee of +each level; the oracle is the documented guarantee, not a generated reference. +ser_write_skew_prevented (under SERIALIZABLE the classic write-skew pair cannot +both commit -- at least one must fail with a serialization error), +rr_snapshot_stable (under REPEATABLE READ a repeated read returns the same +value despite a concurrent committed update), and rc_sees_committed (under READ +COMMITTED a new statement sees a concurrently committed change and does not see +an uncommitted one). The harness's per-case cleanup closes every connection a +body opened, so the extra sessions never leak across cases. + + The skip / availability framework ================================= diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 88c31ae401f85..85b1c4da0d2f3 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -285,7 +285,9 @@ OBJS = \ sql/prop_index_am.o \ sql/prop_table_am.o \ sql/prop_pl_langs.o \ - sql/prop_integrity.o + sql/prop_integrity.o \ + sql/prop_constraints.o \ + sql/prop_isolation.o PG_CPPFLAGS = -I$(libpq_srcdir) -I$(srcdir)/sql PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 531d61cbe0431..97b21e6005e3b 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -291,6 +291,8 @@ prop_query_sources = files( 'sql/prop_table_am.c', 'sql/prop_pl_langs.c', 'sql/prop_integrity.c', + 'sql/prop_constraints.c', + 'sql/prop_isolation.c', ) pg_prop = executable('pg_prop', diff --git a/src/test/prop/prop_arrayslice.c b/src/test/prop/prop_arrayslice.c index 869d1e2a705b4..e77abcf6cd5d3 100644 --- a/src/test/prop/prop_arrayslice.c +++ b/src/test/prop/prop_arrayslice.c @@ -38,31 +38,7 @@ #include "lib/stringinfo.h" /* - * Draw a random SQL integer-array literal into buf as an ARRAY[...] expression - * typed ::int[]. n is drawn in 0..10 elements, each a small int in -20..20, - * with NO NULL elements so every = comparison and slice reconstruction stays - * exact. The ::int[] cast is always emitted so an empty array - * (ARRAY[]::int[]) still carries a type. *n receives the element count. - */ -static void -draw_arr(pg_prop_tc *tc, StringInfo buf, int *n) -{ - int len = (int) pg_prop_draw_int(tc, 0, 10); - - appendStringInfoString(buf, "ARRAY["); - for (int i = 0; i < len; i++) - { - if (i > 0) - appendStringInfoString(buf, ", "); - appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, -20, 20)); - } - appendStringInfoString(buf, "]::int[]"); - - *n = len; -} - -/* - * Draw an array like draw_arr but forced non-empty (n in 1..10), for the + * Draw a non-empty array (n in 1..10), for the * bound/dims/slice-length properties: an empty 1-D array has NULL bounds and * NULL array_dims, so those identities only hold for n >= 1. */ diff --git a/src/test/prop/prop_dateparts.c b/src/test/prop/prop_dateparts.c index f51208167e58f..1c95e7ae18fe9 100644 --- a/src/test/prop/prop_dateparts.c +++ b/src/test/prop/prop_dateparts.c @@ -31,7 +31,7 @@ #include "lib/stringinfo.h" /* - * Draw a (year, month, day) triple into *y/*mo/*d, with y 2000..2030, + * Draw a (year, month, day) triple into y, mo and d, with y 2000..2030, * mo 1..12 and d 1..28 (so every month is valid without leap/length cases). */ static void diff --git a/src/test/prop/prop_like.c b/src/test/prop/prop_like.c index 3b4b8e35d1c62..5f704fd52fd52 100644 --- a/src/test/prop/prop_like.c +++ b/src/test/prop/prop_like.c @@ -65,30 +65,6 @@ draw_text(pg_prop_tc *tc, StringInfo buf) appendStringInfoChar(buf, letters[pg_prop_draw_int(tc, 0, 2)]); } -/* - * Draw 0..8 tokens into buf as a bare LIKE pattern. Each token is a literal - * letter from "abc", a '%' (match any run), or a '_' (match any one char). - * The tiny alphabet keeps patterns matching often. - */ -static void -draw_pattern(pg_prop_tc *tc, StringInfo buf) -{ - static const char letters[] = "abc"; - int n = (int) pg_prop_draw_int(tc, 0, 8); - - for (int i = 0; i < n; i++) - { - int kind = (int) pg_prop_draw_int(tc, 0, 4); - - if (kind == 3) - appendStringInfoChar(buf, '%'); - else if (kind == 4) - appendStringInfoChar(buf, '_'); - else - appendStringInfoChar(buf, letters[kind]); - } -} - /* * Draw 0..10 mixed-case characters from "abAB" into buf as a bare string, for * exercising case folding (ILIKE vs lower()+LIKE). Letters only. diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 3fe8a517ed944..115e42ce356c4 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep index_am table_am pl_langs integrity); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep index_am table_am pl_langs integrity constraints isolation); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 0ecf8fa1b089d28abd4354001d7cb68a538d0362 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 17:54:42 -0400 Subject: [PATCH 279/338] prop: deepen integrity area (GIN amcheck, heapallindexed, pageinspect) Three stronger integrity oracles. gin_amcheck runs gin_index_check over a GIN index built on a generated int[] column (GIN posting/entry-tree consistency). btree_amcheck_heapallindexed runs bt_index_check with heapallindexed => true -- which also verifies every heap tuple is represented in the index -- over btree, multicolumn, expression, partial and covering index shapes. pageinspect_header_bounds asserts a heap page's header offsets satisfy 0 <= lower <= upper <= special <= pagesize and that every line pointer's tuple extent stays within the page. Verified the check functions genuinely execute (error when passed the wrong relation kind), not elided. --- src/test/prop/sql/prop_integrity.c | 147 +++++++++++++++++++++++++++++ 1 file changed, 147 insertions(+) diff --git a/src/test/prop/sql/prop_integrity.c b/src/test/prop/sql/prop_integrity.c index d8053b2b781e2..2d86bd16b81f6 100644 --- a/src/test/prop/sql/prop_integrity.c +++ b/src/test/prop/sql/prop_integrity.c @@ -193,3 +193,150 @@ PG_PROP(stattuple_bounds) pg_prop_fail(tc, "pgstattuple byte accounting inconsistent"); pg_prop_disconnect(conn); } + +/* + * A GIN index over an int[] column built from random data passes amcheck's + * gin_index_check (GIN posting-tree / entry-tree structural consistency). + * Requires the btree_gin opclass extension for a scalar int, or a native array + * GIN opclass -- we use an int[] column with the built-in array_ops. + */ +PG_PROP(gin_amcheck) +{ + pg_prop_conn *conn = connect_integ(tc); + char *err = NULL; + int n; + int i; + StringInfoData ddl; + + if (!pg_prop_require_extension(conn, "amcheck")) + pg_prop_skip(tc, "amcheck not available"); + + /* build a table with an int[] column and random small arrays */ + n = (int) pg_prop_draw_int(tc, 0, 120); + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "SET client_min_messages=warning; " + "DROP TABLE IF EXISTS gin_t; " + "CREATE TABLE gin_t (id int PRIMARY KEY, a int[]); "); + for (i = 0; i < n; i++) + { + int m = (int) pg_prop_draw_int(tc, 0, 5); + int j; + + appendStringInfo(&ddl, "INSERT INTO gin_t VALUES (%d, ARRAY[", i); + for (j = 0; j < m; j++) + { + if (j > 0) + appendStringInfoString(&ddl, ","); + appendStringInfo(&ddl, "%d", (int) pg_prop_draw_int(tc, -10, 10)); + } + appendStringInfoString(&ddl, "]::int[]); "); + } + appendStringInfoString(&ddl, + "CREATE INDEX gin_t_idx ON gin_t USING gin (a); " + "ANALYZE gin_t;"); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "GIN build failed: %s", err ? err : ""); + pfree(ddl.data); + + /* gin_index_check raises on structural corruption */ + if (!pg_prop_sql_exec(conn, + "SELECT gin_index_check('gin_t_idx'::regclass)", &err)) + pg_prop_fail(tc, "gin_index_check failed: %s", err ? err : ""); + if (err) + pg_free(err); + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS gin_t", &err); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} + +/* + * The stronger btree check: bt_index_check(index, heapallindexed => true) also + * verifies that every heap tuple is represented in the index. Run it over the + * generated btree plus multicolumn, expression and partial variants -- each a + * different index shape that must still be internally consistent and complete. + */ +PG_PROP(btree_amcheck_heapallindexed) +{ + pg_prop_conn *conn = connect_integ(tc); + char *err = NULL; + static const char *const defs[] = { + "CREATE INDEX integ_idx ON integ_t (k)", + "CREATE INDEX integ_idx ON integ_t (k, id)", /* multicolumn */ + "CREATE INDEX integ_idx ON integ_t ((k * k))", /* expression */ + "CREATE INDEX integ_idx ON integ_t (k) WHERE k > 0", /* partial */ + "CREATE INDEX integ_idx ON integ_t (k) INCLUDE (pl)", /* covering */ + }; + int which; + StringInfoData ddl; + + if (!pg_prop_require_extension(conn, "amcheck")) + pg_prop_skip(tc, "amcheck not available"); + build_and_churn(tc, conn); + + which = (int) pg_prop_draw_int(tc, 0, 4); + initStringInfo(&ddl); + appendStringInfo(&ddl, "DROP INDEX IF EXISTS integ_idx; %s;", defs[which]); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "index build failed (%s): %s", defs[which], + err ? err : ""); + pfree(ddl.data); + + /* heapallindexed = true: full index-vs-heap consistency */ + if (!pg_prop_sql_exec(conn, + "SELECT bt_index_check('integ_idx'::regclass, true)", &err)) + pg_prop_fail(tc, "bt_index_check(heapallindexed) failed on %s: %s", + defs[which], err ? err : ""); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} + +/* + * pageinspect page-structure invariants: for a heap page, the page_header + * offsets must satisfy 0 <= lower <= upper <= special <= pagesize, and every + * line pointer in heap_page_items that points to a tuple must have its + * [lp_off, lp_off+lp_len) within the page. A violated offset means a corrupt + * or misread page -- caught before it becomes a wrong answer. + */ +PG_PROP(pageinspect_header_bounds) +{ + pg_prop_conn *conn = connect_integ(tc); + char *err = NULL; + long badhdr, + baditem; + + if (!pg_prop_require_extension(conn, "pageinspect")) + pg_prop_skip(tc, "pageinspect not available"); + build_and_churn(tc, conn); + + /* + * Check page 0's header offsets. (A tiny table may have no page 0 if + * empty; get_raw_page then errors, so guard with a row count.) + */ + if (scalar_long(tc, conn, "SELECT count(*) FROM integ_t") == 0) + { + pg_prop_disconnect(conn); + return; + } + + badhdr = scalar_long(tc, conn, + "SELECT count(*) FROM page_header(get_raw_page('integ_t', 0)) h " + "WHERE NOT (h.lower >= 0 AND h.lower <= h.upper " + "AND h.upper <= h.special AND h.special <= 8192)"); + if (badhdr != 0) + pg_prop_fail(tc, "heap page 0 header offsets out of range"); + + /* every normal line pointer's tuple extent stays within the page */ + baditem = scalar_long(tc, conn, + "SELECT count(*) FROM heap_page_items(get_raw_page('integ_t', 0)) i " + "WHERE i.lp_off <> 0 " + "AND (i.lp_off < 0 OR i.lp_off + i.lp_len > 8192)"); + if (baditem != 0) + pg_prop_fail(tc, "heap page 0 line pointer extent outside the page"); + + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} From a317889dd82530da97a1d85fadf9d3d3ec25867d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 17:54:42 -0400 Subject: [PATCH 280/338] prop: deepen isolation area (predicate-lock phantom, row-lock timeout) ser_phantom_prevented: under SERIALIZABLE, two transactions that each read a range and insert a phantom into the other's range cannot both commit -- predicate locking must abort one. row_lock_conflict_times_out: a row locked FOR UPDATE by one session blocks a conflicting write from another, which with a short lock_timeout fails with 55P03 rather than waiting forever, and succeeds once released -- exercising the row-lock machinery synchronously without a second thread. --- src/test/prop/sql/prop_isolation.c | 109 +++++++++++++++++++++++++++++ 1 file changed, 109 insertions(+) diff --git a/src/test/prop/sql/prop_isolation.c b/src/test/prop/sql/prop_isolation.c index e2a285a1bc9ba..6e1c848af55f5 100644 --- a/src/test/prop/sql/prop_isolation.c +++ b/src/test/prop/sql/prop_isolation.c @@ -252,3 +252,112 @@ PG_PROP(rc_sees_committed) pg_prop_disconnect(t2); pg_prop_disconnect(admin); } + +/* + * SERIALIZABLE phantom / predicate-lock anomaly: T1 and T2 each count the rows + * matching a range predicate and then insert a new row that falls in the + * *other* transaction's predicate. Serially one would see the other's insert; + * concurrently this is a write-skew-via-phantom, and SERIALIZABLE's predicate + * locking must stop both from committing. At least one COMMIT must fail with a + * serialization error. + */ +PG_PROP(ser_phantom_prevented) +{ + pg_prop_conn *admin = open_conn(tc); + pg_prop_conn *t1 = open_conn(tc); + pg_prop_conn *t2 = open_conn(tc); + char *err = NULL, + *e1 = NULL, + *e2 = NULL; + bool c1, + c2; + + /* a table of integer "marks"; T1 works the even side, T2 the odd side */ + if (!run(admin, + "DROP TABLE IF EXISTS ph_t; " + "CREATE TABLE ph_t (id serial PRIMARY KEY, g int); " + "INSERT INTO ph_t (g) SELECT (i % 4) FROM generate_series(1, 8) i;", + &err)) + pg_prop_fail(tc, "phantom setup failed: %s", err ? err : ""); + + run(t1, "BEGIN ISOLATION LEVEL SERIALIZABLE", &err); + run(t2, "BEGIN ISOLATION LEVEL SERIALIZABLE", &err); + + /* each reads a predicate range (establishing a predicate lock) ... */ + (void) scalar_long(tc, t1, "SELECT count(*) FROM ph_t WHERE g = 0"); + (void) scalar_long(tc, t2, "SELECT count(*) FROM ph_t WHERE g = 2"); + + /* ... then each inserts a phantom into the other's range */ + run(t1, "INSERT INTO ph_t (g) VALUES (2)", &err); + run(t2, "INSERT INTO ph_t (g) VALUES (0)", &err); + + c1 = run(t1, "COMMIT", &e1); + c2 = run(t2, "COMMIT", &e2); + + if (c1 && c2) + pg_prop_fail(tc, + "SERIALIZABLE allowed a phantom anomaly: both committed"); + + if (e1) + pg_free(e1); + if (e2) + pg_free(e2); + if (err) + pg_free(err); + pg_prop_disconnect(t1); + pg_prop_disconnect(t2); + pg_prop_disconnect(admin); +} + +/* + * Lock conflicts are detected and bounded: a row locked by one session blocks + * a conflicting write from another, and with a short lock_timeout that write + * fails (SQLSTATE 55P03) rather than waiting forever. This exercises the + * row-lock machinery synchronously (no second thread): T2 holds the lock, T1 + * tries to take it with a timeout and must fail; after T2 commits, T1 can take + * it. A write that neither waits nor fails would be a lock-correctness bug. + */ +PG_PROP(row_lock_conflict_times_out) +{ + pg_prop_conn *admin = open_conn(tc); + pg_prop_conn *t1 = open_conn(tc); + pg_prop_conn *t2 = open_conn(tc); + char *err = NULL; + bool blocked_update_ok; + int delta = (int) pg_prop_draw_int(tc, 1, 100); + StringInfoData q; + + setup_iso(tc, admin); + + /* T2 locks row 1 for update and holds it (open transaction) */ + run(t2, "BEGIN", &err); + (void) scalar_long(tc, t2, "SELECT v FROM iso_t WHERE id = 1 FOR UPDATE"); + + /* T1, with a short lock_timeout, tries to update the same row: must fail */ + run(t1, "BEGIN", &err); + run(t1, "SET lock_timeout = '300ms'", &err); + initStringInfo(&q); + appendStringInfo(&q, "UPDATE iso_t SET v = v + %d WHERE id = 1", delta); + blocked_update_ok = run(t1, q.data, &err); + if (blocked_update_ok) + pg_prop_fail(tc, + "conflicting UPDATE succeeded while row was locked " + "(expected lock_timeout)"); + run(t1, "ROLLBACK", &err); + + /* T2 releases; now T1 can take the lock and update */ + run(t2, "COMMIT", &err); + run(t1, "BEGIN", &err); + run(t1, "SET lock_timeout = '5s'", &err); + if (!run(t1, q.data, &err)) + pg_prop_fail(tc, "UPDATE failed after lock released: %s", + err ? err : ""); + run(t1, "COMMIT", &err); + + if (err) + pg_free(err); + pfree(q.data); + pg_prop_disconnect(t1); + pg_prop_disconnect(t2); + pg_prop_disconnect(admin); +} From fe6f47cb4363fb14a794e5cc2b77e0ed367da99e Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 17:54:43 -0400 Subject: [PATCH 281/338] prop: add contrib-breadth area (tablefunc, TSMs, walinspect, dblink) New 'contrib' area with bounded invariants over smaller contrib modules, each skip-if-absent: tablefunc_normal_rand (normal_rand returns exactly n finite rows), tsm_system_rows_bound / tsm_system_time_bound (a TABLESAMPLE returns at most the requested count and only genuine table rows), walinspect_monotonic (pg_walinspect records over an LSN range have non-decreasing start_lsn), and dblink_roundtrip (a value sent through a dblink loopback -- conninfo built from the server's own socket dir and port so it reaches this node -- returns unchanged). Register the area and document the new integrity, isolation and contrib coverage in DESIGN. Full suite green at 255 Tier-B areas. --- src/test/prop/DESIGN | 43 ++++- src/test/prop/Makefile | 3 +- src/test/prop/meson.build | 1 + src/test/prop/sql/prop_contrib.c | 314 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 5 files changed, 354 insertions(+), 9 deletions(-) create mode 100644 src/test/prop/sql/prop_contrib.c diff --git a/src/test/prop/DESIGN b/src/test/prop/DESIGN index d831561180aaf..858aeb2a7d8ab 100644 --- a/src/test/prop/DESIGN +++ b/src/test/prop/DESIGN @@ -424,11 +424,18 @@ valuable class from wrong results. btree_amcheck (bt_index_check on a freshly built index), heap_verify (verify_heapam finds no corruption after churn), vm_consistent (pg_check_visible / pg_check_frozen find no wrongly-set visibility-map bits after VACUUM FREEZE), and stattuple_bounds (pgstattuple's -byte accounting is non-negative and bounded by the table size). Each requires -its contrib extension and skips cleanly if absent. (A check function that -reports via RAISE, like bt_index_check, is invoked as a plain top-level SELECT, -never buried in a subquery a constant-false outer query could optimize away -- -that elision would make the oracle vacuous; see finding 024.) +byte accounting is non-negative and bounded by the table size). Beyond those, +gin_amcheck (gin_index_check on a GIN index over a generated int[] column), +btree_amcheck_heapallindexed (bt_index_check with heapallindexed => true, which +also verifies every heap tuple is represented in the index, over btree / +multicolumn / expression / partial / covering shapes), and +pageinspect_header_bounds (a heap page's header offsets satisfy 0 <= lower <= +upper <= special <= pagesize, and every line pointer's tuple extent stays within +the page). Each requires its contrib extension and skips cleanly if absent. +(A check function that reports via RAISE, like bt_index_check, is invoked as a +plain top-level SELECT, never buried in a subquery a constant-false outer query +could optimize away -- that elision would make the oracle vacuous; see finding +024.) Constraint enforcement (sql/prop_constraints.c) @@ -461,8 +468,30 @@ both commit -- at least one must fail with a serialization error), rr_snapshot_stable (under REPEATABLE READ a repeated read returns the same value despite a concurrent committed update), and rc_sees_committed (under READ COMMITTED a new statement sees a concurrently committed change and does not see -an uncommitted one). The harness's per-case cleanup closes every connection a -body opened, so the extra sessions never leak across cases. +an uncommitted one). Two more: ser_phantom_prevented (SERIALIZABLE's predicate +locking stops a phantom/write-skew anomaly where each txn reads a range and +inserts into the other's range from both committing), and +row_lock_conflict_times_out (a row locked FOR UPDATE by one session blocks a +conflicting write from another, which with a short lock_timeout fails with +55P03 rather than waiting forever, and succeeds once the lock is released -- +exercising the row-lock machinery synchronously, without a second thread). The +harness's per-case cleanup closes every connection a body opened, so the extra +sessions never leak across cases. + + +Contrib breadth (sql/prop_contrib.c) +==================================== + +A grab-bag of smaller contrib modules, each a bounded invariant with a +skip-if-absent guard: tablefunc_normal_rand (normal_rand(n, mean, sd) returns +exactly n finite rows), tsm_system_rows_bound (TABLESAMPLE SYSTEM_ROWS(n) +returns at most n rows, all genuine table rows), tsm_system_time_bound (every +SYSTEM_TIME sample is a real table row), walinspect_monotonic (records read +over a generated LSN range have non-decreasing start_lsn -- WAL is an ordered +append-only log), and dblink_roundtrip (a value sent through a dblink loopback +back to this very node -- conninfo built from the server's own +unix_socket_directories and port -- returns unchanged, exercising connection +setup and result marshalling). The skip / availability framework diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 85b1c4da0d2f3..9a51cb222ceed 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -287,7 +287,8 @@ OBJS = \ sql/prop_pl_langs.o \ sql/prop_integrity.o \ sql/prop_constraints.o \ - sql/prop_isolation.o + sql/prop_isolation.o \ + sql/prop_contrib.o PG_CPPFLAGS = -I$(libpq_srcdir) -I$(srcdir)/sql PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 97b21e6005e3b..3164576b1bfb9 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -293,6 +293,7 @@ prop_query_sources = files( 'sql/prop_integrity.c', 'sql/prop_constraints.c', 'sql/prop_isolation.c', + 'sql/prop_contrib.c', ) pg_prop = executable('pg_prop', diff --git a/src/test/prop/sql/prop_contrib.c b/src/test/prop/sql/prop_contrib.c new file mode 100644 index 0000000000000..9dd756444eefb --- /dev/null +++ b/src/test/prop/sql/prop_contrib.c @@ -0,0 +1,314 @@ +/*------------------------------------------------------------------------- + * + * prop_contrib.c + * Property tests over assorted contrib extensions. + * + * A grab-bag of smaller contrib modules, each with a bounded invariant and a + * skip-if-absent guard so the area runs whatever the server offers: + * + * tablefunc_normal_rand -- normal_rand(n, mean, sd) returns exactly n rows, + * all finite; and crosstab's row count is bounded by + * the source. + * tsm_system_rows_bound -- TABLESAMPLE SYSTEM_ROWS(n) returns at most n rows, + * all of which are genuine rows of the table. + * tsm_system_time_bound -- TABLESAMPLE SYSTEM_TIME(ms) returns a subset of + * the table (every sampled id exists). + * walinspect_monotonic -- pg_get_wal_records_info over a generated LSN range + * returns records whose LSNs are within the range and + * non-decreasing (WAL is an append-only sequence). + * dblink_roundtrip -- a value sent through a dblink loopback query comes + * back unchanged (connection + result marshalling). + * + * Tier B: runs SQL against the cluster the TAP driver started. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_contrib.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "contrib" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +static long +scalar_long(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *v; + long n; + + v = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "scalar failed: %s\n SQL: %s", + err ? err : "(unknown)", sql); + n = v ? atol(v) : 0; + if (v) + pg_free(v); + if (err) + pg_free(err); + return n; +} + +static pg_prop_conn * +connect_c(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + pg_prop_sql_exec(conn, "SET client_min_messages=warning", &err); + if (err) + pg_free(err); + return conn; +} + +/* normal_rand(n, mean, sd) yields exactly n finite rows. */ +PG_PROP(tablefunc_normal_rand) +{ + pg_prop_conn *conn = connect_c(tc); + int n = (int) pg_prop_draw_int(tc, 0, 500); + double mean = (double) pg_prop_draw_int(tc, -100, 100); + int sd = (int) pg_prop_draw_int(tc, 0, 50); + StringInfoData q; + long cnt, + nonfinite; + + if (!pg_prop_require_extension(conn, "tablefunc")) + pg_prop_skip(tc, "tablefunc not available"); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT count(*) FROM normal_rand(%d, %g, %d)", + n, mean, sd); + cnt = scalar_long(tc, conn, q.data); + if (cnt != n) + pg_prop_fail(tc, "normal_rand returned %ld rows, expected %d", cnt, n); + + /* every value is finite (normal_rand must not emit NaN/Inf) */ + resetStringInfo(&q); + appendStringInfo(&q, + "SELECT count(*) FROM normal_rand(%d, %g, %d) v " + "WHERE v = 'NaN'::float8 OR v = 'Infinity'::float8 " + "OR v = '-Infinity'::float8", + n, mean, sd); + nonfinite = scalar_long(tc, conn, q.data); + if (nonfinite != 0) + pg_prop_fail(tc, "normal_rand produced %ld non-finite values", nonfinite); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* TABLESAMPLE SYSTEM_ROWS(n): at most n rows, all genuine table rows. */ +PG_PROP(tsm_system_rows_bound) +{ + pg_prop_conn *conn = connect_c(tc); + char *err = NULL; + int total = (int) pg_prop_draw_int(tc, 0, 300); + int want = (int) pg_prop_draw_int(tc, 0, 400); + int i; + StringInfoData ddl; + long got, + bogus; + + if (!pg_prop_require_extension(conn, "tsm_system_rows")) + pg_prop_skip(tc, "tsm_system_rows not available"); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS smp_t; " + "CREATE TABLE smp_t (id int PRIMARY KEY); "); + for (i = 0; i < total; i++) + appendStringInfo(&ddl, "INSERT INTO smp_t VALUES (%d); ", i); + appendStringInfoString(&ddl, "ANALYZE smp_t;"); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "sample table build failed: %s", err ? err : ""); + pfree(ddl.data); + + initStringInfo(&ddl); + appendStringInfo(&ddl, + "SELECT count(*) FROM smp_t TABLESAMPLE SYSTEM_ROWS(%d)", + want); + got = scalar_long(tc, conn, ddl.data); + /* result is bounded by both the requested count and the table size */ + if (got > want || got > total) + pg_prop_fail(tc, + "SYSTEM_ROWS(%d) returned %ld rows (table has %d)", + want, got, total); + + /* every sampled id must be a real id in the table */ + resetStringInfo(&ddl); + appendStringInfo(&ddl, + "SELECT count(*) FROM " + "(SELECT id FROM smp_t TABLESAMPLE SYSTEM_ROWS(%d)) s " + "WHERE NOT EXISTS (SELECT 1 FROM smp_t t WHERE t.id = s.id)", + want); + bogus = scalar_long(tc, conn, ddl.data); + if (bogus != 0) + pg_prop_fail(tc, "SYSTEM_ROWS returned %ld ids not in the table", bogus); + + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS smp_t", &err); + if (err) + pg_free(err); + pfree(ddl.data); + pg_prop_disconnect(conn); +} + +/* TABLESAMPLE SYSTEM_TIME(ms): every sampled id is a genuine table row. */ +PG_PROP(tsm_system_time_bound) +{ + pg_prop_conn *conn = connect_c(tc); + char *err = NULL; + int total = (int) pg_prop_draw_int(tc, 0, 300); + int i; + StringInfoData ddl; + long bogus; + + if (!pg_prop_require_extension(conn, "tsm_system_time")) + pg_prop_skip(tc, "tsm_system_time not available"); + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS smt_t; " + "CREATE TABLE smt_t (id int PRIMARY KEY); "); + for (i = 0; i < total; i++) + appendStringInfo(&ddl, "INSERT INTO smt_t VALUES (%d); ", i); + appendStringInfoString(&ddl, "ANALYZE smt_t;"); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "sample table build failed: %s", err ? err : ""); + pfree(ddl.data); + + /* a short time budget; the sample is a subset of the table */ + bogus = scalar_long(tc, conn, + "SELECT count(*) FROM " + "(SELECT id FROM smt_t TABLESAMPLE SYSTEM_TIME(10)) s " + "WHERE NOT EXISTS (SELECT 1 FROM smt_t t WHERE t.id = s.id)"); + if (bogus != 0) + pg_prop_fail(tc, "SYSTEM_TIME returned %ld ids not in the table", bogus); + + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS smt_t", &err); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} + +/* + * pg_walinspect: records read over [start, current] all have LSNs within the + * range and non-decreasing start_lsn (WAL is an ordered append-only log). We + * capture a start LSN, generate some WAL (a tiny table + inserts), then read + * the records in between and check the ordering invariant. + */ +PG_PROP(walinspect_monotonic) +{ + pg_prop_conn *conn = connect_c(tc); + char *err = NULL; + char *start_lsn; + bool failed = false; + int i; + int n = (int) pg_prop_draw_int(tc, 1, 50); + StringInfoData ddl; + long disordered; + + if (!pg_prop_require_extension(conn, "pg_walinspect")) + pg_prop_skip(tc, "pg_walinspect not available"); + + start_lsn = pg_prop_sql_scalar(conn, "SELECT pg_current_wal_lsn()::text", + &failed, &err); + if (failed || start_lsn == NULL) + pg_prop_skip(tc, "cannot read current WAL LSN (not in recovery?)"); + + /* generate some WAL */ + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS wal_t; " + "CREATE TABLE wal_t (id int); "); + for (i = 0; i < n; i++) + appendStringInfo(&ddl, "INSERT INTO wal_t VALUES (%d); ", i); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "WAL-generating DML failed: %s", err ? err : ""); + pfree(ddl.data); + + /* + * Read the records between start_lsn and the current LSN; their start_lsn + * values must be strictly increasing and within the queried range. Count + * any adjacent pair that is out of order (expected zero). + */ + initStringInfo(&ddl); + appendStringInfo(&ddl, + "SELECT count(*) FROM (" + "SELECT start_lsn, lag(start_lsn) OVER (ORDER BY start_lsn) AS prev " + "FROM pg_get_wal_records_info('%s', pg_current_wal_lsn())) s " + "WHERE s.prev IS NOT NULL AND s.start_lsn <= s.prev", + start_lsn); + disordered = scalar_long(tc, conn, ddl.data); + if (disordered != 0) + pg_prop_fail(tc, "pg_walinspect reported %ld out-of-order records", + disordered); + + pg_free(start_lsn); + if (err) + pg_free(err); + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS wal_t", &err); + if (err) + pg_free(err); + pfree(ddl.data); + pg_prop_disconnect(conn); +} + +/* + * dblink loopback round-trip: a value sent through a dblink connection back to + * the same cluster and returned must be unchanged. Exercises connection setup + * and result marshalling. The loopback connection string reuses the current + * session's socket directory and port via the PG* environment the harness runs + * under, passed as a literal. + */ +PG_PROP(dblink_roundtrip) +{ + pg_prop_conn *conn = connect_c(tc); + char *err = NULL; + int a = (int) pg_prop_draw_int(tc, -1000000, 1000000); + StringInfoData q; + char *v; + bool failed = false; + + if (!pg_prop_require_extension(conn, "dblink")) + pg_prop_skip(tc, "dblink not available"); + + /* + * Build the loopback conninfo from the server's own settings so it points + * at this node (host = first unix_socket_directories entry, port = + * current_setting('port')), and read a value back with an explicit column + * definition list. Compare to the sent value. + */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT x = %d FROM dblink(" + "'host=' || split_part(current_setting('unix_socket_directories'), ',', 1) " + "|| ' port=' || current_setting('port') " + "|| ' dbname=' || current_database() " + "|| ' user=' || current_user, " + "'SELECT %d') AS t(x int)", + a, a); + v = pg_prop_sql_scalar(conn, q.data, &failed, &err); + if (failed) + { + /* dblink loopback can be refused by auth config; skip rather than fail */ + pg_prop_skip(tc, "dblink loopback not permitted here: %s", + err ? err : ""); + } + if (v == NULL || strcmp(v, "t") != 0) + pg_prop_fail(tc, "dblink did not round-trip %d unchanged", a); + + if (v) + pg_free(v); + if (err) + pg_free(err); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 115e42ce356c4..e87ef3c398b15 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep index_am table_am pl_langs integrity constraints isolation); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep index_am table_am pl_langs integrity constraints isolation contrib); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From ee3f2d65d41465a47f2cc3f84266ddd801aecb6c Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 22:47:02 -0400 Subject: [PATCH 282/338] prop: add window-function and grouping-sets area (query_window) Phase-4 query shapes -- the transformations with the richest wrong-results history -- as metamorphic oracles over a generated table. Window: count(*) OVER () equals the total; a running sum of a non-negative summand is monotone and its max equals the plain sum; row_number() is a permutation of 1..n; dense_rank <= rank <= row_number. Grouping sets: ROLLUP grand total equals the ungrouped aggregate; ROLLUP detail equals plain GROUP BY; CUBE is a superset of ROLLUP; GROUPING SETS equals the UNION ALL of its component groupings. Fixes found while writing these (finding 026): use the generator's real columns, keep window ORDER BY columns in scope, and take a running sum's final value as max(rs) not ORDER BY ... LIMIT 1. --- src/test/prop/sql/prop_query_window.c | 294 ++++++++++++++++++++++++++ 1 file changed, 294 insertions(+) create mode 100644 src/test/prop/sql/prop_query_window.c diff --git a/src/test/prop/sql/prop_query_window.c b/src/test/prop/sql/prop_query_window.c new file mode 100644 index 0000000000000..a28bdd4a706fe --- /dev/null +++ b/src/test/prop/sql/prop_query_window.c @@ -0,0 +1,294 @@ +/*------------------------------------------------------------------------- + * + * prop_query_window.c + * Query-level property tests for window functions and grouping sets. + * + * These are the planner/executor transformations with the richest history of + * wrong-results bugs, exercised here as metamorphic oracles over a generated + * table (query_gen.h): no reference answer, only a relationship that must hold. + * + * Window functions: + * window_count_over_all -- count(*) OVER () on every row equals the total + * row count; a running count ends at the total. + * window_running_sum -- the last value of sum(x) OVER (ORDER BY a, id) + * equals the plain sum(x); the running sum is + * monotone when x >= 0. + * window_row_number_perm -- row_number() OVER (ORDER BY a, id) is a + * permutation of 1..n (each appears exactly once). + * window_rank_bounds -- rank() <= row_number() and dense_rank() <= + * rank(), per the standard window-rank ordering. + * + * Grouping sets: + * rollup_grand_total -- the all-NULL grouping row of GROUP BY ROLLUP(a,b) + * equals the ungrouped aggregate over the table. + * rollup_detail_matches -- the detail rows of ROLLUP(a,b) equal plain + * GROUP BY a, b. + * cube_superset_rollup -- GROUP BY CUBE(a,b) returns at least as many + * grouping rows as ROLLUP(a,b) (cube is a superset). + * groupingsets_union -- GROUP BY GROUPING SETS ((a),(b),()) equals the + * UNION ALL of GROUP BY a, GROUP BY b, and the grand + * total -- a tautology the executor must reproduce. + * + * Tier B: runs SQL against the cluster the TAP driver started. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_query_window.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "query_window" +#include "pg_prop.h" + +#include "lib/stringinfo.h" +#include "query_gen.h" + +static long +scalar_long(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *v; + long n; + + v = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "scalar failed: %s\n SQL: %s", + err ? err : "(unknown)", sql); + n = v ? atol(v) : 0; + if (v) + pg_free(v); + if (err) + pg_free(err); + return n; +} + +static void +require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, + const char *a, const char *b) +{ + bool fa = false, + fb = false; + char *ea = NULL, + *eb = NULL, + *ra, + *rb; + + ra = pg_prop_sql_rows(conn, a, &fa, &ea); + if (fa) + pg_prop_fail(tc, "query a failed: %s\n SQL: %s", ea ? ea : "", a); + rb = pg_prop_sql_rows(conn, b, &fb, &eb); + if (fb) + pg_prop_fail(tc, "query b failed: %s\n SQL: %s", eb ? eb : "", b); + if (strcmp(ra, rb) != 0) + pg_prop_fail(tc, "mismatch:\n a: %s\n b: %s\n ra:\n%s\n rb:\n%s", + a, b, ra, rb); + pg_free(ra); + pg_free(rb); + if (ea) + pg_free(ea); + if (eb) + pg_free(eb); +} + +static pg_prop_conn * +connect_build(pg_prop_tc *tc, int *nrows) +{ + pg_prop_conn *conn; + char *err = NULL; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + qg_build_table(tc, conn, nrows); + return conn; +} + +/* count(*) OVER () on every row equals the table's row count. */ +PG_PROP(window_count_over_all) +{ + int nrows; + pg_prop_conn *conn = connect_build(tc, &nrows); + long bad; + + /* a row whose window count differs from the total is a bug */ + bad = scalar_long(tc, conn, + "SELECT count(*) FROM " + "(SELECT count(*) OVER () AS c FROM qg_t) s " + "WHERE s.c <> (SELECT count(*) FROM qg_t)"); + if (bad != 0) + pg_prop_fail(tc, "count(*) OVER () disagreed with total on %ld rows", + bad); + pg_prop_disconnect(conn); +} + +/* + * The last value of a running sum over a total order equals the plain sum, and + * the running sum is non-decreasing when the summed column is non-negative + * (use coalesce(abs(a),0) so it is). + */ +PG_PROP(window_running_sum) +{ + int nrows; + pg_prop_conn *conn = connect_build(tc, &nrows); + long last_running, + plain_sum, + nonmonotone; + + if (nrows == 0) + { + pg_prop_disconnect(conn); + return; + } + + last_running = scalar_long(tc, conn, + "SELECT coalesce(max(rs), 0) FROM " + "(SELECT sum(coalesce(abs(a),0)) OVER " + "(ORDER BY a, id) AS rs FROM qg_t) s"); + plain_sum = scalar_long(tc, conn, + "SELECT coalesce(sum(coalesce(abs(a),0)),0) FROM qg_t"); + if (last_running != plain_sum) + pg_prop_fail(tc, "running sum last=%ld != plain sum=%ld", + last_running, plain_sum); + + /* non-negative summand => running sum never decreases */ + nonmonotone = scalar_long(tc, conn, + "SELECT count(*) FROM " + "(SELECT rs, lag(rs) OVER (ORDER BY a, id) AS prev FROM " + "(SELECT a, id, sum(coalesce(abs(a),0)) OVER (ORDER BY a, id) AS rs " + "FROM qg_t) t) u WHERE prev IS NOT NULL AND rs < prev"); + if (nonmonotone != 0) + pg_prop_fail(tc, "running sum of non-negative values decreased %ld times", + nonmonotone); + pg_prop_disconnect(conn); +} + +/* row_number() over a total order is a permutation of 1..n. */ +PG_PROP(window_row_number_perm) +{ + int nrows; + pg_prop_conn *conn = connect_build(tc, &nrows); + long bad; + + /* the set of row numbers must be exactly {1..nrows} */ + bad = scalar_long(tc, conn, + "SELECT count(*) FROM " + "(SELECT row_number() OVER (ORDER BY a, id) AS rn FROM qg_t) s " + "FULL JOIN generate_series(1, (SELECT count(*) FROM qg_t)) g(n) " + "ON s.rn = g.n WHERE s.rn IS NULL OR g.n IS NULL"); + if (bad != 0) + pg_prop_fail(tc, "row_number() is not a permutation of 1..n (%ld off)", + bad); + pg_prop_disconnect(conn); +} + +/* dense_rank() <= rank() <= row_number() over the same ordering. */ +PG_PROP(window_rank_bounds) +{ + int nrows; + pg_prop_conn *conn = connect_build(tc, &nrows); + long bad; + + bad = scalar_long(tc, conn, + "SELECT count(*) FROM " + "(SELECT row_number() OVER w AS rn, rank() OVER w AS rk, " + "dense_rank() OVER w AS dr FROM qg_t " + "WINDOW w AS (ORDER BY a)) s " + "WHERE NOT (s.dr <= s.rk AND s.rk <= s.rn)"); + if (bad != 0) + pg_prop_fail(tc, "rank ordering violated on %ld rows " + "(dense_rank <= rank <= row_number)", bad); + pg_prop_disconnect(conn); +} + +/* ROLLUP(a,b) grand-total row equals the ungrouped aggregate. */ +PG_PROP(rollup_grand_total) +{ + int nrows; + pg_prop_conn *conn = connect_build(tc, &nrows); + long rollup_total, + plain_total; + + /* the grand-total row of ROLLUP has a IS NULL and b IS NULL (and GROUPING + * bits set); its sum equals the whole-table sum */ + rollup_total = scalar_long(tc, conn, + "SELECT coalesce(s,0) FROM " + "(SELECT sum(coalesce(a,0)) AS s, " + "grouping(a) AS ga, grouping(b) AS gb " + "FROM qg_t GROUP BY ROLLUP(a, b)) r " + "WHERE ga = 1 AND gb = 1"); + plain_total = scalar_long(tc, conn, + "SELECT coalesce(sum(coalesce(a,0)),0) FROM qg_t"); + if (rollup_total != plain_total) + pg_prop_fail(tc, "ROLLUP grand total=%ld != plain total=%ld", + rollup_total, plain_total); + pg_prop_disconnect(conn); +} + +/* ROLLUP(a,b) detail rows equal plain GROUP BY a, b. */ +PG_PROP(rollup_detail_matches) +{ + int nrows; + pg_prop_conn *conn = connect_build(tc, &nrows); + + require_same_rows(tc, conn, + /* ROLLUP detail = rows with no grouping bit set, projected like GROUP BY */ + "SELECT a, b, sum(coalesce(a,0)) AS s, count(*) AS c " + "FROM qg_t GROUP BY a, b ORDER BY a, b", + "SELECT a, b, s, c FROM " + "(SELECT a, b, sum(coalesce(a,0)) AS s, count(*) AS c, " + "grouping(a) AS ga, grouping(b) AS gb " + "FROM qg_t GROUP BY ROLLUP(a, b)) r " + "WHERE ga = 0 AND gb = 0 ORDER BY a, b"); + pg_prop_disconnect(conn); +} + +/* CUBE(a,b) returns at least as many grouping rows as ROLLUP(a,b). */ +PG_PROP(cube_superset_rollup) +{ + int nrows; + pg_prop_conn *conn = connect_build(tc, &nrows); + long ncube, + nrollup; + + ncube = scalar_long(tc, conn, + "SELECT count(*) FROM " + "(SELECT 1 FROM qg_t GROUP BY CUBE(a, b)) s"); + nrollup = scalar_long(tc, conn, + "SELECT count(*) FROM " + "(SELECT 1 FROM qg_t GROUP BY ROLLUP(a, b)) s"); + if (ncube < nrollup) + pg_prop_fail(tc, "CUBE rows %ld < ROLLUP rows %ld", ncube, nrollup); + pg_prop_disconnect(conn); +} + +/* + * GROUPING SETS ((a),(b),()) equals the UNION ALL of GROUP BY a, GROUP BY b, + * and the grand total -- a tautology of grouping-sets semantics. Compare the + * (grouping-set, key, aggregate) rows via a normalizing projection. + */ +PG_PROP(groupingsets_union) +{ + int nrows; + pg_prop_conn *conn = connect_build(tc, &nrows); + + require_same_rows(tc, conn, + /* explicit grouping sets, normalized: tag the set by its grouping bits */ + "SELECT grouping(a) AS ga, grouping(b) AS gb, " + "a, b, count(*) AS c " + "FROM qg_t GROUP BY GROUPING SETS ((a),(b),()) " + "ORDER BY ga, gb, a, b", + /* the equivalent union of three plain groupings */ + "SELECT ga, gb, a, b, c FROM (" + "SELECT 0 AS ga, 1 AS gb, a, NULL::int AS b, count(*) AS c " + "FROM qg_t GROUP BY a " + "UNION ALL " + "SELECT 1, 0, NULL, b, count(*) FROM qg_t GROUP BY b " + "UNION ALL " + "SELECT 1, 1, NULL, NULL, count(*) FROM qg_t" + ") u ORDER BY ga, gb, a, b"); + pg_prop_disconnect(conn); +} From 43508ac3d456293cc3af997dd9c56ee0284fe723 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 22:47:02 -0400 Subject: [PATCH 283/338] prop: add outer-join-order invariant to the plan-advice area A three-way LEFT JOIN chain steered to a specific JOIN_ORDER must return the same rows as the planner's free choice. Outer-join reordering is a distinct, historically bug-prone transformation (the planner must respect outer-join identities and not reorder across a nullable side); forcing a legal order must not change the result. --- src/test/prop/sql/prop_query_planadvice.c | 35 +++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/src/test/prop/sql/prop_query_planadvice.c b/src/test/prop/sql/prop_query_planadvice.c index 284796d6193b0..32f75b66292c8 100644 --- a/src/test/prop/sql/prop_query_planadvice.c +++ b/src/test/prop/sql/prop_query_planadvice.c @@ -336,3 +336,38 @@ PG_PROP(distinct_limit_plan_invariant) pfree(q.data); pg_prop_disconnect(conn); } + +/* + * Outer-join reordering: a three-way LEFT JOIN chain, steered to a specific + * JOIN_ORDER, must return the same rows as the planner's free choice. Outer- + * join reordering is a distinct and historically bug-prone transformation + * (the planner must respect outer-join identities and not reorder across a + * nullable side incorrectly); forcing a legal order must not change the result. + * Built over three self-joined copies of the generated table. + */ +PG_PROP(outer_join_order_invariant) +{ + pg_prop_conn *conn; + int nrows; + StringInfoData q; + + conn = connect_advice(tc); + qg_build_table(tc, conn, &nrows); + + /* + * x LEFT JOIN y on y.id = x.a, that LEFT JOIN z on z.id = x.b. Project + * all three ids (NULL-extended on the right where no match), ordered for a + * stable comparison. Steer a left-deep order over the aliases. + */ + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT x.id, y.id, z.id FROM " QG_TABLE " x " + "LEFT JOIN " QG_TABLE " y ON y.id = x.a " + "LEFT JOIN " QG_TABLE " z ON z.id = x.b " + "ORDER BY x.id, y.id NULLS FIRST, z.id NULLS FIRST"); + + require_plan_invariant(tc, conn, q.data, "JOIN_ORDER((x y) z)", ""); + + pfree(q.data); + pg_prop_disconnect(conn); +} From 4e13193b4dd07fb61f303d7fe69e40078731d91e Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Wed, 14 Oct 2026 22:47:03 -0400 Subject: [PATCH 284/338] prop: add SP-GiST AM-specific area; wire window/spgist; docs SP-GiST ships no integer opclass, so it is outside the generic int-column index trait suite; this AM-specific area exercises the opclasses it supports (text_ ops prefix/equality/range, point box-containment, inet network-containment) with the index-vs-seqscan differential. Register the query_window and spgist areas and document them plus the outer-join-order plan-advice property in DESIGN. Full suite green at 257 Tier-B areas, zero compiler warnings. --- src/test/prop/DESIGN | 28 ++++ src/test/prop/Makefile | 4 +- src/test/prop/meson.build | 2 + src/test/prop/sql/prop_spgist.c | 267 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 5 files changed, 301 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/sql/prop_spgist.c diff --git a/src/test/prop/DESIGN b/src/test/prop/DESIGN index 858aeb2a7d8ab..69c4d09fc4e16 100644 --- a/src/test/prop/DESIGN +++ b/src/test/prop/DESIGN @@ -494,6 +494,34 @@ unix_socket_directories and port -- returns unchanged, exercising connection setup and result marshalling). +Window functions and grouping sets (sql/prop_query_window.c) +=========================================================== + +The planner/executor transformations with the richest wrong-results history, +as metamorphic oracles over a generated table. Window: window_count_over_all +(count(*) OVER () equals the total on every row), window_running_sum (the final +value of a running sum over a total order equals the plain sum -- taken as +max(rs) since a non-negative summand makes it monotone -- and it never +decreases), window_row_number_perm (row_number() is a permutation of 1..n), and +window_rank_bounds (dense_rank <= rank <= row_number). Grouping sets: +rollup_grand_total (the all-NULL grouping row equals the ungrouped aggregate), +rollup_detail_matches (ROLLUP detail rows equal plain GROUP BY), cube_superset_ +rollup (CUBE returns at least as many grouping rows as ROLLUP), and +groupingsets_union (GROUPING SETS ((a),(b),()) equals the UNION ALL of the +three component groupings). + + +SP-GiST (sql/prop_spgist.c) +=========================== + +SP-GiST ships no integer opclass, so it is outside the generic int-column index +trait suite; this AM-specific area exercises the opclasses it does support with +the same index-vs-seqscan differential: text_prefix (text_ops over equality, +LIKE prefix and range), point_contains (a box-containment query over point), +and inet_contains (a network-containment query over inet, skipped if the inet +opclass is absent). + + The skip / availability framework ================================= diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 9a51cb222ceed..781c24d0c2350 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -288,7 +288,9 @@ OBJS = \ sql/prop_integrity.o \ sql/prop_constraints.o \ sql/prop_isolation.o \ - sql/prop_contrib.o + sql/prop_contrib.o \ + sql/prop_query_window.o \ + sql/prop_spgist.o PG_CPPFLAGS = -I$(libpq_srcdir) -I$(srcdir)/sql PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 3164576b1bfb9..a0196ec276e8d 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -294,6 +294,8 @@ prop_query_sources = files( 'sql/prop_constraints.c', 'sql/prop_isolation.c', 'sql/prop_contrib.c', + 'sql/prop_query_window.c', + 'sql/prop_spgist.c', ) pg_prop = executable('pg_prop', diff --git a/src/test/prop/sql/prop_spgist.c b/src/test/prop/sql/prop_spgist.c new file mode 100644 index 0000000000000..502fdb28b8826 --- /dev/null +++ b/src/test/prop/sql/prop_spgist.c @@ -0,0 +1,267 @@ +/*------------------------------------------------------------------------- + * + * prop_spgist.c + * AM-specific property tests for SP-GiST. + * + * SP-GiST ships no integer opclass, so it is not in the generic int-column + * index trait suite (prop_index_am.c). It does ship opclasses for text (the + * radix-tree text_ops), point (quad/kd trees) and inet; those are exercised + * here with the same core oracle used everywhere for indexes -- an index scan + * must return exactly the rows a sequential scan does -- over the opclasses + * SP-GiST actually supports. + * + * text_prefix -- SP-GiST text_ops answering a prefix (LIKE 'p%') and an + * equality (c = 'p') and a range (c < 'p') matches a seqscan. + * point_contains -- SP-GiST over point answering a containment (p <@ box) + * matches a seqscan. + * inet_contains -- SP-GiST over inet answering a network containment matches + * a seqscan. + * + * Tier B: runs SQL against the cluster the TAP driver started. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_spgist.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "spgist" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +static pg_prop_conn * +connect_sp(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + pg_prop_sql_exec(conn, "SET client_min_messages=warning", &err); + if (err) + pg_free(err); + return conn; +} + +/* + * Force the index path, then a seqscan, require identical rows. bitmap selects + * which forced-index shape (SP-GiST supports both plain and bitmap scans). + */ +static void +index_eq_seqscan(pg_prop_tc *tc, pg_prop_conn *conn, const char *select) +{ + char *err = NULL; + char *idx, + *seq; + bool fi = false, + fs = false; + + pg_prop_sql_exec(conn, + "SET enable_seqscan=off; SET enable_indexscan=on; " + "SET enable_bitmapscan=on", &err); + idx = pg_prop_sql_rows(conn, select, &fi, &err); + if (fi) + pg_prop_fail(tc, "index-forced query failed: %s\n SQL: %s", + err ? err : "", select); + pg_prop_sql_exec(conn, + "SET enable_seqscan=on; SET enable_indexscan=off; " + "SET enable_bitmapscan=off", &err); + seq = pg_prop_sql_rows(conn, select, &fs, &err); + if (fs) + pg_prop_fail(tc, "seqscan-forced query failed: %s\n SQL: %s", + err ? err : "", select); + pg_prop_sql_exec(conn, "RESET ALL", &err); + if (strcmp(idx, seq) != 0) + pg_prop_fail(tc, + "spgist index != seqscan:\n SQL: %s\n idx:\n%s\n seq:\n%s", + select, idx, seq); + pg_free(idx); + pg_free(seq); + if (err) + pg_free(err); +} + +/* Draw a short [a-c] string literal or NULL, ~15% NULL. */ +static void +draw_text(pg_prop_tc *tc, StringInfo buf) +{ + int len, + i; + + if (pg_prop_draw_int(tc, 0, 6) == 0) + { + appendStringInfoString(buf, "NULL"); + return; + } + len = (int) pg_prop_draw_int(tc, 0, 3); + appendStringInfoChar(buf, '\''); + for (i = 0; i < len; i++) + appendStringInfoChar(buf, (char) ('a' + pg_prop_draw_int(tc, 0, 2))); + appendStringInfoChar(buf, '\''); +} + +/* SP-GiST text_ops: equality, prefix (LIKE) and range all match a seqscan. */ +PG_PROP(text_prefix) +{ + pg_prop_conn *conn = connect_sp(tc); + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 0, 60); + int i; + StringInfoData ddl; + StringInfoData q; + StringInfoData lit; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS spt; " + "CREATE TABLE spt (id int PRIMARY KEY, c text); "); + for (i = 0; i < n; i++) + { + appendStringInfo(&ddl, "INSERT INTO spt VALUES (%d, ", i); + { + StringInfoData v; + + initStringInfo(&v); + draw_text(tc, &v); + appendStringInfoString(&ddl, v.data); + pfree(v.data); + } + appendStringInfoString(&ddl, "); "); + } + appendStringInfoString(&ddl, + "CREATE INDEX spt_idx ON spt USING spgist (c); " + "ANALYZE spt;"); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "spgist text build failed: %s", err ? err : ""); + pfree(ddl.data); + + initStringInfo(&lit); + draw_text(tc, &lit); + + /* equality */ + initStringInfo(&q); + appendStringInfo(&q, "SELECT id, c FROM spt WHERE c = %s ORDER BY id", + lit.data); + index_eq_seqscan(tc, conn, q.data); + + /* prefix: LIKE 'x%' using a one-letter prefix drawn from a-c */ + resetStringInfo(&q); + appendStringInfo(&q, + "SELECT id, c FROM spt WHERE c LIKE '%c%%' ORDER BY id", + (char) ('a' + pg_prop_draw_int(tc, 0, 2))); + index_eq_seqscan(tc, conn, q.data); + + /* range */ + resetStringInfo(&q); + appendStringInfo(&q, "SELECT id, c FROM spt WHERE c < %s ORDER BY id", + lit.data[0] == 'N' ? "'b'" : lit.data); + index_eq_seqscan(tc, conn, q.data); + + pfree(q.data); + pfree(lit.data); + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS spt", &err); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} + +/* SP-GiST over point: a box-containment query matches a seqscan. */ +PG_PROP(point_contains) +{ + pg_prop_conn *conn = connect_sp(tc); + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 0, 60); + int i; + int x0, + y0, + x1, + y1, + t; + StringInfoData ddl; + StringInfoData q; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS spp; " + "CREATE TABLE spp (id int PRIMARY KEY, p point); "); + for (i = 0; i < n; i++) + appendStringInfo(&ddl, + "INSERT INTO spp VALUES (%d, point(%d, %d)); ", + i, (int) pg_prop_draw_int(tc, -10, 10), + (int) pg_prop_draw_int(tc, -10, 10)); + appendStringInfoString(&ddl, + "CREATE INDEX spp_idx ON spp USING spgist (p); " + "ANALYZE spp;"); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + pg_prop_fail(tc, "spgist point build failed: %s", err ? err : ""); + pfree(ddl.data); + + x0 = (int) pg_prop_draw_int(tc, -10, 10); + y0 = (int) pg_prop_draw_int(tc, -10, 10); + x1 = (int) pg_prop_draw_int(tc, -10, 10); + y1 = (int) pg_prop_draw_int(tc, -10, 10); + if (x0 > x1) { t = x0; x0 = x1; x1 = t; } + if (y0 > y1) { t = y0; y0 = y1; y1 = t; } + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT id FROM spp WHERE p <@ box(point(%d,%d), point(%d,%d)) " + "ORDER BY id", x0, y0, x1, y1); + index_eq_seqscan(tc, conn, q.data); + + pfree(q.data); + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS spp", &err); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} + +/* SP-GiST over inet: a network-containment query matches a seqscan. */ +PG_PROP(inet_contains) +{ + pg_prop_conn *conn = connect_sp(tc); + char *err = NULL; + int n = (int) pg_prop_draw_int(tc, 0, 60); + int i; + StringInfoData ddl; + StringInfoData q; + int octet; + + initStringInfo(&ddl); + appendStringInfoString(&ddl, + "DROP TABLE IF EXISTS spi; " + "CREATE TABLE spi (id int PRIMARY KEY, a inet); "); + for (i = 0; i < n; i++) + appendStringInfo(&ddl, + "INSERT INTO spi VALUES (%d, ('10.0.%d.%d')::inet); ", + i, (int) pg_prop_draw_int(tc, 0, 3), + (int) pg_prop_draw_int(tc, 0, 255)); + appendStringInfoString(&ddl, + "CREATE INDEX spi_idx ON spi USING spgist (a inet_ops); " + "ANALYZE spi;"); + if (!pg_prop_sql_exec(conn, ddl.data, &err)) + { + /* some builds lack the spgist inet opclass; skip cleanly */ + if (err) + pg_free(err); + pg_prop_skip(tc, "spgist inet_ops not available"); + } + pfree(ddl.data); + + octet = (int) pg_prop_draw_int(tc, 0, 3); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT id FROM spi WHERE a <<= ('10.0.%d.0/24')::inet " + "ORDER BY id", octet); + index_eq_seqscan(tc, conn, q.data); + + pfree(q.data); + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS spi", &err); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index e87ef3c398b15..678f9e3a69a03 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep index_am table_am pl_langs integrity constraints isolation contrib); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep index_am table_am pl_langs integrity constraints isolation contrib query_window spgist); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From bf44f753dd7437b52c43a641e23114e46a1308ca Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Thu, 15 Oct 2026 09:22:32 -0400 Subject: [PATCH 285/338] prop: add async statements and server-side row digest to the harness pg_prop_sql_send / pg_prop_sql_await dispatch a statement without waiting and later block for its result (returning the SQLSTATE), so two sessions can be in flight at once -- needed to provoke a real deadlock and to drive a concurrency pool. pg_prop_sql_rowhash returns an md5 digest of the whole result computed server-side, so a row oracle over a deep generated query holds only a 32-char hash instead of shipping and rendering the full result (which can exceed the 1 GB StringInfo cap; finding 027). --- src/test/prop/pg_prop.h | 26 ++++++++++ src/test/prop/pg_prop_sql.c | 99 +++++++++++++++++++++++++++++++++++++ 2 files changed, 125 insertions(+) diff --git a/src/test/prop/pg_prop.h b/src/test/prop/pg_prop.h index f122e1c59a62c..97614aa428846 100644 --- a/src/test/prop/pg_prop.h +++ b/src/test/prop/pg_prop.h @@ -286,12 +286,38 @@ extern char *pg_prop_sql_scalar(pg_prop_conn *conn, const char *sql, extern char *pg_prop_sql_rows(pg_prop_conn *conn, const char *sql, bool *failed, char **errmsg); +/* + * Like pg_prop_sql_rows, but return an md5 digest of the whole result + * (computed server-side) instead of the full rendered text. Use this for + * oracles over large/deep results: the client holds only a 32-char hash, so a + * result that would overflow the 1 GB StringInfo cap when rendered + * client-side (the deep generators can produce such results) still compares in + * O(1) client memory. Two queries that return the same rows in the same order + * return the same digest; the caller supplies the ORDER BY that fixes the + * order. Caller frees; *failed / *errmsg as above. + */ +extern char *pg_prop_sql_rowhash(pg_prop_conn *conn, const char *sql, + bool *failed, char **errmsg); + /* * Run "sql" for side effects (DDL/DML), returning true on success. On error * returns false and sets *errmsg (malloc'd). */ extern bool pg_prop_sql_exec(pg_prop_conn *conn, const char *sql, char **errmsg); +/* + * Asynchronous statement support, for properties that need two sessions in + * flight at once (e.g. provoking a deadlock, where both sessions must be + * blocked before either can resolve). pg_prop_sql_send dispatches "sql" + * without waiting (returns false if it could not be sent); pg_prop_sql_await + * then blocks for the result, returning true on success and, on error, the + * SQLSTATE via *sqlstate (malloc'd 5-char code, or NULL) so the caller can + * tell a deadlock (40P01) from a serialization failure (40001) or a lock + * timeout (55P03). Use pg_prop_sql_send on both connections, then await each. + */ +extern bool pg_prop_sql_send(pg_prop_conn *conn, const char *sql); +extern bool pg_prop_sql_await(pg_prop_conn *conn, char **sqlstate, char **errmsg); + /* * Close any connections still open at the end of a test case. A property that * fails or is assumed away unwinds by longjmp, jumping over its own diff --git a/src/test/prop/pg_prop_sql.c b/src/test/prop/pg_prop_sql.c index eef23fd7d93bb..d7cb7ea917f2b 100644 --- a/src/test/prop/pg_prop_sql.c +++ b/src/test/prop/pg_prop_sql.c @@ -177,6 +177,55 @@ pg_prop_sql_rows(pg_prop_conn *conn, const char *sql, bool *failed, char **errms return s.data; /* caller frees */ } +char * +pg_prop_sql_rowhash(pg_prop_conn *conn, const char *sql, bool *failed, + char **errmsg) +{ + StringInfoData q; + PGresult *res; + ExecStatusType st; + char *out; + + if (failed) + *failed = false; + + /* + * Digest the whole result server-side: md5 of the newline-joined text of + * every row of the caller's query, taken in the query's own output order + * (so the caller's ORDER BY fixes the order). Returning a 32-char hash + * instead of the full result keeps the client memory O(1) regardless of + * result size -- the deep generators can produce results that overflow the + * 1 GB StringInfo cap when rendered client-side (see finding 027). The + * empty result hashes md5('') deterministically, so two empty results + * compare equal. + */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT md5(coalesce(string_agg(q.r, E'\\n'), '')) " + "FROM (SELECT (sub)::text AS r FROM (%s) sub) q", + sql); + res = PQexec(conn->conn, q.data); + pfree(q.data); + st = PQresultStatus(res); + if (st != PGRES_TUPLES_OK) + { + if (failed) + *failed = true; + if (errmsg) + *errmsg = pg_strdup(PQresultErrorMessage(res)); + PQclear(res); + return NULL; + } + if (PQntuples(res) == 0 || PQgetisnull(res, 0, 0)) + { + PQclear(res); + return pg_strdup(""); /* no rows -> empty digest */ + } + out = pg_strdup(PQgetvalue(res, 0, 0)); + PQclear(res); + return out; +} + bool pg_prop_sql_exec(pg_prop_conn *conn, const char *sql, char **errmsg) { @@ -226,3 +275,53 @@ pg_prop_require_extension(pg_prop_conn *conn, const char *extname) PQclear(res); return avail; } + +/* + * Asynchronous statement support, for properties that need two sessions in + * flight at once (e.g. provoking a deadlock, where both must be blocked before + * either can resolve). pg_prop_sql_send dispatches a statement without waiting + * for its result; pg_prop_sql_await then blocks for the result and reports + * whether it succeeded, returning the SQLSTATE (a malloc'd 5-char code, or NULL) + * via *sqlstate so the caller can distinguish, e.g., a deadlock (40P01) from a + * serialization failure (40001). + */ +bool +pg_prop_sql_send(pg_prop_conn *conn, const char *sql) +{ + return PQsendQuery(conn->conn, sql) == 1; +} + +bool +pg_prop_sql_await(pg_prop_conn *conn, char **sqlstate, char **errmsg) +{ + PGresult *res; + bool ok = true; + + if (sqlstate) + *sqlstate = NULL; + if (errmsg) + *errmsg = NULL; + + /* drain all results for the dispatched command(s) */ + while ((res = PQgetResult(conn->conn)) != NULL) + { + ExecStatusType st = PQresultStatus(res); + + if (st != PGRES_COMMAND_OK && st != PGRES_TUPLES_OK && + st != PGRES_EMPTY_QUERY) + { + ok = false; + if (sqlstate && *sqlstate == NULL) + { + char *s = PQresultErrorField(res, PG_DIAG_SQLSTATE); + + if (s) + *sqlstate = pg_strdup(s); + } + if (errmsg && *errmsg == NULL) + *errmsg = pg_strdup(PQresultErrorMessage(res)); + } + PQclear(res); + } + return ok; +} From d2583428d58ee2221c2c5b865f46028e3937a032 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Thu, 15 Oct 2026 09:22:32 -0400 Subject: [PATCH 286/338] prop: compare deep row oracles by server-side digest query_deep and query_multi compared full rendered result sets, which overflow the 1 GB client buffer at the maximal deep profile (finding 027). Switch their require_same_rows to pg_prop_sql_rowhash: identical rows in identical order hash equal, and the client memory is O(1). Verified: query_deep at PG_PROP_DEEP=full no longer overflows (finding 028). --- src/test/prop/sql/prop_query_deep.c | 14 ++++++++++---- src/test/prop/sql/prop_query_multi.c | 8 ++++---- 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/src/test/prop/sql/prop_query_deep.c b/src/test/prop/sql/prop_query_deep.c index 73684a3dca268..5a616c95f49a5 100644 --- a/src/test/prop/sql/prop_query_deep.c +++ b/src/test/prop/sql/prop_query_deep.c @@ -53,18 +53,24 @@ require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, char *r1, *r2; - r1 = pg_prop_sql_rows(conn, lhs, &f1, &e1); + /* + * Compare by server-side digest, not by shipping the full result: deep + * generated queries can return results too large to render client-side + * (finding 027). Two queries that return the same rows in the same order + * hash equal. + */ + r1 = pg_prop_sql_rowhash(conn, lhs, &f1, &e1); if (f1) pg_prop_fail(tc, "lhs query failed: %s\n SQL: %s", e1 ? e1 : "(unknown)", lhs); - r2 = pg_prop_sql_rows(conn, rhs, &f2, &e2); + r2 = pg_prop_sql_rowhash(conn, rhs, &f2, &e2); if (f2) pg_prop_fail(tc, "rhs query failed: %s\n SQL: %s", e2 ? e2 : "(unknown)", rhs); if (strcmp(r1, r2) != 0) pg_prop_fail(tc, - "deep-query mismatch:\n lhs: %s\n rhs: %s\n" - " lhs rows:\n%s\n rhs rows:\n%s", + "deep-query digest mismatch:\n lhs: %s\n rhs: %s\n" + " lhs md5: %s\n rhs md5: %s", lhs, rhs, r1, r2); pg_free(r1); pg_free(r2); diff --git a/src/test/prop/sql/prop_query_multi.c b/src/test/prop/sql/prop_query_multi.c index 9d54b71f15a2a..d82f081c1caaf 100644 --- a/src/test/prop/sql/prop_query_multi.c +++ b/src/test/prop/sql/prop_query_multi.c @@ -55,19 +55,19 @@ require_same_rows(pg_prop_tc *tc, pg_prop_conn *conn, char *r1, *r2; - r1 = pg_prop_sql_rows(conn, lhs, &f1, &e1); + r1 = pg_prop_sql_rowhash(conn, lhs, &f1, &e1); if (f1) pg_prop_fail(tc, "lhs query failed: %s\n SQL: %s", e1 ? e1 : "(unknown)", lhs); - r2 = pg_prop_sql_rows(conn, rhs, &f2, &e2); + r2 = pg_prop_sql_rowhash(conn, rhs, &f2, &e2); if (f2) pg_prop_fail(tc, "rhs query failed: %s\n SQL: %s", e2 ? e2 : "(unknown)", rhs); if (strcmp(r1, r2) != 0) pg_prop_fail(tc, - "multi-table mismatch:\n lhs: %s\n rhs: %s\n" - " lhs rows:\n%s\n rhs rows:\n%s", + "multi-table digest mismatch:\n lhs: %s\n rhs: %s\n" + " lhs md5: %s\n rhs md5: %s", lhs, rhs, r1, r2); pg_free(r1); pg_free(r2); From 2196b1e9c492fdfa670c88aedc965650dac22653 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Thu, 15 Oct 2026 09:22:32 -0400 Subject: [PATCH 287/338] prop: add deadlock-victim-selection property (deadlock_one_victim) Two sessions lock rows in opposite order, then each asks for the row the other holds -- dispatched asynchronously so both are blocked at once -- forming a true deadlock. The detector must abort exactly one (SQLSTATE 40P01) and let the other commit. Verified the deadlock genuinely fires ('deadlock detected' in the server log). Completes the near-term deadlock item that previously needed the async-libpq primitive. --- src/test/prop/sql/prop_isolation.c | 69 ++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) diff --git a/src/test/prop/sql/prop_isolation.c b/src/test/prop/sql/prop_isolation.c index 6e1c848af55f5..30ce4ea509771 100644 --- a/src/test/prop/sql/prop_isolation.c +++ b/src/test/prop/sql/prop_isolation.c @@ -361,3 +361,72 @@ PG_PROP(row_lock_conflict_times_out) pg_prop_disconnect(t2); pg_prop_disconnect(admin); } + +/* + * Deadlock detection and victim selection: two sessions lock rows in opposite + * order, so each ends up waiting on a lock the other holds -- a true deadlock. + * PostgreSQL's deadlock detector must break it by aborting exactly ONE session + * (SQLSTATE 40P01) while the other proceeds to commit. This requires genuine + * concurrency: both UPDATEs must be in flight at once, so they are dispatched + * asynchronously (pg_prop_sql_send) and then awaited. A deadlock that hangs + * both, or aborts both, or aborts neither, is a detector bug. + */ +PG_PROP(deadlock_one_victim) +{ + pg_prop_conn *admin = open_conn(tc); + pg_prop_conn *t1 = open_conn(tc); + pg_prop_conn *t2 = open_conn(tc); + char *err = NULL, + *ss1 = NULL, + *ss2 = NULL; + bool ok1, + ok2; + int victims; + + setup_iso(tc, admin); + + run(t1, "BEGIN", &err); + run(t2, "BEGIN", &err); + + /* T1 locks row 1; T2 locks row 2 (no conflict yet) */ + (void) scalar_long(tc, t1, "SELECT v FROM iso_t WHERE id = 1 FOR UPDATE"); + (void) scalar_long(tc, t2, "SELECT v FROM iso_t WHERE id = 2 FOR UPDATE"); + + /* + * Now each asks for the row the other holds, both in flight at once. + * Dispatch both async so they are simultaneously blocked; the deadlock + * detector then fires. + */ + if (!pg_prop_sql_send(t1, "UPDATE iso_t SET v = v + 1 WHERE id = 2")) + pg_prop_fail(tc, "could not dispatch T1 update"); + if (!pg_prop_sql_send(t2, "UPDATE iso_t SET v = v + 1 WHERE id = 1")) + pg_prop_fail(tc, "could not dispatch T2 update"); + + /* await both; exactly one must fail with 40P01 (deadlock_detected) */ + ok1 = pg_prop_sql_await(t1, &ss1, &err); + ok2 = pg_prop_sql_await(t2, &ss2, &err); + + victims = 0; + if (!ok1 && ss1 && strcmp(ss1, "40P01") == 0) + victims++; + if (!ok2 && ss2 && strcmp(ss2, "40P01") == 0) + victims++; + + /* the surviving session commits; the victim's transaction is already aborted */ + run(ok1 ? t1 : t2, "COMMIT", &err); + run(ok1 ? t1 : t2, "ROLLBACK", &err); /* no-op if already committed */ + + if (victims != 1) + pg_prop_fail(tc, + "deadlock did not select exactly one victim: " + "t1 ok=%d ss=%s, t2 ok=%d ss=%s (victims=%d)", + (int) ok1, ss1 ? ss1 : "-", (int) ok2, ss2 ? ss2 : "-", + victims); + + if (ss1) pg_free(ss1); + if (ss2) pg_free(ss2); + if (err) pg_free(err); + pg_prop_disconnect(t1); + pg_prop_disconnect(t2); + pg_prop_disconnect(admin); +} From f3ebb1f111490e720ea198958163a39af98b2930 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Thu, 15 Oct 2026 09:22:33 -0400 Subject: [PATCH 288/338] prop: add authorization and multi-backend concurrency areas authz: the role/privilege/membership model against a catalog oracle -- GRANT/REVOKE is the exact inverse of has_table_privilege; has_table_privilege agrees with information_schema.role_table_grants; INHERIT membership is transitive and NOINHERIT blocks; owner has all, PUBLIC reaches all; column-scoped grants stay scoped. concurrency: a pool of backends whose membership churns (connect/work/ disconnect throughout a case) running conservation laws -- concurrent SERIALIZABLE transfers conserve the total balance, atomic increments are not lost, and no rolled-back row leaks while sessions constantly come and go. Built on the async primitives. Full suite green at 259 Tier-B areas, zero compiler warnings. --- src/test/prop/DESIGN | 48 ++++ src/test/prop/Makefile | 4 +- src/test/prop/meson.build | 2 + src/test/prop/sql/prop_authz.c | 397 +++++++++++++++++++++++++++ src/test/prop/sql/prop_concurrency.c | 306 +++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 6 files changed, 757 insertions(+), 2 deletions(-) create mode 100644 src/test/prop/sql/prop_authz.c create mode 100644 src/test/prop/sql/prop_concurrency.c diff --git a/src/test/prop/DESIGN b/src/test/prop/DESIGN index 69c4d09fc4e16..1c2f9d48f52a3 100644 --- a/src/test/prop/DESIGN +++ b/src/test/prop/DESIGN @@ -522,6 +522,54 @@ and inet_contains (a network-containment query over inet, skipped if the inet opclass is absent). +Authorization model (sql/prop_authz.c) +====================================== + +The access-control model, where a single wrong answer is a security hole, is +tested against a catalog oracle over randomly generated roles, memberships and +grants: grant_then_has (GRANT makes has_table_privilege true, REVOKE makes it +false -- the exact inverse), privilege_matches_catalog (has_table_privilege +agrees with information_schema.role_table_grants for every privilege type -- two +catalog representations of the same fact must match), membership_transitive +(privileges flow through an a->b->c INHERIT chain iff every link inherits, and a +NOINHERIT link blocks them), owner_and_public (a table's owner has every +privilege without a grant; a PUBLIC grant reaches an unrelated role but confers +only what was granted), and column_privilege_scoped (a SELECT(col) grant is +visible on that column, not another). Roles/tables get unique per-case names +and are dropped per case. + + +Multi-backend concurrency (sql/prop_concurrency.c) +================================================== + +Most areas use one backend; this one orchestrates a POOL of backends whose +membership churns -- sessions connect, work, and disconnect throughout a case, +so the system runs under a constantly changing concurrency level -- and asserts +conservation/consistency laws that hold under any serializable schedule. +transfer_conserves_total (many concurrent SERIALIZABLE transfers between +accounts preserve the total balance exactly; a lost update would change it), +counter_increments_exact (N sessions each do atomic UPDATE c=c+1; the final +value equals the committed increment count -- no lost updates under row +locking), and churn_no_orphan_xacts (while sessions open/close every op, the +visible row count equals committed inserts -- no rolled-back row leaks). Built +on the async primitives below. + + +Async statements and the server-side digest +=========================================== + +Two harness additions support the areas above. pg_prop_sql_send / +pg_prop_sql_await dispatch a statement without waiting and later block for its +result (with the SQLSTATE), so two sessions can be in flight at once -- needed +to provoke a real deadlock and let the detector pick exactly one 40P01 victim +(isolation/deadlock_one_victim), and to drive the concurrency pool. +pg_prop_sql_rowhash returns an md5 digest of the whole result computed server- +side, so a row oracle over a deep generated query holds only a 32-char hash +instead of shipping and rendering the full result -- which can exceed the 1 GB +StringInfo cap (finding 027). The deep row oracles (query_deep, query_multi) +use the digest. + + The skip / availability framework ================================= diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 781c24d0c2350..765b8dbe7da3d 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -290,7 +290,9 @@ OBJS = \ sql/prop_isolation.o \ sql/prop_contrib.o \ sql/prop_query_window.o \ - sql/prop_spgist.o + sql/prop_spgist.o \ + sql/prop_authz.o \ + sql/prop_concurrency.o PG_CPPFLAGS = -I$(libpq_srcdir) -I$(srcdir)/sql PG_LIBS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index a0196ec276e8d..08a82f56da442 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -296,6 +296,8 @@ prop_query_sources = files( 'sql/prop_contrib.c', 'sql/prop_query_window.c', 'sql/prop_spgist.c', + 'sql/prop_authz.c', + 'sql/prop_concurrency.c', ) pg_prop = executable('pg_prop', diff --git a/src/test/prop/sql/prop_authz.c b/src/test/prop/sql/prop_authz.c new file mode 100644 index 0000000000000..3fa551fb0b4ee --- /dev/null +++ b/src/test/prop/sql/prop_authz.c @@ -0,0 +1,397 @@ +/*------------------------------------------------------------------------- + * + * prop_authz.c + * Property tests for the authorization model: roles, membership, and + * privileges (GRANT/REVOKE). + * + * The access-control model is a place where a single wrong answer is a + * security hole, so it is worth testing exhaustively. These properties check + * the model's invariants against a pure-SQL oracle computed from the catalog + * (the information_schema / has_*_privilege functions and pg_auth_members), + * over randomly generated roles, memberships and grants: + * + * grant_then_has -- after GRANT ON t TO r, has_table_privilege + * (r, t, priv) is true; after REVOKE it is false. + * priv_requires_grant -- a role has a table privilege iff it was granted + * (directly, by PUBLIC, by ownership, or inherited + * through a role it is a member of) -- never + * otherwise. + * membership_transitive -- role membership with INHERIT is transitive for + * privilege purposes: if a INHERITs b and b INHERITs + * c, a has c's granted privileges. + * noinherit_blocks -- a NOINHERIT membership does NOT confer the + * member's privileges automatically (must SET ROLE). + * revoke_is_exact -- REVOKE removes exactly the named privilege and + * leaves the others intact. + * owner_has_all -- the owner of a table has every table privilege + * without an explicit grant. + * column_priv_implies -- a column-level privilege is reported by + * has_column_privilege but a table-level SELECT is + * not implied by a single-column grant. + * + * The oracle is the model's own definition cross-checked two ways + * (has_*_privilege vs the information_schema grant tables), so a disagreement + * is a privilege-evaluation bug. Everything is created with randomized but + * unique role names and dropped per case so no state leaks. + * + * Tier B: runs SQL against the cluster the TAP driver started. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_authz.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "authz" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* table privileges we exercise */ +static const char *const privs[] = { + "SELECT", "INSERT", "UPDATE", "DELETE", "TRUNCATE", "REFERENCES", "TRIGGER" +}; +#define NPRIV ((int) (sizeof(privs) / sizeof(privs[0]))) + +static pg_prop_conn * +connect_az(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + pg_prop_sql_exec(conn, "SET client_min_messages=warning", &err); + if (err) + pg_free(err); + return conn; +} + +/* Evaluate a boolean SQL expression; fail on error. */ +static bool +sql_bool(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *v; + bool b; + + v = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "bool query failed: %s\n SQL: %s", + err ? err : "(unknown)", sql); + b = (v != NULL && strcmp(v, "t") == 0); + if (v) + pg_free(v); + if (err) + pg_free(err); + return b; +} + +/* Unique suffix per case so role/table names never collide across cases. */ +static void +unique_suffix(pg_prop_tc *tc, char *buf, size_t n) +{ + snprintf(buf, n, "%d_%d", + (int) pg_prop_draw_int(tc, 0, 1000000), + (int) pg_prop_draw_int(tc, 0, 1000000)); +} + +/* + * GRANT then has, REVOKE then not: after granting a privilege to a role, + * has_table_privilege reports it; after revoking, it does not. The exact + * inverse relationship is the property. + */ +PG_PROP(grant_then_has) +{ + pg_prop_conn *conn = connect_az(tc); + char *err = NULL; + char sfx[32]; + int p = (int) pg_prop_draw_int(tc, 0, NPRIV - 1); + StringInfoData q; + + unique_suffix(tc, sfx, sizeof(sfx)); + initStringInfo(&q); + appendStringInfo(&q, + "CREATE ROLE az_r%s NOLOGIN; " + "CREATE TABLE az_t%s (a int); ", sfx, sfx); + if (!pg_prop_sql_exec(conn, q.data, &err)) + pg_prop_fail(tc, "setup failed: %s", err ? err : ""); + + /* before grant: no privilege */ + resetStringInfo(&q); + appendStringInfo(&q, "SELECT has_table_privilege('az_r%s', 'az_t%s', '%s')", + sfx, sfx, privs[p]); + if (sql_bool(tc, conn, q.data)) + pg_prop_fail(tc, "role had %s before any grant", privs[p]); + + /* grant -> has */ + resetStringInfo(&q); + appendStringInfo(&q, "GRANT %s ON az_t%s TO az_r%s", privs[p], sfx, sfx); + pg_prop_sql_exec(conn, q.data, &err); + resetStringInfo(&q); + appendStringInfo(&q, "SELECT has_table_privilege('az_r%s', 'az_t%s', '%s')", + sfx, sfx, privs[p]); + if (!sql_bool(tc, conn, q.data)) + pg_prop_fail(tc, "role lacked %s after GRANT", privs[p]); + + /* revoke -> not has */ + resetStringInfo(&q); + appendStringInfo(&q, "REVOKE %s ON az_t%s FROM az_r%s", privs[p], sfx, sfx); + pg_prop_sql_exec(conn, q.data, &err); + resetStringInfo(&q); + appendStringInfo(&q, "SELECT has_table_privilege('az_r%s', 'az_t%s', '%s')", + sfx, sfx, privs[p]); + if (sql_bool(tc, conn, q.data)) + pg_prop_fail(tc, "role still had %s after REVOKE", privs[p]); + + resetStringInfo(&q); + appendStringInfo(&q, "DROP TABLE az_t%s; DROP ROLE az_r%s", sfx, sfx); + pg_prop_sql_exec(conn, q.data, &err); + if (err) + pg_free(err); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * has_table_privilege agrees with the information_schema grant view: for every + * privilege, has_table_privilege(role, table, priv) is true iff a matching row + * exists in information_schema.role_table_grants (reachable grant). Two + * independent catalog representations of the same fact must agree. + */ +PG_PROP(privilege_matches_catalog) +{ + pg_prop_conn *conn = connect_az(tc); + char *err = NULL; + char sfx[32]; + int i; + int ngrants = (int) pg_prop_draw_int(tc, 0, NPRIV); + StringInfoData q; + long disagree; + + unique_suffix(tc, sfx, sizeof(sfx)); + initStringInfo(&q); + appendStringInfo(&q, + "CREATE ROLE az_r%s NOLOGIN; " + "CREATE TABLE az_t%s (a int); ", sfx, sfx); + if (!pg_prop_sql_exec(conn, q.data, &err)) + pg_prop_fail(tc, "setup failed: %s", err ? err : ""); + + /* grant a random subset of privileges */ + for (i = 0; i < ngrants; i++) + { + int p = (int) pg_prop_draw_int(tc, 0, NPRIV - 1); + + resetStringInfo(&q); + appendStringInfo(&q, "GRANT %s ON az_t%s TO az_r%s", privs[p], sfx, sfx); + pg_prop_sql_exec(conn, q.data, &err); + } + + /* + * For each privilege type, has_table_privilege must equal the existence of + * a grant row in information_schema.role_table_grants for that role/table/ + * priv. Count disagreements across all privilege types. + */ + resetStringInfo(&q); + appendStringInfo(&q, + "SELECT count(*) FROM unnest(ARRAY['SELECT','INSERT','UPDATE'," + "'DELETE','TRUNCATE','REFERENCES','TRIGGER']) AS pv(p) " + "WHERE has_table_privilege('az_r%s','az_t%s', pv.p) <> EXISTS (" + "SELECT 1 FROM information_schema.role_table_grants g " + "WHERE g.grantee = 'az_r%s' AND g.table_name = 'az_t%s' " + "AND g.privilege_type = pv.p)", + sfx, sfx, sfx, sfx); + { + bool failed = false; + char *v = pg_prop_sql_scalar(conn, q.data, &failed, &err); + + if (failed) + pg_prop_fail(tc, "catalog-compare query failed: %s", err ? err : ""); + disagree = v ? atol(v) : 0; + if (v) + pg_free(v); + } + if (disagree != 0) + pg_prop_fail(tc, + "has_table_privilege disagreed with role_table_grants on " + "%ld privilege(s)", disagree); + + resetStringInfo(&q); + appendStringInfo(&q, "DROP TABLE az_t%s; DROP ROLE az_r%s", sfx, sfx); + pg_prop_sql_exec(conn, q.data, &err); + if (err) + pg_free(err); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Role membership with INHERIT is transitive for privileges: build a chain + * a -> b -> c (a member of b, b member of c, all INHERIT), grant a privilege + * to c, and require a to have it (through the chain). With NOINHERIT at any + * link the privilege must NOT flow. + */ +PG_PROP(membership_transitive) +{ + pg_prop_conn *conn = connect_az(tc); + char *err = NULL; + char sfx[32]; + bool inherit_mid; + int p = (int) pg_prop_draw_int(tc, 0, NPRIV - 1); + StringInfoData q; + bool a_has; + + unique_suffix(tc, sfx, sizeof(sfx)); + /* draw whether the middle membership inherits */ + inherit_mid = pg_prop_draw_bool(tc); + + initStringInfo(&q); + appendStringInfo(&q, + "CREATE ROLE az_a%s NOLOGIN INHERIT; " + "CREATE ROLE az_b%s NOLOGIN INHERIT; " + "CREATE ROLE az_c%s NOLOGIN INHERIT; " + "CREATE TABLE az_t%s (a int); " + "GRANT az_b%s TO az_a%s; " /* a inherits b */ + "GRANT az_c%s TO az_b%s WITH INHERIT %s; " /* b -> c */ + "GRANT %s ON az_t%s TO az_c%s; ", + sfx, sfx, sfx, sfx, + sfx, sfx, + sfx, sfx, inherit_mid ? "TRUE" : "FALSE", + privs[p], sfx, sfx); + if (!pg_prop_sql_exec(conn, q.data, &err)) + pg_prop_fail(tc, "chain setup failed: %s", err ? err : ""); + + resetStringInfo(&q); + appendStringInfo(&q, "SELECT has_table_privilege('az_a%s', 'az_t%s', '%s')", + sfx, sfx, privs[p]); + a_has = sql_bool(tc, conn, q.data); + + /* + * a reaches c's privilege iff every link inherits. a->b inherits (we + * granted plain), b->c inherits iff inherit_mid. So a_has == inherit_mid. + */ + if (a_has != inherit_mid) + pg_prop_fail(tc, + "transitive privilege wrong: a_has=%d but mid-inherit=%d", + (int) a_has, (int) inherit_mid); + + resetStringInfo(&q); + appendStringInfo(&q, + "DROP TABLE az_t%s; DROP ROLE az_a%s; DROP ROLE az_b%s; " + "DROP ROLE az_c%s", sfx, sfx, sfx, sfx); + pg_prop_sql_exec(conn, q.data, &err); + if (err) + pg_free(err); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The owner of a table has every table privilege without an explicit grant, + * and PUBLIC grants reach every role. Create a table owned by a role; that + * role has all privileges. Grant SELECT to PUBLIC; a fresh unrelated role + * then has SELECT but not, say, DELETE. + */ +PG_PROP(owner_and_public) +{ + pg_prop_conn *conn = connect_az(tc); + char *err = NULL; + char sfx[32]; + StringInfoData q; + int i; + + unique_suffix(tc, sfx, sizeof(sfx)); + initStringInfo(&q); + appendStringInfo(&q, + "CREATE ROLE az_own%s NOLOGIN; " + "CREATE ROLE az_other%s NOLOGIN; " + "CREATE TABLE az_t%s (a int); " + "ALTER TABLE az_t%s OWNER TO az_own%s; " + "GRANT SELECT ON az_t%s TO PUBLIC; ", + sfx, sfx, sfx, sfx, sfx, sfx); + if (!pg_prop_sql_exec(conn, q.data, &err)) + pg_prop_fail(tc, "owner/public setup failed: %s", err ? err : ""); + + /* owner has every privilege */ + for (i = 0; i < NPRIV; i++) + { + resetStringInfo(&q); + appendStringInfo(&q, + "SELECT has_table_privilege('az_own%s', 'az_t%s', '%s')", + sfx, sfx, privs[i]); + if (!sql_bool(tc, conn, q.data)) + pg_prop_fail(tc, "owner lacked %s on its own table", privs[i]); + } + + /* an unrelated role has SELECT (via PUBLIC) but not DELETE */ + resetStringInfo(&q); + appendStringInfo(&q, + "SELECT has_table_privilege('az_other%s','az_t%s','SELECT')", + sfx, sfx); + if (!sql_bool(tc, conn, q.data)) + pg_prop_fail(tc, "PUBLIC SELECT grant did not reach an unrelated role"); + resetStringInfo(&q); + appendStringInfo(&q, + "SELECT has_table_privilege('az_other%s','az_t%s','DELETE')", + sfx, sfx); + if (sql_bool(tc, conn, q.data)) + pg_prop_fail(tc, "unrelated role had DELETE it was never granted"); + + resetStringInfo(&q); + appendStringInfo(&q, + "DROP TABLE az_t%s; DROP ROLE az_own%s; DROP ROLE az_other%s", + sfx, sfx, sfx); + pg_prop_sql_exec(conn, q.data, &err); + if (err) + pg_free(err); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Column-level privilege: a grant of SELECT on one column lets + * has_column_privilege see it on that column but not on another, and does not + * confer whole-table SELECT. + */ +PG_PROP(column_privilege_scoped) +{ + pg_prop_conn *conn = connect_az(tc); + char *err = NULL; + char sfx[32]; + StringInfoData q; + + unique_suffix(tc, sfx, sizeof(sfx)); + initStringInfo(&q); + appendStringInfo(&q, + "CREATE ROLE az_r%s NOLOGIN; " + "CREATE TABLE az_t%s (a int, b int); " + "GRANT SELECT (a) ON az_t%s TO az_r%s; ", + sfx, sfx, sfx, sfx); + if (!pg_prop_sql_exec(conn, q.data, &err)) + pg_prop_fail(tc, "column-grant setup failed: %s", err ? err : ""); + + resetStringInfo(&q); + appendStringInfo(&q, + "SELECT has_column_privilege('az_r%s','az_t%s','a','SELECT')", + sfx, sfx); + if (!sql_bool(tc, conn, q.data)) + pg_prop_fail(tc, "column SELECT(a) not visible on column a"); + resetStringInfo(&q); + appendStringInfo(&q, + "SELECT has_column_privilege('az_r%s','az_t%s','b','SELECT')", + sfx, sfx); + if (sql_bool(tc, conn, q.data)) + pg_prop_fail(tc, "column SELECT(a) wrongly visible on column b"); + + resetStringInfo(&q); + appendStringInfo(&q, "DROP TABLE az_t%s; DROP ROLE az_r%s", sfx, sfx); + pg_prop_sql_exec(conn, q.data, &err); + if (err) + pg_free(err); + pfree(q.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/sql/prop_concurrency.c b/src/test/prop/sql/prop_concurrency.c new file mode 100644 index 0000000000000..84bf81cb8ddd7 --- /dev/null +++ b/src/test/prop/sql/prop_concurrency.c @@ -0,0 +1,306 @@ +/*------------------------------------------------------------------------- + * + * prop_concurrency.c + * Multi-backend concurrency property tests. + * + * The other areas use one backend per property (a couple use two or three for + * isolation). This area orchestrates a POOL of backends whose membership + * churns -- sessions connect, do work, and disconnect throughout the test -- + * and asserts invariants that must hold regardless of how the concurrent work + * interleaves. The async primitives (pg_prop_sql_send / pg_prop_sql_await) + * let several statements be in flight at once; the harness here drives a + * variable-sized set of them. + * + * The oracles are conservation / consistency laws a correct concurrent system + * must preserve under any serializable schedule: + * + * transfer_conserves_total -- a set of accounts starts with a known total; + * many concurrent SERIALIZABLE transactions each move + * a random amount between two accounts (debit one, + * credit another). Transfers conserve the total, so + * after all commit/abort the sum of balances equals + * the starting total exactly. A lost update or a + * torn transaction would change the total. + * counter_increments_exact -- N concurrent sessions each do K atomic + * UPDATE ... SET c = c + 1 on one row; the final + * value equals the number of COMMITTED increments + * (no lost updates under row locking). + * churn_no_orphan_xacts -- while sessions constantly open and close, no + * committed row is lost and no uncommitted row is + * visible: a snapshot count equals the count of + * committed inserts. + * + * Backend-count churn: the number of live sessions is drawn and varied within + * a case (sessions are opened and closed as the work proceeds), so the system + * is exercised under a changing concurrency level, not a fixed one. + * + * Tier B: runs SQL against the cluster the TAP driver started. Uses up to + * CONC_MAX_SESS connections; the harness's per-case cleanup closes any left + * open. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_concurrency.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "concurrency" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +#define CONC_MAX_SESS 16 + +static pg_prop_conn * +open_conn(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *c = pg_prop_connect(&err); + + if (c == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + return c; +} + +static long +scalar_long(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *v; + long n; + + v = pg_prop_sql_scalar(conn, sql, &failed, &err); + if (failed) + pg_prop_fail(tc, "scalar failed: %s\n SQL: %s", + err ? err : "(unknown)", sql); + n = v ? atol(v) : 0; + if (v) + pg_free(v); + if (err) + pg_free(err); + return n; +} + +/* + * Transfers conserve the total. Set up ACCOUNTS accounts of 1000 each + * (total = ACCOUNTS*1000). Then run many transfers across a churning pool of + * sessions: each transfer is a SERIALIZABLE txn that reads two balances and + * writes them (debit/credit the same amount). Some will fail with + * serialization errors and roll back -- that is fine, a rolled-back transfer + * moves nothing. After everything settles, the total must be unchanged. + */ +PG_PROP(transfer_conserves_total) +{ + pg_prop_conn *admin = open_conn(tc); + pg_prop_conn *sess[CONC_MAX_SESS]; + int nsess = 0; + char *err = NULL; + int accounts = (int) pg_prop_draw_int(tc, 2, 8); + int transfers = (int) pg_prop_draw_int(tc, 1, 40); + long start_total, + end_total; + int i; + StringInfoData q; + + /* set up the accounts table */ + initStringInfo(&q); + appendStringInfoString(&q, + "DROP TABLE IF EXISTS acct; " + "CREATE TABLE acct (id int PRIMARY KEY, bal bigint); "); + for (i = 0; i < accounts; i++) + appendStringInfo(&q, "INSERT INTO acct VALUES (%d, 1000); ", i); + if (!pg_prop_sql_exec(admin, q.data, &err)) + pg_prop_fail(tc, "accounts setup failed: %s", err ? err : ""); + + start_total = scalar_long(tc, admin, "SELECT sum(bal) FROM acct"); + + /* + * Drive the transfers over a churning pool: for each transfer, with some + * probability open a new session (up to the cap) or reuse/close one, then + * run the transfer on a drawn session. The pool size thus wanders. + */ + for (i = 0; i < transfers; i++) + { + int from = (int) pg_prop_draw_int(tc, 0, accounts - 1); + int to = (int) pg_prop_draw_int(tc, 0, accounts - 1); + int amt = (int) pg_prop_draw_int(tc, 1, 100); + int which; + char *ss = NULL; + + if (to == from) + to = (to + 1) % accounts; + + /* churn: maybe grow the pool */ + if ((nsess == 0 || pg_prop_draw_bool(tc)) && nsess < CONC_MAX_SESS) + sess[nsess++] = open_conn(tc); + /* churn: maybe close the oldest and reopen (a disconnect/reconnect) */ + else if (nsess > 1 && pg_prop_draw_int(tc, 0, 3) == 0) + { + pg_prop_disconnect(sess[0]); + for (int j = 1; j < nsess; j++) + sess[j - 1] = sess[j]; + sess[nsess - 1] = open_conn(tc); + } + + which = (int) pg_prop_draw_int(tc, 0, nsess - 1); + + /* a serializable transfer: debit then credit the same amount */ + resetStringInfo(&q); + appendStringInfo(&q, + "BEGIN ISOLATION LEVEL SERIALIZABLE; " + "UPDATE acct SET bal = bal - %d WHERE id = %d; " + "UPDATE acct SET bal = bal + %d WHERE id = %d; " + "COMMIT", + amt, from, amt, to); + /* run it; a serialization failure rolls the whole txn back (no move) */ + if (!pg_prop_sql_exec(sess[which], q.data, &err)) + { + /* ensure the aborted session is usable again */ + pg_prop_sql_exec(sess[which], "ROLLBACK", &ss); + if (ss) + pg_free(ss); + } + if (err) + { + pg_free(err); + err = NULL; + } + } + + /* close the pool */ + for (i = 0; i < nsess; i++) + pg_prop_disconnect(sess[i]); + + end_total = scalar_long(tc, admin, "SELECT sum(bal) FROM acct"); + if (end_total != start_total) + pg_prop_fail(tc, + "transfers did not conserve total: start=%ld end=%ld " + "(%d accounts, %d transfers)", + start_total, end_total, accounts, transfers); + + pg_prop_sql_exec(admin, "DROP TABLE IF EXISTS acct", &err); + if (err) + pg_free(err); + pfree(q.data); + pg_prop_disconnect(admin); +} + +/* + * No lost updates: a single counter row, incremented by many transactions + * spread over a churning pool. Each increment is its own autocommit + * UPDATE ... SET c = c + 1 (row lock serializes them). The final value equals + * the number of increments that COMMITTED. Since each is a single autocommit + * statement it cannot partially apply, so committed == attempted here, and the + * final counter must equal the attempt count. + */ +PG_PROP(counter_increments_exact) +{ + pg_prop_conn *admin = open_conn(tc); + pg_prop_conn *sess[CONC_MAX_SESS]; + int nsess = 0; + char *err = NULL; + int incs = (int) pg_prop_draw_int(tc, 0, 60); + int i; + long final; + + if (!pg_prop_sql_exec(admin, + "DROP TABLE IF EXISTS ctr; " + "CREATE TABLE ctr (c bigint); INSERT INTO ctr VALUES (0)", + &err)) + pg_prop_fail(tc, "counter setup failed: %s", err ? err : ""); + + for (i = 0; i < incs; i++) + { + int which; + + if ((nsess == 0 || pg_prop_draw_bool(tc)) && nsess < CONC_MAX_SESS) + sess[nsess++] = open_conn(tc); + else if (nsess > 1 && pg_prop_draw_int(tc, 0, 4) == 0) + { + pg_prop_disconnect(sess[nsess - 1]); + sess[nsess - 1] = open_conn(tc); + } + which = (int) pg_prop_draw_int(tc, 0, nsess - 1); + if (!pg_prop_sql_exec(sess[which], "UPDATE ctr SET c = c + 1", &err)) + pg_prop_fail(tc, "increment failed: %s", err ? err : ""); + } + + for (i = 0; i < nsess; i++) + pg_prop_disconnect(sess[i]); + + final = scalar_long(tc, admin, "SELECT c FROM ctr"); + if (final != incs) + pg_prop_fail(tc, "lost updates: counter=%ld != increments=%d", + final, incs); + + pg_prop_sql_exec(admin, "DROP TABLE IF EXISTS ctr", &err); + if (err) + pg_free(err); + pg_prop_disconnect(admin); +} + +/* + * Churn with no orphan transactions: sessions open, insert a committed row, + * sometimes open a transaction that inserts a row and ROLLS BACK, then close. + * The number of rows visible afterward equals the number of committed inserts; + * no rolled-back row survives. + */ +PG_PROP(churn_no_orphan_xacts) +{ + pg_prop_conn *admin = open_conn(tc); + char *err = NULL; + int ops = (int) pg_prop_draw_int(tc, 0, 50); + int committed = 0; + int i; + long visible; + + if (!pg_prop_sql_exec(admin, + "DROP TABLE IF EXISTS churn; " + "CREATE TABLE churn (id serial PRIMARY KEY, v int)", + &err)) + pg_prop_fail(tc, "churn setup failed: %s", err ? err : ""); + + for (i = 0; i < ops; i++) + { + pg_prop_conn *s = open_conn(tc); + int v = (int) pg_prop_draw_int(tc, 0, 1000); + StringInfoData q; + + initStringInfo(&q); + if (pg_prop_draw_bool(tc)) + { + /* committed insert */ + appendStringInfo(&q, "INSERT INTO churn (v) VALUES (%d)", v); + if (pg_prop_sql_exec(s, q.data, &err)) + committed++; + else if (err) { pg_free(err); err = NULL; } + } + else + { + /* rolled-back insert: must leave nothing */ + resetStringInfo(&q); + appendStringInfo(&q, + "BEGIN; INSERT INTO churn (v) VALUES (%d); ROLLBACK", + v); + pg_prop_sql_exec(s, q.data, &err); + if (err) { pg_free(err); err = NULL; } + } + pfree(q.data); + pg_prop_disconnect(s); /* backend count churns every op */ + } + + visible = scalar_long(tc, admin, "SELECT count(*) FROM churn"); + if (visible != committed) + pg_prop_fail(tc, + "visible rows %ld != committed inserts %d (rolled-back row leaked?)", + visible, committed); + + pg_prop_sql_exec(admin, "DROP TABLE IF EXISTS churn", &err); + if (err) + pg_free(err); + pg_prop_disconnect(admin); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 678f9e3a69a03..73337af07892b 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep index_am table_am pl_langs integrity constraints isolation contrib query_window spgist); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 6e93940134cfcf0a50586298b3a97fd95f3af97e Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Thu, 15 Oct 2026 09:35:20 -0400 Subject: [PATCH 289/338] prop: add GEQO equivalence property The genetic query optimizer searches join orders heuristically instead of exhaustively; it must change which plan is chosen, never the answer. geqo_equivalence runs a 4-way join with the exhaustive planner and again with GEQO forced (geqo on, threshold 2, fixed seed) and requires identical rows. A difference would mean GEQO produced a wrong-results plan. Verified at 2000 iterations. --- src/test/prop/sql/prop_query_planadvice.c | 40 +++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/src/test/prop/sql/prop_query_planadvice.c b/src/test/prop/sql/prop_query_planadvice.c index 32f75b66292c8..42ee94c18295d 100644 --- a/src/test/prop/sql/prop_query_planadvice.c +++ b/src/test/prop/sql/prop_query_planadvice.c @@ -371,3 +371,43 @@ PG_PROP(outer_join_order_invariant) pfree(q.data); pg_prop_disconnect(conn); } + +/* + * GEQO equivalence: the genetic query optimizer searches the join-order space + * heuristically instead of exhaustively, but it must never change the ANSWER + * -- only which plan is chosen. Run the same multi-way self-join with the + * exhaustive planner (geqo off) and with GEQO forced (geqo on, threshold 2), + * and require identical rows. A difference would mean GEQO produced a plan + * that computes the wrong result -- a serious optimizer bug. Uses the plan- + * invariant comparator with GEQO toggled via the extra SET. + */ +PG_PROP(geqo_equivalence) +{ + pg_prop_conn *conn; + int nrows; + StringInfoData q; + + conn = connect_advice(tc); + qg_build_table(tc, conn, &nrows); + + /* a 4-way self-join: well above geqo_threshold=2 when GEQO is forced */ + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT w.id, x.id, y.id, z.id FROM " QG_TABLE " w " + "JOIN " QG_TABLE " x ON x.id = w.a " + "JOIN " QG_TABLE " y ON y.id = w.b " + "JOIN " QG_TABLE " z ON z.id = x.a " + "ORDER BY w.id, x.id, y.id, z.id"); + + /* + * Default run has geqo off (RESET ALL in require_plan_invariant); the + * steered run forces GEQO on with the minimum threshold so the genetic + * search is used for this join. No plan advice -- just the GEQO toggle. + */ + require_plan_invariant(tc, conn, q.data, "", + "SET geqo = on; SET geqo_threshold = 2; " + "SET geqo_seed = 0.5;"); + + pfree(q.data); + pg_prop_disconnect(conn); +} From ec01810f94b42a01b4839b76bb03bbea4d51f9fd Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Fri, 16 Oct 2026 13:47:28 -0400 Subject: [PATCH 290/338] prop: make the query generators adversarial, not uniform-small The single-table and multi-table generators produced tiny uniform data (ints in -5..5, text from {a,b,c}), which never pushes a mature planner to a wrong plan or an executor edge. Fatten and skew the distributions where bugs live: - ints: ~22% boundary (INT_MIN, INT_MAX, 0, -1, 1) and ~35% a dominant skew value, so selectivity estimates diverge from reality and plan choice flips; - text: empty string distinct from NULL, a dominant value, and occasional uppercase / trailing-space (collation and case/space edges); - float8: +-0.0, +-Infinity, NaN alongside small values; - numeric: 0, NaN and the typmod boundary (999.99 for numeric(5,2)). Enrich the predicate grammar with the type-interaction forms bugs hide in: IS [NOT] DISTINCT FROM, CASE-as-boolean, cross-type comparison (int::numeric, int::text vs text), and COALESCE in the comparand. Harden the affected oracles so the new boundary data cannot false-alarm: sums and abs() cast to bigint (INT_MIN/MAX would overflow int), and PQS quotes and casts the float pivot value so Infinity/NaN are literals, not identifiers (Postgres defines 'NaN' = 'NaN' TRUE, so pivot containment stays sound). --- src/test/prop/sql/prop_query_planadvice.c | 2 +- src/test/prop/sql/prop_query_pqs.c | 8 +- src/test/prop/sql/prop_query_window.c | 14 +- src/test/prop/sql/query_gen.c | 149 +++++++++++++++++++--- 4 files changed, 145 insertions(+), 28 deletions(-) diff --git a/src/test/prop/sql/prop_query_planadvice.c b/src/test/prop/sql/prop_query_planadvice.c index 42ee94c18295d..9c6c5e3a07e40 100644 --- a/src/test/prop/sql/prop_query_planadvice.c +++ b/src/test/prop/sql/prop_query_planadvice.c @@ -182,7 +182,7 @@ PG_PROP(groupagg_plan_invariant) initStringInfo(&q); appendStringInfo(&q, - "SELECT g, count(*), sum(a), min(b), max(b), " + "SELECT g, count(*), sum(a::bigint), min(b), max(b), " "count(DISTINCT c) FROM " "(SELECT (%s) AS g, a, b, c FROM " QG_TABLE ") s " "GROUP BY g ORDER BY g", diff --git a/src/test/prop/sql/prop_query_pqs.c b/src/test/prop/sql/prop_query_pqs.c index 79f0bb47d5da5..c955466fb1fc7 100644 --- a/src/test/prop/sql/prop_query_pqs.c +++ b/src/test/prop/sql/prop_query_pqs.c @@ -160,11 +160,15 @@ append_true_leaf(pg_prop_tc *tc, StringInfo buf, const PivotRow *pv) append_text_leaf(tc, buf, pv->c); break; default: - /* f: only an equality or IS NULL; avoid float range edge cases. */ + /* f: only an equality or IS NULL; avoid float range edge cases. + * Quote and cast the value so special floats (Infinity, -Infinity, + * NaN) are valid literals rather than bare identifiers. Postgres + * defines 'NaN'::float8 = 'NaN'::float8 as TRUE, so pivot + * containment stays sound even for NaN. */ if (pv->f == NULL) appendStringInfoString(buf, "f IS NULL"); else - appendStringInfo(buf, "f = %s", pv->f); + appendStringInfo(buf, "f = '%s'::float8", pv->f); break; } } diff --git a/src/test/prop/sql/prop_query_window.c b/src/test/prop/sql/prop_query_window.c index a28bdd4a706fe..36376759ba44f 100644 --- a/src/test/prop/sql/prop_query_window.c +++ b/src/test/prop/sql/prop_query_window.c @@ -146,10 +146,10 @@ PG_PROP(window_running_sum) last_running = scalar_long(tc, conn, "SELECT coalesce(max(rs), 0) FROM " - "(SELECT sum(coalesce(abs(a),0)) OVER " + "(SELECT sum(coalesce(abs(a::bigint),0)) OVER " "(ORDER BY a, id) AS rs FROM qg_t) s"); plain_sum = scalar_long(tc, conn, - "SELECT coalesce(sum(coalesce(abs(a),0)),0) FROM qg_t"); + "SELECT coalesce(sum(coalesce(abs(a::bigint),0)),0) FROM qg_t"); if (last_running != plain_sum) pg_prop_fail(tc, "running sum last=%ld != plain sum=%ld", last_running, plain_sum); @@ -158,7 +158,7 @@ PG_PROP(window_running_sum) nonmonotone = scalar_long(tc, conn, "SELECT count(*) FROM " "(SELECT rs, lag(rs) OVER (ORDER BY a, id) AS prev FROM " - "(SELECT a, id, sum(coalesce(abs(a),0)) OVER (ORDER BY a, id) AS rs " + "(SELECT a, id, sum(coalesce(abs(a::bigint),0)) OVER (ORDER BY a, id) AS rs " "FROM qg_t) t) u WHERE prev IS NOT NULL AND rs < prev"); if (nonmonotone != 0) pg_prop_fail(tc, "running sum of non-negative values decreased %ld times", @@ -216,12 +216,12 @@ PG_PROP(rollup_grand_total) * bits set); its sum equals the whole-table sum */ rollup_total = scalar_long(tc, conn, "SELECT coalesce(s,0) FROM " - "(SELECT sum(coalesce(a,0)) AS s, " + "(SELECT sum(coalesce(a::bigint,0)) AS s, " "grouping(a) AS ga, grouping(b) AS gb " "FROM qg_t GROUP BY ROLLUP(a, b)) r " "WHERE ga = 1 AND gb = 1"); plain_total = scalar_long(tc, conn, - "SELECT coalesce(sum(coalesce(a,0)),0) FROM qg_t"); + "SELECT coalesce(sum(coalesce(a::bigint,0)),0) FROM qg_t"); if (rollup_total != plain_total) pg_prop_fail(tc, "ROLLUP grand total=%ld != plain total=%ld", rollup_total, plain_total); @@ -236,10 +236,10 @@ PG_PROP(rollup_detail_matches) require_same_rows(tc, conn, /* ROLLUP detail = rows with no grouping bit set, projected like GROUP BY */ - "SELECT a, b, sum(coalesce(a,0)) AS s, count(*) AS c " + "SELECT a, b, sum(coalesce(a::bigint,0)) AS s, count(*) AS c " "FROM qg_t GROUP BY a, b ORDER BY a, b", "SELECT a, b, s, c FROM " - "(SELECT a, b, sum(coalesce(a,0)) AS s, count(*) AS c, " + "(SELECT a, b, sum(coalesce(a::bigint,0)) AS s, count(*) AS c, " "grouping(a) AS ga, grouping(b) AS gb " "FROM qg_t GROUP BY ROLLUP(a, b)) r " "WHERE ga = 0 AND gb = 0 ORDER BY a, b"); diff --git a/src/test/prop/sql/query_gen.c b/src/test/prop/sql/query_gen.c index d83b72c914591..4ab6db35f9539 100644 --- a/src/test/prop/sql/query_gen.c +++ b/src/test/prop/sql/query_gen.c @@ -27,67 +27,139 @@ #include "query_gen.h" -/* Small domains keep counterexamples legible and cases fast. */ +/* + * Value domains deliberately favour the cases that break planners and + * executors, not uniform small noise: boundary integers (INT_MIN/MAX, 0, +-1), + * heavy NULL, and skew (one value dominating) so selectivity estimates diverge + * from reality and plan choice flips. Legible counterexamples still shrink + * because the engine prefers the small/boundary draws. + */ #define QG_VAL_MIN (-5) #define QG_VAL_MAX 5 +/* INT_MIN / INT_MAX as SQL text (INT_MIN written to avoid the unary-minus + * literal-overflow trap). */ +#define SQL_INT_MIN "(-2147483647 - 1)" +#define SQL_INT_MAX "2147483647" + /* ---- core column value generators ------------------------------------ */ -/* int in [QG_VAL_MIN, QG_VAL_MAX] or SQL NULL, ~15% NULL. */ +/* + * Draw an int as SQL text with an adversarial, skewed distribution: + * ~18% NULL, + * ~22% a boundary value (INT_MIN, INT_MAX, 0, -1, 1), + * ~35% the skew value 0 (so one value dominates -> bad selectivity est), + * else a small value in [-5,5]. + */ static void draw_int_or_null(pg_prop_tc *tc, StringInfo buf) { - if (pg_prop_draw_int(tc, 0, 6) == 0) + int roll = (int) pg_prop_draw_int(tc, 0, 99); + + if (roll < 18) appendStringInfoString(buf, "NULL"); + else if (roll < 40) + { + static const char *const edge[] = + {SQL_INT_MIN, SQL_INT_MAX, "0", "(-1)", "1"}; + + appendStringInfoString(buf, edge[pg_prop_draw_int(tc, 0, 4)]); + } + else if (roll < 75) + appendStringInfoString(buf, "0"); /* skew: 0 dominates */ else appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX)); } -/* short [a-c] string literal or SQL NULL, ~15% NULL. */ +/* + * Draw a text literal, adversarial: ~18% NULL, ~15% empty string, often a + * repeated short run (skew + collation edge cases), occasionally a value with + * a trailing space or an upper-case letter (case/space sensitivity). + */ static void draw_text_or_null(pg_prop_tc *tc, StringInfo buf) { + int roll = (int) pg_prop_draw_int(tc, 0, 99); int len; int i; - if (pg_prop_draw_int(tc, 0, 6) == 0) + if (roll < 18) { appendStringInfoString(buf, "NULL"); return; } + if (roll < 33) + { + appendStringInfoString(buf, "''"); /* empty string (distinct from NULL) */ + return; + } + if (roll < 55) + { + appendStringInfoString(buf, "'a'"); /* skew: 'a' dominates */ + return; + } len = (int) pg_prop_draw_int(tc, 0, 3); appendStringInfoChar(buf, '\''); for (i = 0; i < len; i++) - appendStringInfoChar(buf, (char) ('a' + pg_prop_draw_int(tc, 0, 2))); + { + /* mostly a-c; occasionally an uppercase letter or a trailing space */ + int k = (int) pg_prop_draw_int(tc, 0, 9); + + if (k == 0) + appendStringInfoChar(buf, (char) ('A' + pg_prop_draw_int(tc, 0, 2))); + else if (k == 1) + appendStringInfoChar(buf, ' '); + else + appendStringInfoChar(buf, (char) ('a' + pg_prop_draw_int(tc, 0, 2))); + } appendStringInfoChar(buf, '\''); } -/* float8 literal (small, one decimal) or SQL NULL, ~15% NULL. */ +/* float8: adversarial -- NULL, +-0.0, +-Infinity, NaN, or a small value. */ static void draw_float_or_null(pg_prop_tc *tc, StringInfo buf) { - if (pg_prop_draw_int(tc, 0, 6) == 0) + int roll = (int) pg_prop_draw_int(tc, 0, 99); + + if (roll < 15) appendStringInfoString(buf, "NULL"); + else if (roll < 35) + { + static const char *const edge[] = + {"'0'::float8", "'-0'::float8", "'Infinity'::float8", + "'-Infinity'::float8", "'NaN'::float8"}; + + appendStringInfoString(buf, edge[pg_prop_draw_int(tc, 0, 4)]); + } else appendStringInfo(buf, "%d.%d", (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), (int) pg_prop_draw_int(tc, 0, 9)); } -/* numeric literal with a scale, or SQL NULL -- exercises exact decimal paths. */ +/* numeric: NULL, boundary (0, huge, tiny-scale), or a scaled value. */ static void draw_numeric_or_null(pg_prop_tc *tc, StringInfo buf) { - if (pg_prop_draw_int(tc, 0, 6) == 0) + int roll = (int) pg_prop_draw_int(tc, 0, 99); + + if (roll < 15) appendStringInfoString(buf, "NULL"); + else if (roll < 30) + { + static const char *const edge[] = + {"0", "0.00", "999.99", "-999.99", "'NaN'::numeric"}; + + appendStringInfoString(buf, edge[pg_prop_draw_int(tc, 0, 4)]); + } else appendStringInfo(buf, "%d.%02d", (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), (int) pg_prop_draw_int(tc, 0, 99)); } -/* boolean literal or SQL NULL. */ +/* boolean literal or SQL NULL (balanced three-valued). */ static void draw_bool_or_null(pg_prop_tc *tc, StringInfo buf) { @@ -209,13 +281,20 @@ pick_num_col(pg_prop_tc *tc) return (col == 0) ? "a" : (col == 1) ? "b" : "f"; } -/* One leaf comparison against a column, or an IS NULL test, or an ext leaf. */ +/* + * One leaf comparison. The grammar is deliberately rich: besides the plain + * comparisons it emits three-valued-logic edge forms (IS DISTINCT FROM, IS + * NULL/NOT NULL), CASE expressions, and cross-type comparisons -- exactly the + * type-interaction and NULL-handling code where planner/executor bugs hide. + */ static void draw_leaf(pg_prop_tc *tc, StringInfo buf) { static const char *const cmp[] = {"=", "<>", "<", "<=", ">", ">="}; - int next_kinds = pg_prop_hooks_count("query_gen.predicate") > 0 ? 8 : 7; - int kind = (int) pg_prop_draw_int(tc, 0, next_kinds - 1); + int have_ext = pg_prop_hooks_count("query_gen.predicate") > 0 ? 1 : 0; + /* kinds 0..10 are built-in; 11 is the extension leaf when present. */ + int n_builtin = 11; + int kind = (int) pg_prop_draw_int(tc, 0, n_builtin - 1 + have_ext); const char *colname; switch (kind) @@ -266,7 +345,41 @@ draw_leaf(pg_prop_tc *tc, StringInfo buf) appendStringInfo(buf, "g IS %s", pg_prop_draw_bool(tc) ? "TRUE" : "NOT TRUE"); break; - case 7: /* extension-contributed leaf */ + case 7: /* col IS [NOT] DISTINCT FROM const (3VL edge) */ + colname = pick_num_col(tc); + appendStringInfo(buf, "%s IS %sDISTINCT FROM ", colname, + pg_prop_draw_bool(tc) ? "" : "NOT "); + if (strcmp(colname, "f") == 0) + draw_float_or_null(tc, buf); + else + draw_int_or_null(tc, buf); + break; + case 8: /* CASE expression used as a boolean */ + appendStringInfoString(buf, "(CASE WHEN "); + appendStringInfo(buf, "a %s ", cmp[pg_prop_draw_int(tc, 0, 5)]); + draw_int_or_null(tc, buf); + appendStringInfoString(buf, " THEN b ELSE "); + draw_int_or_null(tc, buf); + appendStringInfo(buf, " END) %s ", cmp[pg_prop_draw_int(tc, 0, 5)]); + draw_int_or_null(tc, buf); + break; + case 9: /* cross-type: int column vs numeric/text cast */ + if (pg_prop_draw_bool(tc)) + appendStringInfo(buf, "a::numeric %s %d.%02d", + cmp[pg_prop_draw_int(tc, 0, 5)], + (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), + (int) pg_prop_draw_int(tc, 0, 99)); + else + appendStringInfo(buf, "a::text %s c", + cmp[pg_prop_draw_int(tc, 0, 5)]); + break; + case 10: /* COALESCE in the comparison (NULL coalescing) */ + appendStringInfo(buf, "coalesce(a, b, %d) %s ", + (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), + cmp[pg_prop_draw_int(tc, 0, 5)]); + draw_int_or_null(tc, buf); + break; + case 11: /* extension-contributed leaf */ { qg_pred_ctx pctx; int n = pg_prop_hooks_count("query_gen.predicate"); @@ -350,11 +463,11 @@ qg_draw_int_expr(pg_prop_tc *tc, StringInfo buf) (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX)); break; case 3: - /* coalesce so NULLs don't dominate; a + b */ - appendStringInfoString(buf, "(coalesce(a,0) + coalesce(b,0))"); + /* coalesce so NULLs don't dominate; a + b, bigint-safe vs INT_MAX */ + appendStringInfoString(buf, "(coalesce(a::bigint,0) + coalesce(b::bigint,0))"); break; default: - appendStringInfoString(buf, "(coalesce(a,0) * 2)"); + appendStringInfoString(buf, "(coalesce(a::bigint,0) * 2)"); break; } } From 013c652b42aba4d0c9efeb38215ac66046ce23cd Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Fri, 16 Oct 2026 13:47:28 -0400 Subject: [PATCH 291/338] prop: add full-configuration differential oracle (query_diff) The strongest wrong-results oracle in the suite, and the SQLancer-proven one: hold a generated query fixed over adversarial data and run it under a battery of forced execution strategies -- seqscan vs index vs bitmap, nestloop vs hash vs merge join, hash vs sort aggregation, GEQO on/off, JIT on/off, parallel on/off, tiny vs large work_mem (spill vs in-memory) -- requiring every strategy that executes to produce a byte-identical result digest. Unlike the metamorphic oracles (TLP/NoREC), which rewrite the query and so are blind to a bug corrupting all rewrites identically, this varies only the plan, not the query, so a disagreement is unambiguous: two execution paths computed different answers for the same query over the same data. Three properties: single-table filter (all columns projected -- full-value, not key-only), GROUP BY aggregation (hash/sort/parallel-partial agg), and deep multi-way join (where GEQO, join-method forcing and spill interact). Comparison is by server-side digest, so results of any size cost O(1) client memory. Verified sound at 500 iterations (all configs agree); the fattened generators flow through it and every existing query area. --- src/test/prop/Makefile | 1 + src/test/prop/meson.build | 1 + src/test/prop/sql/prop_query_diff.c | 276 ++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 279 insertions(+), 1 deletion(-) create mode 100644 src/test/prop/sql/prop_query_diff.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 765b8dbe7da3d..c5a4882933245 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -281,6 +281,7 @@ OBJS = \ sql/prop_query_ext_citext.o \ sql/prop_query_multi.o \ sql/prop_query_deep.o \ + sql/prop_query_diff.o \ sql/index_traits.o \ sql/prop_index_am.o \ sql/prop_table_am.o \ diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 08a82f56da442..31527c68ee6eb 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -286,6 +286,7 @@ prop_query_sources = files( 'sql/prop_query_ext_citext.c', 'sql/prop_query_multi.c', 'sql/prop_query_deep.c', + 'sql/prop_query_diff.c', 'sql/index_traits.c', 'sql/prop_index_am.c', 'sql/prop_table_am.c', diff --git a/src/test/prop/sql/prop_query_diff.c b/src/test/prop/sql/prop_query_diff.c new file mode 100644 index 0000000000000..d1ef3c95abf23 --- /dev/null +++ b/src/test/prop/sql/prop_query_diff.c @@ -0,0 +1,276 @@ +/*------------------------------------------------------------------------- + * + * prop_query_diff.c + * Full-configuration differential oracle. + * + * This is the strongest wrong-results oracle in the suite. The idea, taken + * from SQLancer's differential / query-plan-guidance mode (which found the + * large majority of real DBMS bugs in that line of work): a correct database + * returns the *same rows* for a query regardless of how the planner and + * executor are forced to compute it. We take one generated query over + * adversarial, skewed, correlated data and run it under a battery of forced + * configurations -- sequential vs index vs bitmap scans, nested-loop vs hash + * vs merge joins, hash vs sort aggregation, GEQO on/off, JIT on/off, parallel + * on/off, tiny vs large work_mem (forcing/avoiding spill) -- and require every + * configuration that executes to produce a byte-identical result digest. + * + * Unlike the metamorphic oracles (TLP/NoREC), which rewrite the *query* and so + * are blind to a bug that corrupts all rewrites identically, this oracle holds + * the query fixed and varies only the *execution strategy*. A disagreement is + * therefore unambiguous: two execution paths computed different answers for the + * same query over the same data -- a server bug, not an oracle artifact. + * + * The configurations are applied with SET LOCAL inside a transaction so they + * cannot leak between runs. Results are compared by server-side digest + * (pg_prop_sql_rowhash), so arbitrarily large results cost O(1) client memory. + * Every query carries a total ORDER BY so "same rows" means "same digest". + * + * Portions Copyright (c) 2024, PostgreSQL Global Development Group + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "query_diff" +#include "pg_prop.h" + +#include "lib/stringinfo.h" +#include "query_gen.h" + +/* + * Each configuration is a bundle of SET LOCAL statements forcing a particular + * class of plan. "" means "planner's free choice" (the reference). We + * deliberately include settings that force expensive, less-exercised paths + * (tiny work_mem -> external sort / batched hash spill; disabled hashing -> + * sort/merge everywhere), because the spill and sort paths are where executor + * bugs hide. + */ +static const char *const configs[] = +{ + "", /* reference: planner chooses freely */ + "SET LOCAL enable_indexscan=off; SET LOCAL enable_indexonlyscan=off;" + "SET LOCAL enable_bitmapscan=off;", + "SET LOCAL enable_seqscan=off;", + "SET LOCAL enable_hashjoin=off; SET LOCAL enable_mergejoin=off;", + "SET LOCAL enable_nestloop=off; SET LOCAL enable_mergejoin=off;", + "SET LOCAL enable_nestloop=off; SET LOCAL enable_hashjoin=off;", + "SET LOCAL enable_hashagg=off;", + "SET LOCAL enable_sort=off;", + "SET LOCAL geqo=on; SET LOCAL geqo_threshold=2; SET LOCAL geqo_seed=0.3;", + "SET LOCAL geqo=off;", + "SET LOCAL work_mem='64kB';", + "SET LOCAL work_mem='256MB';", + "SET LOCAL parallel_setup_cost=0; SET LOCAL parallel_tuple_cost=0;" + "SET LOCAL min_parallel_table_scan_size=0;" + "SET LOCAL max_parallel_workers_per_gather=4;", + "SET LOCAL max_parallel_workers_per_gather=0;", + "SET LOCAL jit=on; SET LOCAL jit_above_cost=0; SET LOCAL jit_inline_above_cost=0;" + "SET LOCAL jit_optimize_above_cost=0;", + "SET LOCAL jit=off;", +}; +#define N_CONFIGS ((int) (sizeof(configs) / sizeof(configs[0]))) + +/* + * Run "query" under configuration setup and return the result digest, or NULL + * on error with *errored set. A SQL error under a forced config is not itself + * a bug (e.g. a value that overflows, or a config making a plan impossible); + * only a *disagreement in results* among the configs that succeeded is. Each + * run is its own transaction so the SET LOCALs are scoped and discarded. + */ +static char * +digest_under(pg_prop_conn *conn, const char *setup, const char *query, + bool *errored) +{ + char *dig; + bool failed = false; + char *err = NULL; + + *errored = false; + + if (!pg_prop_sql_exec(conn, "BEGIN", &err)) + { + if (err) + free(err); + *errored = true; + return NULL; + } + if (setup[0] != '\0' && !pg_prop_sql_exec(conn, setup, &err)) + { + if (err) + free(err); + pg_prop_sql_exec(conn, "ROLLBACK", NULL); + *errored = true; + return NULL; + } + dig = pg_prop_sql_rowhash(conn, query, &failed, &err); + pg_prop_sql_exec(conn, "ROLLBACK", NULL); + + if (failed) + { + if (err) + free(err); + if (dig) + free(dig); + *errored = true; + return NULL; + } + return dig; +} + +/* + * The core differential property: take the reference digest (first config that + * executes), then require every other config that executes to agree. + */ +static void +run_diff(pg_prop_tc *tc, pg_prop_conn *conn, const char *query) +{ + char *ref = NULL; + bool errored; + int i; + int ref_cfg = -1; + + for (i = 0; i < N_CONFIGS; i++) + { + ref = digest_under(conn, configs[i], query, &errored); + if (!errored && ref != NULL) + { + ref_cfg = i; + break; + } + if (ref) + { + free(ref); + ref = NULL; + } + } + if (ref == NULL) + return; /* query errors under every config: vacuous */ + + for (i = ref_cfg + 1; i < N_CONFIGS; i++) + { + char *d = digest_under(conn, configs[i], query, &errored); + + if (errored || d == NULL) + { + if (d) + free(d); + continue; /* config not applicable; skip */ + } + if (strcmp(ref, d) != 0) + { + char *rc = pstrdup(ref); + char *dc = pstrdup(d); + + free(ref); + free(d); + pg_prop_fail(tc, + "two execution strategies disagree on the same query.\n" + " config[%d] %s -> %s\n" + " config[%d] %s -> %s\n" + " query: %s", + ref_cfg, configs[ref_cfg][0] ? configs[ref_cfg] : "(free choice)", rc, + i, configs[i], dc, + query); + } + free(d); + } + free(ref); +} + +/* + * Single-table differential: SELECT every column WHERE over + * the adversarial single table, totally ordered. + */ +PG_PROP(diff_single_table_filter) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int nrows; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + qg_build_table(tc, conn, &nrows); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT id, a, b, c, d, f, g FROM " QG_TABLE + " WHERE "); + qg_draw_predicate(tc, &q); + appendStringInfoString(&q, " ORDER BY id, a, b, c, d, f, g NULLS FIRST"); + + run_diff(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Aggregation differential: GROUP BY with a spread of aggregates (hash-vs-sort + * aggregation and parallel partial aggregation are the executor code most + * likely to disagree), full aggregate tuple ordered. + */ +PG_PROP(diff_group_aggregate) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int nrows; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + qg_build_table(tc, conn, &nrows); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT c, count(*), count(a), sum(a), avg(b)::numeric(30,6), " + "min(d), max(d), bool_or(g), string_agg(c, ',' ORDER BY c) " + "FROM " QG_TABLE " WHERE "); + qg_draw_predicate(tc, &q); + appendStringInfoString(&q, " GROUP BY c ORDER BY c NULLS FIRST"); + + run_diff(tc, conn, q.data); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Deep multi-table differential: a generated multi-way join over the related + * database, every key projected, totally ordered. This is where GEQO, + * join-method forcing, and spill all interact at once. + */ +PG_PROP(diff_deep_join) +{ + pg_prop_conn *conn; + char *err = NULL; + qg_db_params p; + qg_db *db; + qg_query q; + StringInfoData whole; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + qg_db_params_for_env(&p); + db = qg_build_db(tc, conn, &p); + + if (!qg_build_query(tc, db, 4, &q)) + { + qg_drop_db(tc, conn, db); + pg_prop_disconnect(conn); + return; /* db too small; vacuous */ + } + + initStringInfo(&whole); + appendStringInfo(&whole, "%s %s", q.select_from, q.order_by); + + run_diff(tc, conn, whole.data); + + pfree(whole.data); + qg_free_query(&q); + qg_drop_db(tc, conn, db); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 73337af07892b..422ce58b70cb9 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 9b550d9bd0d2c6329117da5b871681011f29069d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Fri, 16 Oct 2026 18:19:34 -0400 Subject: [PATCH 292/338] prop: abort the run and save the statement when a backend crashes A backend that dies (assertion failure, segfault) took its connection with it, and the property saw only a SQL failure. The engine then shrank by trying other inputs until the case stopped reproducing, losing the crashing statement, while the postmaster sat in crash recovery and every later area failed to connect. One assertion failure in an 8-hour hunt surfaced as 430 connection "failures" and no reproducer. Every synchronous statement helper now checks the connection after PQexec. If it died, the statement is appended to $PG_PROP_CRASH_LOG (default pg_prop_crashes.sql), stamped with area/property and the server message, and pg_prop exits with status 3 at once. The harness records the running area/property in pg_prop_current_area for this. --- src/test/prop/pg_prop.c | 6 ++++ src/test/prop/pg_prop.h | 3 ++ src/test/prop/pg_prop_sql.c | 68 +++++++++++++++++++++++++++++++++++-- 3 files changed, 74 insertions(+), 3 deletions(-) diff --git a/src/test/prop/pg_prop.c b/src/test/prop/pg_prop.c index d3274e5378719..2dd8e99a46f20 100644 --- a/src/test/prop/pg_prop.c +++ b/src/test/prop/pg_prop.c @@ -217,6 +217,10 @@ static bool overrun; * is skipped, not one case; run_property reads and clears it. */ static bool skip_requested; + +/* "area/property" now running, for diagnostics (see pg_prop_sql.c). */ +const char *pg_prop_current_area = NULL; +static char current_area_buf[256]; static StringInfoData skip_reason; static bool skip_reason_inited; @@ -630,6 +634,8 @@ run_property(PropEntry *prop, uint64 iterations, bool have_seed, uint64 seed, check_rc(ctx, hegel_settings_set_verbosity(ctx, s, verbosity), "set_verbosity"); check_rc(ctx, hegel_settings_set_seed(ctx, s, seed, have_seed), "set_seed"); snprintf(dbkey, sizeof(dbkey), "%s/%s", prop->area, prop->name); + snprintf(current_area_buf, sizeof(current_area_buf), "%s", dbkey); + pg_prop_current_area = current_area_buf; check_rc(ctx, hegel_settings_set_database_key(ctx, s, dbkey), "set_db_key"); check_rc(ctx, hegel_settings_set_database(ctx, s, dbpath ? dbpath : ""), "set_database"); diff --git a/src/test/prop/pg_prop.h b/src/test/prop/pg_prop.h index 97614aa428846..a8b46fd5be4f0 100644 --- a/src/test/prop/pg_prop.h +++ b/src/test/prop/pg_prop.h @@ -305,6 +305,9 @@ extern char *pg_prop_sql_rowhash(pg_prop_conn *conn, const char *sql, */ extern bool pg_prop_sql_exec(pg_prop_conn *conn, const char *sql, char **errmsg); +/* "area/property" currently running (NULL outside a run); for diagnostics. */ +extern const char *pg_prop_current_area; + /* * Asynchronous statement support, for properties that need two sessions in * flight at once (e.g. provoking a deadlock, where both sessions must be diff --git a/src/test/prop/pg_prop_sql.c b/src/test/prop/pg_prop_sql.c index d7cb7ea917f2b..bfa8f17a5fe1a 100644 --- a/src/test/prop/pg_prop_sql.c +++ b/src/test/prop/pg_prop_sql.c @@ -18,6 +18,9 @@ */ #include "postgres_fe.h" +#include +#include + #include "lib/stringinfo.h" #include "libpq-fe.h" #include "pg_prop.h" @@ -68,6 +71,55 @@ pg_prop_connect(char **err) return c; } +/* Unlink c from the live list, if present. */ +static void +unlink_conn(pg_prop_conn *c); + +/* + * Crash tripwire. A backend that dies (assertion failure, segfault) takes + * its connection with it, and the property would otherwise see only a + * confusing SQL failure, which the engine then "shrinks" by trying other + * inputs until the case stops reproducing -- by which time the postmaster is + * in crash recovery and every following area fails to connect. So every + * statement helper calls this after PQexec: if the connection is now bad, + * the statement that killed it is appended to $PG_PROP_CRASH_LOG (default + * pg_prop_crashes.sql in the working directory), stamped with the area + * and time, and the run aborts at once with a distinct exit code. The + * crashing statement is the reproducer; it is captured before shrinking can + * lose it, and nothing downstream runs against a recovering server. + */ +static void +check_backend_alive(pg_prop_conn *conn, const char *sql) +{ + FILE *f; + const char *path; + time_t now; + + if (PQstatus(conn->conn) == CONNECTION_OK) + return; + + path = getenv("PG_PROP_CRASH_LOG"); + if (path == NULL || path[0] == '\0') + path = "pg_prop_crashes.sql"; + now = time(NULL); + f = fopen(path, "a"); + if (f != NULL) + { + fprintf(f, "-- backend died: area=%s pid=%d at %s" + "-- server said: %s\n%s;\n\n", + pg_prop_current_area ? pg_prop_current_area : "?", + (int) getpid(), ctime(&now), + PQerrorMessage(conn->conn), sql); + fclose(f); + } + fprintf(stderr, + "pg_prop: BACKEND CRASH -- the server connection died while running:\n" + "%s\n(statement saved to %s; aborting so no further area runs " + "against a recovering server)\n", sql, path); + fflush(stdout); + exit(3); +} + /* Unlink c from the live list, if present. */ static void unlink_conn(pg_prop_conn *c) @@ -112,9 +164,12 @@ char * pg_prop_sql_scalar(pg_prop_conn *conn, const char *sql, bool *failed, char **errmsg) { PGresult *res = PQexec(conn->conn, sql); - ExecStatusType st = PQresultStatus(res); + ExecStatusType st; char *out; + check_backend_alive(conn, sql); + st = PQresultStatus(res); + if (failed) *failed = false; if (st != PGRES_TUPLES_OK) @@ -140,11 +195,14 @@ char * pg_prop_sql_rows(pg_prop_conn *conn, const char *sql, bool *failed, char **errmsg) { PGresult *res = PQexec(conn->conn, sql); - ExecStatusType st = PQresultStatus(res); + ExecStatusType st; StringInfoData s; int nrows, ncols; + check_backend_alive(conn, sql); + st = PQresultStatus(res); + if (failed) *failed = false; if (st != PGRES_TUPLES_OK) @@ -205,6 +263,7 @@ pg_prop_sql_rowhash(pg_prop_conn *conn, const char *sql, bool *failed, "FROM (SELECT (sub)::text AS r FROM (%s) sub) q", sql); res = PQexec(conn->conn, q.data); + check_backend_alive(conn, sql); pfree(q.data); st = PQresultStatus(res); if (st != PGRES_TUPLES_OK) @@ -230,7 +289,10 @@ bool pg_prop_sql_exec(pg_prop_conn *conn, const char *sql, char **errmsg) { PGresult *res = PQexec(conn->conn, sql); - ExecStatusType st = PQresultStatus(res); + ExecStatusType st; + + check_backend_alive(conn, sql); + st = PQresultStatus(res); if (st != PGRES_COMMAND_OK && st != PGRES_TUPLES_OK) { From c6daf292cbe20f1f15d1f11095028880c0c63f34 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Fri, 16 Oct 2026 18:19:34 -0400 Subject: [PATCH 293/338] prop: generate the plan shapes the planner's fragile code acts on The deep generator only built FK chains of plain tables joined INNER or LEFT, with no secondary indexes and at most a few hundred rows, so the planner had one sensible plan and most of a differential oracle's forced configurations compared that plan with itself. Two real bugs present in the tree under test (self-join elimination across inheritance; a LATERAL UNION ALL assertion) were unreachable from it. Generator: - secondary indexes on FK and scalar columns: single, composite, partial, and expression indexes on the single table; - inheritance children holding rows whose ids duplicate the parent's (a parent PK does not constrain children), so uniqueness proofs are unsound across the parent scan; - join members that are not plain relations: ONLY rel, DISTINCT derived table, UNION ALL of two filtered halves (an appendrel), LATERAL subquery with OFFSET 0, and LATERAL UNION ALL whose first arm is a DISTINCT subquery, each exposing the same id/fk columns so oracles are unchanged; - under the deep profiles, an optional server-generated bulk of 500..20000 skewed rows (dominant 0, NULLs, INT_MIN/MAX, NaN) so index, bitmap, parallel and spill paths are actually chosen. Oracle fixes for the new shapes: the cross-product bound reads table sizes from the server (they now include inheritance children), and the FK-join count uses ONLY, since the FK and PK hold for the declared tables only. --- src/test/prop/sql/prop_query_multi.c | 22 ++- src/test/prop/sql/query_gen.c | 217 ++++++++++++++++++++++++++- src/test/prop/sql/query_gen.h | 2 + 3 files changed, 232 insertions(+), 9 deletions(-) diff --git a/src/test/prop/sql/prop_query_multi.c b/src/test/prop/sql/prop_query_multi.c index d82f081c1caaf..b7a0510dcc238 100644 --- a/src/test/prop/sql/prop_query_multi.c +++ b/src/test/prop/sql/prop_query_multi.c @@ -228,8 +228,17 @@ PG_PROP(join_card_commutes_and_bounded) parent->name, child->name); ba = count_of(tc, conn, sql.data); - na = child->nrows; - nb = parent->nrows; + /* + * The cross-product bound is over what the query actually scans, which + * includes any inheritance child of either table -- so read the sizes from + * the server, not from the generator's per-table row counts. + */ + resetStringInfo(&sql); + appendStringInfo(&sql, "SELECT count(*) FROM %s", child->name); + na = count_of(tc, conn, sql.data); + resetStringInfo(&sql); + appendStringInfo(&sql, "SELECT count(*) FROM %s", parent->name); + nb = count_of(tc, conn, sql.data); if (ab != ba) pg_prop_fail(tc, @@ -248,7 +257,10 @@ PG_PROP(join_card_commutes_and_bounded) /* * A FK equi-join to the parent's primary key returns exactly the child rows * whose fk is non-NULL (each matches exactly one parent, since fk references a - * unique id and the FK is enforced so no orphans exist). + * unique id and the FK is enforced so no orphans exist). ONLY on both sides: + * the uniqueness and the FK are properties of the declared tables, not of any + * inheritance child the generator may have attached, whose rows a plain scan + * would also return. */ PG_PROP(fk_join_eq_nonnull_children) { @@ -282,11 +294,11 @@ PG_PROP(fk_join_eq_nonnull_children) initStringInfo(&sql); appendStringInfo(&sql, - "SELECT count(*) FROM %s ch JOIN %s pa ON ch.fk = pa.id", + "SELECT count(*) FROM ONLY %s ch JOIN ONLY %s pa ON ch.fk = pa.id", child->name, parent->name); joined = count_of(tc, conn, sql.data); resetStringInfo(&sql); - appendStringInfo(&sql, "SELECT count(*) FROM %s WHERE fk IS NOT NULL", + appendStringInfo(&sql, "SELECT count(*) FROM ONLY %s WHERE fk IS NOT NULL", child->name); nonnull = count_of(tc, conn, sql.data); diff --git a/src/test/prop/sql/query_gen.c b/src/test/prop/sql/query_gen.c index 4ab6db35f9539..3017d22393d78 100644 --- a/src/test/prop/sql/query_gen.c +++ b/src/test/prop/sql/query_gen.c @@ -260,6 +260,53 @@ qg_build_table(pg_prop_tc *tc, pg_prop_conn *conn, int *nrows) appendStringInfoString(&ddl, "); "); } + /* + * Under the deep-hunt profiles, sometimes append a server-generated bulk + * of rows after the drawn ones. A planner does not consider an index for + * a 40-row table, so without volume the index, bitmap, parallel and spill + * paths a differential oracle wants to compare never get chosen. The bulk + * is skewed the same way the drawn values are (a dominant 0, NULLs, the + * int extremes) and deterministic in the drawn size, so cases replay. + */ + if (qg_deep_requested() && pg_prop_draw_bool(tc)) + { + int bulk = (int) pg_prop_draw_int(tc, 500, 20000); + + appendStringInfo(&ddl, + "INSERT INTO " QG_TABLE " (id, a, b, c, d, f, g) " + "SELECT %d + i, " + "CASE WHEN i %% 10 < 6 THEN 0 WHEN i %% 10 = 6 THEN NULL " + "WHEN i %% 97 = 0 THEN 2147483647 WHEN i %% 89 = 0 THEN (-2147483647 - 1) " + "ELSE (hashint4(i) %% 1000) END, " + "CASE WHEN i %% 7 = 0 THEN NULL ELSE hashint4(i * 31) %% 50 END, " + "CASE WHEN i %% 11 = 0 THEN NULL WHEN i %% 3 = 0 THEN 'a' " + "ELSE chr(97 + abs(hashint4(i)) %% 3) || repeat('b', i %% 3) END, " + "CASE WHEN i %% 13 = 0 THEN NULL ELSE ((hashint4(i * 7) %% 99999) / 100.0)::numeric(5,2) END, " + "CASE WHEN i %% 17 = 0 THEN NULL WHEN i %% 101 = 0 THEN 'NaN'::float8 " + "ELSE (hashint4(i * 13) %% 1000) / 10.0 END, " + "CASE WHEN i %% 5 = 0 THEN NULL ELSE i %% 2 = 0 END " + "FROM generate_series(1, %d) i; ", + n, bulk); + n += bulk; + } + + /* + * Secondary indexes, drawn per case: without them the planner has a single + * path for a one-table query (seqscan), and a differential oracle compares + * a plan with itself. Single-column, composite, partial and expression + * indexes open index, index-only, bitmap-AND/OR and presorted paths. + */ + if (pg_prop_draw_bool(tc)) + appendStringInfoString(&ddl, "CREATE INDEX ON " QG_TABLE " (a); "); + if (pg_prop_draw_bool(tc)) + appendStringInfoString(&ddl, "CREATE INDEX ON " QG_TABLE " (b, a); "); + if (pg_prop_draw_bool(tc)) + appendStringInfoString(&ddl, "CREATE INDEX ON " QG_TABLE " (c); "); + if (pg_prop_draw_int(tc, 0, 2) == 0) + appendStringInfoString(&ddl, "CREATE INDEX ON " QG_TABLE " (d) WHERE d IS NOT NULL; "); + if (pg_prop_draw_int(tc, 0, 2) == 0) + appendStringInfoString(&ddl, "CREATE INDEX ON " QG_TABLE " ((coalesce(a, 0) + 1)); "); + /* ANALYZE so the planner has stats and will actually consider plans. */ appendStringInfoString(&ddl, "ANALYZE " QG_TABLE ";"); @@ -566,6 +613,7 @@ qg_db_params_default(qg_db_params *p) p->max_rows = 30; p->allow_fk = true; p->allow_triggers = false; + p->allow_inherit = false; p->toast_profile = 0; p->cardinality_bias = 0; } @@ -581,6 +629,7 @@ qg_db_params_deep(qg_db_params *p) p->max_rows = 2000; /* volume extremes: some empty, some large */ p->allow_fk = true; p->allow_triggers = true; /* result-neutral triggers in the deep hunt */ + p->allow_inherit = true; p->toast_profile = 2; p->cardinality_bias = 0; } @@ -604,6 +653,7 @@ qg_db_params_deep_fast(qg_db_params *p) p->max_rows = 200; p->allow_fk = true; p->allow_triggers = true; + p->allow_inherit = true; p->toast_profile = 1; /* ~10% large values: TOAST path, less bulk */ p->cardinality_bias = 0; } @@ -615,6 +665,15 @@ qg_db_params_deep_fast(qg_db_params *p) * the maximal deep profile. Areas call this instead of hard-coding a profile * so the same binary runs the quick suite and the deep hunt. */ +/* true when PG_PROP_DEEP asks for one of the deep-hunt profiles */ +bool +qg_deep_requested(void) +{ + const char *v = getenv("PG_PROP_DEEP"); + + return !(v == NULL || v[0] == '\0' || strcmp(v, "0") == 0); +} + void qg_db_params_for_env(qg_db_params *p) { @@ -855,6 +914,78 @@ qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) } appendStringInfoString(&ddl, "); "); } + + /* + * Secondary indexes. Without them the planner has nothing to choose + * between on a single relation (seqscan is the only path), so most of + * a differential oracle's forced configurations collapse onto the same + * plan. An index on the FK and on one or two data columns (sometimes + * composite, sometimes partial) opens index, index-only, bitmap and + * merge-join-on-presorted paths, plus Memoize over a parameterized + * inner index scan. + */ + if (t->fkcol >= 0 && pg_prop_draw_int(tc, 0, 3) != 0) + appendStringInfo(&ddl, "CREATE INDEX ON %s (%s); ", t->name, + t->schema.cols[t->fkcol].name); + for (col = 0; col < t->schema.ncols; col++) + { + int k; + + /* + * Text-like columns can carry TOAST-sized values that exceed the + * btree tuple limit, so only scalar columns are indexed. + */ + if (col == t->fkcol || t->schema.cols[col].is_text_like || + pg_prop_draw_int(tc, 0, 2) != 0) + continue; + k = (int) pg_prop_draw_int(tc, 0, 2); + if (k == 1 && (col + 1 >= t->schema.ncols || + t->schema.cols[col + 1].is_text_like)) + k = 0; + if (k == 0) + appendStringInfo(&ddl, "CREATE INDEX ON %s (%s); ", t->name, + t->schema.cols[col].name); + else if (k == 1) + appendStringInfo(&ddl, "CREATE INDEX ON %s (%s, %s); ", t->name, + t->schema.cols[col].name, + t->schema.cols[col + 1].name); + else + appendStringInfo(&ddl, + "CREATE INDEX ON %s (%s) WHERE %s IS NOT NULL; ", + t->name, t->schema.cols[col].name, + t->schema.cols[col].name); + } + + /* + * Inheritance child. Some tables get a child holding a few rows that + * duplicate the parent's ids (a parent PRIMARY KEY does not constrain + * its children). A query over the parent scans both, so a planner + * proof that leans on the parent's unique index -- self-join + * elimination, uniqueness-based join removal, DISTINCT elision -- is + * unsound there, while "ONLY parent" must still see only the parent. + */ + if (p->allow_inherit && n > 0 && pg_prop_draw_int(tc, 0, 2) == 0) + { + int m = (int) pg_prop_draw_int(tc, 1, n < 5 ? n : 5); + + appendStringInfo(&ddl, "CREATE TABLE %s_c () INHERITS (%s); ", + t->name, t->name); + for (r = 1; r <= m; r++) + { + appendStringInfo(&ddl, "INSERT INTO %s_c VALUES (%d", t->name, + (int) pg_prop_draw_int(tc, 1, n)); + for (col = 0; col < t->schema.ncols; col++) + { + appendStringInfoString(&ddl, ", "); + if (col == t->fkcol) + appendStringInfoString(&ddl, "NULL"); /* FK not inherited */ + else + t->schema.cols[col].draw(tc, &ddl); + } + appendStringInfoString(&ddl, "); "); + } + appendStringInfo(&ddl, "ANALYZE %s_c; ", t->name); + } appendStringInfo(&ddl, "ANALYZE %s; ", t->name); } @@ -1021,10 +1152,88 @@ build_join_chain(pg_prop_tc *tc, const qg_db *db, int start, int max_joins, used[n] = &db->tables[nexttab]; snprintf(aliases[n], QG_NAME_LEN, "r%d", n); - /* inner or left join, drawn */ - appendStringInfo(from, " %s JOIN %s %s ON ", - pg_prop_draw_bool(tc) ? "" : "LEFT", - db->tables[nexttab].name, aliases[n]); + /* + * The join member is not always a plain relation. The planner's most + * fragile code is in pulling up subqueries, translating appendrel + * (UNION ALL / inheritance) members, and placing lateral references -- + * so the member is drawn from those shapes too. Every shape exposes + * the columns id and fk under the member alias, so the join clause, + * the key projection and the predicates stay unchanged. + * + * 0 plain relation (inner or LEFT join, as before) + * 1 ONLY relation (excludes any inheritance child) + * 2 derived table with DISTINCT (subquery with sorted paths) + * 3 UNION ALL of two filtered halves of the relation (appendrel) + * 4 LATERAL subquery correlated to the previous member, OFFSET 0 + * so it is not flattened + * 5 LATERAL UNION ALL whose first arm is itself a DISTINCT subquery + * and whose correlation references the previous member + */ + { + const char *tn = db->tables[nexttab].name; + const char *prev = aliases[n - 1]; + const char *jt = pg_prop_draw_bool(tc) ? "" : "LEFT"; + int shape = (int) pg_prop_draw_int(tc, 0, 5); + /* derived shapes expose id, plus fk when the table has one */ + const char *cols = db->tables[nexttab].fkcol >= 0 ? "id, fk" : "id"; + const char *xcols = db->tables[nexttab].fkcol >= 0 ? "x.id, x.fk" : "x.id"; + + switch (shape) + { + case 1: + appendStringInfo(from, " %s JOIN ONLY %s %s ON ", jt, tn, + aliases[n]); + break; + case 2: + appendStringInfo(from, + " %s JOIN (SELECT DISTINCT %s FROM %s) %s ON ", + jt, cols, tn, aliases[n]); + break; + case 3: + appendStringInfo(from, + " %s JOIN (SELECT %s FROM %s WHERE id %% 2 = 0 " + "UNION ALL SELECT %s FROM %s WHERE id %% 2 <> 0) %s ON ", + jt, cols, tn, cols, tn, aliases[n]); + break; + case 4: + case 5: + { + /* + * The correlation is the FK equality itself, moved + * inside the lateral subquery; the outer ON is TRUE. + * Same rows as the plain join, different tree. + */ + StringInfoData corr; + + initStringInfo(&corr); + if (frontier_is_child) + appendStringInfo(&corr, "%s.fk = x.id", prev); + else + appendStringInfo(&corr, "x.fk = %s.id", prev); + if (shape == 4) + appendStringInfo(from, + " %s JOIN LATERAL (SELECT %s FROM %s x " + "WHERE %s OFFSET 0) %s ON true", + jt, xcols, tn, corr.data, aliases[n]); + else + appendStringInfo(from, + " %s JOIN LATERAL (SELECT %s FROM " + "(SELECT DISTINCT %s FROM %s) x WHERE %s " + "AND x.id %% 2 = 0 " + "UNION ALL SELECT %s FROM %s x WHERE %s " + "AND x.id %% 2 <> 0) %s ON true", + jt, xcols, cols, tn, corr.data, + xcols, tn, corr.data, aliases[n]); + pfree(corr.data); + n++; + continue; /* join condition already placed */ + } + default: + appendStringInfo(from, " %s JOIN %s %s ON ", jt, tn, + aliases[n]); + break; + } + } if (frontier_is_child) /* frontier.fk = new.id */ appendStringInfo(from, "%s.fk = %s.id", aliases[n - 1], aliases[n]); diff --git a/src/test/prop/sql/query_gen.h b/src/test/prop/sql/query_gen.h index c56f38717d6a2..1cc5bf05cbada 100644 --- a/src/test/prop/sql/query_gen.h +++ b/src/test/prop/sql/query_gen.h @@ -173,6 +173,7 @@ typedef struct qg_db_params max_rows; /* per-table row count range */ bool allow_fk; /* declare + correlate FKs between tables */ bool allow_triggers; /* attach result-neutral triggers (0c) */ + bool allow_inherit; /* give some tables an inheritance child */ int toast_profile; /* 0 none .. 3 always (0b) */ int cardinality_bias; /* 0 mixed, 1 favour low-card columns */ } qg_db_params; @@ -190,6 +191,7 @@ extern void qg_db_params_deep_fast(qg_db_params *p); * Query areas call this so one binary serves both the quick suite and the hunt. */ extern void qg_db_params_for_env(qg_db_params *p); +extern bool qg_deep_requested(void); /* * Build a random database per the params (dropping any prior qg_t* tables). From 0ffacf6fa8c233a3c9a1d1c185d947315ee1c557 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Fri, 16 Oct 2026 18:19:34 -0400 Subject: [PATCH 294/338] prop: widen the differential oracle to the planner's proof-based rewrites Add a configuration per optimization that removes or restructures work on the strength of a proof -- self-join elimination, Memoize, Material, incremental sort, eager aggregation, GROUP BY / DISTINCT reordering, presorted aggregate, groupagg, parallel hash/append/gather merge, partitionwise join/aggregate, partition pruning -- plus collapse limits of 1 and an everything-off nested-loop plan. A wrong proof is a wrong answer, and switching the rewrite off yields the plan to compare against. Three new deep properties: aggregation over the deep join (count(*) vs count(kN) separates NULL-extended rows; the input eager aggregation and GROUP BY reordering act on), EXISTS / NOT EXISTS / IN / NOT IN over it (semi/anti join conversion, NOT IN three-valued logic), and window functions over it (partitioned count, row_number, lag over a total order). Against the 2026-10-13 base this found the self-join-elimination wrong-results bug fixed upstream by 89e06d78298 within 600 cases. --- src/test/prop/sql/prop_query_diff.c | 188 ++++++++++++++++++++++++++++ 1 file changed, 188 insertions(+) diff --git a/src/test/prop/sql/prop_query_diff.c b/src/test/prop/sql/prop_query_diff.c index d1ef3c95abf23..a887830134613 100644 --- a/src/test/prop/sql/prop_query_diff.c +++ b/src/test/prop/sql/prop_query_diff.c @@ -67,6 +67,34 @@ static const char *const configs[] = "SET LOCAL jit=on; SET LOCAL jit_above_cost=0; SET LOCAL jit_inline_above_cost=0;" "SET LOCAL jit_optimize_above_cost=0;", "SET LOCAL jit=off;", + + /* + * The planner's proof-based rewrites, each switched off on its own. These + * rewrites delete or restructure work only when they can prove it + * redundant (a join to a uniquely-keyed self, a GROUP BY order, a Memoize + * cache hit), so a wrong proof is a wrong answer. Turning one off gives + * the unrewritten plan to compare against. + */ + "SET LOCAL enable_self_join_elimination=off;", + "SET LOCAL enable_memoize=off;", + "SET LOCAL enable_material=off;", + "SET LOCAL enable_incremental_sort=off;", + "SET LOCAL enable_eager_aggregate=off;", + "SET LOCAL enable_group_by_reordering=off; SET LOCAL enable_distinct_reordering=off;", + "SET LOCAL enable_presorted_aggregate=off;", + "SET LOCAL enable_groupagg=off;", + "SET LOCAL enable_gathermerge=off; SET LOCAL enable_parallel_hash=off;" + "SET LOCAL enable_parallel_append=off;", + "SET LOCAL enable_partitionwise_join=on; SET LOCAL enable_partitionwise_aggregate=on;", + "SET LOCAL enable_partition_pruning=off;", + /* join-search limits: no subquery/join flattening into one search */ + "SET LOCAL from_collapse_limit=1; SET LOCAL join_collapse_limit=1;", + /* every rewrite at once off, nested loops only: the most naive plan */ + "SET LOCAL enable_self_join_elimination=off; SET LOCAL enable_memoize=off;" + "SET LOCAL enable_hashjoin=off; SET LOCAL enable_mergejoin=off;" + "SET LOCAL enable_hashagg=off; SET LOCAL enable_eager_aggregate=off;" + "SET LOCAL enable_incremental_sort=off; SET LOCAL enable_indexscan=off;" + "SET LOCAL enable_bitmapscan=off; SET LOCAL enable_indexonlyscan=off;", }; #define N_CONFIGS ((int) (sizeof(configs) / sizeof(configs[0]))) @@ -274,3 +302,163 @@ PG_PROP(diff_deep_join) qg_drop_db(tc, conn, db); pg_prop_disconnect(conn); } + +/* + * Shared setup for the shapes below: build the related database and one deep + * join query over it. Returns false (case vacuous) if the database is too + * small to join. + */ +static bool +deep_setup(pg_prop_tc *tc, pg_prop_conn **connp, qg_db **dbp, qg_query *q) +{ + char *err = NULL; + qg_db_params p; + + *connp = pg_prop_connect(&err); + if (*connp == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + qg_db_params_for_env(&p); + *dbp = qg_build_db(tc, *connp, &p); + if (!qg_build_query(tc, *dbp, 4, q)) + { + qg_drop_db(tc, *connp, *dbp); + pg_prop_disconnect(*connp); + return false; + } + return true; +} + +static void +deep_teardown(pg_prop_tc *tc, pg_prop_conn *conn, qg_db *db, qg_query *q) +{ + qg_free_query(q); + qg_drop_db(tc, conn, db); + pg_prop_disconnect(conn); +} + +/* + * Aggregation over a deep join, grouped by the anchor's key. This is the + * input eager aggregation (pushing a partial aggregate below a join) and + * GROUP BY reordering act on, and where hash vs sort vs presorted aggregation + * all apply. count(kN) for each joined key counts the NULL-extended rows of + * the LEFT JOINs differently from count(*), so a wrong join-null placement + * shows up as a different count. + */ +PG_PROP(diff_deep_aggregate) +{ + pg_prop_conn *conn; + qg_db *db; + qg_query q; + StringInfoData sql; + StringInfoData aggs; + char *keys; + char *tok; + char *save = NULL; + + if (!deep_setup(tc, &conn, &db, &q)) + return; + + /* one count(kN) per projected key: "count(k0), count(k1), ..." */ + initStringInfo(&aggs); + keys = pstrdup(q.keys); + for (tok = strtok_r(keys, ", ", &save); tok; tok = strtok_r(NULL, ", ", &save)) + appendStringInfo(&aggs, "%scount(%s)", aggs.len ? ", " : "", tok); + pfree(keys); + + initStringInfo(&sql); + appendStringInfo(&sql, + "SELECT k0, count(*), %s, count(DISTINCT k1) " + "FROM (%s) j GROUP BY k0 ORDER BY k0 NULLS FIRST", + aggs.data, q.select_from); + run_diff(tc, conn, sql.data); + + pfree(aggs.data); + pfree(sql.data); + deep_teardown(tc, conn, db, &q); +} + +/* + * Semijoin / antijoin over the deep join: rows of the anchor table that do (or + * do not) have a match in the join, written with EXISTS, NOT EXISTS, IN and + * NOT IN. The planner turns these into semi/anti joins (and right-semi / + * right-anti hash joins, unique-ified inner rels, ...), so each form exercises + * a different conversion while the answer is fixed by the data. + */ +PG_PROP(diff_semi_anti_join) +{ + pg_prop_conn *conn; + qg_db *db; + qg_query q; + StringInfoData sql; + const char *anchor; + int form; + + if (!deep_setup(tc, &conn, &db, &q)) + return; + + anchor = q.pred_table->name; + form = (int) pg_prop_draw_int(tc, 0, 3); + initStringInfo(&sql); + switch (form) + { + case 0: + appendStringInfo(&sql, + "SELECT a.id FROM %s a WHERE EXISTS " + "(SELECT 1 FROM (%s) j WHERE j.k0 = a.id) ORDER BY a.id", + anchor, q.select_from); + break; + case 1: + appendStringInfo(&sql, + "SELECT a.id FROM %s a WHERE NOT EXISTS " + "(SELECT 1 FROM (%s) j WHERE j.k1 = a.id) ORDER BY a.id", + anchor, q.select_from); + break; + case 2: + appendStringInfo(&sql, + "SELECT a.id FROM %s a WHERE a.id IN " + "(SELECT j.k0 FROM (%s) j) ORDER BY a.id", + anchor, q.select_from); + break; + default: + /* NOT IN with a possibly-NULL subquery column: three-valued */ + appendStringInfo(&sql, + "SELECT a.id FROM %s a WHERE a.id NOT IN " + "(SELECT j.k1 FROM (%s) j) ORDER BY a.id", + anchor, q.select_from); + break; + } + run_diff(tc, conn, sql.data); + + pfree(sql.data); + deep_teardown(tc, conn, db, &q); +} + +/* + * Window functions over the deep join: a partitioned running count, row + * numbers, and a lag over a total order. Window evaluation runs above the + * join, so its input order and partitioning come from whatever plan produced + * the join, and incremental sort / presorted paths feed it differently. + */ +PG_PROP(diff_deep_window) +{ + pg_prop_conn *conn; + qg_db *db; + qg_query q; + StringInfoData sql; + + if (!deep_setup(tc, &conn, &db, &q)) + return; + + initStringInfo(&sql); + appendStringInfo(&sql, + "SELECT %s, " + "count(*) OVER (PARTITION BY k0), " + "row_number() OVER (PARTITION BY k0 ORDER BY %s), " + "lag(k0) OVER (ORDER BY %s) " + "FROM (%s) j %s", + q.keys, q.keys, q.keys, q.select_from, q.order_by); + run_diff(tc, conn, sql.data); + + pfree(sql.data); + deep_teardown(tc, conn, db, &q); +} From db837dae3f2d8d5a7b967da761bb6c2b789ff45d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Fri, 16 Oct 2026 19:25:25 -0400 Subject: [PATCH 295/338] prop: keep the expression index bigint-safe against INT_MAX data coalesce(a, 0) + 1 overflows int4 for the INT_MAX value the adversarial generator now emits, failing table setup. Index the bigint expression. --- src/test/prop/sql/query_gen.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/test/prop/sql/query_gen.c b/src/test/prop/sql/query_gen.c index 3017d22393d78..d6333aad23ff3 100644 --- a/src/test/prop/sql/query_gen.c +++ b/src/test/prop/sql/query_gen.c @@ -305,7 +305,7 @@ qg_build_table(pg_prop_tc *tc, pg_prop_conn *conn, int *nrows) if (pg_prop_draw_int(tc, 0, 2) == 0) appendStringInfoString(&ddl, "CREATE INDEX ON " QG_TABLE " (d) WHERE d IS NOT NULL; "); if (pg_prop_draw_int(tc, 0, 2) == 0) - appendStringInfoString(&ddl, "CREATE INDEX ON " QG_TABLE " ((coalesce(a, 0) + 1)); "); + appendStringInfoString(&ddl, "CREATE INDEX ON " QG_TABLE " ((coalesce(a, 0)::bigint + 1)); "); /* ANALYZE so the planner has stats and will actually consider plans. */ appendStringInfoString(&ddl, "ANALYZE " QG_TABLE ";"); From 7bc62a54d84a3505895887c32d4157fb17a8d137 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Fri, 16 Oct 2026 21:34:31 -0400 Subject: [PATCH 296/338] prop: suppress the TooSlow health check in deep-hunt mode only Under PG_PROP_DEEP a test case's generation time is dominated by the server building its database; on a loaded hunt machine that exceeds the engine's 30-second TooSlow threshold and the run aborts with a health-check error that measures load, not generation. Suppress that one check when PG_PROP_DEEP is set; the committed TAP run keeps all health checks. --- src/test/prop/pg_prop.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/src/test/prop/pg_prop.c b/src/test/prop/pg_prop.c index 2dd8e99a46f20..dff2d79ca7425 100644 --- a/src/test/prop/pg_prop.c +++ b/src/test/prop/pg_prop.c @@ -633,6 +633,19 @@ run_property(PropEntry *prop, uint64 iterations, bool have_seed, uint64 seed, check_rc(ctx, hegel_settings_set_test_cases(ctx, s, iterations), "set_test_cases"); check_rc(ctx, hegel_settings_set_verbosity(ctx, s, verbosity), "set_verbosity"); check_rc(ctx, hegel_settings_set_seed(ctx, s, seed, have_seed), "set_seed"); + + /* + * Under a deep hunt (PG_PROP_DEEP set), a test case's "generation" is + * dominated by server-side DDL/DML building the database, which on a + * loaded machine routinely exceeds the engine's 30-second TooSlow + * threshold -- the check then measures server load, not the engine, and + * aborts the run. Suppress just that check in hunt mode; the committed + * TAP run (PG_PROP_DEEP unset) keeps every health check. + */ + if (getenv("PG_PROP_DEEP") != NULL && getenv("PG_PROP_DEEP")[0] != '\0' && + strcmp(getenv("PG_PROP_DEEP"), "0") != 0) + check_rc(ctx, hegel_settings_set_suppress_health_check(ctx, s, HEGEL_HC_TOO_SLOW), + "suppress_health_check"); snprintf(dbkey, sizeof(dbkey), "%s/%s", prop->area, prop->name); snprintf(current_area_buf, sizeof(current_area_buf), "%s", dbkey); pg_prop_current_area = current_area_buf; From bd3ebc37cd859fcef10800f2c69d8b2a9d261233 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Fri, 16 Oct 2026 21:34:31 -0400 Subject: [PATCH 297/338] prop: add a stateful model test of the write path (dml_model) Every other oracle checks a read. dml_model checks writes: a random sequence of INSERT, value and key UPDATE (single- and multi-row), DELETE (by key, by value, of NULLs), INSERT ON CONFLICT DO UPDATE / DO NOTHING, MERGE with matched-delete / matched-update / not-matched-insert, TRUNCATE, savepoint rollback/release and commit, with VACUUM or REINDEX between transactions, is applied to a real table and to an in-memory C model (key -> value). After every statement the table is read back three ways -- forced seqscan, index scan and bitmap scan on the value index -- and each must equal the model. Keys are drawn from a 24-value domain so statements collide constantly (conflicts, HOT updates, MERGE matching). The table sometimes has fillfactor 50 (room for HOT) and a partial index. The shrunk statement sequence is the reproducer. Verified at 500 cases on FreeBSD/aarch64. --- src/test/prop/Makefile | 1 + src/test/prop/meson.build | 1 + src/test/prop/sql/prop_dml_model.c | 414 +++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 417 insertions(+), 1 deletion(-) create mode 100644 src/test/prop/sql/prop_dml_model.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index c5a4882933245..909147826827b 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -282,6 +282,7 @@ OBJS = \ sql/prop_query_multi.o \ sql/prop_query_deep.o \ sql/prop_query_diff.o \ + sql/prop_dml_model.o \ sql/index_traits.o \ sql/prop_index_am.o \ sql/prop_table_am.o \ diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 31527c68ee6eb..b96a0d9826eed 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -287,6 +287,7 @@ prop_query_sources = files( 'sql/prop_query_multi.c', 'sql/prop_query_deep.c', 'sql/prop_query_diff.c', + 'sql/prop_dml_model.c', 'sql/index_traits.c', 'sql/prop_index_am.c', 'sql/prop_table_am.c', diff --git a/src/test/prop/sql/prop_dml_model.c b/src/test/prop/sql/prop_dml_model.c new file mode 100644 index 0000000000000..64b453dc115c4 --- /dev/null +++ b/src/test/prop/sql/prop_dml_model.c @@ -0,0 +1,414 @@ +/*------------------------------------------------------------------------- + * + * prop_dml_model.c + * Stateful model test of the write path. + * + * Every other oracle in the suite checks a *read* (two ways of computing a + * query agree). This one checks *writes*: a random sequence of statements -- + * INSERT, UPDATE of key and value, DELETE, INSERT ... ON CONFLICT DO UPDATE / + * DO NOTHING, MERGE with all three action kinds, savepoint/rollback, VACUUM, + * index creation -- is applied both to a real table and to an in-memory model + * of it (a plain C array: key -> value). After every step the table, read + * back in full, must equal the model. Statements are drawn with the keys + * colliding on purpose, so the conflict, HOT-update, index-maintenance and + * MERGE-matching paths are all hit. + * + * The model is the specification: it is ten lines of obviously-correct C per + * statement kind. A disagreement means the server applied a write wrongly (or + * a read after a write is wrong), and the shrunk statement sequence is the + * reproducer. + * + * The table carries a secondary index on the value and (sometimes) a partial + * index, so the read-back can be steered through an index scan as well as a + * seqscan; both must agree with the model. That catches an index left out of + * step with the heap -- the classic write-path corruption. + * + * Portions Copyright (c) 2024, PostgreSQL Global Development Group + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "dml_model" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* key domain: small, so statements collide on keys constantly */ +#define DM_KMAX 24 +/* steps per case */ +#define DM_MAX_STEPS 40 + +typedef struct DmModel +{ + bool present[DM_KMAX + 1]; + int val[DM_KMAX + 1]; /* value; meaningful only if present */ + bool isnull[DM_KMAX + 1]; /* value IS NULL */ +} DmModel; + +/* the model's committed state, plus a copy taken at the open savepoint */ +typedef struct DmState +{ + DmModel cur; + DmModel saved; + bool in_savepoint; +} DmState; + +static void +dm_exec(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "statement failed: %s\n SQL: %s", err ? err : "", sql); +} + +/* render the model as the same text the server's read-back produces */ +static char * +dm_render(const DmModel *m) +{ + StringInfoData s; + + initStringInfo(&s); + for (int k = 1; k <= DM_KMAX; k++) + { + if (!m->present[k]) + continue; + if (m->isnull[k]) + appendStringInfo(&s, "%d\t\\N\n", k); + else + appendStringInfo(&s, "%d\t%d\n", k, m->val[k]); + } + return s.data; +} + +/* + * Compare the table with the model, reading the table back two ways: a + * forced seqscan and a forced index scan on the value index (ordered by key in + * both cases). Either disagreeing with the model fails the case. + */ +static void +dm_check(pg_prop_tc *tc, pg_prop_conn *conn, const DmModel *m, + const StringInfo log) +{ + static const char *const reads[] = { + "SET LOCAL enable_indexscan=off; SET LOCAL enable_bitmapscan=off; " + "SET LOCAL enable_indexonlyscan=off;", + "SET LOCAL enable_seqscan=off; SET LOCAL enable_bitmapscan=off;", + "SET LOCAL enable_seqscan=off; SET LOCAL enable_indexscan=off;", + }; + char *want = dm_render(m); + + for (int i = 0; i < (int) lengthof(reads); i++) + { + bool failed = false; + char *err = NULL; + char *got; + StringInfoData q; + + dm_exec(tc, conn, "BEGIN"); + dm_exec(tc, conn, reads[i]); + initStringInfo(&q); + /* v >= minint OR v IS NULL is true for every row, but lets an index on v drive the scan */ + appendStringInfoString(&q, + "SELECT k, v FROM dm WHERE v >= -2147483648 OR v IS NULL " + "ORDER BY k"); + got = pg_prop_sql_rows(conn, q.data, &failed, &err); + pfree(q.data); + dm_exec(tc, conn, "ROLLBACK"); + if (failed) + pg_prop_fail(tc, "read-back failed: %s", err ? err : ""); + if (strcmp(got, want) != 0) + pg_prop_fail(tc, + "table does not match the model (read %d: %s)\n" + " model:\n%s table:\n%s statements so far:\n%s", + i, reads[i], want, got, log->data); + pg_free(got); + } + pfree(want); +} + +/* draw a value: small ints with heavy repeats, sometimes NULL */ +static void +dm_draw_val(pg_prop_tc *tc, int *v, bool *isnull) +{ + *isnull = pg_prop_draw_int(tc, 0, 7) == 0; + *v = (int) pg_prop_draw_int(tc, -3, 9); +} + +static void +dm_val_sql(StringInfo b, int v, bool isnull) +{ + if (isnull) + appendStringInfoString(b, "NULL"); + else + appendStringInfo(b, "%d", v); +} + +/* + * One random step: emit its SQL into "sql" and apply its effect to the model. + * Statements that the model predicts will fail (a duplicate-key INSERT, a key + * UPDATE onto an existing key) are not emitted; the model only covers + * successful writes, and the constraint area tests the failures. + */ +static void +dm_step(pg_prop_tc *tc, DmState *st, StringInfo sql) +{ + DmModel *m = &st->cur; + int kind = (int) pg_prop_draw_int(tc, 0, 11); + int k = (int) pg_prop_draw_int(tc, 1, DM_KMAX); + int v; + bool vn; + + dm_draw_val(tc, &v, &vn); + + switch (kind) + { + case 0: /* INSERT (only if absent) */ + if (m->present[k]) + goto upsert; + appendStringInfo(sql, "INSERT INTO dm VALUES (%d, ", k); + dm_val_sql(sql, v, vn); + appendStringInfoString(sql, ")"); + m->present[k] = true; + m->val[k] = v; + m->isnull[k] = vn; + break; + case 1: /* UPDATE value of one key */ + appendStringInfoString(sql, "UPDATE dm SET v = "); + dm_val_sql(sql, v, vn); + appendStringInfo(sql, " WHERE k = %d", k); + if (m->present[k]) + { + m->val[k] = v; + m->isnull[k] = vn; + } + break; + case 2: /* UPDATE v = v + 1 over a value range (multi-row) */ + { + int lo = (int) pg_prop_draw_int(tc, -3, 9); + + appendStringInfo(sql, "UPDATE dm SET v = v + 1 WHERE v >= %d", lo); + for (int i = 1; i <= DM_KMAX; i++) + if (m->present[i] && !m->isnull[i] && m->val[i] >= lo) + m->val[i]++; + } + break; + case 3: /* UPDATE the key itself, when the target is free */ + { + int k2 = (int) pg_prop_draw_int(tc, 1, DM_KMAX); + + if (!m->present[k] || m->present[k2] || k == k2) + { + appendStringInfo(sql, "SELECT 1"); + break; + } + appendStringInfo(sql, "UPDATE dm SET k = %d WHERE k = %d", k2, k); + m->present[k2] = true; + m->val[k2] = m->val[k]; + m->isnull[k2] = m->isnull[k]; + m->present[k] = false; + } + break; + case 4: /* DELETE one key */ + appendStringInfo(sql, "DELETE FROM dm WHERE k = %d", k); + m->present[k] = false; + break; + case 5: /* DELETE by value predicate, incl. NULLs */ + if (pg_prop_draw_bool(tc)) + { + appendStringInfoString(sql, "DELETE FROM dm WHERE v IS NULL"); + for (int i = 1; i <= DM_KMAX; i++) + if (m->present[i] && m->isnull[i]) + m->present[i] = false; + } + else + { + appendStringInfo(sql, "DELETE FROM dm WHERE v = %d", v); + for (int i = 1; i <= DM_KMAX; i++) + if (m->present[i] && !m->isnull[i] && m->val[i] == v) + m->present[i] = false; + } + break; + case 6: /* INSERT ... ON CONFLICT DO UPDATE */ + upsert: + appendStringInfo(sql, "INSERT INTO dm VALUES (%d, ", k); + dm_val_sql(sql, v, vn); + appendStringInfoString(sql, + ") ON CONFLICT (k) DO UPDATE SET v = EXCLUDED.v"); + m->present[k] = true; + m->val[k] = v; + m->isnull[k] = vn; + break; + case 7: /* INSERT ... ON CONFLICT DO NOTHING */ + appendStringInfo(sql, "INSERT INTO dm VALUES (%d, ", k); + dm_val_sql(sql, v, vn); + appendStringInfoString(sql, ") ON CONFLICT DO NOTHING"); + if (!m->present[k]) + { + m->present[k] = true; + m->val[k] = v; + m->isnull[k] = vn; + } + break; + case 8: /* MERGE over a small source of keys */ + { + int n = (int) pg_prop_draw_int(tc, 1, 4); + int sk[4]; + int sv[4]; + bool svn[4]; + int nsrc = 0; + bool seen[DM_KMAX + 1] = {0}; + + /* distinct source keys: MERGE rejects a row matched twice */ + for (int i = 0; i < n; i++) + { + int kk = (int) pg_prop_draw_int(tc, 1, DM_KMAX); + + if (seen[kk]) + continue; + seen[kk] = true; + sk[nsrc] = kk; + dm_draw_val(tc, &sv[nsrc], &svn[nsrc]); + nsrc++; + } + appendStringInfoString(sql, "MERGE INTO dm t USING (VALUES "); + for (int i = 0; i < nsrc; i++) + { + appendStringInfo(sql, "%s(%d, ", i ? ", " : "", sk[i]); + dm_val_sql(sql, sv[i], svn[i]); + appendStringInfoString(sql, "::int)"); + } + /* + * matched & source value NULL -> delete; matched -> update; + * not matched -> insert. + */ + appendStringInfoString(sql, + ") AS s(k, v) ON t.k = s.k " + "WHEN MATCHED AND s.v IS NULL THEN DELETE " + "WHEN MATCHED THEN UPDATE SET v = s.v " + "WHEN NOT MATCHED THEN INSERT VALUES (s.k, s.v)"); + for (int i = 0; i < nsrc; i++) + { + int kk = sk[i]; + + if (m->present[kk] && svn[i]) + m->present[kk] = false; + else if (m->present[kk]) + { + m->val[kk] = sv[i]; + m->isnull[kk] = false; + } + else + { + m->present[kk] = true; + m->val[kk] = sv[i]; + m->isnull[kk] = svn[i]; + } + } + } + break; + case 9: /* savepoint open / rollback-to / release */ + if (!st->in_savepoint) + { + appendStringInfoString(sql, "SAVEPOINT sp"); + st->saved = st->cur; + st->in_savepoint = true; + } + else if (pg_prop_draw_bool(tc)) + { + appendStringInfoString(sql, "ROLLBACK TO SAVEPOINT sp; RELEASE SAVEPOINT sp"); + st->cur = st->saved; + st->in_savepoint = false; + } + else + { + appendStringInfoString(sql, "RELEASE SAVEPOINT sp"); + st->in_savepoint = false; + } + break; + case 10: /* COMMIT and start a new transaction (VACUUM needs out-of-txn) */ + appendStringInfoString(sql, "COMMIT; BEGIN"); + st->in_savepoint = false; + break; + default: /* TRUNCATE (transactional; undone by ROLLBACK TO) */ + appendStringInfoString(sql, "TRUNCATE dm"); + for (int i = 1; i <= DM_KMAX; i++) + m->present[i] = false; + break; + } +} + +/* + * The property: run a random statement sequence inside transactions, and after + * every statement compare the table with the model. Every few steps the + * transaction commits, sometimes followed by VACUUM (so pruning, HOT chains + * and index cleanup run between writes) or by a fresh index build. + */ +PG_PROP(dml_matches_model) +{ + pg_prop_conn *conn; + char *err = NULL; + DmState st; + StringInfoData log; + StringInfoData sql; + int nsteps; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + + memset(&st, 0, sizeof(st)); + initStringInfo(&log); + initStringInfo(&sql); + + /* fillfactor < 100 leaves room on the page for HOT updates */ + appendStringInfo(&sql, + "DROP TABLE IF EXISTS dm; " + "CREATE TABLE dm (k int PRIMARY KEY, v int) WITH (fillfactor = %d); " + "CREATE INDEX dm_v ON dm (v)", + pg_prop_draw_bool(tc) ? 100 : 50); + if (pg_prop_draw_bool(tc)) + appendStringInfoString(&sql, "; CREATE INDEX dm_vp ON dm (v) WHERE v > 3"); + dm_exec(tc, conn, sql.data); + dm_exec(tc, conn, "BEGIN"); + + nsteps = (int) pg_prop_draw_int(tc, 1, DM_MAX_STEPS); + for (int i = 0; i < nsteps; i++) + { + resetStringInfo(&sql); + dm_step(tc, &st, &sql); + appendStringInfo(&log, " %s;\n", sql.data); + dm_exec(tc, conn, sql.data); + + /* occasionally, between transactions, VACUUM or REINDEX */ + if (strncmp(sql.data, "COMMIT", 6) == 0 && pg_prop_draw_bool(tc)) + { + const char *maint = pg_prop_draw_bool(tc) ? "VACUUM dm" : "REINDEX TABLE dm"; + + dm_exec(tc, conn, "COMMIT"); + appendStringInfo(&log, " COMMIT; %s; BEGIN;\n", maint); + dm_exec(tc, conn, maint); + dm_exec(tc, conn, "BEGIN"); + } + /* + * The read-back runs in its own BEGIN/ROLLBACK, which would end the + * open transaction; wrap the check in a savepoint instead when one is + * not already in use, else check after the next COMMIT. + */ + if (!st.in_savepoint) + { + dm_exec(tc, conn, "COMMIT"); + dm_check(tc, conn, &st.cur, &log); + dm_exec(tc, conn, "BEGIN"); + } + } + dm_exec(tc, conn, "COMMIT"); + st.in_savepoint = false; + dm_check(tc, conn, &st.cur, &log); + + dm_exec(tc, conn, "DROP TABLE dm"); + pfree(log.data); + pfree(sql.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 422ce58b70cb9..311b9d0a539a7 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 71750b4b07a601dfaefc6b8de551bbe3000bd58b Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 17 Oct 2026 02:13:08 -0400 Subject: [PATCH 298/338] prop: make the differential oracle fail on malformed queries run_diff took the first configuration that executed as its reference and treated a query that errored under every configuration as vacuous. That silently passed a generator bug: diff_deep_aggregate and diff_semi_anti_join hard-coded the key k1, which does not exist when the generated join chain has a single table, so those cases errored everywhere and "passed" -- 2 million "column k1 does not exist" errors in one night's server log, no failure. The reference is now always config[0] (the planner's free choice). Only a plan-dependent runtime error (out of range, overflow, division by zero, which a correct server raises or not depending on whether a filter runs first) makes the case vacuous or skips a config; any other error fails the case with the query. The two properties now use the last projected key instead of k1. --- src/test/prop/sql/prop_query_diff.c | 117 +++++++++++++++++++--------- 1 file changed, 80 insertions(+), 37 deletions(-) diff --git a/src/test/prop/sql/prop_query_diff.c b/src/test/prop/sql/prop_query_diff.c index a887830134613..eef2ef35a6c2b 100644 --- a/src/test/prop/sql/prop_query_diff.c +++ b/src/test/prop/sql/prop_query_diff.c @@ -107,27 +107,26 @@ static const char *const configs[] = */ static char * digest_under(pg_prop_conn *conn, const char *setup, const char *query, - bool *errored) + bool *errored, char **errmsg) { char *dig; bool failed = false; char *err = NULL; *errored = false; + *errmsg = NULL; if (!pg_prop_sql_exec(conn, "BEGIN", &err)) { - if (err) - free(err); *errored = true; + *errmsg = err; return NULL; } if (setup[0] != '\0' && !pg_prop_sql_exec(conn, setup, &err)) { - if (err) - free(err); pg_prop_sql_exec(conn, "ROLLBACK", NULL); *errored = true; + *errmsg = err; return NULL; } dig = pg_prop_sql_rowhash(conn, query, &failed, &err); @@ -135,16 +134,32 @@ digest_under(pg_prop_conn *conn, const char *setup, const char *query, if (failed) { - if (err) - free(err); if (dig) free(dig); *errored = true; + *errmsg = err; return NULL; } return dig; } +/* + * Errors a correct server may raise or not depending on the plan: arithmetic + * overflow and division by zero are evaluated per row, so a plan that filters + * first never evaluates the expression on the offending row. Anything else -- + * an unknown column, a syntax error, an internal error -- means the query + * itself is malformed (a generator bug) or the server failed, and is never + * waved through as "vacuous". + */ +static bool +plan_dependent_error(const char *msg) +{ + return msg != NULL && + (strstr(msg, "out of range") != NULL || + strstr(msg, "division by zero") != NULL || + strstr(msg, "overflow") != NULL); +} + /* * The core differential property: take the reference digest (first config that * executes), then require every other config that executes to agree. @@ -152,38 +167,50 @@ digest_under(pg_prop_conn *conn, const char *setup, const char *query, static void run_diff(pg_prop_tc *tc, pg_prop_conn *conn, const char *query) { - char *ref = NULL; + char *ref; + char *err = NULL; bool errored; int i; - int ref_cfg = -1; - for (i = 0; i < N_CONFIGS; i++) + /* + * The reference is config[0], the planner's free choice. If it errors, + * the case is vacuous only for a plan-dependent runtime error; anything + * else fails the case so a malformed generated query cannot pass silently. + */ + ref = digest_under(conn, configs[0], query, &errored, &err); + if (errored) { - ref = digest_under(conn, configs[i], query, &errored); - if (!errored && ref != NULL) + if (plan_dependent_error(err)) { - ref_cfg = i; - break; - } - if (ref) - { - free(ref); - ref = NULL; + pg_free(err); + return; } + pg_prop_fail(tc, "reference query failed: %s\n query: %s", + err ? err : "", query); } - if (ref == NULL) - return; /* query errors under every config: vacuous */ + pg_free(err); - for (i = ref_cfg + 1; i < N_CONFIGS; i++) + for (i = 1; i < N_CONFIGS; i++) { - char *d = digest_under(conn, configs[i], query, &errored); + char *d = digest_under(conn, configs[i], query, &errored, &err); - if (errored || d == NULL) + if (errored) { - if (d) - free(d); - continue; /* config not applicable; skip */ + /* + * A forced config may legitimately hit a runtime error the + * reference plan avoided (see plan_dependent_error). Any other + * error under a config whose reference succeeded is a bug: the + * same query must not become invalid because of a planner GUC. + */ + if (!plan_dependent_error(err)) + pg_prop_fail(tc, + "config[%d] errored where the reference succeeded: %s\n" + " config: %s\n query: %s", + i, err ? err : "", configs[i], query); + pg_free(err); + continue; } + pg_free(err); if (strcmp(ref, d) != 0) { char *rc = pstrdup(ref); @@ -193,12 +220,10 @@ run_diff(pg_prop_tc *tc, pg_prop_conn *conn, const char *query) free(d); pg_prop_fail(tc, "two execution strategies disagree on the same query.\n" - " config[%d] %s -> %s\n" + " config[0] (free choice) -> %s\n" " config[%d] %s -> %s\n" " query: %s", - ref_cfg, configs[ref_cfg][0] ? configs[ref_cfg] : "(free choice)", rc, - i, configs[i], dc, - query); + rc, i, configs[i], dc, query); } free(d); } @@ -303,6 +328,21 @@ PG_PROP(diff_deep_join) pg_prop_disconnect(conn); } + +/* + * The name of the last projected key ("k0" for a one-table chain, "kN" for an + * N+1-table chain). Properties that need a second key use this rather than + * hard-coding k1, which does not exist when the chain has one table. + */ +static const char * +last_key(const qg_query *q, char *buf, size_t len) +{ + const char *p = strrchr(q->keys, 'k'); + + snprintf(buf, len, "%s", p ? p : "k0"); + return buf; +} + /* * Shared setup for the shapes below: build the related database and one deep * join query over it. Returns false (case vacuous) if the database is too @@ -354,6 +394,7 @@ PG_PROP(diff_deep_aggregate) char *keys; char *tok; char *save = NULL; + char lk[16]; if (!deep_setup(tc, &conn, &db, &q)) return; @@ -367,9 +408,9 @@ PG_PROP(diff_deep_aggregate) initStringInfo(&sql); appendStringInfo(&sql, - "SELECT k0, count(*), %s, count(DISTINCT k1) " + "SELECT k0, count(*), %s, count(DISTINCT %s) " "FROM (%s) j GROUP BY k0 ORDER BY k0 NULLS FIRST", - aggs.data, q.select_from); + aggs.data, last_key(&q, lk, sizeof(lk)), q.select_from); run_diff(tc, conn, sql.data); pfree(aggs.data); @@ -392,9 +433,11 @@ PG_PROP(diff_semi_anti_join) StringInfoData sql; const char *anchor; int form; + char lk[16]; if (!deep_setup(tc, &conn, &db, &q)) return; + last_key(&q, lk, sizeof(lk)); anchor = q.pred_table->name; form = (int) pg_prop_draw_int(tc, 0, 3); @@ -410,8 +453,8 @@ PG_PROP(diff_semi_anti_join) case 1: appendStringInfo(&sql, "SELECT a.id FROM %s a WHERE NOT EXISTS " - "(SELECT 1 FROM (%s) j WHERE j.k1 = a.id) ORDER BY a.id", - anchor, q.select_from); + "(SELECT 1 FROM (%s) j WHERE j.%s = a.id) ORDER BY a.id", + anchor, q.select_from, lk); break; case 2: appendStringInfo(&sql, @@ -423,8 +466,8 @@ PG_PROP(diff_semi_anti_join) /* NOT IN with a possibly-NULL subquery column: three-valued */ appendStringInfo(&sql, "SELECT a.id FROM %s a WHERE a.id NOT IN " - "(SELECT j.k1 FROM (%s) j) ORDER BY a.id", - anchor, q.select_from); + "(SELECT j.%s FROM (%s) j) ORDER BY a.id", + anchor, lk, q.select_from); break; } run_diff(tc, conn, sql.data); From 8f80aa3706360df8999117442e1f7cff51c9b234 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 17 Oct 2026 02:18:40 -0400 Subject: [PATCH 299/338] prop: give deep-join predicates the nullability-sensitive forms The table-qualified predicate grammar the deep oracles use had only NULL tests and plain comparisons, so the planner's strictness and nullability proofs -- outer-join reduction, qual pushdown across a LEFT JOIN, index-qual matching -- saw nothing hard. Add IS [NOT] DISTINCT FROM, = ANY / <> ALL over typed arrays that may hold NULL or be empty, and cross-type comparisons through a cast. diff_deep_filtered puts such a predicate on the anchor of a deep join and runs it through every differential configuration. --- src/test/prop/sql/prop_query_diff.c | 30 ++++++++++++++++++++++ src/test/prop/sql/query_gen.c | 40 ++++++++++++++++++++++++++++- 2 files changed, 69 insertions(+), 1 deletion(-) diff --git a/src/test/prop/sql/prop_query_diff.c b/src/test/prop/sql/prop_query_diff.c index eef2ef35a6c2b..be72057fbd927 100644 --- a/src/test/prop/sql/prop_query_diff.c +++ b/src/test/prop/sql/prop_query_diff.c @@ -505,3 +505,33 @@ PG_PROP(diff_deep_window) pfree(sql.data); deep_teardown(tc, conn, db, &q); } + +/* + * Deep join filtered by a generated predicate on the anchor. The predicate + * grammar (IS DISTINCT FROM, = ANY / <> ALL over arrays holding NULL, + * cross-type casts, NULL tests) is where the planner's nullability and + * strictness reasoning runs: outer-join reduction, qual pushdown below or + * above a LEFT JOIN, and index-qual matching all depend on proving a + * predicate strict. A wrong proof changes the rows. + */ +PG_PROP(diff_deep_filtered) +{ + pg_prop_conn *conn; + qg_db *db; + qg_query q; + StringInfoData pred; + StringInfoData sql; + + if (!deep_setup(tc, &conn, &db, &q)) + return; + + initStringInfo(&pred); + qg_draw_predicate_on(tc, &pred, q.pred_table, q.pred_alias); + initStringInfo(&sql); + appendStringInfo(&sql, "%s WHERE %s %s", q.select_from, pred.data, q.order_by); + run_diff(tc, conn, sql.data); + + pfree(pred.data); + pfree(sql.data); + deep_teardown(tc, conn, db, &q); +} diff --git a/src/test/prop/sql/query_gen.c b/src/test/prop/sql/query_gen.c index d6333aad23ff3..1d756a87cafe1 100644 --- a/src/test/prop/sql/query_gen.c +++ b/src/test/prop/sql/query_gen.c @@ -1039,7 +1039,7 @@ draw_leaf_on(pg_prop_tc *tc, StringInfo buf, const qg_table *t, ci = (int) pg_prop_draw_int(tc, 0, t->schema.ncols - 1); c = &t->schema.cols[ci]; - switch ((int) pg_prop_draw_int(tc, 0, 3)) + switch ((int) pg_prop_draw_int(tc, 0, 7)) { case 0: appendStringInfo(buf, "%s.%s IS NULL", alias, c->name); @@ -1047,6 +1047,44 @@ draw_leaf_on(pg_prop_tc *tc, StringInfo buf, const qg_table *t, case 1: appendStringInfo(buf, "%s.%s IS NOT NULL", alias, c->name); break; + case 2: /* IS [NOT] DISTINCT FROM: null-safe equality */ + appendStringInfo(buf, "%s.%s IS %sDISTINCT FROM ", alias, c->name, + pg_prop_draw_bool(tc) ? "NOT " : ""); + if (c->is_text_like) + draw_text_or_null(tc, buf); + else + draw_int_or_null(tc, buf); + break; + case 3: /* = ANY / <> ALL over an array that may hold NULL */ + { + int n = (int) pg_prop_draw_int(tc, 0, 3); + bool any = pg_prop_draw_bool(tc); + + appendStringInfo(buf, "%s.%s %s (ARRAY[", alias, c->name, + any ? "= ANY" : "<> ALL"); + for (int i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(buf, ", "); + if (c->is_text_like) + draw_text_or_null(tc, buf); + else + draw_int_or_null(tc, buf); + } + /* typed so an empty / all-NULL array still resolves */ + appendStringInfo(buf, "]::%s[])", + c->is_text_like ? "text" : (strstr(c->sqltype, "numeric") ? "numeric" : "int")); + } + break; + case 4: /* cross-type: compare through a cast */ + if (c->is_text_like) + appendStringInfo(buf, "%s.%s::varchar %s 'b'", alias, c->name, + cmp[pg_prop_draw_int(tc, 0, 5)]); + else + appendStringInfo(buf, "%s.%s::numeric %s %d.5", alias, c->name, + cmp[pg_prop_draw_int(tc, 0, 5)], + (int) pg_prop_draw_int(tc, -3, 9)); + break; default: appendStringInfo(buf, "%s.%s %s ", alias, c->name, cmp[pg_prop_draw_int(tc, 0, 5)]); From 14d1f9ba0a1696b1f0057127b9a6520d2da48642 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 17 Oct 2026 08:32:18 -0400 Subject: [PATCH 300/338] prop: add a LATERAL UNION ALL member over a sorted uncorrelated arm The generator's existing LATERAL UNION ALL member correlated its DISTINCT arm with the outer rel, which pushes the qual into the subquery and keeps the planner off the path where an appendrel child's translated tlist is a Var of another rel. Checked against a canary server built at 6296e78c3d2 (before 11b56ee2731 "Fix bogus Assert in find_var_for_subquery_tle"): 0 crashes in 900 cases. New join member: LATERAL (SELECT prev.fk FROM (SELECT DISTINCT id FROM t ORDER BY id LIMIT 1) s UNION ALL SELECT id FROM t WHERE id < 0) ON prev.fk = member.id. The first arm is uncorrelated and sorted, its only output the outer reference; it yields prev.fk once, and the ON clause keeps it exactly where the FK guarantees a referenced row, so the rows equal the plain join's. Used only where the frontier holds the FK and the member table has no fk column of its own (the member exposes only id). Against the canary the assertion now fires within 3-6 s (query_deep/deep_tlp, query_diff/diff_deep_join); the current server stays green at 300 cases on every deep oracle. --- src/test/prop/sql/query_gen.c | 52 ++++++++++++++++++++++++++++++++++- 1 file changed, 51 insertions(+), 1 deletion(-) diff --git a/src/test/prop/sql/query_gen.c b/src/test/prop/sql/query_gen.c index 1d756a87cafe1..a9433d728e57e 100644 --- a/src/test/prop/sql/query_gen.c +++ b/src/test/prop/sql/query_gen.c @@ -1211,7 +1211,7 @@ build_join_chain(pg_prop_tc *tc, const qg_db *db, int start, int max_joins, const char *tn = db->tables[nexttab].name; const char *prev = aliases[n - 1]; const char *jt = pg_prop_draw_bool(tc) ? "" : "LEFT"; - int shape = (int) pg_prop_draw_int(tc, 0, 5); + int shape = (int) pg_prop_draw_int(tc, 0, 6); /* derived shapes expose id, plus fk when the table has one */ const char *cols = db->tables[nexttab].fkcol >= 0 ? "id, fk" : "id"; const char *xcols = db->tables[nexttab].fkcol >= 0 ? "x.id, x.fk" : "x.id"; @@ -1266,6 +1266,56 @@ build_join_chain(pg_prop_tc *tc, const qg_db *db, int start, int max_joins, n++; continue; /* join condition already placed */ } + case 6: + { + /* + * LATERAL UNION ALL whose first arm is an uncorrelated + * sorted subquery (a DISTINCT ... ORDER BY ... LIMIT 1) + * projecting only the outer reference. After pull-up + * the appendrel child's translated tlist is a Var of + * the outer rel, and the child has sorted paths, which + * is the combination find_var_for_subquery_tle + * mishandled (assertion failure in cassert builds, + * possible wrong Var match otherwise; fixed upstream in + * 11b56ee2731). The arm must not be correlated or + * filtered, or the planner never builds that path. + * + * Rows stay those of the plain FK join. The first arm + * yields the outer key exactly once when the member + * table is non-empty (LIMIT 1 over a DISTINCT); the + * second arm is the member's rows with that key, so the + * union is {prev.fk} plus every matching member row. + * The second arm is filtered to an impossible id so the + * union is {prev.fk} alone, and the ON clause keeps it + * only where a member row with that id exists -- which + * the enforced FK guarantees for every non-NULL fk. + * Used only where the frontier holds the FK (prev.fk = + * new.id), so that argument applies. + */ + /* + * The member exposes only id, so it is used only for a + * parent table with no fk column of its own: the chain + * then continues through a child referencing it (new.fk + * = this.id), never through this.fk. + */ + if (!frontier_is_child || db->tables[nexttab].nrows == 0 || + db->tables[nexttab].fkcol >= 0) + { + appendStringInfo(from, " %s JOIN %s %s ON ", jt, tn, + aliases[n]); + break; + } + appendStringInfo(from, + " %s JOIN LATERAL (" + "SELECT %s.fk AS id FROM " + "(SELECT DISTINCT id FROM %s ORDER BY id LIMIT 1) s " + "UNION ALL SELECT id FROM %s WHERE id < 0" + ") %s ON %s.fk = %s.id", + jt, prev, tn, tn, + aliases[n], prev, aliases[n]); + n++; + continue; + } default: appendStringInfo(from, " %s JOIN %s %s ON ", jt, tn, aliases[n]); From 427a064a43d1c79e7bdba8ae9cb78dc401442b7b Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 17 Oct 2026 09:10:22 -0400 Subject: [PATCH 301/338] prop: never walk the join chain through an fk the member does not expose A derived join member exposes only the columns it projects. The new LATERAL UNION ALL member projects id alone, but the chain walk still continued through member.fk when the member's table had one, producing "column rN.fk does not exist". The differential oracle now fails on such errors instead of passing them as vacuous, so the overnight hunt reported them (7 hits, all this). Track per member whether its alias exposes fk, and continue the walk through a member's fk only when it does. --- src/test/prop/sql/query_gen.c | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/src/test/prop/sql/query_gen.c b/src/test/prop/sql/query_gen.c index a9433d728e57e..212c44a37c349 100644 --- a/src/test/prop/sql/query_gen.c +++ b/src/test/prop/sql/query_gen.c @@ -1155,11 +1155,20 @@ build_join_chain(pg_prop_tc *tc, const qg_db *db, int start, int max_joins, int n = 0; int cur = start; + /* + * exposes_fk[i]: member i's alias has an fk column. A plain or ONLY + * member has whatever its table has; a derived member has only what it + * projects. The walk continues through a member's fk only if exposed. + */ + bool exposes_fk[QG_MAX_TABLES]; + used[0] = &db->tables[cur]; snprintf(aliases[0], QG_NAME_LEN, "r0"); appendStringInfo(from, "%s r0", db->tables[cur].name); n = 1; + exposes_fk[0] = db->tables[cur].fkcol >= 0; + while (n <= max_joins) { const qg_table *frontier = used[n - 1]; @@ -1167,7 +1176,7 @@ build_join_chain(pg_prop_tc *tc, const qg_db *db, int start, int max_joins, bool frontier_is_child = false; /* prefer the frontier's own parent (frontier.fk -> parent.id) */ - if (frontier->fkref >= 0) + if (frontier->fkref >= 0 && exposes_fk[n - 1]) { nexttab = frontier->fkref; frontier_is_child = true; @@ -1189,6 +1198,7 @@ build_join_chain(pg_prop_tc *tc, const qg_db *db, int start, int max_joins, used[n] = &db->tables[nexttab]; snprintf(aliases[n], QG_NAME_LEN, "r%d", n); + exposes_fk[n] = db->tables[nexttab].fkcol >= 0; /* * The join member is not always a plain relation. The planner's most @@ -1293,13 +1303,10 @@ build_join_chain(pg_prop_tc *tc, const qg_db *db, int start, int max_joins, * new.id), so that argument applies. */ /* - * The member exposes only id, so it is used only for a - * parent table with no fk column of its own: the chain - * then continues through a child referencing it (new.fk - * = this.id), never through this.fk. + * The member projects only id; exposes_fk records that + * so the walk never continues through this.fk. */ - if (!frontier_is_child || db->tables[nexttab].nrows == 0 || - db->tables[nexttab].fkcol >= 0) + if (!frontier_is_child || db->tables[nexttab].nrows == 0) { appendStringInfo(from, " %s JOIN %s %s ON ", jt, tn, aliases[n]); @@ -1313,6 +1320,7 @@ build_join_chain(pg_prop_tc *tc, const qg_db *db, int start, int max_joins, ") %s ON %s.fk = %s.id", jt, prev, tn, tn, aliases[n], prev, aliases[n]); + exposes_fk[n] = false; /* projects id only */ n++; continue; } From 28ddb7466ea9b9812742cce6964721f17065036d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 17 Oct 2026 12:10:22 -0400 Subject: [PATCH 302/338] prop: check that system columns identify the row (syscol) No query oracle in the suite projected ctid, tableoid or xmin, so a bug that leaves a tuple's system columns wrong while its data is right was invisible. CF 7371 is such a bug: ExecForceStoreHeapTuple() drops tts_tid on its buffer- and virtual-slot paths, so an ordering-operator index scan with lossy distances returns ctid (4294967295,0) for every row that goes through the reorder queue, row locking hands that TID to heap_lock_tuple(), and a routed INSERT into a postgres_fdw partition returns it from RETURNING ctid. The invariant: every row a query returns names, through ctid / tableoid / xmin, a live row of the table holding exactly its data. Results are captured into a temp table and verified afterwards, so no verification qual can reach the scan under test. Four properties: knn_scan_syscols_identify_row KNN ORDER BY <-> LIMIT n over drawn GiST/SP-GiST opclasses (exact and lossy), plain/qual/subquery/CTE knn_row_lock_locks_the_returned_rows the same with FOR UPDATE / NO KEY UPDATE / SHARE / KEY SHARE; the lock must succeed dml_returning_syscols_identify_row UPDATE/DELETE through a KNN subquery and INSERT through a NEW-modifying trigger, RETURNING system columns fdw_routed_insert_returning_ctid routing into loopback postgres_fdw partitions (matching or permuted column order) and local ones Against upstream master 45fc51f (cassert, FreeBSD/aarch64): knn_scan falsified in 10/10 seeds (shrunk to 17 polygon rows, GiST, LIMIT > 1, ctid (4294967295,0)); knn_row_lock crashed the backend in 10/10 seeds (TRAP ItemPointerIsValid); fdw_routed_insert falsified at the first case. With the CF 7371 v8 patch applied all four pass. --- src/test/prop/Makefile | 1 + src/test/prop/meson.build | 1 + src/test/prop/sql/prop_syscol.c | 514 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 517 insertions(+), 1 deletion(-) create mode 100644 src/test/prop/sql/prop_syscol.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 909147826827b..739e760ede130 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -283,6 +283,7 @@ OBJS = \ sql/prop_query_deep.o \ sql/prop_query_diff.o \ sql/prop_dml_model.o \ + sql/prop_syscol.o \ sql/index_traits.o \ sql/prop_index_am.o \ sql/prop_table_am.o \ diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index b96a0d9826eed..4ad44ddc12a04 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -288,6 +288,7 @@ prop_query_sources = files( 'sql/prop_query_deep.c', 'sql/prop_query_diff.c', 'sql/prop_dml_model.c', + 'sql/prop_syscol.c', 'sql/index_traits.c', 'sql/prop_index_am.c', 'sql/prop_table_am.c', diff --git a/src/test/prop/sql/prop_syscol.c b/src/test/prop/sql/prop_syscol.c new file mode 100644 index 0000000000000..bcb5e7635a64f --- /dev/null +++ b/src/test/prop/sql/prop_syscol.c @@ -0,0 +1,514 @@ +/*------------------------------------------------------------------------- + * + * prop_syscol.c + * System columns must identify the row they came from. + * + * Every other query oracle in the suite compares user columns. None of them + * project ctid, tableoid or xmin, so any bug that leaves a tuple's system + * columns wrong while its data is right is invisible to them -- and such bugs + * are real: ExecForceStoreHeapTuple() drops tts_tid on a buffer slot, so an + * ordering-operator index scan whose distances are lossy (GiST circle_ops / + * poly_ops, SP-GiST poly_ops) returns ctid = (4294967295,0) for every row it + * routes through the reorder queue, and SELECT ... FOR UPDATE then hands that + * TID to heap_lock_tuple() (an assertion failure, or P_NEW relation extension + * without assertions). See CF 7371. + * + * The invariant here, checked over a randomized trigger space: + * + * For every row R returned by any query over table T, the system columns R + * reports must name a live row of T holding exactly R's data: + * R.ctid is a valid TID, and T WHERE ctid = R.ctid has R's id and data; + * R.tableoid = T's oid (or the oid of the partition/child that holds it); + * R.xmin equals that row's xmin. + * + * Each case builds a table of geometric values with an ordering-capable index + * (GiST or SP-GiST, with an opclass whose distances may be exact or lossy), + * then draws a query shape that reads system columns through a path the + * planner is pushed onto with plan advice-free GUCs (seqscan off): + * + * - KNN: ORDER BY v <-> point LIMIT n (n drawn; n = 1 never reorders) + * - KNN with a qual on top, or under a subquery / CTE + * - KNN ... FOR UPDATE / FOR SHARE / FOR NO KEY UPDATE (the lock path) + * - UPDATE ... RETURNING ctid / DELETE ... RETURNING ctid, routed through a + * KNN subquery + * - INSERT ... RETURNING ctid into a table with a BEFORE ROW trigger that + * modifies NEW (the trigger path goes through ExecForceStoreHeapTuple too) + * + * The result is captured into a temp table first and verified against the + * base table afterwards, so no verification qual can be pushed into the scan + * under test. A lock-path case that raises an error is also a failure: a + * correct server locks the rows the query returned. + * + * Portions Copyright (c) 2024, PostgreSQL Global Development Group + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "syscol" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* the index/opclass/type combinations that support ORDER BY <-> point */ +typedef struct KnnAm +{ + const char *am; /* gist | spgist */ + const char *opclass; /* "" = default */ + const char *type; /* column type */ + bool lossy; /* distance needs recheck -> reorder queue */ +} KnnAm; + +static const KnnAm knn_ams[] = { + {"gist", "", "point", false}, + {"gist", "", "box", false}, + {"gist", "", "circle", true}, + {"gist", "", "polygon", true}, + {"spgist", "", "point", false}, + {"spgist", "kd_point_ops", "point", false}, + {"spgist", "", "box", false}, + {"spgist", "", "polygon", true}, +}; +#define N_KNN_AMS ((int) lengthof(knn_ams)) + +static void +sc_exec(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "setup failed: %s\n SQL: %s", err ? err : "", sql); +} + +static long +sc_count(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *v = pg_prop_sql_scalar(conn, sql, &failed, &err); + long n; + + if (failed || v == NULL) + pg_prop_fail(tc, "check failed: %s\n SQL: %s", err ? err : "", sql); + n = atol(v); + pg_free(v); + return n; +} + +/* append a literal geometric value of the given type around (x, y) */ +static void +sc_geom(StringInfo b, const char *type, int x, int y, int sz) +{ + if (strcmp(type, "point") == 0) + appendStringInfo(b, "point(%d, %d)", x, y); + else if (strcmp(type, "box") == 0) + appendStringInfo(b, "box(point(%d, %d), point(%d, %d))", x, y, x + sz, y + sz); + else if (strcmp(type, "circle") == 0) + appendStringInfo(b, "circle(point(%d, %d), %d.5)", x, y, sz); + else /* polygon: a triangle, so bbox distance != true distance */ + appendStringInfo(b, "polygon(path '((%d,%d),(%d,%d),(%d,%d))')", + x, y, x + sz, y + sz, x + sz, y); +} + +/* + * Build sc_t(id int primary key, g , v int) with n rows of drawn + * geometry and an ordering-capable index; return the chosen AM. + */ +static const KnnAm * +sc_build(pg_prop_tc *tc, pg_prop_conn *conn) +{ + const KnnAm *k = &knn_ams[pg_prop_draw_int(tc, 0, N_KNN_AMS - 1)]; + int n = (int) pg_prop_draw_int(tc, 2, 300); + int grid = (int) pg_prop_draw_int(tc, 1, 50); + StringInfoData s; + + initStringInfo(&s); + appendStringInfo(&s, + "DROP TABLE IF EXISTS sc_t CASCADE; " + "CREATE TABLE sc_t (id int PRIMARY KEY, g %s, v int); " + "INSERT INTO sc_t VALUES ", k->type); + for (int i = 1; i <= n; i++) + { + /* clustered coordinates with ties, so distances collide */ + int x = (int) pg_prop_draw_int(tc, 0, grid); + int y = (int) pg_prop_draw_int(tc, 0, grid); + int sz = (int) pg_prop_draw_int(tc, 0, 9); + + appendStringInfo(&s, "%s(%d, ", i > 1 ? ", " : "", i); + sc_geom(&s, k->type, x, y, sz); + appendStringInfo(&s, ", %d)", (int) pg_prop_draw_int(tc, -5, 5)); + } + appendStringInfo(&s, "; CREATE INDEX sc_t_g ON sc_t USING %s (g %s); ", + k->am, k->opclass); + /* sometimes dead tuples / HOT chains in the heap under the index */ + if (pg_prop_draw_bool(tc)) + appendStringInfoString(&s, "UPDATE sc_t SET v = v + 1 WHERE id % 3 = 0; "); + if (pg_prop_draw_bool(tc)) + appendStringInfoString(&s, "DELETE FROM sc_t WHERE id % 7 = 0; "); + sc_exec(tc, conn, s.data); + /* VACUUM cannot share a multi-statement (implicit-transaction) string */ + sc_exec(tc, conn, "VACUUM ANALYZE sc_t"); + pfree(s.data); + return k; +} + +/* + * Verify the captured result sc_res(c tid, toid oid, xm text, id int, v int) + * against sc_t: every row's system columns must name the live row with the + * same id and data. + */ +static void +sc_verify(pg_prop_tc *tc, pg_prop_conn *conn, const char *what) +{ + long total = sc_count(tc, conn, "SELECT count(*) FROM sc_res"); + long bad_tid = sc_count(tc, conn, + "SELECT count(*) FROM sc_res WHERE c IS NULL " + "OR c = '(4294967295,0)'::tid OR c::text LIKE '(4294967295,%'"); + long not_found = sc_count(tc, conn, + "SELECT count(*) FROM sc_res r WHERE NOT EXISTS " + "(SELECT 1 FROM sc_t t WHERE t.ctid = r.c " + "AND t.id = r.id AND t.v IS NOT DISTINCT FROM r.v)"); + long bad_oid = sc_count(tc, conn, + "SELECT count(*) FROM sc_res WHERE toid IS DISTINCT FROM " + "'sc_t'::regclass::oid"); + long bad_xmin = sc_count(tc, conn, + "SELECT count(*) FROM sc_res r WHERE NOT EXISTS " + "(SELECT 1 FROM sc_t t WHERE t.id = r.id " + "AND t.xmin::text = r.xm)"); + + if (bad_tid || not_found || bad_oid || bad_xmin) + { + bool failed = false; + char *sample = pg_prop_sql_rows(conn, + "SELECT c, toid, xm, id, v FROM sc_res ORDER BY id LIMIT 8", + &failed, NULL); + + pg_prop_fail(tc, + "%s returned rows whose system columns do not identify them\n" + " rows=%ld invalid ctid=%ld ctid->wrong/missing row=%ld " + "wrong tableoid=%ld wrong xmin=%ld\n sample (ctid, tableoid, xmin, id, v):\n%s", + what, total, bad_tid, not_found, bad_oid, bad_xmin, + sample ? sample : "(unavailable)"); + } +} + +/* draw a KNN ORDER BY clause: v <-> point [ASC], tie-broken by id */ +static void +sc_orderby(pg_prop_tc *tc, StringInfo b) +{ + appendStringInfo(b, "g <-> point(%d, %d)", + (int) pg_prop_draw_int(tc, -5, 55), + (int) pg_prop_draw_int(tc, -5, 55)); +} + +/* + * The read path: capture a KNN query's system columns, possibly wrapped in a + * subquery or CTE and possibly with a qual, then verify them. + */ +PG_PROP(knn_scan_syscols_identify_row) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int lim; + int shape; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + sc_build(tc, conn); + + lim = (int) pg_prop_draw_int(tc, 1, 40); + shape = (int) pg_prop_draw_int(tc, 0, 3); + + initStringInfo(&q); + appendStringInfoString(&q, + "SET enable_seqscan = off; SET enable_bitmapscan = off; " + "DROP TABLE IF EXISTS sc_res; " + "CREATE TEMP TABLE sc_res AS "); + switch (shape) + { + case 0: /* plain KNN */ + appendStringInfoString(&q, + "SELECT ctid AS c, tableoid AS toid, xmin::text AS xm, id, v " + "FROM sc_t ORDER BY "); + sc_orderby(tc, &q); + appendStringInfo(&q, " LIMIT %d", lim); + break; + case 1: /* KNN with a qual the index does not evaluate */ + appendStringInfoString(&q, + "SELECT ctid AS c, tableoid AS toid, xmin::text AS xm, id, v " + "FROM sc_t WHERE v >= -3 ORDER BY "); + sc_orderby(tc, &q); + appendStringInfo(&q, " LIMIT %d", lim); + break; + case 2: /* KNN under a subquery, system columns projected through */ + appendStringInfoString(&q, + "SELECT c, toid, xm, id, v FROM (SELECT ctid AS c, tableoid AS toid, " + "xmin::text AS xm, id, v FROM sc_t ORDER BY "); + sc_orderby(tc, &q); + appendStringInfo(&q, " LIMIT %d) s", lim); + break; + default: /* KNN in a materialized CTE */ + appendStringInfoString(&q, + "WITH k AS MATERIALIZED (SELECT ctid AS c, tableoid AS toid, " + "xmin::text AS xm, id, v FROM sc_t ORDER BY "); + sc_orderby(tc, &q); + appendStringInfo(&q, " LIMIT %d) SELECT * FROM k", lim); + break; + } + appendStringInfoString(&q, "; RESET enable_seqscan; RESET enable_bitmapscan"); + sc_exec(tc, conn, q.data); + sc_verify(tc, conn, "a KNN index scan"); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The lock path: SELECT ... ORDER BY <-> LIMIT n FOR . The rows + * must be lockable (the query must succeed) and must be the rows they say. + */ +PG_PROP(knn_row_lock_locks_the_returned_rows) +{ + static const char *const strength[] = { + "FOR UPDATE", "FOR NO KEY UPDATE", "FOR SHARE", "FOR KEY SHARE" + }; + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + const char *st; + int lim; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + sc_build(tc, conn); + + st = strength[pg_prop_draw_int(tc, 0, lengthof(strength) - 1)]; + lim = (int) pg_prop_draw_int(tc, 1, 40); + + initStringInfo(&q); + appendStringInfoString(&q, + "BEGIN; SET LOCAL enable_seqscan = off; SET LOCAL enable_bitmapscan = off; " + "DROP TABLE IF EXISTS sc_res; " + "CREATE TEMP TABLE sc_res AS " + "SELECT ctid AS c, tableoid AS toid, xmin::text AS xm, id, v " + "FROM sc_t ORDER BY "); + sc_orderby(tc, &q); + appendStringInfo(&q, " LIMIT %d %s", lim, st); + + if (!pg_prop_sql_exec(conn, q.data, &err)) + { + pg_prop_sql_exec(conn, "ROLLBACK", NULL); + pg_prop_fail(tc, + "row-locking KNN query failed: %s\n SQL: %s", + err ? err : "", q.data); + } + sc_exec(tc, conn, "COMMIT"); + sc_verify(tc, conn, "a row-locking KNN index scan"); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The write path: UPDATE / DELETE whose target rows come from a KNN subquery, + * RETURNING system columns, and INSERT ... RETURNING through a BEFORE ROW + * trigger that rewrites NEW (the trigger code stores the modified tuple with + * ExecForceStoreHeapTuple). RETURNING ctid must name the new (or, for DELETE, + * the just-deleted) version. + */ +PG_PROP(dml_returning_syscols_identify_row) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + int kind; + int lim; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + sc_build(tc, conn); + + kind = (int) pg_prop_draw_int(tc, 0, 2); + lim = (int) pg_prop_draw_int(tc, 1, 40); + + initStringInfo(&q); + if (kind == 2) + { + /* trigger that modifies NEW: INSERT ... RETURNING */ + sc_exec(tc, conn, + "CREATE OR REPLACE FUNCTION sc_bump() RETURNS trigger LANGUAGE plpgsql AS " + "$$ BEGIN NEW.v := coalesce(NEW.v, 0) + 100; RETURN NEW; END $$; " + "CREATE TRIGGER sc_bump BEFORE INSERT OR UPDATE ON sc_t " + "FOR EACH ROW EXECUTE FUNCTION sc_bump()"); + appendStringInfo(&q, + "DROP TABLE IF EXISTS sc_res; CREATE TEMP TABLE sc_res " + "(c tid, toid oid, xm text, id int, v int); " + "WITH ins AS (INSERT INTO sc_t SELECT id + 100000, g, v FROM sc_t " + "ORDER BY id LIMIT %d " + "RETURNING ctid, tableoid, xmin::text, id, v) " + "INSERT INTO sc_res SELECT * FROM ins", lim); + } + else + { + appendStringInfoString(&q, + "SET enable_seqscan = off; SET enable_bitmapscan = off; " + "DROP TABLE IF EXISTS sc_res; CREATE TEMP TABLE sc_res " + "(c tid, toid oid, xm text, id int, v int); "); + appendStringInfo(&q, + "WITH m AS (%s sc_t %s WHERE id IN (SELECT id FROM sc_t ORDER BY ", + kind == 0 ? "UPDATE" : "DELETE FROM", + kind == 0 ? "SET v = v + 1" : ""); + sc_orderby(tc, &q); + appendStringInfo(&q, + " LIMIT %d) RETURNING ctid, tableoid, xmin::text, id, v) " + "INSERT INTO sc_res SELECT * FROM m; " + "RESET enable_seqscan; RESET enable_bitmapscan", lim); + } + sc_exec(tc, conn, q.data); + + if (kind == 1) + { + /* + * DELETE: the returned ctids named rows that are now gone, so check + * only that they were valid and distinct (each deleted version once). + */ + long bad = sc_count(tc, conn, + "SELECT count(*) FROM sc_res WHERE c IS NULL " + "OR c::text LIKE '(4294967295,%'"); + long dup = sc_count(tc, conn, + "SELECT count(*) - count(DISTINCT c) FROM sc_res"); + + if (bad || dup) + pg_prop_fail(tc, "DELETE ... RETURNING ctid: %ld invalid, %ld duplicate", bad, dup); + } + else + sc_verify(tc, conn, kind == 0 ? "UPDATE ... RETURNING" : "INSERT ... RETURNING via a modifying trigger"); + + sc_exec(tc, conn, "DROP TRIGGER IF EXISTS sc_bump ON sc_t"); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Tuple routing into a postgres_fdw foreign partition, RETURNING ctid. A + * foreign-table partition whose columns match the parent gets a virtual + * result slot, and the remote row comes back through ExecForceStoreHeapTuple() + * -- whose virtual-slot branch also never set tts_tid (the second path in CF + * 7371). The returned ctid must name the row on the remote side. + * + * The "remote" is a loopback server in the same cluster. Partitions are drawn: + * some foreign (matching or permuted column order, which forces a conversion + * map and a heap slot instead), some local; rows are routed by a drawn key. + */ +PG_PROP(fdw_routed_insert_returning_ctid) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData s; + int nparts; + int nrows; + long bad; + int kinds[4]; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + if (!pg_prop_require_extension(conn, "postgres_fdw")) + pg_prop_skip(tc, "postgres_fdw not installed"); + + nparts = (int) pg_prop_draw_int(tc, 1, 4); + nrows = (int) pg_prop_draw_int(tc, 1, 40); + + initStringInfo(&s); + appendStringInfoString(&s, + "DROP TABLE IF EXISTS sc_par CASCADE; " + "DROP SERVER IF EXISTS sc_loop CASCADE; " + "DO $$ BEGIN EXECUTE format('CREATE SERVER sc_loop FOREIGN DATA WRAPPER postgres_fdw " + "OPTIONS (dbname %L, port %L)', current_database(), current_setting('port')); END $$; " + "DO $$ BEGIN EXECUTE format('CREATE USER MAPPING FOR CURRENT_USER SERVER sc_loop OPTIONS (user %L)', current_user); END $$; " + "CREATE TABLE sc_par (a int, b text) PARTITION BY LIST (a); "); + for (int i = 0; i < nparts; i++) + { + int kind = (int) pg_prop_draw_int(tc, 0, 2); + + kinds[i] = kind; + appendStringInfo(&s, "DROP TABLE IF EXISTS sc_rem%d; ", i); + if (kind == 2) + appendStringInfo(&s, "CREATE TABLE sc_par%d PARTITION OF sc_par FOR VALUES IN (%d); ", i, i); + else + { + /* kind 0: same column order (virtual slot); 1: permuted (conversion map) */ + appendStringInfo(&s, "CREATE TABLE sc_rem%d (%s); ", i, + kind == 0 ? "a int, b text" : "b text, a int"); + appendStringInfo(&s, + "CREATE FOREIGN TABLE sc_par%d PARTITION OF sc_par FOR VALUES IN (%d) " + "SERVER sc_loop OPTIONS (table_name 'sc_rem%d'); ", i, i, i); + } + } + /* + * Commit the DDL before the insert: the loopback connection is a separate + * session and cannot see uncommitted remote tables. (A multi-statement + * string is one implicit transaction.) + */ + sc_exec(tc, conn, s.data); + resetStringInfo(&s); + appendStringInfoString(&s, + "DROP TABLE IF EXISTS sc_res; CREATE TEMP TABLE sc_res (a int, b text, c tid, foreign_part bool); " + "WITH ins AS (INSERT INTO sc_par VALUES "); + for (int r = 0; r < nrows; r++) + appendStringInfo(&s, "%s(%d, 'r%d')", r ? ", " : "", + (int) pg_prop_draw_int(tc, 0, nparts - 1), r); + appendStringInfoString(&s, + " RETURNING a, b, ctid, tableoid) " + "INSERT INTO sc_res SELECT a, b, ctid, " + "(SELECT relkind = 'f' FROM pg_class WHERE oid = ins.tableoid) FROM ins"); + if (!pg_prop_sql_exec(conn, s.data, &err)) + pg_prop_fail(tc, "routed insert failed: %s", err ? err : ""); + + /* + * Every returned ctid must name the row: in the remote table for a foreign + * partition, in the partition itself for a local one. + */ + resetStringInfo(&s); + appendStringInfoString(&s, "SELECT count(*) FROM sc_res r WHERE NOT ("); + for (int i = 0; i < nparts; i++) + { + if (i > 0) + appendStringInfoString(&s, " OR "); + if (kinds[i] == 2) + appendStringInfo(&s, + "(r.a = %d AND EXISTS (SELECT 1 FROM sc_par%d x " + "WHERE x.ctid = r.c AND x.b = r.b))", i, i); + else + appendStringInfo(&s, + "(r.a = %d AND EXISTS (SELECT 1 FROM sc_rem%d x " + "WHERE x.ctid = r.c AND x.b = r.b))", i, i); + } + appendStringInfoChar(&s, ')'); + bad = sc_count(tc, conn, s.data); + if (bad) + { + bool failed = false; + char *sample = pg_prop_sql_rows(conn, + "SELECT a, b, c, foreign_part FROM sc_res ORDER BY b LIMIT 8", + &failed, NULL); + + pg_prop_fail(tc, "routed INSERT ... RETURNING ctid: %ld of %d rows report a ctid " + "that does not name them\n sample (a, b, ctid, foreign):\n%s", + bad, nrows, sample ? sample : ""); + } + + sc_exec(tc, conn, "DROP TABLE sc_par CASCADE; DROP SERVER sc_loop CASCADE"); + for (int i = 0; i < nparts; i++) + { + resetStringInfo(&s); + appendStringInfo(&s, "DROP TABLE IF EXISTS sc_rem%d", i); + sc_exec(tc, conn, s.data); + } + pfree(s.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 311b9d0a539a7..c697f896d39e1 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model syscol index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 0816e395511f95fdae5c7674ee007785b3da925d Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 17 Oct 2026 15:32:12 -0400 Subject: [PATCH 303/338] prop: check case-insensitive matching across encodings and locales (collation) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every other area runs in the default UTF-8 database over ASCII text, but collation code has one implementation per (encoding, ctype, provider), with hand-written fast paths: ILIKE in a single-byte C-ctype database never calls lower() and folds case byte by byte inside like_match.c. A bug in one path is invisible from all the others. Each case picks one of eight database flavours -- LATIN1 / LATIN9 / WIN1252 / SQL_ASCII with LC_CTYPE=C, UTF-8 under libc C, builtin C.UTF-8 and ICU, and LATIN1 under a libc locale -- creating it on first use and skipping it when the host lacks the locale. Text is built server-side from byte values with chr(), so no client encoding conversion intervenes; half the bytes are high-bit (letters, case pairs and caseless letters in LATIN1). Patterns are drawn by shape (%lit%, %lit, lit%, the text with an _ and its case swapped, free form) from substrings of the text, because independent per-byte draws almost never reach the matcher's special cases. Properties, each a definitional identity: ilike_equals_lower_like s ILIKE p = lower(s) LIKE lower(p) regex_icase_equals_lower_regex s ~* r = lower(s) ~ lower(r) ilike_invariant_under_pattern_case like_contains_equals_position position(p in s) > 0 = s LIKE '%p%' collate_c_matches_c_database C-ctype database vs COLLATE "C" ilike_equals_lower_like finds the single-byte C-locale ILIKE bug on x86-64 upstream master in 10 of 10 seeds, shrunk to 'Æ' ILIKE '%Æ%' (LATIN1, C): like_match.c stores pg_ascii_tolower()'s unsigned result in a plain char firstpat, which sign-extends on signed-char platforms and never equals the folded text byte. On aarch64, where char is unsigned, everything passes. --- src/test/prop/Makefile | 1 + src/test/prop/meson.build | 1 + src/test/prop/sql/prop_collation.c | 580 +++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 583 insertions(+), 1 deletion(-) create mode 100644 src/test/prop/sql/prop_collation.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 739e760ede130..69afd76f8ad5f 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -284,6 +284,7 @@ OBJS = \ sql/prop_query_diff.o \ sql/prop_dml_model.o \ sql/prop_syscol.o \ + sql/prop_collation.o \ sql/index_traits.o \ sql/prop_index_am.o \ sql/prop_table_am.o \ diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 4ad44ddc12a04..b03508b6790d6 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -289,6 +289,7 @@ prop_query_sources = files( 'sql/prop_query_diff.c', 'sql/prop_dml_model.c', 'sql/prop_syscol.c', + 'sql/prop_collation.c', 'sql/index_traits.c', 'sql/prop_index_am.c', 'sql/prop_table_am.c', diff --git a/src/test/prop/sql/prop_collation.c b/src/test/prop/sql/prop_collation.c new file mode 100644 index 0000000000000..bd632e54b45a7 --- /dev/null +++ b/src/test/prop/sql/prop_collation.c @@ -0,0 +1,580 @@ +/*------------------------------------------------------------------------- + * + * prop_collation.c + * Case-insensitive and collation-dependent operators across encodings, + * locales and collations. + * + * The rest of the suite runs every query in the default database (UTF-8, + * whatever locale initdb picked) over ASCII text. Collation code, though, + * has one implementation per (encoding, ctype, provider) combination, with + * hand-written fast paths for some of them -- ILIKE in a C-ctype single-byte + * database never calls lower() at all, but folds case byte by byte inside + * like_match.c. A bug in one path is invisible from every other. + * + * So each case here builds its own databases, one per drawn (encoding, + * locale) pair, and draws text from the bytes that differ between paths: + * ASCII letters, high-bit bytes (which are letters in LATIN1/WIN1252 and + * case-pair with each other: 0xC6/0xE6 is AE/ae), and the LIKE metacharacters. + * Text is built server-side from byte values (chr() in single-byte encodings, + * so no client encoding conversion can intervene), and every check is a + * definitional identity that must hold for any encoding and collation: + * + * s ILIKE p = lower(s) LIKE lower(p) (ILIKE's definition) + * s ~* r = lower(s) ~ lower(r) for a literal r + * s ILIKE p = s ILIKE p with p's literal characters case-swapped + * s LIKE p is decided by the same rows under COLLATE "C" and the + * database default when the collation is deterministic + * position(p in s) > 0 = s LIKE '%' || escaped(p) || '%' + * upper(lower(s)) = upper(s) and lower(upper(s)) = lower(s) + * (holds for C ctype; checked only there) + * + * The first identity is exactly what the documentation says ILIKE is. It + * finds the single-byte C-locale ILIKE bug ('xÆy' ILIKE '%Æ%' is false in a + * LATIN1 database with LC_CTYPE=C on a signed-char platform): like_match.c + * stores the folded first pattern byte in a plain char, so a high-bit byte + * sign-extends and never equals the unsigned folded text byte. + * + * Portions Copyright (c) 2024, PostgreSQL Global Development Group + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "collation" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* + * Database flavours. Each is (name suffix, encoding, locale clause). A + * flavour whose locale is not installed on the host is skipped at run time. + */ +typedef struct CollDb +{ + const char *suffix; + const char *encoding; + const char *locale_sql; /* appended to CREATE DATABASE */ + bool single_byte; + bool c_ctype; +} CollDb; + +static const CollDb coll_dbs[] = { + {"latin1_c", "LATIN1", "LOCALE 'C'", true, true}, + {"latin9_c", "LATIN9", "LOCALE 'C'", true, true}, + {"win1252_c", "WIN1252", "LOCALE 'C'", true, true}, + {"sqlascii_c", "SQL_ASCII", "LOCALE 'C'", true, true}, + {"utf8_c", "UTF8", "LOCALE 'C'", false, true}, + {"utf8_builtin", "UTF8", "LOCALE_PROVIDER builtin BUILTIN_LOCALE 'C.UTF-8'", false, false}, + {"utf8_icu", "UTF8", "LOCALE_PROVIDER icu ICU_LOCALE 'en' LOCALE 'C'", false, false}, + {"latin1_libc", "LATIN1", "LOCALE 'en_US.ISO8859-1'", true, false}, +}; +#define N_COLL_DBS ((int) lengthof(coll_dbs)) + +/* + * Bytes text is drawn from. For single-byte databases the high-bit bytes are + * real characters; for UTF-8 databases each entry is rendered as the code + * point of the same value (chr(198) is U+00C6 there too), so the same drawn + * string means the same characters in every flavour. + */ +static const unsigned char coll_alphabet[] = { + 'a', 'b', 'x', 'A', 'B', 'X', 'z', 'Z', '0', ' ', + 0xC6, 0xE6, /* AE ae */ + 0xC9, 0xE9, /* E-acute e-acute */ + 0xD0, 0xF0, /* ETH eth */ + 0xDF, /* sharp s (no single-char upper) */ + 0xFF, /* y-diaeresis (upper is outside LATIN1) */ + 0xB5, /* micro sign */ + 0xAA, 0xBA, /* ordinal indicators (letters, no case) */ + 0xC0, 0xE0, 0xD8, 0xF8, 0xDE, 0xFE, +}; +#define N_ALPHA ((int) lengthof(coll_alphabet)) + +/* Open (creating if needed) the database for a flavour; NULL if unavailable. */ +static pg_prop_conn * +coll_connect(pg_prop_tc *tc, const CollDb *d) +{ + char *err = NULL; + pg_prop_conn *admin; + pg_prop_conn *conn; + char dbname[64]; + const char *saved; + StringInfoData q; + bool failed = false; + char *exists; + + snprintf(dbname, sizeof(dbname), "pgprop_coll_%s", d->suffix); + + admin = pg_prop_connect(&err); + if (admin == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + initStringInfo(&q); + appendStringInfo(&q, "SELECT count(*) FROM pg_database WHERE datname = '%s'", dbname); + exists = pg_prop_sql_scalar(admin, q.data, &failed, NULL); + if (!failed && exists && strcmp(exists, "0") == 0) + { + resetStringInfo(&q); + appendStringInfo(&q, "CREATE DATABASE %s ENCODING '%s' %s TEMPLATE template0", + dbname, d->encoding, d->locale_sql); + if (!pg_prop_sql_exec(admin, q.data, &err)) + { + /* locale not installed on this host, or provider unavailable */ + pg_free(err); + pg_free(exists); + pfree(q.data); + pg_prop_disconnect(admin); + return NULL; + } + } + if (exists) + pg_free(exists); + pfree(q.data); + pg_prop_disconnect(admin); + + saved = getenv("PGDATABASE"); + setenv("PGDATABASE", dbname, 1); + conn = pg_prop_connect(&err); + if (saved) + setenv("PGDATABASE", saved, 1); + else + unsetenv("PGDATABASE"); + if (conn == NULL) + pg_prop_fail(tc, "connect to %s failed: %s", dbname, err ? err : ""); + return conn; +} + +/* append chr(b) || ... for a drawn byte string; '' when empty */ +static void +coll_text_sql(StringInfo b, const unsigned char *s, int n) +{ + if (n == 0) + { + appendStringInfoString(b, "''"); + return; + } + appendStringInfoChar(b, '('); + for (int i = 0; i < n; i++) + appendStringInfo(b, "%schr(%d)", i ? "||" : "", s[i]); + appendStringInfoChar(b, ')'); +} + +/* + * Draw n bytes from the alphabet; with wildcards, also '%' and '_'. About + * half the bytes are high-bit (the half of the alphabet where the encodings + * and ctypes disagree), and a pattern starts with '%' a third of the time, + * because a leading '%' is what sends like_match.c to its "scan for the first + * literal" loop -- the code that differs most between the matcher variants. + */ +static int +coll_draw(pg_prop_tc *tc, unsigned char *out, int maxn, bool wildcards) +{ + int n = (int) pg_prop_draw_int(tc, 0, maxn); + int i = 0; + + if (wildcards && n > 0 && pg_prop_draw_int(tc, 0, 2) == 0) + out[i++] = '%'; + for (; i < n; i++) + { + if (wildcards && pg_prop_draw_int(tc, 0, 4) == 0) + out[i] = pg_prop_draw_bool(tc) ? '%' : '_'; + else if (pg_prop_draw_bool(tc)) + out[i] = coll_alphabet[pg_prop_draw_int(tc, 10, N_ALPHA - 1)]; /* high-bit */ + else + out[i] = coll_alphabet[pg_prop_draw_int(tc, 0, 9)]; /* ASCII */ + } + return n; +} + +/* + * Draw a LIKE pattern by shape rather than byte by byte. An unstructured + * draw almost never produces the combinations the matchers special-case, so + * the shape is chosen first: + * + * 0 '%' || lit || '%' lit is a substring of s, maybe case-swapped + * 1 '%' || lit the "scan for the first literal" loop + * 2 lit || '%' + * 3 s with one char replaced by '_' and its case swapped + * 4 s case-swapped (whole-string match) + * 5 free-form (the old byte-by-byte draw) + * + * "Case-swapped" maps each byte to its LATIN1 case partner (A<->a, and + * 0xC0-0xDE <-> 0xE0-0xFE except the multiplication/division signs), which + * is the pair ILIKE must treat as equal in a single-byte database. + */ +static unsigned char +coll_swapcase(unsigned char c) +{ + if (c >= 'A' && c <= 'Z') + return c + 32; + if (c >= 'a' && c <= 'z') + return c - 32; + if (c >= 0xC0 && c <= 0xDE && c != 0xD7) + return c + 32; + if (c >= 0xE0 && c <= 0xFE && c != 0xF7) + return c - 32; + return c; +} + +static int +coll_draw_pattern(pg_prop_tc *tc, unsigned char *p, int maxn, + const unsigned char *s, int sn) +{ + int shape = (int) pg_prop_draw_int(tc, 0, 5); + int n = 0; + int from; + int len; + bool swap = pg_prop_draw_bool(tc); + + if (sn == 0 && shape != 5) + shape = 5; + switch (shape) + { + case 0: + case 1: + case 2: + from = (int) pg_prop_draw_int(tc, 0, sn - 1); + len = (int) pg_prop_draw_int(tc, 1, Min(sn - from, maxn - 2)); + if (shape != 2) + p[n++] = '%'; + for (int i = 0; i < len; i++) + p[n++] = swap ? coll_swapcase(s[from + i]) : s[from + i]; + if (shape != 1) + p[n++] = '%'; + return n; + case 3: + case 4: + len = Min(sn, maxn); + for (int i = 0; i < len; i++) + p[n++] = coll_swapcase(s[i]); + if (shape == 3 && len > 0) + p[pg_prop_draw_int(tc, 0, len - 1)] = '_'; + return n; + default: + return coll_draw(tc, p, maxn, true); + } +} + +/* + * Single-byte C-ctype databases get half the draws: they are the only ones + * with a hand-written case-folding matcher (C_IMatchText), and four of the + * eight flavours. + */ +static const CollDb * +coll_pick(pg_prop_tc *tc) +{ + if (pg_prop_draw_bool(tc)) + return &coll_dbs[pg_prop_draw_int(tc, 0, 3)]; + return &coll_dbs[pg_prop_draw_int(tc, 0, N_COLL_DBS - 1)]; +} + +static bool +coll_bool(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql, const char *ctx) +{ + bool failed = false; + char *err = NULL; + char *v = pg_prop_sql_scalar(conn, sql, &failed, &err); + bool r; + + if (failed || v == NULL) + pg_prop_fail(tc, "%s: query failed: %s\n SQL: %s", ctx, err ? err : "", sql); + r = (strcmp(v, "t") == 0); + pg_free(v); + return r; +} + +/* describe the drawn bytes for a failure message */ +static char * +coll_hex(const unsigned char *s, int n) +{ + StringInfoData b; + + initStringInfo(&b); + for (int i = 0; i < n; i++) + appendStringInfo(&b, "%s%02X", i ? " " : "", s[i]); + return b.data; +} + +/* + * s ILIKE p == lower(s) LIKE lower(p) + * + * This is ILIKE's documented definition. Both sides are computed by the + * server under the same database collation; any disagreement means the ILIKE + * implementation for this (encoding, ctype) path is wrong. + */ +PG_PROP(ilike_equals_lower_like) +{ + const CollDb *d = coll_pick(tc); + pg_prop_conn *conn = coll_connect(tc, d); + unsigned char s[12]; + unsigned char p[8]; + int sn; + int pn; + StringInfoData q; + bool a; + bool b; + + if (conn == NULL) + pg_prop_skip(tc, "database flavour %s not available here", d->suffix); + + sn = coll_draw(tc, s, sizeof(s), false); + pn = coll_draw_pattern(tc, p, sizeof(p), s, sn); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + coll_text_sql(&q, s, sn); + appendStringInfoString(&q, " ILIKE "); + coll_text_sql(&q, p, pn); + a = coll_bool(tc, conn, q.data, "ILIKE"); + + resetStringInfo(&q); + appendStringInfoString(&q, "SELECT lower("); + coll_text_sql(&q, s, sn); + appendStringInfoString(&q, ") LIKE lower("); + coll_text_sql(&q, p, pn); + appendStringInfoChar(&q, ')'); + b = coll_bool(tc, conn, q.data, "lower LIKE lower"); + + if (a != b) + pg_prop_fail(tc, + "s ILIKE p = %s but lower(s) LIKE lower(p) = %s\n" + " database: %s (encoding %s, %s)\n" + " s bytes: [%s]\n p bytes: [%s]", + a ? "true" : "false", b ? "true" : "false", + d->suffix, d->encoding, d->locale_sql, + coll_hex(s, sn), coll_hex(p, pn)); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * s ~* r == lower(s) ~ lower(r), for r a literal (no regex metacharacters: + * the alphabet has none). Same definition, the regex engine's own + * case-insensitive path. + */ +PG_PROP(regex_icase_equals_lower_regex) +{ + const CollDb *d = coll_pick(tc); + pg_prop_conn *conn = coll_connect(tc, d); + unsigned char s[12]; + unsigned char r[6]; + int sn; + int rn; + StringInfoData q; + bool a; + bool b; + + if (conn == NULL) + pg_prop_skip(tc, "database flavour %s not available here", d->suffix); + + sn = coll_draw(tc, s, sizeof(s), false); + rn = coll_draw(tc, r, sizeof(r), false); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + coll_text_sql(&q, s, sn); + appendStringInfoString(&q, " ~* "); + coll_text_sql(&q, r, rn); + a = coll_bool(tc, conn, q.data, "~*"); + + resetStringInfo(&q); + appendStringInfoString(&q, "SELECT lower("); + coll_text_sql(&q, s, sn); + appendStringInfoString(&q, ") ~ lower("); + coll_text_sql(&q, r, rn); + appendStringInfoChar(&q, ')'); + b = coll_bool(tc, conn, q.data, "lower ~ lower"); + + if (a != b) + pg_prop_fail(tc, + "s ~* r = %s but lower(s) ~ lower(r) = %s\n" + " database: %s (encoding %s, %s)\n" + " s bytes: [%s]\n r bytes: [%s]", + a ? "true" : "false", b ? "true" : "false", + d->suffix, d->encoding, d->locale_sql, + coll_hex(s, sn), coll_hex(r, rn)); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * ILIKE is invariant under changing the case of the pattern: s ILIKE p == + * s ILIKE upper(p) == s ILIKE lower(p). (Fold-equivalent patterns must + * match the same strings.) Independent of lower()'s own correctness. + */ +PG_PROP(ilike_invariant_under_pattern_case) +{ + const CollDb *d = coll_pick(tc); + pg_prop_conn *conn = coll_connect(tc, d); + unsigned char s[12]; + unsigned char p[8]; + int sn; + int pn; + StringInfoData q; + bool a; + bool up; + bool lo; + + if (conn == NULL) + pg_prop_skip(tc, "database flavour %s not available here", d->suffix); + + sn = coll_draw(tc, s, sizeof(s), false); + pn = coll_draw_pattern(tc, p, sizeof(p), s, sn); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + coll_text_sql(&q, s, sn); + appendStringInfoString(&q, " ILIKE "); + coll_text_sql(&q, p, pn); + a = coll_bool(tc, conn, q.data, "ILIKE p"); + + resetStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + coll_text_sql(&q, s, sn); + appendStringInfoString(&q, " ILIKE upper("); + coll_text_sql(&q, p, pn); + appendStringInfoChar(&q, ')'); + up = coll_bool(tc, conn, q.data, "ILIKE upper(p)"); + + resetStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + coll_text_sql(&q, s, sn); + appendStringInfoString(&q, " ILIKE lower("); + coll_text_sql(&q, p, pn); + appendStringInfoChar(&q, ')'); + lo = coll_bool(tc, conn, q.data, "ILIKE lower(p)"); + + /* + * Only fold-preserving pairs are required to agree: if upper() maps a + * character to something lower() does not map back (sharp s, y-diaeresis + * in LATIN1), the patterns are not fold-equivalent. Restrict to the + * case where lower(upper(p)) = lower(p). + */ + resetStringInfo(&q); + appendStringInfoString(&q, "SELECT lower(upper("); + coll_text_sql(&q, p, pn); + appendStringInfoString(&q, ")) = lower("); + coll_text_sql(&q, p, pn); + appendStringInfoChar(&q, ')'); + if (coll_bool(tc, conn, q.data, "fold check") && (a != up || a != lo)) + pg_prop_fail(tc, + "ILIKE depends on the pattern's case: p=%s upper(p)=%s lower(p)=%s\n" + " database: %s (encoding %s, %s)\n" + " s bytes: [%s]\n p bytes: [%s]", + a ? "t" : "f", up ? "t" : "f", lo ? "t" : "f", + d->suffix, d->encoding, d->locale_sql, + coll_hex(s, sn), coll_hex(p, pn)); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Containment, three ways: position(p in s) > 0, strpos(s, p) > 0, and + * s LIKE '%' || p || '%' (p drawn without wildcards) must agree under a + * deterministic collation. + */ +PG_PROP(like_contains_equals_position) +{ + const CollDb *d = coll_pick(tc); + pg_prop_conn *conn = coll_connect(tc, d); + unsigned char s[12]; + unsigned char p[5]; + int sn; + int pn; + StringInfoData q; + bool pos; + bool lk; + + if (conn == NULL) + pg_prop_skip(tc, "database flavour %s not available here", d->suffix); + + sn = coll_draw(tc, s, sizeof(s), false); + pn = coll_draw(tc, p, sizeof(p), false); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT position("); + coll_text_sql(&q, p, pn); + appendStringInfoString(&q, " in "); + coll_text_sql(&q, s, sn); + appendStringInfoString(&q, ") > 0 OR length("); + coll_text_sql(&q, p, pn); + appendStringInfoString(&q, ") = 0"); + pos = coll_bool(tc, conn, q.data, "position"); + + resetStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + coll_text_sql(&q, s, sn); + appendStringInfoString(&q, " LIKE ('%' || "); + coll_text_sql(&q, p, pn); + appendStringInfoString(&q, " || '%')"); + lk = coll_bool(tc, conn, q.data, "LIKE %p%"); + + if (pos != lk) + pg_prop_fail(tc, + "position(p in s) > 0 = %s but s LIKE '%%p%%' = %s\n" + " database: %s (encoding %s, %s)\n" + " s bytes: [%s]\n p bytes: [%s]", + pos ? "true" : "false", lk ? "true" : "false", + d->suffix, d->encoding, d->locale_sql, + coll_hex(s, sn), coll_hex(p, pn)); + + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * The case-insensitive operators must not depend on whether the comparison + * runs in a C-ctype database or in a non-C database under COLLATE "C": both + * are ctype C. (Two different code paths in like.c / regc_pg_locale.c.) + * Bytes are restricted to what both databases can represent: the + * single-byte C database and a single-byte libc database of the same + * encoding. + */ +PG_PROP(collate_c_matches_c_database) +{ + pg_prop_conn *cdb; + pg_prop_conn *ldb; + unsigned char s[12]; + unsigned char p[8]; + int sn; + int pn; + StringInfoData q; + bool a; + bool b; + bool like = pg_prop_draw_bool(tc); + + cdb = coll_connect(tc, &coll_dbs[0]); /* latin1_c */ + ldb = coll_connect(tc, &coll_dbs[N_COLL_DBS - 1]); /* latin1_libc */ + if (cdb == NULL || ldb == NULL) + { + if (cdb) + pg_prop_disconnect(cdb); + if (ldb) + pg_prop_disconnect(ldb); + pg_prop_skip(tc, "a LATIN1 libc locale is not available here"); + } + + sn = coll_draw(tc, s, sizeof(s), false); + pn = coll_draw(tc, p, sizeof(p), like); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT "); + coll_text_sql(&q, s, sn); + appendStringInfoString(&q, like ? " ILIKE " : " ~* "); + coll_text_sql(&q, p, pn); + a = coll_bool(tc, cdb, q.data, "C database"); + appendStringInfoString(&q, " COLLATE \"C\""); + b = coll_bool(tc, ldb, q.data, "COLLATE C"); + + if (a != b) + pg_prop_fail(tc, + "%s in a C-ctype LATIN1 database = %s, but under COLLATE \"C\" " + "in a LATIN1 libc database = %s\n s bytes: [%s]\n p bytes: [%s]", + like ? "ILIKE" : "~*", a ? "true" : "false", b ? "true" : "false", + coll_hex(s, sn), coll_hex(p, pn)); + + pfree(q.data); + pg_prop_disconnect(cdb); + pg_prop_disconnect(ldb); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index c697f896d39e1..82aa0a7584e58 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model syscol index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model syscol collation index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From f7d81276f6a3f318547bd3fa8cff3254d089bd69 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 17 Oct 2026 16:00:20 -0400 Subject: [PATCH 304/338] prop: do not drop failure messages longer than the initial buffer pg_prop_fail, pg_prop_note and pg_prop_skip formatted their message with a single appendStringInfoVA() call and ignored its return value. When the text does not fit in the StringInfo's free space (about 1 kB at first), appendStringInfoVA appends nothing and returns the size it needs, so the message stayed empty and the report printed "(counterexample did not reproduce)" for a case that failed deterministically. Properties whose failure includes a statement history or row sample hit this. Use the standard enlarge-and-retry loop. --- src/test/prop/pg_prop.c | 42 ++++++++++++++++++++++++++++++++--------- 1 file changed, 33 insertions(+), 9 deletions(-) diff --git a/src/test/prop/pg_prop.c b/src/test/prop/pg_prop.c index dff2d79ca7425..e9ad992a63cfe 100644 --- a/src/test/prop/pg_prop.c +++ b/src/test/prop/pg_prop.c @@ -386,9 +386,17 @@ pg_prop_note(pg_prop_tc *tc, const char *fmt,...) StringInfoData s; initStringInfo(&s); - va_start(ap, fmt); - appendStringInfoVA(&s, fmt, ap); - va_end(ap); + for (;;) + { + int needed; + + va_start(ap, fmt); + needed = appendStringInfoVA(&s, fmt, ap); + va_end(ap); + if (needed == 0) + break; + enlargeStringInfo(&s, needed); + } hegel_note(tc->ctx, tc->tc, (const unsigned char *) s.data, s.len); pfree(s.data); } @@ -399,9 +407,17 @@ pg_prop_fail(pg_prop_tc *tc, const char *fmt,...) va_list ap; resetStringInfo(&tc->failmsg); - va_start(ap, fmt); - appendStringInfoVA(&tc->failmsg, fmt, ap); - va_end(ap); + for (;;) + { + int needed; + + va_start(ap, fmt); + needed = appendStringInfoVA(&tc->failmsg, fmt, ap); + va_end(ap); + if (needed == 0) + break; + enlargeStringInfo(&tc->failmsg, needed); + } tc->failed = true; siglongjmp(tc->jmp, 1); /* 1 == interesting/failure */ } @@ -417,9 +433,17 @@ pg_prop_skip(pg_prop_tc *tc, const char *fmt,...) skip_reason_inited = true; } resetStringInfo(&skip_reason); - va_start(ap, fmt); - appendStringInfoVA(&skip_reason, fmt, ap); - va_end(ap); + for (;;) + { + int needed; + + va_start(ap, fmt); + needed = appendStringInfoVA(&skip_reason, fmt, ap); + va_end(ap); + if (needed == 0) + break; + enlargeStringInfo(&skip_reason, needed); + } skip_requested = true; siglongjmp(tc->jmp, 3); /* 3 == skip the whole property */ } From d392e41e2b361a240653eca65569ec597e19f006 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 17 Oct 2026 17:00:32 -0400 Subject: [PATCH 305/338] prop: properties for open planner wrong-results fixes (cf_planner) One property per open commitfest bug fix in the planner/executor, each over a drawn input space around the patch's own reproducer, each failing on upstream master 45fc51f: 7175 memoize_buried_param_matches_no_memoize Memoize on vs off 7384 hashed_subplan_rescan_matches_array_any IN (subq) vs = ANY(ARRAY(subq)) 7421 memoize_collation_matches_no_memoize nondeterministic collation keys 7186 jsonexpr_pullup_matches_offset0_fence JsonExpr strictness in pull-up strict_sql_function_with_jsonexpr_returns_null_on_null 7400 on_conflict_infers_index_on_same_expression NOT NULL-based simplification 7448 eager_agg_volatile_clause_runs_per_joined_row volatile clause under eager agg 7466 without_overlaps_pk_grouping_matches_full_grouping All eight falsify within -n 200 on master (seeds noted per property). --- src/test/prop/Makefile | 5 + src/test/prop/meson.build | 5 + src/test/prop/sql/prop_cf_planner.c | 1134 +++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 1145 insertions(+), 1 deletion(-) create mode 100644 src/test/prop/sql/prop_cf_planner.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 69afd76f8ad5f..61fcd4d64ab46 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -285,6 +285,11 @@ OBJS = \ sql/prop_dml_model.o \ sql/prop_syscol.o \ sql/prop_collation.o \ + sql/prop_cf_types.o \ + sql/prop_cf_index.o \ + sql/prop_cf_exec.o \ + sql/prop_cf_ddl2.o \ + sql/prop_cf_planner.o \ sql/index_traits.o \ sql/prop_index_am.o \ sql/prop_table_am.o \ diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index b03508b6790d6..7dabe039f56b2 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -290,6 +290,11 @@ prop_query_sources = files( 'sql/prop_dml_model.c', 'sql/prop_syscol.c', 'sql/prop_collation.c', + 'sql/prop_cf_types.c', + 'sql/prop_cf_index.c', + 'sql/prop_cf_exec.c', + 'sql/prop_cf_ddl2.c', + 'sql/prop_cf_planner.c', 'sql/index_traits.c', 'sql/prop_index_am.c', 'sql/prop_table_am.c', diff --git a/src/test/prop/sql/prop_cf_planner.c b/src/test/prop/sql/prop_cf_planner.c new file mode 100644 index 0000000000000..80519d85098b5 --- /dev/null +++ b/src/test/prop/sql/prop_cf_planner.c @@ -0,0 +1,1134 @@ +/*------------------------------------------------------------------------- + * + * prop_cf_planner.c + * Plan differentials around open commitfest wrong-results bugs. + * + * Every property here draws a small schema and a query shape around the + * trigger of one CF bug fix, then runs the same query under two settings + * that may only change the plan (a planner GUC, an OFFSET 0 fence) and + * requires the result digests to agree. Each run is its own transaction + * (SET LOCAL), the query carries an ORDER BY on a total key, and the + * digest is md5 of the rendered rows. Tables are temp tables, so they + * vanish with the per-case connection even when a case fails. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_cf_planner.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "cf_planner" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* last schema/data setup of this case, printed with a counterexample */ +static const char *cfp_last_setup = ""; + +static pg_prop_conn * +cfp_connect(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + cfp_last_setup = ""; + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + return conn; +} + +static void +cfp_exec(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "setup failed: %s\n SQL: %s", err ? err : "", sql); +} + +/* schema/data setup: like cfp_exec, and remembered for the failure message */ +static void +cfp_setup(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + cfp_last_setup = sql; + cfp_exec(tc, conn, sql); +} + +/* + * Run "query" in its own transaction after "setup" (SET LOCALs) and return + * the result digest. Any error fails the case: these queries are built so + * that no plan can raise a data-dependent error. + */ +static char * +cfp_digest(pg_prop_tc *tc, pg_prop_conn *conn, const char *setup, + const char *query) +{ + char *err = NULL; + bool failed = false; + char *dig; + + cfp_exec(tc, conn, "BEGIN"); + if (setup[0] != '\0') + cfp_exec(tc, conn, setup); + dig = pg_prop_sql_rowhash(conn, query, &failed, &err); + pg_prop_sql_exec(conn, "ROLLBACK", NULL); + if (failed || dig == NULL) + pg_prop_fail(tc, "query failed: %s\n setup: %s\n query: %s", + err ? err : "", setup, query); + return dig; +} + +/* the first rows of "query" under "setup", for the failure message */ +static char * +cfp_sample(pg_prop_conn *conn, const char *setup, const char *query) +{ + StringInfoData q; + char *rows; + bool failed = false; + + pg_prop_sql_exec(conn, "BEGIN", NULL); + if (setup[0] != '\0') + pg_prop_sql_exec(conn, setup, NULL); + initStringInfo(&q); + appendStringInfo(&q, "SELECT * FROM (%s) s LIMIT 30", query); + rows = pg_prop_sql_rows(conn, q.data, &failed, NULL); + pg_prop_sql_exec(conn, "ROLLBACK", NULL); + pfree(q.data); + return rows ? rows : pg_strdup("(unavailable)"); +} + +/* + * The differential: "qa" under "sa" and "qb" under "sb" must return the same + * rows in the same order. qb may be NULL (same query as qa). + */ +static void +cfp_diff(pg_prop_tc *tc, pg_prop_conn *conn, const char *what, + const char *sa, const char *qa, const char *sb, const char *qb) +{ + char *da; + char *db; + + if (qb == NULL) + qb = qa; + da = cfp_digest(tc, conn, sa, qa); + db = cfp_digest(tc, conn, sb, qb); + if (strcmp(da, db) != 0) + pg_prop_fail(tc, + "%s: two plans of the same query disagree\n" + " data: %s\n" + " A setup: %s\n A query: %s\n A rows:\n%s\n" + " B setup: %s\n B query: %s\n B rows:\n%s", + what, cfp_last_setup, sa, qa, cfp_sample(conn, sa, qa), + sb, qb, cfp_sample(conn, sb, qb)); + free(da); + free(db); +} + +/* nested-loop-only plan shape that makes Memoize the cheap inner path */ +#define CFP_NESTLOOP_ONLY \ + "SET LOCAL enable_seqscan = off; SET LOCAL enable_material = off; " \ + "SET LOCAL enable_mergejoin = off; SET LOCAL enable_hashjoin = off; " + +/* + * CF 7175: Memoize whose cache key is an expression containing an outer + * Param (t2.x + t0.p) while the same Param also filters the inner scan + * elsewhere (t1.y = t0.p). Two (t2.x, t0.p) pairs with the same key value + * select different inner rows, but master keeps the Param in keyparamids and + * never flushes the cache, so one entry serves both. + * + * Drawn: table size, column moduli, the key expression (+, -, *), the shape + * of the extra use of the Param (=, <, <> on another column), and how many + * outer rows drive the correlated subquery. Invariant: enable_memoize only + * changes the plan, so on and off must give the same per-outer-row counts. + */ +PG_PROP(memoize_buried_param_matches_no_memoize) +{ + static const char *const keyop[] = {"+", "-", "*"}; + static const char *const filt[] = {"=", "<", "<>", ">="}; + pg_prop_conn *conn = cfp_connect(tc); + StringInfoData s; + StringInfoData q; + int n = (int) pg_prop_draw_int(tc, 20, 2000); + int ma = (int) pg_prop_draw_int(tc, 1, 40); + int mb = (int) pg_prop_draw_int(tc, 1, 40); + int mc = (int) pg_prop_draw_int(tc, 1, 40); + int nout = (int) pg_prop_draw_int(tc, 2, 200); + const char *op = keyop[pg_prop_draw_int(tc, 0, lengthof(keyop) - 1)]; + const char *fo = filt[pg_prop_draw_int(tc, 0, lengthof(filt) - 1)]; + bool useb = pg_prop_draw_bool(tc); + bool agg = pg_prop_draw_bool(tc); + bool filtb = pg_prop_draw_bool(tc); + + initStringInfo(&s); + appendStringInfo(&s, + "CREATE TEMP TABLE cfa1_m AS SELECT g AS id, g %% %d AS a, " + "g %% %d AS b, (g / 3) %% %d AS c FROM generate_series(0, %d) g; " + "CREATE INDEX ON cfa1_m (id); ANALYZE cfa1_m", + ma, mb, mc, n - 1); + cfp_setup(tc, conn, s.data); + + /* + * Outer: t0 (rows id < nout). Subquery: t2 drives a nested loop whose + * inner side is an index scan on t1.id = with filter + * t1.Q fo t0.P, where P is the outer column buried in the key. + */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT %s FROM (SELECT t0.id, " + "(SELECT count(*) FROM cfa1_m t2 JOIN cfa1_m t1 " + "ON t1.id = t2.a %s t0.%s WHERE t1.%s %s t0.%s) AS c " + "FROM cfa1_m t0 WHERE t0.id < %d) s %s", + agg ? "sum(c)" : "id, c", + op, useb ? "b" : "c", + filtb ? "b" : "c", fo, useb ? "b" : "c", + nout, agg ? "" : "ORDER BY id"); + + cfp_diff(tc, conn, "CF 7175 Memoize with a Param buried in the cache key", + CFP_NESTLOOP_ONLY, q.data, + CFP_NESTLOOP_ONLY "SET LOCAL enable_memoize = off;", NULL); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* append "INSERT INTO VALUES" with n rows of ncol ints in [0, dom], some NULL */ +static void +cfp_rows(pg_prop_tc *tc, StringInfo b, const char *tab, int n, int ncol, + int dom, bool serial_first) +{ + appendStringInfo(b, "INSERT INTO %s VALUES ", tab); + for (int i = 0; i < n; i++) + { + appendStringInfoString(b, i ? ", (" : "("); + for (int c = 0; c < ncol; c++) + { + if (c > 0) + appendStringInfoString(b, ", "); + if (c == 0 && serial_first) + appendStringInfo(b, "%d", i + 1); + else if (pg_prop_draw_int(tc, 0, 9) == 0) + appendStringInfoString(b, "NULL"); + else + appendStringInfo(b, "%d", (int) pg_prop_draw_int(tc, 0, dom)); + } + appendStringInfoChar(b, ')'); + } + appendStringInfoString(b, "; "); +} + +/* + * CF 7384: a hashed SubPlan correlated to an outer query level, inside a join + * OR clause from which extract_restriction_or_clauses() pulls a restriction on + * one side. The same SubPlan then sits in two quals with two SubPlanStates + * sharing one PlanState; the first to rebuild its hash table clears chgParam, + * and the other keeps probing a table built for an earlier outer row. + * + * SELECT s.id, EXISTS (SELECT 1 FROM p JOIN c ON c.id = p.cid + * WHERE p.id = s.pid AND + * (c.owner = s.uid OR (p.pub AND c.K [NOT] IN + * (SELECT f.cid FROM f WHERE f.uid = s.uid)))) + * FROM s ORDER BY s.id + * + * Reference: the same query with "c.K IN (SELECT ...)" written + * "c.K = ANY (ARRAY(SELECT ...))" (NOT IN as "<> ALL (ARRAY(...))"). Both + * are the same ANY/ALL test with the same NULL semantics, but an ARRAY + * sublink is never hashed. Drawn: table contents (small domains, NULLs), + * which c column is tested, IN vs NOT IN, EXISTS vs count(*), the plan + * shape GUCs, and an extra OR arm. + */ +PG_PROP(hashed_subplan_rescan_matches_array_any) +{ + static const char *const setups[] = { + "", + "SET LOCAL enable_hashjoin = off; SET LOCAL enable_mergejoin = off;", + "SET LOCAL enable_nestloop = off;", + "SET LOCAL enable_indexscan = off; SET LOCAL enable_bitmapscan = off;", + }; + pg_prop_conn *conn = cfp_connect(tc); + StringInfoData s; + StringInfoData qa; + StringInfoData qb; + int dom = (int) pg_prop_draw_int(tc, 1, 12); + int nc = (int) pg_prop_draw_int(tc, 1, 12); + int np = (int) pg_prop_draw_int(tc, 1, 20); + int nf = (int) pg_prop_draw_int(tc, 0, 20); + int ns = (int) pg_prop_draw_int(tc, 1, 20); + bool notin = pg_prop_draw_int(tc, 0, 3) == 0; + bool cnt = pg_prop_draw_bool(tc); + bool extra = pg_prop_draw_bool(tc); + const char *key = pg_prop_draw_bool(tc) ? "id" : "owner"; + const char *setup = setups[pg_prop_draw_int(tc, 0, lengthof(setups) - 1)]; + + initStringInfo(&s); + appendStringInfoString(&s, + "CREATE TEMP TABLE cfa1_hc (id int PRIMARY KEY, owner int); " + "CREATE TEMP TABLE cfa1_hp (id int PRIMARY KEY, cid int, pub bool); " + "CREATE TEMP TABLE cfa1_hf (cid int, uid int); " + "CREATE TEMP TABLE cfa1_hs (id int PRIMARY KEY, pid int, uid int); "); + cfp_rows(tc, &s, "cfa1_hc", nc, 2, dom, true); + appendStringInfoString(&s, "INSERT INTO cfa1_hp VALUES "); + for (int i = 0; i < np; i++) + appendStringInfo(&s, "%s(%d, %d, %s)", i ? ", " : "", i + 1, + (int) pg_prop_draw_int(tc, 0, nc + 1), + pg_prop_draw_int(tc, 0, 3) ? "true" : "false"); + appendStringInfoString(&s, "; "); + if (nf > 0) + cfp_rows(tc, &s, "cfa1_hf", nf, 2, dom, false); + cfp_rows(tc, &s, "cfa1_hs", ns, 3, np + 1, true); + appendStringInfoString(&s, "ANALYZE cfa1_hc, cfa1_hp, cfa1_hf, cfa1_hs"); + cfp_setup(tc, conn, s.data); + + initStringInfo(&qa); + initStringInfo(&qb); + for (int v = 0; v < 2; v++) + { + StringInfo q = v == 0 ? &qa : &qb; + + appendStringInfo(q, + "SELECT s.id, %s FROM cfa1_hp p JOIN cfa1_hc c ON c.id = p.cid " + "WHERE p.id = s.pid AND (c.owner = s.uid %s OR (p.pub AND c.%s ", + cnt ? "(SELECT count(*)" : "EXISTS (SELECT 1", + extra ? "OR (NOT p.pub AND c.owner = s.pid)" : "", + key); + if (v == 0) + appendStringInfo(q, "%s (SELECT f.cid FROM cfa1_hf f WHERE f.uid = s.uid)", + notin ? "NOT IN" : "IN"); + else + appendStringInfo(q, "%s (ARRAY(SELECT f.cid FROM cfa1_hf f WHERE f.uid = s.uid))", + notin ? "<> ALL" : "= ANY"); + appendStringInfoString(q, "))) AS r FROM cfa1_hs s ORDER BY s.id"); + } + + cfp_diff(tc, conn, "CF 7384 duplicated hashed SubPlan rescan", + setup, qa.data, setup, qb.data); + + pfree(s.data); + pfree(qa.data); + pfree(qb.data); + pg_prop_disconnect(conn); +} + +/* + * Append a SQL string literal of 1..maxlen letters drawn from "alpha" (ASCII + * only). With "accents", 'a'/'A' may also come out as U+00E1/U+00C1, so + * case- and accent-variants of one word collide under a level-1 collation. + */ +static void +cfp_word(pg_prop_tc *tc, StringInfo b, const char *alpha, int maxlen, + bool accents) +{ + size_t len; + char *w = pg_prop_draw_from(tc, alpha, 1, maxlen, &len); + + appendStringInfoChar(b, '\''); + for (size_t i = 0; i < len; i++) + { + if (accents && (w[i] == 'a' || w[i] == 'A') && pg_prop_draw_bool(tc)) + appendStringInfoString(b, w[i] == 'a' ? "\xc3\xa1" : "\xc3\x81"); + else + appendStringInfoChar(b, w[i]); + } + appendStringInfoChar(b, '\''); + free(w); +} + +/* + * CF 7421: Memoize keyed on an outer expression whose collation is + * nondeterministic while the join clause compares under a different + * (deterministic) collation, e.g. i.x COLLATE "C" = o.x with o.x + * case-insensitive. Master hashes/compares cache keys with the outer + * expression's collation, so 'ab' and 'AB' share one cache entry although + * the clause tells them apart: wrong LEFT JOIN rows, wrong EXISTS / NOT + * EXISTS (PG19 semi/anti-join Memoize). + * + * Drawn: the nondeterministic collation (case-, or case+accent- + * insensitive), whether the outer collation comes from the column or a + * COLLATE in a derived table, the clause collation, which side carries the + * COLLATE, the alphabet and lengths of the strings (so case/accent + * variants collide), table sizes, and the join shape. Invariant: + * enable_memoize only changes the plan. + */ +PG_PROP(memoize_collation_matches_no_memoize) +{ + static const char *const ndcoll[] = { + "und-u-ks-level2", /* case-insensitive */ + "und-u-ks-level1", /* case- and accent-insensitive */ + }; + static const char *const dcoll[] = {"\"C\"", "\"POSIX\"", "\"und-x-icu\""}; + static const char *const alpha[] = {"aAbB", "aA", "abAB "}; + pg_prop_conn *conn = cfp_connect(tc); + StringInfoData s; + StringInfoData q; + const char *nd = ndcoll[pg_prop_draw_int(tc, 0, lengthof(ndcoll) - 1)]; + const char *dc = dcoll[pg_prop_draw_int(tc, 0, lengthof(dcoll) - 1)]; + const char *al = alpha[pg_prop_draw_int(tc, 0, lengthof(alpha) - 1)]; + bool accents = pg_prop_draw_bool(tc); + int maxlen = (int) pg_prop_draw_int(tc, 1, 3); + int no = (int) pg_prop_draw_int(tc, 5, 300); + int ni = (int) pg_prop_draw_int(tc, 1, 30); + int nfill = (int) pg_prop_draw_int(tc, 0, 2000); + bool colcoll = pg_prop_draw_bool(tc); /* ci on the column */ + bool inner_side = pg_prop_draw_bool(tc); /* COLLATE on inner side */ + int shape = (int) pg_prop_draw_int(tc, 0, 3); + const char *orel; + const char *clause; + + initStringInfo(&s); + appendStringInfo(&s, + "CREATE COLLATION pg_temp.cfa1_nd (provider = icu, locale = '%s', " + "deterministic = false); " + "CREATE TEMP TABLE cfa1_co (id int, x text%s); " + "CREATE TEMP TABLE cfa1_cs (id int, x text COLLATE %s); " + "INSERT INTO cfa1_co VALUES ", + nd, colcoll ? " COLLATE pg_temp.cfa1_nd" : "", dc); + for (int i = 0; i < no; i++) + { + appendStringInfo(&s, "%s(%d, ", i ? ", " : "", i); + cfp_word(tc, &s, al, maxlen, accents); + appendStringInfoChar(&s, ')'); + } + appendStringInfoString(&s, "; INSERT INTO cfa1_cs VALUES "); + for (int i = 0; i < ni; i++) + { + appendStringInfo(&s, "%s(%d, ", i ? ", " : "", i); + cfp_word(tc, &s, al, maxlen, accents); + appendStringInfoChar(&s, ')'); + } + appendStringInfo(&s, "; INSERT INTO cfa1_cs SELECT g, 'z' || g " + "FROM generate_series(%d, %d) g; " + "CREATE INDEX ON cfa1_cs (x); ANALYZE cfa1_co, cfa1_cs", + ni, ni + nfill - 1); + cfp_setup(tc, conn, s.data); + + /* + * The outer key o.x is nondeterministic, either by the column's collation + * or implicitly through a derived table's COLLATE. + */ + orel = colcoll ? "cfa1_co o" : + "(SELECT id, x COLLATE pg_temp.cfa1_nd AS x FROM cfa1_co) o"; + + /* + * The clause: compare under the deterministic collation. With the + * COLLATE on the inner side the outer key keeps its nondeterministic + * collation (the bug); on the outer side the key carries the clause's + * collation (control). + */ + initStringInfo(&q); + if (inner_side) + appendStringInfo(&q, "i.x COLLATE %s = o.x", dc); + else + appendStringInfo(&q, "i.x = o.x COLLATE %s", dc); + clause = pstrdup(q.data); + resetStringInfo(&q); + + switch (shape) + { + case 0: + appendStringInfo(&q, + "SELECT o.id, i.id FROM %s LEFT JOIN cfa1_cs i ON %s " + "ORDER BY 1, 2", orel, clause); + break; + case 1: + appendStringInfo(&q, + "SELECT o.id FROM %s WHERE EXISTS " + "(SELECT 1 FROM cfa1_cs i WHERE %s) ORDER BY 1", orel, clause); + break; + case 2: + appendStringInfo(&q, + "SELECT o.id FROM %s WHERE NOT EXISTS " + "(SELECT 1 FROM cfa1_cs i WHERE %s) ORDER BY 1", orel, clause); + break; + default: + appendStringInfo(&q, + "SELECT o.id, i.id FROM %s JOIN cfa1_cs i ON %s " + "ORDER BY 1, 2", orel, clause); + break; + } + + cfp_diff(tc, conn, "CF 7421 Memoize cache key collation", + CFP_NESTLOOP_ONLY, q.data, + CFP_NESTLOOP_ONLY "SET LOCAL enable_memoize = off;", NULL); + + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * A SQL/JSON query expression over context item CTX with PASSING argument + * ARG, as a format with two %s (CTX, ARG), and text output. None of these + * is strict in ARG: a NULL PASSING value is a JSON null variable, and the + * result does not depend on it at all in several of them. + */ +static const char *const cfp_jsonexpr[] = { + "JSON_VALUE(%s, '$' PASSING %s AS p)", + "JSON_VALUE(%s, 'strict $.a' PASSING %s AS p DEFAULT 'd' ON ERROR)", + "JSON_VALUE(%s, '$.a' PASSING %s AS p DEFAULT 'd' ON EMPTY)", + "JSON_EXISTS(%s, '$ ? ($p == null)' PASSING %s AS p)::text", + "JSON_EXISTS(%s, '$ ? (@ == $p)' PASSING %s AS p)::text", + "JSON_EXISTS(%s, '$.a' PASSING %s AS p)::text", + "JSON_QUERY(%s, '$' PASSING %s AS p WITH WRAPPER)::text", + "JSON_QUERY(%s, '$.a' PASSING %s AS p EMPTY OBJECT ON EMPTY)::text", + "JSON_QUERY(%s, '$ ? (@ != $p)' PASSING %s AS p WITH CONDITIONAL WRAPPER)::text", +}; + +static const char *const cfp_jsonctx[] = { + "jsonb '1'", "jsonb '{\"a\": 2}'", "jsonb '[1, null]'", "jsonb 'null'", + "jsonb '{}'", "jsonb '\"s\"'", +}; + +/* append one drawn SQL/JSON expression with the given PASSING arg; ctx may be a column */ +static void +cfp_draw_jsonexpr(pg_prop_tc *tc, StringInfo b, const char *arg, + const char *ctxcol) +{ + const char *fmt = cfp_jsonexpr[pg_prop_draw_int(tc, 0, lengthof(cfp_jsonexpr) - 1)]; + const char *ctx = (ctxcol && pg_prop_draw_int(tc, 0, 3) == 0) ? ctxcol : + cfp_jsonctx[pg_prop_draw_int(tc, 0, lengthof(cfp_jsonctx) - 1)]; + + appendStringInfo(b, fmt, ctx, arg); +} + +/* + * CF 7186, outer-join pull-up: a subquery whose target list holds a SQL/JSON + * query function with a PASSING argument is pulled up under an outer join. + * Master thinks JsonExpr is strict, so pullup_replace_vars() wraps no + * PlaceHolderVar around it and evaluates it above the join, giving a non-NULL + * value for null-extended rows. Reference: the same query with OFFSET 0 in + * the subquery, which blocks pull-up. + * + * Drawn: the JsonExpr (kind, path, ON EMPTY/ERROR, context item constant or + * column), the join kind (LEFT / FULL / RIGHT) and condition (false, key + * equality, equality plus a filter), and the table contents (with NULLs). + */ +PG_PROP(jsonexpr_pullup_matches_offset0_fence) +{ + static const char *const joins[] = {"LEFT JOIN", "FULL JOIN", "RIGHT JOIN"}; + static const char *const conds[] = { + "false", "a.k = ss.y", "a.k = ss.y AND a.k > 1", "a.k IS NOT DISTINCT FROM ss.y", + }; + pg_prop_conn *conn = cfp_connect(tc); + StringInfoData s; + StringInfoData e; + StringInfoData qa; + StringInfoData qb; + int na = (int) pg_prop_draw_int(tc, 1, 10); + int nb = (int) pg_prop_draw_int(tc, 1, 10); + const char *jn = joins[pg_prop_draw_int(tc, 0, lengthof(joins) - 1)]; + const char *cond = conds[pg_prop_draw_int(tc, 0, lengthof(conds) - 1)]; + bool wrap = pg_prop_draw_bool(tc); + bool lateral = pg_prop_draw_int(tc, 0, 3) == 0; + + initStringInfo(&s); + appendStringInfoString(&s, + "CREATE TEMP TABLE cfa1_ja (k int); " + "CREATE TEMP TABLE cfa1_jb (y int, j jsonb); "); + cfp_rows(tc, &s, "cfa1_ja", na, 1, 4, false); + appendStringInfoString(&s, "INSERT INTO cfa1_jb VALUES "); + for (int i = 0; i < nb; i++) + { + int y = (int) pg_prop_draw_int(tc, -1, 4); + + appendStringInfo(&s, "%s(", i ? ", " : ""); + if (y < 0) + appendStringInfoString(&s, "NULL"); + else + appendStringInfo(&s, "%d", y); + appendStringInfo(&s, ", %s)", + pg_prop_draw_int(tc, 0, 3) == 0 ? "NULL" : + cfp_jsonctx[pg_prop_draw_int(tc, 0, lengthof(cfp_jsonctx) - 1)]); + } + appendStringInfoString(&s, "; ANALYZE cfa1_ja, cfa1_jb"); + cfp_setup(tc, conn, s.data); + + /* the expression, optionally inside a strict operator */ + initStringInfo(&e); + if (wrap) + appendStringInfoString(&e, "'<' || "); + cfp_draw_jsonexpr(tc, &e, lateral && pg_prop_draw_bool(tc) ? "a.k" : "y", "j"); + if (wrap) + appendStringInfoString(&e, " || '>'"); + + initStringInfo(&qa); + initStringInfo(&qb); + for (int v = 0; v < 2; v++) + { + StringInfo q = v == 0 ? &qa : &qb; + + if (lateral) + { + /* LATERAL only allows LEFT JOIN with a reference to a */ + appendStringInfo(q, + "SELECT a.k, ss.y, ss.v FROM cfa1_ja a LEFT JOIN LATERAL " + "(SELECT y, %s AS v FROM cfa1_jb%s) ss ON %s " + "ORDER BY 1 NULLS FIRST, 2 NULLS FIRST, 3 NULLS FIRST", + e.data, v ? " OFFSET 0" : "", cond); + } + else + appendStringInfo(q, + "SELECT a.k, ss.y, ss.v FROM cfa1_ja a %s " + "(SELECT y, %s AS v FROM cfa1_jb%s) ss ON %s " + "ORDER BY 1 NULLS FIRST, 2 NULLS FIRST, 3 NULLS FIRST", + jn, e.data, v ? " OFFSET 0" : "", cond); + } + + cfp_diff(tc, conn, "CF 7186 JsonExpr pulled up under an outer join", + "", qa.data, "", qb.data); + + pfree(s.data); + pfree(e.data); + pfree(qa.data); + pfree(qb.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7186, inlining: a STRICT SQL function whose body is a SQL/JSON query + * function of its arguments. Master inlines it (believing the body strict), + * which drops the strictness check, so a NULL argument yields non-NULL. + * Invariant (definition of STRICT): the call is NULL whenever an argument is + * NULL. Drawn: the JsonExpr, whether the context item is the jsonb argument + * or a constant, a strict wrapper, and the argument values. + */ +PG_PROP(strict_sql_function_with_jsonexpr_returns_null_on_null) +{ + pg_prop_conn *conn = cfp_connect(tc); + StringInfoData s; + char *v; + bool failed = false; + char *err = NULL; + bool wrap = pg_prop_draw_bool(tc); + int nb = (int) pg_prop_draw_int(tc, 1, 10); + + initStringInfo(&s); + appendStringInfoString(&s, + "CREATE FUNCTION pg_temp.cfa1_jf(int, jsonb) RETURNS text " + "STRICT IMMUTABLE LANGUAGE sql AS $$ SELECT "); + if (wrap) + appendStringInfoString(&s, "'<' || "); + cfp_draw_jsonexpr(tc, &s, "$1", "$2"); + if (wrap) + appendStringInfoString(&s, " || '>'"); + appendStringInfoString(&s, " $$; CREATE TEMP TABLE cfa1_jb (y int, j jsonb); " + "INSERT INTO cfa1_jb VALUES (NULL, NULL), (1, NULL), (NULL, '1')"); + for (int i = 0; i < nb; i++) + { + int y = (int) pg_prop_draw_int(tc, -1, 4); + + if (y < 0) + appendStringInfoString(&s, ", (NULL, "); + else + appendStringInfo(&s, ", (%d, ", y); + appendStringInfo(&s, "%s)", + pg_prop_draw_int(tc, 0, 3) == 0 ? "NULL" : + cfp_jsonctx[pg_prop_draw_int(tc, 0, lengthof(cfp_jsonctx) - 1)]); + } + cfp_setup(tc, conn, s.data); + + v = pg_prop_sql_scalar(conn, + "SELECT count(*) FROM cfa1_jb " + "WHERE (y IS NULL OR j IS NULL) AND pg_temp.cfa1_jf(y, j) IS NOT NULL", + &failed, &err); + if (failed || v == NULL) + pg_prop_fail(tc, "query failed: %s\n data: %s", err ? err : "", s.data); + if (strcmp(v, "0") != 0) + { + char *rows = pg_prop_sql_rows(conn, + "SELECT y, j, pg_temp.cfa1_jf(y, j) FROM cfa1_jb " + "WHERE (y IS NULL OR j IS NULL) LIMIT 8", + &failed, NULL); + + pg_prop_fail(tc, + "CF 7186 STRICT SQL function returned non-NULL for a NULL argument (%s rows)\n" + " setup: %s\n (y, j, f(y, j)):\n%s", + v, s.data, rows ? rows : ""); + } + pg_free(v); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* + * Expression generator for CF 7400: a tree of COALESCE / CASE / NULLIF / + * typmod casts / an inlinable identity SQL function over a NOT NULL column a + * and a nullable column b of one type family, with typmods drawn so the + * typmod the tree exposes can change when the planner reduces it. + */ +typedef struct CfpTyp +{ + const char *const *types; /* spellings with and without typmod */ + int ntypes; + const char *const *lits; + int nlits; + const char *fn; /* identity SQL function for this family */ +} CfpTyp; + +static const char *const cfp_vtypes[] = {"varchar(5)", "varchar(10)", "varchar(20)", "varchar", "text"}; +static const char *const cfp_vlits[] = {"'x'", "'yy'", "''"}; +static const char *const cfp_ntypes[] = {"numeric(10)", "numeric(10,2)", "numeric(5,1)", "numeric"}; +static const char *const cfp_nlits[] = {"1", "2.5", "0"}; +static const CfpTyp cfp_typs[] = { + {cfp_vtypes, lengthof(cfp_vtypes), cfp_vlits, lengthof(cfp_vlits), "pg_temp.cfa1_fv"}, + {cfp_ntypes, lengthof(cfp_ntypes), cfp_nlits, lengthof(cfp_nlits), "pg_temp.cfa1_fn"}, +}; + +static void cfp_typmod_expr(pg_prop_tc *tc, StringInfo b, const CfpTyp *t, int depth); + +/* a boolean test: the NOT NULL column's IS [NOT] NULL folds to a constant */ +static void +cfp_typmod_cond(pg_prop_tc *tc, StringInfo b, const CfpTyp *t, int depth) +{ + switch (pg_prop_draw_int(tc, 0, 4)) + { + case 0: + appendStringInfoString(b, "a IS NULL"); + break; + case 1: + appendStringInfoString(b, "a IS NOT NULL"); + break; + case 2: + appendStringInfoString(b, pg_prop_draw_bool(tc) ? "false" : "true"); + break; + case 3: + appendStringInfoString(b, "b IS NULL"); + break; + default: + appendStringInfoChar(b, '('); + cfp_typmod_expr(tc, b, t, depth - 1); + appendStringInfoString(b, ") IS NULL"); + break; + } +} + +static void +cfp_typmod_expr(pg_prop_tc *tc, StringInfo b, const CfpTyp *t, int depth) +{ + /* depth > 3 marks the root: always a node the planner may reduce */ + int k = depth > 3 ? (int) pg_prop_draw_int(tc, 3, 8) : + (int) pg_prop_draw_int(tc, 0, depth <= 0 ? 2 : 8); + const char *ty = t->types[pg_prop_draw_int(tc, 0, t->ntypes - 1)]; + + if (depth > 3) + depth = 3; + + switch (k) + { + case 0: + appendStringInfoString(b, pg_prop_draw_bool(tc) ? "a" : "b"); + break; + case 1: + appendStringInfo(b, "null::%s", ty); + break; + case 2: + appendStringInfo(b, "%s::%s", t->lits[pg_prop_draw_int(tc, 0, t->nlits - 1)], ty); + break; + case 3: + case 4: + { + int n = (int) pg_prop_draw_int(tc, 2, 3); + + appendStringInfoString(b, "COALESCE("); + for (int i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoString(b, ", "); + cfp_typmod_expr(tc, b, t, depth - 1); + } + appendStringInfoChar(b, ')'); + } + break; + case 5: + { + int n = (int) pg_prop_draw_int(tc, 1, 2); + + appendStringInfoString(b, "CASE"); + for (int i = 0; i < n; i++) + { + appendStringInfoString(b, " WHEN "); + cfp_typmod_cond(tc, b, t, depth); + appendStringInfoString(b, " THEN "); + cfp_typmod_expr(tc, b, t, depth - 1); + } + appendStringInfoString(b, " ELSE "); + cfp_typmod_expr(tc, b, t, depth - 1); + appendStringInfoString(b, " END"); + } + break; + case 6: + appendStringInfoString(b, "NULLIF("); + cfp_typmod_expr(tc, b, t, depth - 1); + appendStringInfoString(b, ", "); + cfp_typmod_expr(tc, b, t, depth - 1); + appendStringInfoChar(b, ')'); + break; + case 7: + appendStringInfoChar(b, '('); + cfp_typmod_expr(tc, b, t, depth - 1); + appendStringInfo(b, ")::%s", ty); + break; + default: + appendStringInfo(b, "%s(", t->fn); + cfp_typmod_expr(tc, b, t, depth - 1); + appendStringInfoChar(b, ')'); + break; + } +} + +/* + * CF 7400 (part 1, still on master): ON CONFLICT inference simplifies the + * index expression twice but the inference expression once, and reducing a + * CASE / COALESCE / NULLIF (PG19 folds "a IS NULL" on a NOT NULL column) can + * change the typmod the node exposes, so the two trees differ and no + * arbiter index is found. + * + * Invariant: given a unique index on exactly expression E, both + * INSERT ... ON CONFLICT ((E)) DO NOTHING and ... DO UPDATE must infer it + * (they must not fail with "no unique or exclusion constraint matching"), + * and a duplicate key inserted that way leaves one row. The index is built + * on an empty table, so it cannot fail on duplicates. + */ +PG_PROP(on_conflict_infers_index_on_same_expression) +{ + pg_prop_conn *conn = cfp_connect(tc); + const CfpTyp *t = &cfp_typs[pg_prop_draw_int(tc, 0, lengthof(cfp_typs) - 1)]; + const char *ta = t->types[pg_prop_draw_int(tc, 0, t->ntypes - 1)]; + const char *tb = t->types[pg_prop_draw_int(tc, 0, t->ntypes - 1)]; + const char *va = t->lits[pg_prop_draw_int(tc, 0, t->nlits - 1)]; + const char *vb = pg_prop_draw_bool(tc) ? "NULL" : + t->lits[pg_prop_draw_int(tc, 0, t->nlits - 1)]; + bool upd = pg_prop_draw_bool(tc); + StringInfoData e; + StringInfoData s; + char *err = NULL; + + initStringInfo(&e); + cfp_typmod_expr(tc, &e, t, 4); + + initStringInfo(&s); + appendStringInfo(&s, + "CREATE FUNCTION pg_temp.cfa1_fv(varchar) RETURNS varchar " + "IMMUTABLE LANGUAGE sql AS 'SELECT $1'; " + "CREATE FUNCTION pg_temp.cfa1_fn(numeric) RETURNS numeric " + "IMMUTABLE LANGUAGE sql AS 'SELECT $1'; " + "CREATE TEMP TABLE cfa1_oc (a %s NOT NULL, b %s)", ta, tb); + cfp_setup(tc, conn, s.data); + + /* + * The index can still be rejected for reasons unrelated to the bug (an + * expression that folds to a constant is fine, but be safe): discard + * such a case rather than count it. + */ + resetStringInfo(&s); + appendStringInfo(&s, "CREATE UNIQUE INDEX cfa1_oc_k ON cfa1_oc ((%s))", e.data); + if (!pg_prop_sql_exec(conn, s.data, &err)) + { + if (err && (strstr(err, "XX000") || strstr(err, "cache lookup") || + strstr(err, "unexpected"))) + pg_prop_fail(tc, "CREATE INDEX internal error: %s\n SQL: %s", err, s.data); + pg_prop_assume(tc, false); + } + + for (int i = 0; i < 2; i++) + { + resetStringInfo(&s); + appendStringInfo(&s, + "INSERT INTO cfa1_oc VALUES (%s, %s) ON CONFLICT ((%s)) %s", + va, vb, e.data, + upd ? "DO UPDATE SET b = EXCLUDED.b" : "DO NOTHING"); + if (!pg_prop_sql_exec(conn, s.data, &err)) + pg_prop_fail(tc, + "CF 7400 ON CONFLICT did not infer the unique index on the same expression\n" + " error: %s index: CREATE UNIQUE INDEX ON cfa1_oc ((%s))\n" + " table: a %s NOT NULL, b %s\n SQL: %s", + err ? err : "", e.data, ta, tb, s.data); + } + + /* the key may be NULL (NULLs are distinct), so 1 or 2 rows */ + { + bool failed = false; + char *n; + + resetStringInfo(&s); + appendStringInfo(&s, + "SELECT count(*) = CASE WHEN (SELECT (%s) IS NULL FROM cfa1_oc LIMIT 1) " + "THEN 2 ELSE 1 END FROM cfa1_oc", e.data); + n = pg_prop_sql_scalar(conn, s.data, &failed, &err); + if (failed || n == NULL || strcmp(n, "t") != 0) + pg_prop_fail(tc, "CF 7400 duplicate insert through ON CONFLICT left the wrong row count " + "(%s)\n index expr: %s", failed ? (err ? err : "") : (n ? n : "NULL"), + e.data); + pg_free(n); + } + + pfree(e.data); + pfree(s.data); + pg_prop_disconnect(conn); +} + +static long +cfp_long(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *v = pg_prop_sql_scalar(conn, sql, &failed, &err); + long n; + + if (failed || v == NULL) + pg_prop_fail(tc, "check failed: %s\n SQL: %s", err ? err : "", sql); + n = atol(v); + pg_free(v); + return n; +} + +/* + * CF 7448: eager aggregation (PG19) pushes a partial aggregate below a join + * that still has a volatile clause to apply, so the clause runs once per + * partial group instead of once per joined row. + * + * The volatile clause is nextval() of a fresh sequence inside a comparison + * that is always true, so the query result is fixed, and the sequence counts + * the evaluations. Invariant (volatile functions are evaluated for every row + * that needs them, and a volatile qual is never a pseudoconstant): every row + * the join produces passed the clause, so the sequence advanced at least + * once per joined row; and enable_eager_aggregate on/off give the same rows. + * + * Drawn: sizes, key and group moduli (so groups are large enough for eager + * aggregation), where the volatile clause sits (ON with no Vars, ON over both + * rels, WHERE with no Vars), the aggregates, the join method GUCs, a third + * joined table, and min_eager_agg_group_size. + */ +PG_PROP(eager_agg_volatile_clause_runs_per_joined_row) +{ + static const char *const vclause[] = { + "nextval('cfa1_vs') > 0", + "nextval('cfa1_vs') + f.c > d.g - 1000000", + "(nextval('cfa1_vs') > 0 OR f.c = d.g)", + }; + static const char *const methods[] = { + "", + "SET LOCAL enable_hashjoin = off;", + "SET LOCAL enable_hashjoin = off; SET LOCAL enable_mergejoin = off;", + "SET LOCAL enable_nestloop = off; SET LOCAL enable_mergejoin = off;", + "SET LOCAL enable_hashagg = off;", + }; + static const char *const aggs[] = { + "sum(f.c)", "count(*)", "count(f.c)", "max(f.c)", "min(f.c) + sum(f.c)", + }; + pg_prop_conn *conn = cfp_connect(tc); + StringInfoData s; + StringInfoData q; + StringInfoData j; + StringInfoData setup; + int nf = (int) pg_prop_draw_int(tc, 100, 3000); + int kmod = (int) pg_prop_draw_int(tc, 1, 20); + int nd = (int) pg_prop_draw_int(tc, 1, 60); + int gmod = (int) pg_prop_draw_int(tc, 1, 5); + int cmod = (int) pg_prop_draw_int(tc, 1, 4); + int where = (int) pg_prop_draw_int(tc, 0, 3); /* 0..2 ON, 3 WHERE */ + const char *vc = vclause[pg_prop_draw_int(tc, 0, lengthof(vclause) - 1)]; + const char *agg = aggs[pg_prop_draw_int(tc, 0, lengthof(aggs) - 1)]; + const char *meth = methods[pg_prop_draw_int(tc, 0, lengthof(methods) - 1)]; + bool third = pg_prop_draw_int(tc, 0, 3) == 0; + bool lowmin = pg_prop_draw_bool(tc); + long joined; + long evals; + + initStringInfo(&s); + appendStringInfo(&s, + "CREATE TEMP TABLE cfa1_vf (k int, c int); " + "CREATE TEMP TABLE cfa1_vd (k int, g int); " + "CREATE TEMP TABLE cfa1_ve (g int, h int); " + "INSERT INTO cfa1_vf SELECT i %% %d, i %% %d FROM generate_series(1, %d) i; " + "INSERT INTO cfa1_vd SELECT i %% %d, i %% %d FROM generate_series(1, %d) i; " + "INSERT INTO cfa1_ve SELECT i %% %d, i FROM generate_series(1, %d) i; " + "CREATE TEMP SEQUENCE cfa1_vs; " + "ANALYZE cfa1_vf, cfa1_vd, cfa1_ve", + kmod, cmod, nf, kmod, gmod, nd, gmod, gmod); + cfp_setup(tc, conn, s.data); + + /* the join, without its volatile clause */ + initStringInfo(&j); + appendStringInfoString(&j, "cfa1_vf f JOIN cfa1_vd d ON f.k = d.k"); + if (third) + appendStringInfoString(&j, " JOIN cfa1_ve e ON e.g = d.g"); + + joined = cfp_long(tc, conn, psprintf("SELECT count(*) FROM %s", j.data)); + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT d.g, "); + appendStringInfoString(&q, agg); + appendStringInfoString(&q, " FROM cfa1_vf f JOIN cfa1_vd d ON f.k = d.k"); + if (where < 3) + appendStringInfo(&q, " AND %s", vc); + if (third) + appendStringInfoString(&q, " JOIN cfa1_ve e ON e.g = d.g"); + if (where == 3) + appendStringInfo(&q, " WHERE %s", vc); + appendStringInfoString(&q, " GROUP BY d.g ORDER BY d.g"); + + initStringInfo(&setup); + appendStringInfo(&setup, "SET LOCAL enable_eager_aggregate = on; %s%s", + meth, lowmin ? " SET LOCAL min_eager_agg_group_size = 0;" : ""); + + /* eager aggregation allowed: count evaluations of the volatile clause */ + cfp_exec(tc, conn, "SELECT setval('cfa1_vs', 1, false)"); + free(cfp_digest(tc, conn, setup.data, q.data)); + evals = cfp_long(tc, conn, + "SELECT CASE WHEN is_called THEN last_value ELSE 0 END FROM cfa1_vs"); + if (evals < joined) + { + char *plan; + bool failed = false; + + cfp_exec(tc, conn, "BEGIN"); + cfp_exec(tc, conn, setup.data); + plan = pg_prop_sql_rows(conn, psprintf("EXPLAIN (COSTS OFF) %s", q.data), + &failed, NULL); + pg_prop_sql_exec(conn, "ROLLBACK", NULL); + pg_prop_fail(tc, + "CF 7448 volatile clause evaluated %ld times for %ld joined rows\n" + " data: %s\n setup: %s\n query: %s\n plan:\n%s", + evals, joined, s.data, setup.data, q.data, plan ? plan : ""); + } + + /* and the same rows with eager aggregation off */ + { + StringInfoData off; + + initStringInfo(&off); + appendStringInfo(&off, "%s SET LOCAL enable_eager_aggregate = off;", setup.data); + cfp_diff(tc, conn, "CF 7448 eager aggregation with a volatile clause", + setup.data, q.data, off.data, NULL); + pfree(off.data); + } + + pfree(s.data); + pfree(q.data); + pfree(j.data); + pfree(setup.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7466: check_functional_grouping() accepts a WITHOUT OVERLAPS primary key + * as proof that the other columns depend on the grouped key columns. The key + * is enforced by a GiST index whose equality need not be GROUP BY's btree + * equality: with citext (btree_gist gist_text_ops through binary coercion) + * 'a' and 'A' are distinct keys but one group, so the query is accepted and + * returns one arbitrary title for two rows. + * + * Invariant: if the server accepts + * SELECT , FROM t GROUP BY + * then the functional dependency holds, so the result equals + * SELECT ... GROUP BY , + * (a correct server either rejects the first form, which makes the case + * vacuous, or accepts it only when the dependency is real). + * + * Drawn: the key column type (citext, or a text / int control), an extra + * leading key column, the range type of the period, the non-key columns, + * and rows with case variants and overlapping or disjoint periods; rows + * that violate the key are dropped by ON CONFLICT DO NOTHING. + */ +PG_PROP(without_overlaps_pk_grouping_matches_full_grouping) +{ + static const char *const idtypes[] = {"citext", "citext", "text", "int"}; + static const char *const ids_txt[] = {"a", "A", "b", "B", "ab", "aB"}; + pg_prop_conn *conn = cfp_connect(tc); + StringInfoData s; + StringInfoData qa; + StringInfoData qb; + const char *idt; + bool extra = pg_prop_draw_bool(tc); + bool multi = pg_prop_draw_int(tc, 0, 3) == 0; + bool two = pg_prop_draw_bool(tc); + int n = (int) pg_prop_draw_int(tc, 1, 30); + const char *keys; + const char *others; + char *err = NULL; + bool failed = false; + char *da; + char *db; + + if (!pg_prop_require_extension(conn, "btree_gist")) + pg_prop_skip(tc, "btree_gist not installed"); + if (!pg_prop_require_extension(conn, "citext")) + pg_prop_skip(tc, "citext not installed"); + idt = idtypes[pg_prop_draw_int(tc, 0, lengthof(idtypes) - 1)]; + + initStringInfo(&s); + appendStringInfo(&s, + "CREATE TEMP TABLE cfa1_tp (%sid %s, valid_at %s, title text NOT NULL, " + "n int, PRIMARY KEY (%sid, valid_at WITHOUT OVERLAPS)); " + "INSERT INTO cfa1_tp VALUES ", + extra ? "k int, " : "", idt, multi ? "int4multirange" : "int4range", + extra ? "k, " : ""); + for (int i = 0; i < n; i++) + { + int lo = (int) pg_prop_draw_int(tc, 0, 8); + int len = (int) pg_prop_draw_int(tc, 1, 4); + + appendStringInfoString(&s, i ? ", (" : "("); + if (extra) + appendStringInfo(&s, "%d, ", (int) pg_prop_draw_int(tc, 0, 1)); + if (strcmp(idt, "int") == 0) + appendStringInfo(&s, "%d", (int) pg_prop_draw_int(tc, 0, 3)); + else + appendStringInfo(&s, "'%s'", ids_txt[pg_prop_draw_int(tc, 0, lengthof(ids_txt) - 1)]); + if (multi) + appendStringInfo(&s, ", '{[%d,%d)}'", lo, lo + len); + else + appendStringInfo(&s, ", '[%d,%d)'", lo, lo + len); + appendStringInfo(&s, ", 't%d', %d)", i, (int) pg_prop_draw_int(tc, 0, 2)); + } + appendStringInfoString(&s, " ON CONFLICT DO NOTHING; ANALYZE cfa1_tp"); + cfp_setup(tc, conn, s.data); + + keys = extra ? "k, id, valid_at" : "id, valid_at"; + others = two ? "title, n" : "title"; + + initStringInfo(&qa); + initStringInfo(&qb); + appendStringInfo(&qa, + "SELECT %s, %s FROM cfa1_tp GROUP BY %s " + "ORDER BY %sid::text COLLATE \"C\", valid_at, title", + keys, others, keys, extra ? "k, " : ""); + appendStringInfo(&qb, + "SELECT %s, %s FROM cfa1_tp GROUP BY %s, %s " + "ORDER BY %sid::text COLLATE \"C\", valid_at, title", + keys, others, keys, others, extra ? "k, " : ""); + + da = pg_prop_sql_rowhash(conn, qa.data, &failed, &err); + if (failed) + { + /* rejecting the key-only grouping is always correct */ + if (err == NULL || strstr(err, "must appear in the GROUP BY clause") == NULL) + pg_prop_fail(tc, "unexpected error: %s\n query: %s", err ? err : "", qa.data); + pg_prop_disconnect(conn); + pfree(s.data); + pfree(qa.data); + pfree(qb.data); + return; + } + db = pg_prop_sql_rowhash(conn, qb.data, &failed, &err); + if (failed || db == NULL) + pg_prop_fail(tc, "query failed: %s\n query: %s", err ? err : "", qb.data); + if (strcmp(da, db) != 0) + pg_prop_fail(tc, + "CF 7466 GROUP BY a WITHOUT OVERLAPS primary key was accepted but the " + "columns are not functionally dependent on it\n" + " data: %s\n key-only grouping:\n%s\n full grouping:\n%s\n" + " A: %s\n B: %s", + s.data, cfp_sample(conn, "", qa.data), cfp_sample(conn, "", qb.data), + qa.data, qb.data); + free(da); + free(db); + pfree(s.data); + pfree(qa.data); + pfree(qb.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 82aa0a7584e58..6773454008587 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model syscol collation index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model syscol collation cf_types cf_index cf_exec cf_ddl2 cf_planner index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 7822275a37c7a1d7ce4e2ad930803b4deb3e1f3c Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 17 Oct 2026 17:00:32 -0400 Subject: [PATCH 306/338] prop: properties for open DDL / partitioning / FK / RLS fixes (cf_ddl2) Fifteen properties, one per open commitfest bug fix, each a catalog or data invariant checked after a drawn DDL sequence: index properties survive a partition-index rebuild (6440), REFRESH MATERIALIZED VIEW CONCURRENTLY equals its query (6579), postgres_fdw DML on remote partitions matches local (6770), auto-named partition constraints never collide (6842), REVOKE keeps grant chains rooted (6961), ALTER PUBLICATION reaches event triggers (7055), publication table lists are order-independent (7210), temporal FK validation matches a range_agg model (7241), rolled-back SET TABLESPACE keeps indexes consistent (7312), DROP SCHEMA pg_temp_N leaves no orphans (7321), a pending DETACH looks the same to every session (7351), CREATE TABLE LIKE touches only the new table (7379), ON CONFLICT WHERE respects RLS (7403), FKs reference only unique constraints (7464), WITHOUT OVERLAPS UNIQUE agrees for range and multirange (7465). Each falsifies on upstream master 45fc51f. --- src/test/prop/sql/prop_cf_ddl2.c | 2406 ++++++++++++++++++++++++++++++ 1 file changed, 2406 insertions(+) create mode 100644 src/test/prop/sql/prop_cf_ddl2.c diff --git a/src/test/prop/sql/prop_cf_ddl2.c b/src/test/prop/sql/prop_cf_ddl2.c new file mode 100644 index 0000000000000..9952f3570e0ee --- /dev/null +++ b/src/test/prop/sql/prop_cf_ddl2.c @@ -0,0 +1,2406 @@ +/*------------------------------------------------------------------------- + * + * prop_cf_ddl2.c + * Catalog and data invariants after DDL: partitioning, constraints, + * RLS, matviews, publications, tablespaces, temp schemas, ACLs. + * + * Each property drives a randomized DDL sequence around the trigger of an + * open commitfest bug fix and checks an invariant that holds on a correct + * server: + * + * partindex_rebuild_preserves_props (CF 6440) + * ALTER COLUMN TYPE / SET EXPRESSION on a partitioned table rebuilds + * every index of the partition tree that uses the column; a rebuild is + * an implementation detail, so every user-visible property of every + * index (name, comment, reloptions, tablespace, per-column statistics + * target, replica identity, cluster mark, extension dependency, + * constraint comment) must be the same afterwards. + * + * refresh_concurrently_matches_query (CF 6579) + * A REFRESH MATERIALIZED VIEW CONCURRENTLY that succeeds leaves exactly + * the rows of the defining query; one that fails changes nothing. + * + * fdw_partitioned_remote_dml_matches_local (CF 6770) + * UPDATE/DELETE through postgres_fdw on a remote partitioned or + * inheritance table changes the same rows as on a local copy. + * + * partition_autoname_constraint_never_collides (CF 6842) + * An auto-named ADD CHECK / SET NOT NULL on a partition root never + * fails with a duplicate-name error, whatever schemas and constraint + * names the partitions have. + * + * revoke_keeps_grant_chains_rooted (CF 6961) + * After any GRANT/REVOKE/membership sequence, every non-owner grantor + * in an ACL directly holds the grant option it granted from. + * + * alter_publication_reported_to_event_triggers (CF 7055) + * An ALTER PUBLICATION that changes the publication is listed by + * pg_event_trigger_ddl_commands() at ddl_command_end. + * + * publication_table_list_order_independent (CF 7210) + * The same TABLE / TABLE ONLY mention list in two orders, when both are + * accepted, publishes the same tables. + * + * temporal_fk_validation_matches_model (CF 7241) + * Every way of validating a PERIOD foreign key against existing rows + * accepts iff the rows are covered (range_agg model). + * + * rollback_set_tablespace_keeps_indexes_consistent (CF 7312) + * After a (sub)transaction with SET TABLESPACE rolls back, index scans + * and the seqscan agree and nothing errors. + * + * drop_temp_schema_keeps_catalog_consistent (CF 7321) + * After DROP SCHEMA pg_temp_N, new temp objects never reference a + * missing namespace. + * + * pending_detach_partition_consistent (CF 7351) + * A partition left pending detach is still covered by FOR ALL TABLES, + * and all sessions agree on its publication membership. + * + * create_like_only_touches_new_table (CF 7379) + * CREATE [TEMP] TABLE ... (LIKE ... INCLUDING COMMENTS) never changes + * another table, whatever search_path says. + * + * on_conflict_where_respects_rls (CF 7403) + * A function in ON CONFLICT DO UPDATE/SELECT WHERE never receives a + * value from a row hidden by RLS. + * + * fk_references_only_unique_constraints (CF 7464) + * An accepted (temporal) FK is backed by a PK / UNIQUE constraint of + * the referenced table, never by an EXCLUDE constraint. + * + * without_overlaps_unique_range_eq_multirange (CF 7465) + * UNIQUE [NULLS NOT DISTINCT] (.., x WITHOUT OVERLAPS) accepts the same + * rows for range and single-range multirange columns. + * + * All objects use the prefix "cfc2_"; each case drops what it created, and + * starts by dropping anything a failed earlier case left behind. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_cf_ddl2.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "cf_ddl2" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +static pg_prop_conn * +c2_connect(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + pg_prop_sql_exec(conn, "SET client_min_messages = warning", NULL); + return conn; +} + +/* run DDL/DML that must succeed on a correct server */ +static void +c2_exec(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "statement failed: %s\n SQL: %s", err ? err : "", sql); +} + +/* run a statement whose failure is acceptable; returns success */ +static bool +c2_try(pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + bool ok = pg_prop_sql_exec(conn, sql, &err); + + if (err) + pg_free(err); + return ok; +} + +static char * +c2_rows(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *r = pg_prop_sql_rows(conn, sql, &failed, &err); + + if (failed) + pg_prop_fail(tc, "query failed: %s\n SQL: %s", err ? err : "", sql); + return r; +} + +static long +c2_count(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *v = pg_prop_sql_scalar(conn, sql, &failed, &err); + long n; + + if (failed) + pg_prop_fail(tc, "query failed: %s\n SQL: %s", err ? err : "", sql); + n = v ? atol(v) : 0; + if (v) + pg_free(v); + return n; +} + +/* ---------------------------------------------------------------------- + * CF 6440: partitioned index rebuild preserves descendant properties + * ---------------------------------------------------------------------- + */ + +typedef struct PiKind +{ + const char *def; /* column list etc. after ON cfc2_r */ + bool constraint; /* ADD CONSTRAINT ... UNIQUE */ + bool uses_g; /* references the generated column */ + bool replident; /* usable as REPLICA IDENTITY */ + bool clusterable; /* CLUSTER ON allowed (btree, not partial) */ + int exprcol; /* 1-based expression column, 0 = none */ + int ncols; +} PiKind; + +static const PiKind pi_kinds[] = { + {"UNIQUE (id, a)", true, false, true, true, 0, 2}, + {"(id, a)", false, false, true, true, 0, 2}, /* UNIQUE INDEX */ + {"(b, (a * 3))", false, false, false, true, 2, 2}, + {"((a + 1)) WHERE b > 0", false, false, false, false, 1, 1}, + {"USING hash (a)", false, false, false, false, 0, 1}, + {"(id, g)", false, true, true, true, 0, 2}, /* UNIQUE INDEX */ + {"((g + a), id)", false, true, false, true, 1, 2}, +}; + +/* the catalog facts about every index in cfc2_r's tree that must survive */ +#define PI_SNAPSHOT \ + "SELECT pg_partition_root(i.indexrelid)::regclass::text, t.relname, c.relname, " \ + " coalesce(obj_description(c.oid, 'pg_class'), '-'), " \ + " coalesce(c.reloptions::text, '-'), coalesce(ts.spcname, 'pg_default'), " \ + " (SELECT string_agg(coalesce(a.attstattarget::text, '-'), ',' ORDER BY a.attnum) " \ + " FROM pg_attribute a WHERE a.attrelid = c.oid AND a.attnum > 0), " \ + " i.indisreplident, i.indisclustered, t.relreplident, " \ + " coalesce((SELECT string_agg(e.extname, ',' ORDER BY e.extname) FROM pg_depend d " \ + " JOIN pg_extension e ON e.oid = d.refobjid WHERE d.classid = 'pg_class'::regclass " \ + " AND d.objid = c.oid AND d.refclassid = 'pg_extension'::regclass AND d.deptype = 'x'), '-'), " \ + " coalesce((SELECT co.conname || ':' || coalesce(obj_description(co.oid, 'pg_constraint'), '-') " \ + " FROM pg_constraint co WHERE co.conindid = c.oid AND co.conrelid = t.oid), '-') " \ + "FROM pg_index i JOIN pg_class c ON c.oid = i.indexrelid " \ + "JOIN pg_class t ON t.oid = i.indrelid " \ + "LEFT JOIN pg_tablespace ts ON ts.oid = c.reltablespace " \ + "WHERE i.indrelid IN (SELECT relid FROM pg_partition_tree('cfc2_r')) " \ + "ORDER BY 1, 2, 3" + +static const char *pi_cleanup = +"DROP TABLE IF EXISTS cfc2_r CASCADE; " +"DROP TABLE IF EXISTS cfc2_r_l0, cfc2_r_l1, cfc2_r_l2, cfc2_r_m, cfc2_r_m0, cfc2_r_m1 CASCADE"; + +PG_PROP(partindex_rebuild_preserves_props) +{ + pg_prop_conn *conn = c2_connect(tc); + StringInfoData s; + bool has_g = pg_prop_draw_bool(tc); + bool use_ts = pg_prop_draw_bool(tc); + int nleaf = (int) pg_prop_draw_int(tc, 1, 3); + bool has_mid = pg_prop_draw_bool(tc); + int nmid = has_mid ? (int) pg_prop_draw_int(tc, 1, 2) : 0; + int nidx = (int) pg_prop_draw_int(tc, 1, 3); + int kinds[3]; + int alter; + char *map; + char *before; + char *after; + char *line; + char *save = NULL; + int nmut = 0; + + c2_exec(tc, conn, pi_cleanup); + /* CREATE/DROP TABLESPACE refuse to run in a multi-statement string */ + c2_try(conn, "DROP TABLESPACE IF EXISTS cfc2_ts"); + if (use_ts) + { + c2_exec(tc, conn, "SET allow_in_place_tablespaces = on"); + c2_exec(tc, conn, "CREATE TABLESPACE cfc2_ts LOCATION ''"); + } + + initStringInfo(&s); + appendStringInfo(&s, + "CREATE TABLE cfc2_r (id int NOT NULL, a int NOT NULL, b int NOT NULL%s) " + "PARTITION BY RANGE (id); ", + has_g ? ", g int GENERATED ALWAYS AS (b * 2) STORED NOT NULL" : ""); + for (int i = 0; i < nleaf; i++) + { + if (pg_prop_draw_bool(tc)) + appendStringInfo(&s, + "CREATE TABLE cfc2_r_l%d PARTITION OF cfc2_r " + "FOR VALUES FROM (%d) TO (%d); ", i, i * 100, i * 100 + 100); + else + { + /* different physical column order, with a dropped column */ + appendStringInfo(&s, + "CREATE TABLE cfc2_r_l%d (x int, b int NOT NULL%s, a int NOT NULL, " + "id int NOT NULL); ALTER TABLE cfc2_r_l%d DROP COLUMN x; " + "ALTER TABLE cfc2_r ATTACH PARTITION cfc2_r_l%d " + "FOR VALUES FROM (%d) TO (%d); ", + i, has_g ? ", g int GENERATED ALWAYS AS (b * 2) STORED NOT NULL" : "", + i, i, i * 100, i * 100 + 100); + } + } + if (has_mid) + { + appendStringInfoString(&s, + "CREATE TABLE cfc2_r_m PARTITION OF cfc2_r FOR VALUES FROM (900) TO (1000) " + "PARTITION BY RANGE (id); "); + for (int i = 0; i < nmid; i++) + appendStringInfo(&s, + "CREATE TABLE cfc2_r_m%d PARTITION OF cfc2_r_m " + "FOR VALUES FROM (%d) TO (%d); ", i, 900 + i * 50, 950 + i * 50); + } + /* a few rows, spread over the partitions */ + appendStringInfo(&s, + "INSERT INTO cfc2_r (id, a, b) SELECT p * 100 + k, k %% 4, k " + "FROM generate_series(0, %d) p, generate_series(0, %d) k; ", + nleaf - 1, (int) pg_prop_draw_int(tc, 0, 5)); + if (has_mid) + appendStringInfoString(&s, "INSERT INTO cfc2_r (id, a, b) VALUES (901, 1, 1), (902, 2, 2); "); + + for (int k = 0; k < nidx; k++) + { + int kind; + + kind = (int) pg_prop_draw_int(tc, 0, lengthof(pi_kinds) - 1); + if (pi_kinds[kind].uses_g && !has_g) + kind -= 5; /* the first five kinds don't use g */ + kinds[k] = kind; + if (pi_kinds[kind].constraint) + appendStringInfo(&s, "ALTER TABLE cfc2_r ADD CONSTRAINT cfc2_rx%d %s; ", + k, pi_kinds[kind].def); + else + appendStringInfo(&s, "CREATE %sINDEX cfc2_rx%d ON cfc2_r %s; ", + pi_kinds[kind].replident ? "UNIQUE " : "", k, + pi_kinds[kind].def); + } + c2_exec(tc, conn, s.data); + + /* + * Every index of the tree: "\t\t\t". + * Mutate each one's user-settable properties at random. + */ + map = c2_rows(tc, conn, + "SELECT substr(pg_partition_root(i.indexrelid)::regclass::text, 8), " + "quote_ident(t.relname), t.relkind, quote_ident(c.relname) " + "FROM pg_index i JOIN pg_class c ON c.oid = i.indexrelid " + "JOIN pg_class t ON t.oid = i.indrelid " + "WHERE i.indrelid IN (SELECT relid FROM pg_partition_tree('cfc2_r')) " + "ORDER BY 1, 2"); + for (line = strtok_r(map, "\n", &save); line; line = strtok_r(NULL, "\n", &save)) + { + char *f[4]; + char *fs = NULL; + const PiKind *pk; + bool leaf; + + f[0] = strtok_r(line, "\t", &fs); + f[1] = strtok_r(NULL, "\t", &fs); + f[2] = strtok_r(NULL, "\t", &fs); + f[3] = strtok_r(NULL, "\t", &fs); + if (!f[0] || !f[1] || !f[2] || !f[3]) + continue; + pk = &pi_kinds[kinds[atoi(f[0]) % nidx]]; + leaf = (f[2][0] == 'r'); + + resetStringInfo(&s); + if (pg_prop_draw_int(tc, 0, 3) == 0) + appendStringInfo(&s, "COMMENT ON INDEX %s IS 'c%d'; ", f[3], nmut); + if (pk->exprcol > 0 && pg_prop_draw_int(tc, 0, 2) == 0) + appendStringInfo(&s, "ALTER INDEX %s ALTER COLUMN %d SET STATISTICS %d; ", + f[3], pk->exprcol, (int) pg_prop_draw_int(tc, 0, 1000)); + if (leaf && pg_prop_draw_int(tc, 0, 3) == 0) + appendStringInfo(&s, "ALTER INDEX %s SET (fillfactor = %d); ", + f[3], (int) pg_prop_draw_int(tc, 50, 99)); + if (use_ts && pg_prop_draw_int(tc, 0, 3) == 0) + appendStringInfo(&s, "ALTER INDEX %s SET TABLESPACE %s; ", f[3], + pg_prop_draw_bool(tc) ? "cfc2_ts" : "pg_default"); + if (pk->replident && pg_prop_draw_int(tc, 0, 2) == 0) + appendStringInfo(&s, "ALTER TABLE %s REPLICA IDENTITY USING INDEX %s; ", + f[1], f[3]); + if (pk->clusterable && leaf && pg_prop_draw_int(tc, 0, 3) == 0) + appendStringInfo(&s, "ALTER TABLE %s CLUSTER ON %s; ", f[1], f[3]); + if (pg_prop_draw_int(tc, 0, 4) == 0) + appendStringInfo(&s, "ALTER INDEX %s DEPENDS ON EXTENSION plpgsql; ", f[3]); + if (pk->constraint && pg_prop_draw_int(tc, 0, 3) == 0) + appendStringInfo(&s, "COMMENT ON CONSTRAINT %s ON %s IS 'k%d'; ", + f[3], f[1], nmut); + /* rename last: the statements above use the current name */ + if (pg_prop_draw_int(tc, 0, 2) == 0) + appendStringInfo(&s, "ALTER INDEX %s RENAME TO cfc2_n%d; ", f[3], nmut); + nmut++; + if (s.len > 0) + c2_exec(tc, conn, s.data); + } + pg_free(map); + + before = c2_rows(tc, conn, PI_SNAPSHOT); + + /* the rebuild-forcing DDL */ + alter = (int) pg_prop_draw_int(tc, 0, has_g ? 2 : 1); + resetStringInfo(&s); + if (use_ts && pg_prop_draw_bool(tc)) + appendStringInfoString(&s, "SET default_tablespace = cfc2_ts; "); + switch (alter) + { + case 0: + appendStringInfoString(&s, "ALTER TABLE cfc2_r ALTER COLUMN a TYPE int"); + break; + case 1: + appendStringInfoString(&s, "ALTER TABLE cfc2_r ALTER COLUMN a TYPE bigint"); + break; + default: + appendStringInfoString(&s, "ALTER TABLE cfc2_r ALTER COLUMN g SET EXPRESSION AS (b * 3)"); + break; + } + pg_prop_note(tc, "rebuild DDL: %s", s.data); + c2_exec(tc, conn, s.data); + c2_exec(tc, conn, "RESET default_tablespace"); + + after = c2_rows(tc, conn, PI_SNAPSHOT); + if (strcmp(before, after) != 0) + pg_prop_fail(tc, + "index rebuild changed index properties of the partition tree\n" + " DDL: %s\n" + " (root, table, index, comment, reloptions, tablespace, stattargets, " + "indisreplident, indisclustered, relreplident, extdeps, constraint)\n" + " before:\n%s after:\n%s", + s.data, before, after); + + c2_exec(tc, conn, pi_cleanup); + if (use_ts) + c2_exec(tc, conn, "DROP TABLESPACE cfc2_ts"); + pg_free(before); + pg_free(after); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------------- + * CF 6579: REFRESH MATERIALIZED VIEW CONCURRENTLY == the defining query + * ---------------------------------------------------------------------- + */ + +/* append a drawn value for a column of the given kind (0 int, 1 text) */ +static void +mv_value(pg_prop_tc *tc, StringInfo b, int kind, int dom) +{ + int v = (int) pg_prop_draw_int(tc, -1, dom); + + if (v < 0) + appendStringInfoString(b, "NULL"); + else if (kind == 0) + appendStringInfo(b, "%d", v); + else + appendStringInfo(b, "'t%d'", v); +} + +/* multiset digest of a relation/query's rows */ +static char * +mv_digest(pg_prop_tc *tc, pg_prop_conn *conn, const char *from) +{ + StringInfoData q; + bool failed = false; + char *err = NULL; + char *v; + + initStringInfo(&q); + appendStringInfo(&q, + "SELECT count(*) || ':' || coalesce(md5(string_agg(x::text, '|' ORDER BY x::text)), '') " + "FROM (%s) x", from); + v = pg_prop_sql_scalar(conn, q.data, &failed, &err); + if (failed || v == NULL) + pg_prop_fail(tc, "digest failed: %s\n SQL: %s", err ? err : "", q.data); + pfree(q.data); + return v; +} + +/* + * After every REFRESH MATERIALIZED VIEW CONCURRENTLY: if it succeeded, the + * matview holds exactly the multiset of rows its query returns now; if it + * failed, the matview is unchanged. (A unique index may make the refresh + * fail; it may never make it silently drop or merge rows.) Base tables get + * duplicate rows, NULLs in indexed and non-indexed columns, and a random set + * of unique indexes on the matview. + */ +PG_PROP(refresh_concurrently_matches_query) +{ + pg_prop_conn *conn = c2_connect(tc); + StringInfoData s; + int ncols = (int) pg_prop_draw_int(tc, 1, 4); + int dom = (int) pg_prop_draw_int(tc, 0, 4); + int kinds[4]; + int nidx; + int rounds = (int) pg_prop_draw_int(tc, 1, 4); + bool filtered = pg_prop_draw_bool(tc); + const char *query; + + c2_exec(tc, conn, "DROP TABLE IF EXISTS cfc2_mf CASCADE"); + initStringInfo(&s); + appendStringInfoString(&s, "CREATE TABLE cfc2_mf ("); + for (int c = 0; c < ncols; c++) + { + kinds[c] = (int) pg_prop_draw_int(tc, 0, 1); + appendStringInfo(&s, "%sc%d %s", c ? ", " : "", c, kinds[c] ? "text" : "int"); + } + appendStringInfoString(&s, ")"); + c2_exec(tc, conn, s.data); + + /* initial rows */ + resetStringInfo(&s); + appendStringInfoString(&s, "INSERT INTO cfc2_mf VALUES "); + { + int n = (int) pg_prop_draw_int(tc, 1, 8); + + for (int r = 0; r < n; r++) + { + appendStringInfoString(&s, r ? ", (" : "("); + for (int c = 0; c < ncols; c++) + { + if (c) + appendStringInfoString(&s, ", "); + mv_value(tc, &s, kinds[c], dom); + } + appendStringInfoChar(&s, ')'); + } + } + /* may violate a unique index created below; then just start empty */ + c2_try(conn, s.data); + + query = filtered ? "SELECT * FROM cfc2_mf WHERE c0::text IS DISTINCT FROM '0'" + : "SELECT * FROM cfc2_mf"; + resetStringInfo(&s); + appendStringInfo(&s, "CREATE MATERIALIZED VIEW cfc2_mv AS %s", query); + c2_exec(tc, conn, s.data); + + /* unique indexes on drawn column subsets; at least one must succeed */ + nidx = 0; + for (int k = 0; k < 3 && (nidx == 0 || pg_prop_draw_bool(tc)); k++) + { + bool first = true; + + resetStringInfo(&s); + appendStringInfo(&s, "CREATE UNIQUE INDEX cfc2_mv_u%d ON cfc2_mv (", k); + for (int c = 0; c < ncols; c++) + { + if (pg_prop_draw_bool(tc) || (first && c == ncols - 1)) + { + appendStringInfo(&s, "%sc%d", first ? "" : ", ", c); + first = false; + } + } + appendStringInfo(&s, ")%s", + pg_prop_draw_int(tc, 0, 3) == 0 ? " NULLS NOT DISTINCT" : ""); + if (c2_try(conn, s.data)) + nidx++; + } + if (nidx == 0) + { + /* initial data violates every drawn index: empty it and retry once */ + c2_exec(tc, conn, "TRUNCATE cfc2_mf; REFRESH MATERIALIZED VIEW cfc2_mv; " + "CREATE UNIQUE INDEX cfc2_mv_u9 ON cfc2_mv (c0)"); + } + + for (int r = 0; r < rounds; r++) + { + int nmod = (int) pg_prop_draw_int(tc, 1, 4); + char *before; + char *after; + char *want; + char *err = NULL; + bool ok; + + /* mutate the base table: duplicate existing rows, insert, update, delete */ + for (int m = 0; m < nmod; m++) + { + resetStringInfo(&s); + switch ((int) pg_prop_draw_int(tc, 0, 3)) + { + case 0: + appendStringInfo(&s, + "INSERT INTO cfc2_mf SELECT * FROM cfc2_mf " + "ORDER BY ctid LIMIT %d OFFSET %d", + (int) pg_prop_draw_int(tc, 1, 3), + (int) pg_prop_draw_int(tc, 0, 6)); + break; + case 1: + appendStringInfoString(&s, "INSERT INTO cfc2_mf VALUES ("); + for (int c = 0; c < ncols; c++) + { + if (c) + appendStringInfoString(&s, ", "); + mv_value(tc, &s, kinds[c], dom); + } + appendStringInfoChar(&s, ')'); + break; + case 2: + { + int c = (int) pg_prop_draw_int(tc, 0, ncols - 1); + + appendStringInfo(&s, "UPDATE cfc2_mf SET c%d = ", c); + mv_value(tc, &s, kinds[c], dom); + appendStringInfo(&s, " WHERE ctid IN (SELECT ctid FROM cfc2_mf " + "ORDER BY ctid LIMIT %d OFFSET %d)", + (int) pg_prop_draw_int(tc, 1, 2), + (int) pg_prop_draw_int(tc, 0, 6)); + } + break; + default: + appendStringInfo(&s, + "DELETE FROM cfc2_mf WHERE ctid IN (SELECT ctid FROM cfc2_mf " + "ORDER BY ctid LIMIT %d OFFSET %d)", + (int) pg_prop_draw_int(tc, 1, 2), + (int) pg_prop_draw_int(tc, 0, 6)); + break; + } + c2_exec(tc, conn, s.data); + } + + before = mv_digest(tc, conn, "SELECT * FROM cfc2_mv"); + ok = pg_prop_sql_exec(conn, "REFRESH MATERIALIZED VIEW CONCURRENTLY cfc2_mv", &err); + after = mv_digest(tc, conn, "SELECT * FROM cfc2_mv"); + want = mv_digest(tc, conn, query); + if (ok && strcmp(after, want) != 0) + { + char *got = c2_rows(tc, conn, "SELECT * FROM cfc2_mv ORDER BY 1 LIMIT 20"); + char *exp = c2_rows(tc, conn, + filtered ? "SELECT * FROM cfc2_mf WHERE c0::text IS DISTINCT FROM '0' ORDER BY 1 LIMIT 20" + : "SELECT * FROM cfc2_mf ORDER BY 1 LIMIT 20"); + + pg_prop_fail(tc, + "REFRESH MATERIALIZED VIEW CONCURRENTLY succeeded but the matview " + "differs from its query (round %d)\n matview %s:\n%s query %s:\n%s", + r, after, got, want, exp); + } + if (!ok && strcmp(before, after) != 0) + pg_prop_fail(tc, "failed REFRESH CONCURRENTLY (%s) changed the matview", + err ? err : ""); + if (err) + pg_free(err); + pg_free(before); + pg_free(after); + pg_free(want); + } + + c2_exec(tc, conn, "DROP TABLE cfc2_mf CASCADE"); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------------- + * CF 6770: UPDATE/DELETE through postgres_fdw on a remote partitioned table + * ---------------------------------------------------------------------- + */ + +/* + * A foreign table whose remote side is a partitioned (or inheritance) table + * must behave like the remote table itself: an UPDATE or DELETE through it + * changes exactly the rows its WHERE clause selects. The model is a plain + * local table holding the same rows, to which the same statement is applied + * directly. The WHERE clause sometimes contains a non-shippable volatile + * call so postgres_fdw cannot push the statement down and must address + * remote rows one by one (by ctid -- which is not unique across partitions). + */ +PG_PROP(fdw_partitioned_remote_dml_matches_local) +{ + pg_prop_conn *conn = c2_connect(tc); + StringInfoData s; + StringInfoData w; + int nparts = (int) pg_prop_draw_int(tc, 1, 4); + bool inherit = pg_prop_draw_bool(tc); + int nrows = (int) pg_prop_draw_int(tc, 1, 12); + int nstmts = (int) pg_prop_draw_int(tc, 1, 3); + + if (!pg_prop_require_extension(conn, "postgres_fdw")) + pg_prop_skip(tc, "postgres_fdw not installed"); + + initStringInfo(&s); + initStringInfo(&w); + c2_exec(tc, conn, + "DROP FOREIGN TABLE IF EXISTS cfc2_ft; " + "DROP TABLE IF EXISTS cfc2_fr, cfc2_fl CASCADE; " + "DROP SERVER IF EXISTS cfc2_loop CASCADE"); + c2_exec(tc, conn, + "DO $$ BEGIN EXECUTE format('CREATE SERVER cfc2_loop FOREIGN DATA WRAPPER postgres_fdw " + "OPTIONS (dbname %L, port %L)', current_database(), current_setting('port')); " + "EXECUTE format('CREATE USER MAPPING FOR CURRENT_USER SERVER cfc2_loop " + "OPTIONS (user %L)', current_user); END $$"); + + if (inherit) + { + appendStringInfoString(&s, "CREATE TABLE cfc2_fr (a int, b int); "); + for (int p = 0; p < nparts; p++) + appendStringInfo(&s, + "CREATE TABLE cfc2_fr%d (CHECK (a = %d)) INHERITS (cfc2_fr); ", p, p); + } + else + { + appendStringInfoString(&s, "CREATE TABLE cfc2_fr (a int, b int) PARTITION BY LIST (a); "); + for (int p = 0; p < nparts; p++) + appendStringInfo(&s, + "CREATE TABLE cfc2_fr%d PARTITION OF cfc2_fr FOR VALUES IN (%d); ", p, p); + } + appendStringInfoString(&s, "CREATE TABLE cfc2_fl (a int, b int); "); + c2_exec(tc, conn, s.data); + + /* the same rows on both sides; rows go straight into their partition */ + for (int r = 0; r < nrows; r++) + { + int a = (int) pg_prop_draw_int(tc, 0, nparts - 1); + int b = (int) pg_prop_draw_int(tc, 0, 5); + + resetStringInfo(&s); + appendStringInfo(&s, + "INSERT INTO cfc2_fr%d VALUES (%d, %d); INSERT INTO cfc2_fl VALUES (%d, %d)", + a, a, b, a, b); + c2_exec(tc, conn, s.data); + } + c2_exec(tc, conn, + "CREATE FOREIGN TABLE cfc2_ft (a int, b int) SERVER cfc2_loop " + "OPTIONS (table_name 'cfc2_fr')"); + + for (int k = 0; k < nstmts; k++) + { + char *got; + char *want; + char *ng; + char *nw; + + resetStringInfo(&w); + switch ((int) pg_prop_draw_int(tc, 0, 3)) + { + case 0: + appendStringInfo(&w, "a = %d", (int) pg_prop_draw_int(tc, 0, nparts - 1)); + break; + case 1: + appendStringInfo(&w, "b = %d", (int) pg_prop_draw_int(tc, 0, 5)); + break; + case 2: + appendStringInfo(&w, "a = %d AND b <= %d", + (int) pg_prop_draw_int(tc, 0, nparts - 1), + (int) pg_prop_draw_int(tc, 0, 5)); + break; + default: + appendStringInfo(&w, "b > %d", (int) pg_prop_draw_int(tc, 0, 5)); + break; + } + if (pg_prop_draw_int(tc, 0, 3) != 0) + appendStringInfoString(&w, " AND random() <= 1"); /* not shippable */ + + resetStringInfo(&s); + if (pg_prop_draw_bool(tc)) + appendStringInfo(&s, + "WITH m AS (UPDATE %%s SET b = b + 10 WHERE %s RETURNING 1) " + "SELECT count(*) FROM m", w.data); + else + appendStringInfo(&s, + "WITH m AS (DELETE FROM %%s WHERE %s RETURNING 1) " + "SELECT count(*) FROM m", w.data); + { + char buf[1024]; + bool failed = false; + char *err = NULL; + + snprintf(buf, sizeof(buf), s.data, "cfc2_ft"); + pg_prop_note(tc, "stmt %d: %s", k, buf); + ng = pg_prop_sql_scalar(conn, buf, &failed, &err); + if (failed) + pg_prop_fail(tc, "foreign DML failed: %s\n SQL: %s", err ? err : "", buf); + snprintf(buf, sizeof(buf), s.data, "cfc2_fl"); + nw = pg_prop_sql_scalar(conn, buf, &failed, &err); + if (failed) + pg_prop_fail(tc, "local DML failed: %s\n SQL: %s", err ? err : "", buf); + } + got = c2_rows(tc, conn, "SELECT a, b FROM cfc2_fr ORDER BY a, b"); + want = c2_rows(tc, conn, "SELECT a, b FROM cfc2_fl ORDER BY a, b"); + if (strcmp(got, want) != 0 || strcmp(ng ? ng : "", nw ? nw : "") != 0) + pg_prop_fail(tc, + "DML through a foreign table on a remote %s table differs from " + "the same DML on a local table\n WHERE %s\n" + " rows reported: foreign %s, local %s\n" + " remote now (a, b):\n%s expected:\n%s", + inherit ? "inheritance" : "partitioned", w.data, + ng ? ng : "?", nw ? nw : "?", got, want); + pg_free(got); + pg_free(want); + } + + c2_exec(tc, conn, + "DROP FOREIGN TABLE cfc2_ft; DROP TABLE cfc2_fr, cfc2_fl CASCADE; " + "DROP SERVER cfc2_loop CASCADE"); + pfree(s.data); + pfree(w.data); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------------- + * CF 6842: auto-named constraints on a partition tree spanning schemas + * ---------------------------------------------------------------------- + */ + +/* + * An auto-named ALTER TABLE root ADD CHECK / SET NOT NULL / ADD NOT NULL + * picks the name itself, so it must never fail with "constraint already + * exists" (42710): the server is free to pick any unused name, whatever + * names the partitions' existing constraints have and whatever schemas the + * partitions live in. Afterwards every partition has the new constraint + * (inherited: same name for CHECK, a NOT NULL on the column for NOT NULL). + * The partitions' pre-existing constraints are drawn to take exactly the + * names the root would pick, or numbered variants of them. + */ +PG_PROP(partition_autoname_constraint_never_collides) +{ + static const char *const schemas[] = {"public", "cfc2_s1", "cfc2_s2"}; + static const char *const ddl[] = { + "ADD CHECK (a > -50)", + "ADD CHECK (a + b > -50)", + "ALTER COLUMN a SET NOT NULL", + "ADD NOT NULL a", + "ALTER COLUMN b SET NOT NULL", + }; + static const char *const suffix[] = { + "_a_check", "_check", "_a_not_null", "_b_not_null", "_b_check", "_a_check1", + "_a_b_check", + }; + pg_prop_conn *conn = c2_connect(tc); + StringInfoData s; + const char *rootschema = schemas[pg_prop_draw_int(tc, 0, 1)]; + bool mid = pg_prop_draw_bool(tc); + int nleaf = (int) pg_prop_draw_int(tc, 1, 3); + int d = (int) pg_prop_draw_int(tc, 0, lengthof(ddl) - 1); + char leafq[4][64]; + char parentq[64]; + char *err = NULL; + long missing; + + initStringInfo(&s); + c2_exec(tc, conn, + "DROP TABLE IF EXISTS public.cfc2_pp, cfc2_s1.cfc2_pp CASCADE; " + "DROP SCHEMA IF EXISTS cfc2_s1, cfc2_s2 CASCADE; " + "DROP TABLE IF EXISTS public.cfc2_pp_mid, public.cfc2_pp_0, public.cfc2_pp_1, " + "public.cfc2_pp_2 CASCADE; " + "CREATE SCHEMA cfc2_s1; CREATE SCHEMA cfc2_s2"); + appendStringInfo(&s, "CREATE TABLE %s.cfc2_pp (a int, b int) PARTITION BY RANGE (a); ", + rootschema); + if (mid) + { + snprintf(parentq, sizeof(parentq), "%s.cfc2_pp_mid", + schemas[pg_prop_draw_int(tc, 0, 2)]); + appendStringInfo(&s, + "CREATE TABLE %s PARTITION OF %s.cfc2_pp " + "FOR VALUES FROM (0) TO (1000) PARTITION BY RANGE (a); ", + parentq, rootschema); + } + else + snprintf(parentq, sizeof(parentq), "%s.cfc2_pp", rootschema); + for (int i = 0; i < nleaf; i++) + { + const char *ls = schemas[pg_prop_draw_int(tc, 0, 2)]; + bool samename = strcmp(ls, rootschema) != 0 && pg_prop_draw_bool(tc); + char cand[64]; + + /* a leaf may share the root's relname in another schema (once) */ + snprintf(cand, sizeof(cand), "%s.cfc2_pp", ls); + for (int j = 0; j < i; j++) + if (strcmp(leafq[j], cand) == 0) + samename = false; + if (samename) + strlcpy(leafq[i], cand, sizeof(leafq[i])); + else + snprintf(leafq[i], sizeof(leafq[i]), "%s.cfc2_pp_%d", ls, i); + appendStringInfo(&s, "CREATE TABLE %s PARTITION OF %s FOR VALUES FROM (%d) TO (%d); ", + leafq[i], parentq, i * 10, i * 10 + 10); + } + appendStringInfo(&s, "INSERT INTO %s.cfc2_pp SELECT g, g FROM generate_series(0, %d) g; ", + rootschema, nleaf * 10 - 1); + c2_exec(tc, conn, s.data); + + /* pre-existing constraints on partitions, named like the root would */ + for (int i = 0; i < nleaf; i++) + { + int nc = (int) pg_prop_draw_int(tc, 0, 2); + + for (int k = 0; k < nc; k++) + { + const char *suf = suffix[pg_prop_draw_int(tc, 0, lengthof(suffix) - 1)]; + + resetStringInfo(&s); + switch ((int) pg_prop_draw_int(tc, 0, 2)) + { + case 0: + appendStringInfo(&s, "ALTER TABLE %s ADD CONSTRAINT cfc2_pp%s CHECK (a > -%d)", + leafq[i], suf, (int) pg_prop_draw_int(tc, 1, 99)); + break; + case 1: + appendStringInfo(&s, "ALTER TABLE %s ADD CONSTRAINT cfc2_pp%s CHECK (a IS NOT NULL)", + leafq[i], suf); + break; + default: + appendStringInfo(&s, "ALTER TABLE %s ADD CHECK (a > -%d)", + leafq[i], (int) pg_prop_draw_int(tc, 1, 99)); + break; + } + /* duplicate names on the same leaf are legitimately refused */ + c2_try(conn, s.data); + } + } + + resetStringInfo(&s); + appendStringInfo(&s, "ALTER TABLE %s.cfc2_pp %s", rootschema, ddl[d]); + if (!pg_prop_sql_exec(conn, s.data, &err)) + { + char *cons; + char *sql = pg_strdup(s.data); + + resetStringInfo(&s); + appendStringInfo(&s, + "SELECT conrelid::regclass, conname, contype FROM pg_constraint " + "WHERE conrelid IN (SELECT relid FROM pg_partition_tree('%s.cfc2_pp')) " + "ORDER BY 1, 2", rootschema); + cons = c2_rows(tc, conn, s.data); + + pg_prop_fail(tc, "auto-named constraint DDL on the partition root failed: %s" + " SQL: %s\n existing constraints:\n%s", + err ? err : "", sql, cons); + } + + /* every partition inherited it */ + resetStringInfo(&s); + if (d <= 1) + appendStringInfo(&s, + "SELECT count(*) FROM pg_partition_tree('%s.cfc2_pp') t " + "WHERE NOT EXISTS (SELECT 1 FROM pg_constraint c, pg_constraint r " + "WHERE r.conrelid = '%s.cfc2_pp'::regclass AND r.contype = 'c' " + "AND r.conislocal AND c.conrelid = t.relid AND c.contype = 'c' " + "AND c.conname = r.conname AND (c.conrelid = r.conrelid OR c.coninhcount > 0))", + rootschema, rootschema); + else + appendStringInfo(&s, + "SELECT count(*) FROM pg_partition_tree('%s.cfc2_pp') t " + "WHERE NOT EXISTS (SELECT 1 FROM pg_constraint c JOIN pg_attribute a " + "ON a.attrelid = c.conrelid AND a.attnum = c.conkey[1] " + "WHERE c.conrelid = t.relid AND c.contype = 'n' AND a.attname = '%c')", + rootschema, d == 4 ? 'b' : 'a'); + missing = c2_count(tc, conn, s.data); + if (missing) + pg_prop_fail(tc, "%ld partitions lack the constraint added by ALTER TABLE ... %s", + missing, ddl[d]); + + c2_exec(tc, conn, + "DROP TABLE IF EXISTS public.cfc2_pp, cfc2_s1.cfc2_pp CASCADE; " + "DROP SCHEMA IF EXISTS cfc2_s1, cfc2_s2 CASCADE"); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------------- + * CF 6961: REVOKE keeps every grant chain rooted at the owner + * ---------------------------------------------------------------------- + */ + +#define ACL_NROLES 4 + +/* entries (grantor, grantee, privilege) whose grantor no longer holds the option */ +#define ACL_ORPHANS \ + "SELECT coalesce(string_agg(format('%s granted %s to %s', a.grantor::regrole, " \ + " a.privilege_type, CASE a.grantee WHEN 0 THEN 'PUBLIC' ELSE a.grantee::regrole::text END), " \ + " '; '), '') " \ + "FROM pg_class c, aclexplode(c.relacl) a " \ + "WHERE c.oid = 'cfc2_acl'::regclass AND a.grantor <> c.relowner " \ + "AND NOT EXISTS (SELECT 1 FROM aclexplode(c.relacl) b WHERE b.grantee = a.grantor " \ + " AND b.privilege_type = a.privilege_type AND b.is_grantable)" + +static const char *acl_cleanup = +"RESET SESSION AUTHORIZATION; DROP TABLE IF EXISTS cfc2_acl; " +"DROP ROLE IF EXISTS cfc2_r0, cfc2_r1, cfc2_r2, cfc2_r3"; + +/* + * A grant made by a non-owner grantor is only legitimate while the grantor + * holds the grant option directly in the ACL; REVOKE without CASCADE must + * refuse to break that ("dependent privileges exist") and REVOKE ... CASCADE + * must remove the dependants. So after any sequence of GRANT / REVOKE / + * role-membership / SUPERUSER changes, issued by drawn roles, every ACL entry + * whose grantor is not the owner is backed by a grant-option entry for the + * same privilege held by that grantor. (This is also what pg_dump needs to + * recreate the ACL.) Statements may legitimately fail (permissions); the + * invariant is checked after every one. + */ +PG_PROP(revoke_keeps_grant_chains_rooted) +{ + static const char *const privs[] = {"SELECT", "UPDATE"}; + pg_prop_conn *conn = c2_connect(tc); + StringInfoData s; + StringInfoData log; + int nsteps = (int) pg_prop_draw_int(tc, 1, 16); + bool holder[2][ACL_NROLES]; + + initStringInfo(&s); + initStringInfo(&log); + /* "no privileges were granted" warnings are expected noise here */ + c2_exec(tc, conn, "SET client_min_messages = error"); + c2_exec(tc, conn, acl_cleanup); + c2_exec(tc, conn, + "CREATE ROLE cfc2_r0; CREATE ROLE cfc2_r1; CREATE ROLE cfc2_r2; CREATE ROLE cfc2_r3; " + "CREATE TABLE cfc2_acl (a int); ALTER TABLE cfc2_acl OWNER TO cfc2_r0"); + + /* random memberships (r_i member of r_j), often of the owner */ + { + int nm = (int) pg_prop_draw_int(tc, 0, 3); + + for (int m = 0; m < nm; m++) + { + int of = pg_prop_draw_bool(tc) ? 0 : (int) pg_prop_draw_int(tc, 0, ACL_NROLES - 1); + int who = (int) pg_prop_draw_int(tc, 1, ACL_NROLES - 1); + + resetStringInfo(&s); + appendStringInfo(&s, "GRANT cfc2_r%d TO cfc2_r%d%s", of, who, + pg_prop_draw_int(tc, 0, 3) == 0 ? " WITH INHERIT FALSE" : ""); + if (of != who && c2_try(conn, s.data)) + appendStringInfo(&log, "%s;\n", s.data); + } + } + + /* + * holder[p][r]: r was (last we know) granted privs[p] WITH GRANT OPTION. + * Only steers the generator toward actors that can extend a chain; the + * invariant is computed from the catalog, not from this. + */ + memset(holder, 0, sizeof(holder)); + for (int p = 0; p < (int) lengthof(privs); p++) + holder[p][0] = true; /* the owner */ + + for (int k = 0; k < nsteps; k++) + { + int pi = (int) pg_prop_draw_int(tc, 0, lengthof(privs) - 1); + const char *priv = privs[pi]; + int actor = (int) pg_prop_draw_int(tc, 0, ACL_NROLES - 1); + int target = (int) pg_prop_draw_int(tc, 1, ACL_NROLES - 1); + int op = (int) pg_prop_draw_int(tc, 0, 5); + bool go = false; + char *orph; + + /* usually let a role that holds the option act */ + if (op <= 3 && pg_prop_draw_int(tc, 0, 3) != 0) + { + int start = actor; + + while (!holder[pi][actor]) + { + actor = (actor + 1) % ACL_NROLES; + if (actor == start) + break; + } + } + resetStringInfo(&s); + switch (op) + { + case 0: + case 1: + go = pg_prop_draw_int(tc, 0, 2) != 0; + appendStringInfo(&s, "SET SESSION AUTHORIZATION cfc2_r%d; " + "GRANT %s ON cfc2_acl TO cfc2_r%d%s", + actor, priv, target, go ? " WITH GRANT OPTION" : ""); + break; + case 2: + case 3: + appendStringInfo(&s, "SET SESSION AUTHORIZATION cfc2_r%d; " + "REVOKE %s%s ON cfc2_acl FROM cfc2_r%d %s", + actor, + pg_prop_draw_bool(tc) ? "GRANT OPTION FOR " : "", + priv, target, + pg_prop_draw_int(tc, 0, 2) != 0 ? "RESTRICT" : "CASCADE"); + break; + case 4: + appendStringInfo(&s, "GRANT cfc2_r%d TO cfc2_r%d", + pg_prop_draw_bool(tc) ? 0 : (int) pg_prop_draw_int(tc, 0, ACL_NROLES - 1), + target); + break; + default: + appendStringInfo(&s, "ALTER ROLE cfc2_r%d %sSUPERUSER", target, + pg_prop_draw_bool(tc) ? "" : "NO"); + break; + } + if (c2_try(conn, s.data)) + { + appendStringInfo(&log, "%s;\n", s.data); + if (go) + holder[pi][target] = true; + } + else + appendStringInfo(&log, "%s; -- failed\n", s.data); + c2_exec(tc, conn, "RESET SESSION AUTHORIZATION"); + + orph = pg_prop_sql_scalar(conn, ACL_ORPHANS, NULL, NULL); + if (orph && orph[0] != '\0') + { + char *acl = pg_prop_sql_scalar(conn, + "SELECT relacl::text FROM pg_class WHERE oid = 'cfc2_acl'::regclass", + NULL, NULL); + + pg_prop_fail(tc, + "ACL has grants whose grantor does not hold the grant option " + "(broken grant chain): %s\n relacl: %s\n statements:\n%s", + orph, acl ? acl : "?", log.data); + } + if (orph) + pg_free(orph); + } + + c2_exec(tc, conn, acl_cleanup); + pfree(s.data); + pfree(log.data); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------------- + * CF 7055: every effective ALTER PUBLICATION is reported to event triggers + * ---------------------------------------------------------------------- + */ + +/* everything ALTER PUBLICATION can change about cfc2_ep */ +#define EP_STATE \ + "SELECT concat_ws(' | ', " \ + " (SELECT format('%s %s %s %s %s %s %s %s %s', puballtables, puballsequences, pubinsert, " \ + " pubupdate, pubdelete, pubtruncate, pubviaroot, pubgencols, pubowner) " \ + " FROM pg_publication WHERE pubname = 'cfc2_ep'), " \ + " (SELECT string_agg(format('%s/%s/%s/%s', prrelid::regclass, prexcept, " \ + " pg_get_expr(prqual, prrelid), prattrs), ',' ORDER BY prrelid::regclass::text) " \ + " FROM pg_publication_rel r JOIN pg_publication p ON p.oid = r.prpubid " \ + " WHERE p.pubname = 'cfc2_ep'), " \ + " (SELECT string_agg(pnnspid::regnamespace::text, ',' ORDER BY pnnspid::regnamespace::text) " \ + " FROM pg_publication_namespace n JOIN pg_publication p ON p.oid = n.pnpubid " \ + " WHERE p.pubname = 'cfc2_ep'))" + +static const char *ep_cleanup = +"DROP EVENT TRIGGER IF EXISTS cfc2_evt; DROP FUNCTION IF EXISTS cfc2_etf(); " +"DROP PUBLICATION IF EXISTS cfc2_ep; DROP TABLE IF EXISTS cfc2_etlog, cfc2_et0, cfc2_et1, cfc2_et2; " +"DROP SCHEMA IF EXISTS cfc2_es0, cfc2_es1 CASCADE"; + +/* + * pg_event_trigger_ddl_commands() at ddl_command_end lists the commands the + * statement executed. So an ALTER PUBLICATION that succeeds and changes the + * publication's catalog state (its flags, its table list with row filters and + * column lists, its schema list) must be reported at least once. A random + * starting publication and a random sequence of ALTER PUBLICATION forms (ADD, + * DROP, SET of tables / schemas, SET (publish = ...), SET ALL TABLES [EXCEPT], + * SET ALL SEQUENCES) are applied; statements may fail legitimately. + */ +PG_PROP(alter_publication_reported_to_event_triggers) +{ + pg_prop_conn *conn = c2_connect(tc); + StringInfoData s; + int nsteps = (int) pg_prop_draw_int(tc, 1, 8); + + initStringInfo(&s); + c2_exec(tc, conn, "SET client_min_messages = error"); + c2_exec(tc, conn, ep_cleanup); + c2_exec(tc, conn, + "CREATE TABLE cfc2_et0 (a int, b int); CREATE TABLE cfc2_et1 (a int, b int); " + "CREATE TABLE cfc2_et2 (a int, b int); " + "CREATE SCHEMA cfc2_es0; CREATE TABLE cfc2_es0.t (a int); " + "CREATE SCHEMA cfc2_es1; CREATE TABLE cfc2_es1.t (a int); " + "CREATE TABLE cfc2_etlog (tag text, otype text, ident text); " + "CREATE FUNCTION cfc2_etf() RETURNS event_trigger LANGUAGE plpgsql AS $$ " + "DECLARE r record; BEGIN FOR r IN SELECT * FROM pg_event_trigger_ddl_commands() LOOP " + "INSERT INTO cfc2_etlog VALUES (r.command_tag, r.object_type, r.object_identity); " + "END LOOP; END $$; " + "CREATE EVENT TRIGGER cfc2_evt ON ddl_command_end WHEN TAG IN ('ALTER PUBLICATION') " + "EXECUTE FUNCTION cfc2_etf()"); + + switch ((int) pg_prop_draw_int(tc, 0, 3)) + { + case 0: + c2_exec(tc, conn, "CREATE PUBLICATION cfc2_ep"); + break; + case 1: + c2_exec(tc, conn, "CREATE PUBLICATION cfc2_ep FOR TABLE cfc2_et0, cfc2_et1, cfc2_et2"); + break; + case 2: + c2_exec(tc, conn, "CREATE PUBLICATION cfc2_ep FOR TABLES IN SCHEMA cfc2_es0, cfc2_es1, " + "TABLE cfc2_et0"); + break; + default: + c2_exec(tc, conn, "CREATE PUBLICATION cfc2_ep FOR ALL TABLES"); + break; + } + + for (int k = 0; k < nsteps; k++) + { + int t1 = (int) pg_prop_draw_int(tc, 0, 2); + int t2 = (int) pg_prop_draw_int(tc, 0, 2); + int sc = (int) pg_prop_draw_int(tc, 0, 1); + char *before; + char *after; + long reported; + char *err = NULL; + bool ok; + + resetStringInfo(&s); + appendStringInfoString(&s, "ALTER PUBLICATION cfc2_ep "); + switch ((int) pg_prop_draw_int(tc, 0, 10)) + { + case 0: + appendStringInfo(&s, "ADD TABLE cfc2_et%d%s", t1, + pg_prop_draw_bool(tc) ? " WHERE (a > 0)" : ""); + break; + case 1: + appendStringInfo(&s, "DROP TABLE cfc2_et%d", t1); + break; + case 2: + appendStringInfo(&s, "DROP TABLE cfc2_et%d, cfc2_et%d", t1, t2); + break; + case 3: + appendStringInfo(&s, "SET TABLE cfc2_et%d", t1); + break; + case 4: + appendStringInfo(&s, "SET TABLE cfc2_et%d (a), cfc2_et%d", t1, t2); + break; + case 5: + appendStringInfo(&s, "ADD TABLES IN SCHEMA cfc2_es%d", sc); + break; + case 6: + appendStringInfo(&s, "DROP TABLES IN SCHEMA cfc2_es%d", sc); + break; + case 7: + appendStringInfo(&s, "SET TABLES IN SCHEMA cfc2_es%d", sc); + break; + case 8: + appendStringInfo(&s, "SET (publish = '%s')", + pg_prop_draw_bool(tc) ? "insert" : "insert, update, delete"); + break; + case 9: + if (pg_prop_draw_bool(tc)) + appendStringInfo(&s, "SET ALL TABLES EXCEPT (TABLE cfc2_et%d)", t1); + else + appendStringInfoString(&s, "SET ALL TABLES"); + break; + default: + appendStringInfoString(&s, "SET ALL SEQUENCES"); + break; + } + + c2_exec(tc, conn, "TRUNCATE cfc2_etlog"); + before = pg_prop_sql_scalar(conn, EP_STATE, NULL, NULL); + ok = pg_prop_sql_exec(conn, s.data, &err); + after = pg_prop_sql_scalar(conn, EP_STATE, NULL, NULL); + reported = c2_count(tc, conn, "SELECT count(*) FROM cfc2_etlog"); + if (ok && reported == 0 && + strcmp(before ? before : "", after ? after : "") != 0) + pg_prop_fail(tc, + "ALTER PUBLICATION changed the publication but " + "pg_event_trigger_ddl_commands() reported nothing\n" + " SQL: %s\n before: %s\n after: %s", + s.data, before ? before : "", after ? after : ""); + if (err) + pg_free(err); + if (before) + pg_free(before); + if (after) + pg_free(after); + } + + c2_exec(tc, conn, ep_cleanup); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------------- + * CF 7210: a publication's table list is order-independent + * ---------------------------------------------------------------------- + */ + +static const char *pl_cleanup = +"DROP PUBLICATION IF EXISTS cfc2_pl0, cfc2_pl1; " +"DROP TABLE IF EXISTS cfc2_pt0, cfc2_pt1, cfc2_pt2, cfc2_pt3, cfc2_pt4 CASCADE"; + +/* + * A publication's table list is a set of mentions; "t" adds t and its + * descendants, "ONLY t" adds t alone. If the server accepts a list, the + * tables it publishes cannot depend on the order of the mentions. The same + * drawn mention list over a drawn inheritance / partition forest is given in + * two orders (via CREATE PUBLICATION or ALTER PUBLICATION SET TABLE); when + * both are accepted, the published-table sets must match. + * + * Accept-vs-reject is deliberately not compared: the CF 7210 patch rejects + * "TABLE parent, TABLE child" ("conflicting ONLY specifications") yet accepts + * "TABLE child, TABLE parent", because a child added by a parent's recursion + * is in relids but not in relids_recursing. That is arguably a patch bug, + * but this property targets the silent first-mention-wins bug. + */ +PG_PROP(publication_table_list_order_independent) +{ + pg_prop_conn *conn = c2_connect(tc); + StringInfoData s; + int ntab = (int) pg_prop_draw_int(tc, 2, 5); + int nment = (int) pg_prop_draw_int(tc, 1, 5); + int ment[5]; + int only[5]; + int perm[5]; + bool ok[2]; + char *set[2]; + char *stmt[2]; + char *err[2] = {NULL, NULL}; + + initStringInfo(&s); + c2_exec(tc, conn, "SET client_min_messages = error"); + c2_exec(tc, conn, pl_cleanup); + + /* table 0 is a root; table i > 0 inherits from (or partitions) an earlier one */ + if (pg_prop_draw_int(tc, 0, 3) == 0) + { + appendStringInfoString(&s, "CREATE TABLE cfc2_pt0 (a int) PARTITION BY LIST (a); "); + for (int i = 1; i < ntab; i++) + appendStringInfo(&s, "CREATE TABLE cfc2_pt%d PARTITION OF cfc2_pt0 FOR VALUES IN (%d); ", + i, i); + } + else + { + appendStringInfoString(&s, "CREATE TABLE cfc2_pt0 (a int); "); + for (int i = 1; i < ntab; i++) + { + int par = (int) pg_prop_draw_int(tc, -1, i - 1); + + if (par < 0) + appendStringInfo(&s, "CREATE TABLE cfc2_pt%d (a int); ", i); + else + appendStringInfo(&s, "CREATE TABLE cfc2_pt%d () INHERITS (cfc2_pt%d); ", i, par); + } + } + c2_exec(tc, conn, s.data); + + for (int m = 0; m < nment; m++) + { + ment[m] = (int) pg_prop_draw_int(tc, 0, ntab - 1); + only[m] = (int) pg_prop_draw_int(tc, 0, 2); /* 0 plain, 1 ONLY, 2 "*" */ + perm[m] = m; + } + /* a drawn permutation (Fisher-Yates); default reverse when it shrinks */ + for (int m = nment - 1; m > 0; m--) + { + int j = (int) pg_prop_draw_int(tc, 0, m); + int t = perm[m]; + + perm[m] = perm[j]; + perm[j] = t; + } + + for (int p = 0; p < 2; p++) + { + bool via_alter = pg_prop_draw_bool(tc); + + resetStringInfo(&s); + if (via_alter) + appendStringInfo(&s, "CREATE PUBLICATION cfc2_pl%d; ALTER PUBLICATION cfc2_pl%d SET ", p, p); + else + appendStringInfo(&s, "CREATE PUBLICATION cfc2_pl%d FOR ", p); + for (int m = 0; m < nment; m++) + { + int x = p == 0 ? m : perm[m]; + + appendStringInfo(&s, "%sTABLE %scfc2_pt%d%s", m ? ", " : "", + only[x] == 1 ? "ONLY " : "", ment[x], + only[x] == 2 ? " *" : ""); + } + stmt[p] = pg_strdup(s.data); + /* the two-statement string is one transaction: a failure leaves no pub */ + ok[p] = pg_prop_sql_exec(conn, s.data, &err[p]); + resetStringInfo(&s); + appendStringInfo(&s, + "SELECT coalesce(string_agg(prrelid::regclass::text, ',' " + "ORDER BY prrelid::regclass::text), '') FROM pg_publication_rel r " + "JOIN pg_publication p ON p.oid = r.prpubid WHERE p.pubname = 'cfc2_pl%d'", p); + set[p] = pg_prop_sql_scalar(conn, s.data, NULL, NULL); + } + + if (ok[0] && ok[1] && strcmp(set[0] ? set[0] : "", set[1] ? set[1] : "") != 0) + pg_prop_fail(tc, + "the same publication table list in two orders gave different results\n" + " A: %s\n %s, tables {%s}%s%s\n B: %s\n %s, tables {%s}%s%s", + stmt[0], ok[0] ? "accepted" : "rejected", set[0] ? set[0] : "", + err[0] ? " " : "", err[0] ? err[0] : "", + stmt[1], ok[1] ? "accepted" : "rejected", set[1] ? set[1] : "", + err[1] ? " " : "", err[1] ? err[1] : ""); + + c2_exec(tc, conn, pl_cleanup); + pg_free(stmt[0]); + pg_free(stmt[1]); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------------- + * CF 7241: temporal FK validation keeps PERIOD semantics on every path + * ---------------------------------------------------------------------- + */ + +static const char *tf_cleanup = +"DROP TABLE IF EXISTS cfc2_tf, cfc2_tf1, cfc2_tp CASCADE"; + +/* rows of cfc2_tf1 that violate FOREIGN KEY (parent_id, PERIOD valid_at) */ +#define TF_VIOLATIONS \ + "SELECT count(*) FROM cfc2_tf1 c WHERE c.parent_id IS NOT NULL AND c.valid_at IS NOT NULL " \ + "AND NOT coalesce((SELECT range_agg(p.valid_at) FROM cfc2_tp p WHERE p.id = c.parent_id) " \ + " @> c.valid_at, false)" + +/* + * A temporal foreign key (parent_id, PERIOD valid_at) holds for a row iff the + * row's valid_at is covered by the union of the referenced rows' ranges. + * However the constraint comes to be checked against existing rows -- ADD + * (validated), ADD NOT VALID + VALIDATE, ADD NOT ENFORCED + ALTER ENFORCED, + * ATTACH PARTITION of a populated table (cloned or reparented constraint) -- + * the server accepts iff no row violates that, per a SQL model with + * range_agg. Parent rows are drawn as runs of adjacent or gapped ranges so + * that coverage often needs several parent rows. + */ +PG_PROP(temporal_fk_validation_matches_model) +{ + static const char *const pathname[] = { + "ADD CONSTRAINT", "ADD NOT VALID + VALIDATE", "ADD NOT ENFORCED + ALTER ENFORCED", + "ATTACH PARTITION (cloned FK)", "ATTACH PARTITION (reparented NOT VALID FK)", + }; + pg_prop_conn *conn = c2_connect(tc); + StringInfoData s; + int path = (int) pg_prop_draw_int(tc, 0, 4); + int nchild = (int) pg_prop_draw_int(tc, 1, 4); + long viol; + char *err = NULL; + bool ok; + const char *fk = "FOREIGN KEY (parent_id, PERIOD valid_at) REFERENCES cfc2_tp (id, PERIOD valid_at)"; + + initStringInfo(&s); + c2_exec(tc, conn, tf_cleanup); + c2_exec(tc, conn, + "CREATE TABLE cfc2_tp (id int4range, valid_at int4range, " + "CONSTRAINT cfc2_tp_pk PRIMARY KEY (id, valid_at WITHOUT OVERLAPS))"); + + /* parents: per id 0..2, a run of non-overlapping ranges */ + for (int id = 0; id < 3; id++) + { + int n = (int) pg_prop_draw_int(tc, 0, 3); + int pos = (int) pg_prop_draw_int(tc, 0, 4); + + for (int k = 0; k < n; k++) + { + int len = (int) pg_prop_draw_int(tc, 1, 5); + + resetStringInfo(&s); + appendStringInfo(&s, "INSERT INTO cfc2_tp VALUES ('[%d,%d)', '[%d,%d)')", + id, id + 1, pos, pos + len); + c2_exec(tc, conn, s.data); + pos += len + (int) pg_prop_draw_int(tc, 0, 2); /* 0 = adjacent */ + } + } + + /* the referencing table that will hold the rows being validated */ + if (path >= 3) + c2_exec(tc, conn, + "CREATE TABLE cfc2_tf (p int, parent_id int4range, valid_at int4range, " + "CONSTRAINT cfc2_tf_fk FOREIGN KEY (parent_id, PERIOD valid_at) " + "REFERENCES cfc2_tp (id, PERIOD valid_at)) PARTITION BY LIST (p); " + "CREATE TABLE cfc2_tf1 (LIKE cfc2_tf)"); + else + c2_exec(tc, conn, "CREATE TABLE cfc2_tf1 (p int, parent_id int4range, valid_at int4range)"); + for (int r = 0; r < nchild; r++) + { + int pid = (int) pg_prop_draw_int(tc, -1, 3); /* -1 NULL, 3 no parent */ + int lo = (int) pg_prop_draw_int(tc, 0, 14); + int len = (int) pg_prop_draw_int(tc, 1, 6); + + resetStringInfo(&s); + if (pid < 0) + appendStringInfo(&s, "INSERT INTO cfc2_tf1 VALUES (1, NULL, '[%d,%d)')", lo, lo + len); + else + appendStringInfo(&s, "INSERT INTO cfc2_tf1 VALUES (1, '[%d,%d)', '[%d,%d)')", + pid, pid + 1, lo, lo + len); + c2_exec(tc, conn, s.data); + } + viol = c2_count(tc, conn, TF_VIOLATIONS); + + resetStringInfo(&s); + switch (path) + { + case 0: + appendStringInfo(&s, "ALTER TABLE cfc2_tf1 ADD CONSTRAINT cfc2_tf_fk %s", fk); + break; + case 1: + c2_exec(tc, conn, psprintf("ALTER TABLE cfc2_tf1 ADD CONSTRAINT cfc2_tf_fk %s NOT VALID", fk)); + appendStringInfoString(&s, "ALTER TABLE cfc2_tf1 VALIDATE CONSTRAINT cfc2_tf_fk"); + break; + case 2: + c2_exec(tc, conn, psprintf("ALTER TABLE cfc2_tf1 ADD CONSTRAINT cfc2_tf_fk %s NOT ENFORCED", fk)); + appendStringInfoString(&s, "ALTER TABLE cfc2_tf1 ALTER CONSTRAINT cfc2_tf_fk ENFORCED"); + break; + default: + /* path 4: an equivalent NOT VALID FK already on the partition */ + if (path == 4) + c2_exec(tc, conn, psprintf("ALTER TABLE cfc2_tf1 ADD CONSTRAINT cfc2_tf_fk %s NOT VALID", fk)); + appendStringInfoString(&s, "ALTER TABLE cfc2_tf ATTACH PARTITION cfc2_tf1 FOR VALUES IN (1)"); + break; + } + ok = pg_prop_sql_exec(conn, s.data, &err); + if (ok != (viol == 0)) + { + char *parents = c2_rows(tc, conn, "SELECT id, valid_at FROM cfc2_tp ORDER BY 1, 2"); + char *rows = c2_rows(tc, conn, "SELECT parent_id, valid_at FROM cfc2_tf1 ORDER BY 1, 2"); + + pg_prop_fail(tc, + "temporal FK via %s was %s, but %ld existing rows violate it\n" + " SQL: %s%s%s\n referenced (id, valid_at):\n%s referencing (parent_id, valid_at):\n%s", + pathname[path], ok ? "accepted" : "rejected", viol, s.data, + err ? "\n error: " : "", err ? err : "", parents, rows); + } + + c2_exec(tc, conn, tf_cleanup); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------------- + * CF 7312: rolled-back SET TABLESPACE leaves indexes consistent + * ---------------------------------------------------------------------- + */ + +/* + * The index-scan answer must equal the seqscan answer for each probe range. + * Values: committed rows are 1..n, rows written inside the transaction under + * test are >= 100000, rows written after it are negative, so a stale index + * entry left behind by a rollback is visible as a wrong count. + */ +static void +ts_check_indexes(pg_prop_tc *tc, pg_prop_conn *conn, const char *log) +{ + static const char *const probes[] = { + "a < 0", "a >= 100000", "a BETWEEN 1 AND 99999", "a > -1000000000", + }; + + for (int p = 0; p < (int) lengthof(probes); p++) + { + char q[512]; + char *seq; + char *idx; + bool failed = false; + char *err = NULL; + + snprintf(q, sizeof(q), + "SET enable_seqscan = off; SET enable_bitmapscan = %s; SET enable_indexscan = on; " + "SET enable_indexonlyscan = %s", + p % 2 ? "on" : "off", p < 2 ? "on" : "off"); + c2_exec(tc, conn, q); + snprintf(q, sizeof(q), "SELECT count(*) || '/' || coalesce(sum(a), 0) FROM cfc2_tb WHERE %s", + probes[p]); + idx = pg_prop_sql_scalar(conn, q, &failed, &err); + if (failed) + pg_prop_fail(tc, "index scan failed after rolled-back SET TABLESPACE: %s" + " probe: %s\n sequence:\n%s", err ? err : "", probes[p], log); + c2_exec(tc, conn, "RESET enable_seqscan; RESET enable_bitmapscan; RESET enable_indexscan; " + "RESET enable_indexonlyscan; SET enable_indexscan = off; SET enable_bitmapscan = off; " + "SET enable_indexonlyscan = off"); + seq = pg_prop_sql_scalar(conn, q, &failed, &err); + c2_exec(tc, conn, "RESET enable_indexscan; RESET enable_bitmapscan; RESET enable_indexonlyscan"); + if (failed) + pg_prop_fail(tc, "seqscan failed: %s", err ? err : ""); + if (strcmp(idx ? idx : "", seq ? seq : "") != 0) + pg_prop_fail(tc, + "index scan and seqscan disagree after SET TABLESPACE in a " + "rolled-back (sub)transaction\n probe %s: index %s, seqscan %s" + " (count/sum)\n sequence:\n%s", + probes[p], idx ? idx : "", seq ? seq : "", log); + pg_free(idx); + pg_free(seq); + } +} + +/* + * A random transaction over a table with indexes: SET TABLESPACE moves + * (there and back), INSERT/DELETE/TRUNCATE, CREATE INDEX / REINDEX, + * SAVEPOINT / ROLLBACK TO / RELEASE, ending in COMMIT or ROLLBACK; then more + * rows are inserted (reusing freed TIDs). Rolling back must undo the index + * changes together with the heap's, so afterwards index scans and the + * seqscan agree on every probe and nothing errors. + */ +PG_PROP(rollback_set_tablespace_keeps_indexes_consistent) +{ + pg_prop_conn *conn = c2_connect(tc); + StringInfoData s; + StringInfoData log; + int n0 = (int) pg_prop_draw_int(tc, 1, 400); + int nops = (int) pg_prop_draw_int(tc, 1, 6); + int nsp = 0; + bool in_ts = false; + int next_hi = 100000; + + initStringInfo(&s); + initStringInfo(&log); + c2_exec(tc, conn, "DROP TABLE IF EXISTS cfc2_tb"); + c2_try(conn, "DROP TABLESPACE IF EXISTS cfc2_ts"); + c2_exec(tc, conn, "SET allow_in_place_tablespaces = on"); + c2_exec(tc, conn, "CREATE TABLESPACE cfc2_ts LOCATION ''"); + + appendStringInfo(&s, + "CREATE TABLE cfc2_tb (a int, b text); " + "INSERT INTO cfc2_tb SELECT g, 'x' || g FROM generate_series(1, %d) g; " + "CREATE INDEX cfc2_tb_a ON cfc2_tb %s (a)", + n0, pg_prop_draw_int(tc, 0, 3) == 0 ? "USING hash" : ""); + if (pg_prop_draw_bool(tc)) + appendStringInfoString(&s, "; CREATE INDEX cfc2_tb_ab ON cfc2_tb (a, b)"); + c2_exec(tc, conn, s.data); + appendStringInfo(&log, "%s;\nBEGIN;\n", s.data); + c2_exec(tc, conn, "BEGIN"); + + for (int k = 0; k < nops; k++) + { + int op = (int) pg_prop_draw_int(tc, 0, 9); + + resetStringInfo(&s); + switch (op) + { + case 0: + case 1: + case 2: + appendStringInfo(&s, "ALTER TABLE cfc2_tb SET TABLESPACE %s", + in_ts ? "pg_default" : "cfc2_ts"); + in_ts = !in_ts; + break; + case 3: + case 4: + { + int m = (int) pg_prop_draw_int(tc, 1, 1500); + + appendStringInfo(&s, "INSERT INTO cfc2_tb SELECT g, 'y' FROM generate_series(%d, %d) g", + next_hi, next_hi + m - 1); + next_hi += m; + } + break; + case 5: + appendStringInfo(&s, "DELETE FROM cfc2_tb WHERE a %% %d = 0", + (int) pg_prop_draw_int(tc, 2, 5)); + break; + case 6: + appendStringInfoString(&s, pg_prop_draw_bool(tc) ? "TRUNCATE cfc2_tb" + : "REINDEX INDEX cfc2_tb_a"); + break; + case 7: + appendStringInfo(&s, "SAVEPOINT s%d", nsp++); + break; + case 8: + if (nsp > 0) + appendStringInfo(&s, "ROLLBACK TO SAVEPOINT s%d", nsp - 1); + else + appendStringInfo(&s, "SAVEPOINT s%d", nsp++); + break; + default: + appendStringInfoString(&s, "CREATE INDEX ON cfc2_tb (a)"); + break; + } + /* + * in_ts is only a generator hint (a rollback may undo a move); a + * "move" to the current tablespace is a legal no-op. + */ + appendStringInfo(&log, "%s;\n", s.data); + c2_exec(tc, conn, s.data); + } + resetStringInfo(&s); + appendStringInfoString(&s, pg_prop_draw_int(tc, 0, 3) == 0 ? "COMMIT" : "ROLLBACK"); + appendStringInfo(&log, "%s;\n", s.data); + c2_exec(tc, conn, s.data); + + resetStringInfo(&s); + appendStringInfo(&s, "INSERT INTO cfc2_tb SELECT -g, 'z' FROM generate_series(1, %d) g", + (int) pg_prop_draw_int(tc, 1, 600)); + appendStringInfo(&log, "%s;\n", s.data); + c2_exec(tc, conn, s.data); + + ts_check_indexes(tc, conn, log.data); + + c2_exec(tc, conn, "DROP TABLE cfc2_tb"); + c2_exec(tc, conn, "DROP TABLESPACE cfc2_ts"); + pfree(s.data); + pfree(log.data); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------------- + * CF 7321: dropping the session's temp schema leaves a consistent catalog + * ---------------------------------------------------------------------- + */ + +/* catalog rows whose namespace does not exist */ +#define TN_ORPHANS \ + "SELECT (SELECT count(*) FROM pg_class c WHERE NOT EXISTS " \ + " (SELECT 1 FROM pg_namespace n WHERE n.oid = c.relnamespace)) + " \ + "(SELECT count(*) FROM pg_type t WHERE NOT EXISTS " \ + " (SELECT 1 FROM pg_namespace n WHERE n.oid = t.typnamespace)) + " \ + "(SELECT count(*) FROM pg_proc p WHERE NOT EXISTS " \ + " (SELECT 1 FROM pg_namespace n WHERE n.oid = p.pronamespace))" + +/* + * A superuser may DROP SCHEMA pg_temp_N CASCADE on its own temp schema. + * Whatever temp objects existed, and wherever the drop happens (same or + * earlier transaction, inside a savepoint that is or is not rolled back), + * temp objects created afterwards must live in a schema that exists: no + * pg_class / pg_type / pg_proc row may reference a missing namespace, and + * pg_my_temp_schema(), when set, must name an existing schema. + * + * (pg_identify_object() on such an orphan also crashes the server; that + * probe is deliberately not made here because the server is shared.) + */ +PG_PROP(drop_temp_schema_keeps_catalog_consistent) +{ + static const char *const mk[] = { + "CREATE TEMP TABLE cfc2_tt%d (f1 int)", + "CREATE TEMP VIEW cfc2_tt%d AS SELECT 1 AS x", + "CREATE TEMP SEQUENCE cfc2_tt%d", + "CREATE FUNCTION pg_temp.cfc2_tt%d() RETURNS int LANGUAGE sql AS 'SELECT 1'", + "CREATE TEMP TABLE cfc2_tt%d (f1 serial PRIMARY KEY, f2 text)", + }; + pg_prop_conn *conn = c2_connect(tc); + StringInfoData s; + StringInfoData log; + int nobj = 0; + int nsteps = (int) pg_prop_draw_int(tc, 1, 7); + bool in_txn = false; + int nsp = 0; + long orph; + long bad_temp; + + initStringInfo(&s); + initStringInfo(&log); + + for (int k = 0; k < nsteps; k++) + { + resetStringInfo(&s); + switch ((int) pg_prop_draw_int(tc, 0, 6)) + { + case 0: + case 1: + appendStringInfo(&s, mk[pg_prop_draw_int(tc, 0, lengthof(mk) - 1)], nobj++); + break; + case 2: + case 3: + /* drop my own temp schema, if I have one */ + appendStringInfoString(&s, + "DO $$ BEGIN IF pg_my_temp_schema() <> 0 AND EXISTS (SELECT 1 FROM " + "pg_namespace WHERE oid = pg_my_temp_schema()) THEN EXECUTE format(" + "'DROP SCHEMA %I CASCADE', pg_my_temp_schema()::regnamespace); " + "END IF; END $$"); + break; + case 4: + appendStringInfoString(&s, in_txn ? "COMMIT" : "BEGIN"); + in_txn = !in_txn; + nsp = 0; + break; + case 5: + if (in_txn) + appendStringInfo(&s, "SAVEPOINT s%d", nsp++); + else + appendStringInfo(&s, mk[0], nobj++); + break; + default: + if (in_txn && nsp > 0) + appendStringInfo(&s, "ROLLBACK TO SAVEPOINT s%d", nsp - 1); + else if (in_txn) + { + appendStringInfoString(&s, "ROLLBACK"); + in_txn = false; + } + else + appendStringInfo(&s, mk[pg_prop_draw_int(tc, 0, lengthof(mk) - 1)], nobj++); + break; + } + appendStringInfo(&log, "%s;\n", s.data); + c2_exec(tc, conn, s.data); + } + /* always create one more temp object at the end */ + resetStringInfo(&s); + appendStringInfo(&s, mk[pg_prop_draw_int(tc, 0, lengthof(mk) - 1)], nobj++); + appendStringInfo(&log, "%s;\n", s.data); + c2_exec(tc, conn, s.data); + + orph = c2_count(tc, conn, TN_ORPHANS); + bad_temp = c2_count(tc, conn, + "SELECT count(*) FROM (SELECT pg_my_temp_schema() AS t) x " + "WHERE t <> 0 AND NOT EXISTS (SELECT 1 FROM pg_namespace WHERE oid = t)"); + if (orph || bad_temp) + pg_prop_fail(tc, + "after dropping the temp schema: %ld catalog rows reference a missing " + "namespace; pg_my_temp_schema() %s\n sequence:\n%s", + orph, bad_temp ? "names a dropped schema" : "ok", log.data); + + if (in_txn) + c2_exec(tc, conn, "COMMIT"); + pfree(s.data); + pfree(log.data); + /* the temp schema's contents go away with the session */ + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------------- + * CF 7351: a partition whose DETACH CONCURRENTLY was interrupted + * ---------------------------------------------------------------------- + */ + +static const char *dp_cleanup = +"DROP PUBLICATION IF EXISTS cfc2_dpa, cfc2_dpr, cfc2_dpt; " +"DROP TABLE IF EXISTS cfc2_dp, cfc2_dp0, cfc2_dp1, cfc2_dp2, cfc2_dp1_l0, cfc2_dp1_l1 CASCADE"; + +/* UPDATE probe: "ok" or the SQLSTATE-free error text */ +static char * +dp_update_probe(pg_prop_conn *conn, const char *rel) +{ + char q[256]; + char *err = NULL; + + snprintf(q, sizeof(q), "UPDATE %s SET b = b WHERE false", rel); + if (pg_prop_sql_exec(conn, q, &err)) + return pg_strdup("ok"); + return err ? err : pg_strdup("error"); +} + +/* + * Leave a drawn partition (leaf, or a partitioned partition with its own + * leaves, possibly with a different column order) pending detach by letting + * DETACH PARTITION ... CONCURRENTLY time out while another session holds the + * parent open. The partition is then "detached" for every purpose except + * that FINALIZE has not run. Invariants: + * + * - once the detach's first transaction has committed, a session that + * looked at the partition before it and a fresh session agree on whether + * UPDATE of the partition is allowed (its publication membership); + * - a FOR ALL TABLES publication publishes every table: each cfc2_dp leaf + * is in pg_get_publication_tables() itself or through one of its current + * pg_partition_ancestors() (publish_via_partition_root on or off); + * - with PG_PROP_CF_CRASH=1 only (it takes the shared server down on an + * unfixed build): INSERT into the partition computing an identity + * default does not crash. + */ +PG_PROP(pending_detach_partition_consistent) +{ + pg_prop_conn *conn = c2_connect(tc); + pg_prop_conn *other = c2_connect(tc); + pg_prop_conn *fresh; + StringInfoData s; + int nparts = (int) pg_prop_draw_int(tc, 2, 3); + int victim = (int) pg_prop_draw_int(tc, 0, nparts - 1); + bool victim_parted = pg_prop_draw_bool(tc); + bool permuted = pg_prop_draw_bool(tc); + bool identity = pg_prop_draw_bool(tc); + bool via_root = pg_prop_draw_bool(tc); + bool pub_root = pg_prop_draw_bool(tc); + bool all_tables = pg_prop_draw_bool(tc); + const char *crash = getenv("PG_PROP_CF_CRASH"); + char *err = NULL; + char victim_name[32]; + char probe_rel[32]; + char *before_other; + char *after_other; + char *after_fresh; + char *uncovered; + + initStringInfo(&s); + c2_exec(tc, conn, dp_cleanup); + appendStringInfo(&s, "CREATE TABLE cfc2_dp (a int, b int%s) PARTITION BY LIST (a); ", + identity ? ", c bigint GENERATED BY DEFAULT AS IDENTITY" : ""); + for (int p = 0; p < nparts; p++) + { + if (p == victim && victim_parted) + { + appendStringInfo(&s, + "CREATE TABLE cfc2_dp%d PARTITION OF cfc2_dp FOR VALUES IN (%d) " + "PARTITION BY LIST (b); " + "CREATE TABLE cfc2_dp%d_l0 PARTITION OF cfc2_dp%d FOR VALUES IN (0); ", + p, p, p, p); + if (permuted) + appendStringInfo(&s, + "CREATE TABLE cfc2_dp%d_l1 (%sb int, a int); " + "ALTER TABLE cfc2_dp%d ATTACH PARTITION cfc2_dp%d_l1 DEFAULT; ", + p, identity ? "c bigint NOT NULL, " : "", p, p); + else + appendStringInfo(&s, "CREATE TABLE cfc2_dp%d_l1 PARTITION OF cfc2_dp%d DEFAULT; ", + p, p); + } + else if (permuted && p == victim) + appendStringInfo(&s, + "CREATE TABLE cfc2_dp%d (%sb int, a int); " + "ALTER TABLE cfc2_dp ATTACH PARTITION cfc2_dp%d FOR VALUES IN (%d); ", + p, identity ? "c bigint NOT NULL, " : "", p, p); + else + appendStringInfo(&s, "CREATE TABLE cfc2_dp%d PARTITION OF cfc2_dp FOR VALUES IN (%d); ", + p, p); + } + appendStringInfo(&s, "INSERT INTO cfc2_dp (a, b) SELECT g %% %d, g %% 2 FROM generate_series(0, 9) g; ", + nparts); + /* FOR ALL TABLES makes UPDATE without replica identity fail everywhere */ + if (all_tables) + appendStringInfo(&s, "CREATE PUBLICATION cfc2_dpa FOR ALL TABLES WITH (publish_via_partition_root = %s); ", + via_root ? "true" : "false"); + if (pub_root || !all_tables) + appendStringInfo(&s, "CREATE PUBLICATION cfc2_dpr FOR TABLE cfc2_dp WITH (publish_via_partition_root = %s); ", + via_root ? "true" : "false"); + c2_exec(tc, conn, s.data); + + snprintf(victim_name, sizeof(victim_name), "cfc2_dp%d", victim); + if (victim_parted) + snprintf(probe_rel, sizeof(probe_rel), "cfc2_dp%d_l%d", victim, + (int) pg_prop_draw_int(tc, 0, 1)); + else + strlcpy(probe_rel, victim_name, sizeof(probe_rel)); + + /* the other session builds its relcache entry for the partition first */ + before_other = dp_update_probe(other, probe_rel); + pg_free(before_other); + + /* interrupt the detach: the other session holds the parent open */ + c2_exec(tc, other, "BEGIN; SELECT count(*) FROM cfc2_dp"); + c2_exec(tc, conn, "SET lock_timeout = '150ms'"); + resetStringInfo(&s); + appendStringInfo(&s, "ALTER TABLE cfc2_dp DETACH PARTITION %s CONCURRENTLY", victim_name); + if (pg_prop_sql_exec(conn, s.data, &err)) + { + /* it did not wait (nothing to wait for): no pending state to test */ + c2_exec(tc, other, "COMMIT"); + c2_exec(tc, conn, "RESET lock_timeout"); + c2_exec(tc, conn, dp_cleanup); + pg_prop_disconnect(other); + pg_prop_disconnect(conn); + pg_prop_assume(tc, false); + } + if (err) + pg_free(err); + c2_exec(tc, conn, "RESET lock_timeout"); + c2_exec(tc, other, "COMMIT"); + + /* + * The timeout may also hit the first phase's lock wait (say, autovacuum + * on the partition), leaving nothing pending: discard such a case. + */ + if (c2_count(tc, conn, + "SELECT count(*) FROM pg_inherits WHERE inhdetachpending AND " + "inhparent = 'cfc2_dp'::regclass") != 1) + { + c2_exec(tc, conn, dp_cleanup); + pg_prop_disconnect(other); + pg_prop_disconnect(conn); + pg_prop_assume(tc, false); + } + + /* 2. a session with a pre-detach relcache entry agrees with a fresh one */ + fresh = c2_connect(tc); + after_other = dp_update_probe(other, probe_rel); + after_fresh = dp_update_probe(fresh, probe_rel); + if ((strcmp(after_other, "ok") == 0) != (strcmp(after_fresh, "ok") == 0)) + pg_prop_fail(tc, + "after DETACH CONCURRENTLY of %s was interrupted, two sessions disagree on " + "UPDATE %s (publications: %s%s, via_root=%s)\n" + " session that saw it before: %s\n fresh session: %s", + victim_name, probe_rel, all_tables ? "ALL TABLES" : "", + (pub_root || !all_tables) ? " FOR TABLE root" : "", + via_root ? "true" : "false", after_other, after_fresh); + pg_free(after_other); + pg_free(after_fresh); + pg_prop_disconnect(fresh); + + /* 1. FOR ALL TABLES covers every leaf */ + uncovered = !all_tables ? NULL : pg_prop_sql_scalar(conn, + "SELECT string_agg(c.relname, ',') FROM pg_class c " + "WHERE c.relname LIKE 'cfc2_dp%' AND c.relkind = 'r' " + "AND NOT EXISTS (SELECT 1 FROM pg_get_publication_tables('cfc2_dpa') g " + " WHERE g.relid = c.oid OR g.relid IN " + " (SELECT relid FROM pg_partition_ancestors(c.oid)))", + NULL, NULL); + if (uncovered != NULL) + pg_prop_fail(tc, + "FOR ALL TABLES publication (publish_via_partition_root = %s) does not " + "publish %s, a %s partition pending detach (or its leaf)", + via_root ? "true" : "false", uncovered, + victim_parted ? "partitioned" : "leaf"); + + /* 3. identity default through the pending partition (crashes unfixed servers) */ + if (identity && crash != NULL && strcmp(crash, "1") == 0) + { + resetStringInfo(&s); + appendStringInfo(&s, "INSERT INTO %s (a, b) VALUES (%d, 0)", probe_rel, victim); + c2_try(conn, s.data); + } + + c2_exec(tc, conn, dp_cleanup); + pfree(s.data); + pg_prop_disconnect(other); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------------- + * CF 7379: CREATE TABLE ... (LIKE ...) only touches the table it creates + * ---------------------------------------------------------------------- + */ + +/* everything about the permanent cfc2_l* tables a LIKE could clobber */ +#define LK_REL "c.relname LIKE 'cfc2_l%' AND c.relnamespace::regnamespace::text IN ('public', 'cfc2_ls')" +#define LK_SNAPSHOT \ + "SELECT c.relnamespace::regnamespace, c.relname, d.objsubid, d.description " \ + "FROM pg_description d JOIN pg_class c ON d.classoid = 'pg_class'::regclass AND d.objoid = c.oid " \ + "WHERE " LK_REL " " \ + "UNION ALL SELECT co.connamespace::regnamespace, co.conname, 0, " \ + " c.relname || ':' || coalesce(obj_description(co.oid, 'pg_constraint'), '-') " \ + "FROM pg_constraint co JOIN pg_class c ON c.oid = co.conrelid WHERE " LK_REL " " \ + "UNION ALL SELECT c.relnamespace::regnamespace, c.relname, c.relnatts, c.relkind::text " \ + "FROM pg_class c WHERE " LK_REL " " \ + "ORDER BY 1, 2, 3, 4" + +static const char *lk_cleanup = +"DROP TABLE IF EXISTS public.cfc2_lsrc, public.cfc2_lt CASCADE; " +"DROP SCHEMA IF EXISTS cfc2_ls, cfc2_ln CASCADE"; + +/* + * CREATE [TEMP] TABLE t (LIKE src INCLUDING ...) creates t and must leave + * every other table alone, whatever search_path says and whatever + * same-named tables exist in schemas that precede the target's. The + * permanent cfc2_l* tables' comments, constraints (with their comments) and + * shape are snapshotted before and after; they must not change, and the + * statement must not fail. The new table carries src's comments when + * INCLUDING COMMENTS is given. + */ +PG_PROP(create_like_only_touches_new_table) +{ + static const char *const paths[] = { + "public, pg_temp", "pg_temp, public", "cfc2_ls, public, pg_temp", + "public, cfc2_ls, pg_temp", "cfc2_ls, pg_temp, public", + }; + static const char *const incl[] = { + "INCLUDING COMMENTS", "INCLUDING ALL", "INCLUDING COMMENTS INCLUDING CONSTRAINTS", + "INCLUDING COMMENTS INCLUDING INDEXES", "INCLUDING ALL EXCLUDING INDEXES", + }; + pg_prop_conn *conn = c2_connect(tc); + StringInfoData s; + const char *path = paths[pg_prop_draw_int(tc, 0, lengthof(paths) - 1)]; + const char *inc = incl[pg_prop_draw_int(tc, 0, lengthof(incl) - 1)]; + bool temp = pg_prop_draw_int(tc, 0, 3) != 0; + bool qualify = pg_prop_draw_int(tc, 0, 3) == 0; + char *before; + char *after; + char *err = NULL; + long srcc; + long newc; + + initStringInfo(&s); + c2_exec(tc, conn, lk_cleanup); + c2_exec(tc, conn, + "CREATE SCHEMA cfc2_ls; " + "CREATE TABLE public.cfc2_lsrc (a int CONSTRAINT cfc2_lk_c CHECK (a > 0), b text); " + "COMMENT ON COLUMN public.cfc2_lsrc.a IS 'src a'; " + "COMMENT ON CONSTRAINT cfc2_lk_c ON public.cfc2_lsrc IS 'src check'; " + "CREATE INDEX cfc2_lsrc_b ON public.cfc2_lsrc (b); " + "COMMENT ON INDEX cfc2_lsrc_b IS 'src index'"); + if (pg_prop_draw_bool(tc)) + c2_exec(tc, conn, "COMMENT ON COLUMN public.cfc2_lsrc.b IS 'src b'"); + + /* unrelated tables named like the target, in drawn schemas */ + for (int k = 0; k < 2; k++) + { + const char *sch = k == 0 ? "public" : "cfc2_ls"; + + if (!pg_prop_draw_int(tc, 0, 3)) + continue; + resetStringInfo(&s); + appendStringInfo(&s, "CREATE TABLE %s.cfc2_lt (a int, b text, c int); ", sch); + if (pg_prop_draw_bool(tc)) + appendStringInfo(&s, "COMMENT ON COLUMN %s.cfc2_lt.a IS 'unrelated %s a'; ", sch, sch); + if (pg_prop_draw_bool(tc)) + appendStringInfo(&s, + "ALTER TABLE %s.cfc2_lt ADD CONSTRAINT cfc2_lk_c CHECK (c <> 0); " + "COMMENT ON CONSTRAINT cfc2_lk_c ON %s.cfc2_lt IS 'unrelated %s check'; ", + sch, sch, sch); + c2_exec(tc, conn, s.data); + } + + before = c2_rows(tc, conn, LK_SNAPSHOT); + resetStringInfo(&s); + if (temp) + appendStringInfo(&s, "BEGIN; SET LOCAL search_path = %s; CREATE TEMP TABLE %scfc2_lt " + "(LIKE public.cfc2_lsrc %s)", path, qualify ? "pg_temp." : "", inc); + else + /* a permanent target goes into a fresh schema that comes first */ + appendStringInfo(&s, "BEGIN; CREATE SCHEMA cfc2_ln; SET LOCAL search_path = cfc2_ln, %s; " + "CREATE TABLE %scfc2_lt (LIKE public.cfc2_lsrc %s)", + path, qualify ? "cfc2_ln." : "", inc); + if (!pg_prop_sql_exec(conn, s.data, &err)) + { + pg_prop_sql_exec(conn, "ROLLBACK", NULL); + pg_prop_fail(tc, "CREATE TABLE ... LIKE failed: %s SQL: %s", err ? err : "", s.data); + } + after = c2_rows(tc, conn, LK_SNAPSHOT); + srcc = c2_count(tc, conn, + "SELECT count(*) FROM pg_description WHERE objoid = 'public.cfc2_lsrc'::regclass " + "AND objsubid > 0"); + newc = c2_count(tc, conn, + "SELECT count(*) FROM pg_description d JOIN pg_class c ON c.oid = d.objoid " + "WHERE c.relname = 'cfc2_lt' AND d.objsubid > 0 AND " + "(c.relpersistence = 't' OR c.relnamespace::regnamespace::text = 'cfc2_ln')"); + c2_exec(tc, conn, "ROLLBACK"); + + /* the snapshot covers public and cfc2_ls only, never the new table */ + if (strcmp(before, after) != 0) + pg_prop_fail(tc, + "%s with search_path = %s changed other tables\n" + " before:\n%s after:\n%s", s.data, path, before, after); + if (newc != srcc) + pg_prop_fail(tc, + "%s: new table has %ld column comments, source has %ld", + s.data, newc, srcc); + + c2_exec(tc, conn, lk_cleanup); + pg_free(before); + pg_free(after); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------------- + * CF 7403: ON CONFLICT ... WHERE never sees rows RLS hides + * ---------------------------------------------------------------------- + */ + +static const char *rl_cleanup = +"RESET ROLE; DROP TABLE IF EXISTS cfc2_doc; DROP FUNCTION IF EXISTS cfc2_leak(text); " +"DROP SEQUENCE IF EXISTS cfc2_leakseq; DROP ROLE IF EXISTS cfc2_rb"; + +/* + * A user function called from a query run as a role subject to RLS must + * never receive a value from a row that the role's policies hide from it. + * The leak function bumps a sequence (non-transactional, so it survives the + * statement's error) whenever it sees a secret of a row the role cannot + * see. Rows are drawn visible/hidden by owner; policies for SELECT / + * UPDATE / INSERT are drawn (permissive or restrictive, by owner or by a + * level column); the statement is INSERT ... ON CONFLICT (did) DO UPDATE ... + * WHERE leak(existing.secret) or DO SELECT WHERE leak(...), on a drawn key. + * Whether the statement succeeds or fails is not checked -- only that no + * hidden secret reached the function. + */ +PG_PROP(on_conflict_where_respects_rls) +{ + pg_prop_conn *conn = c2_connect(tc); + StringInfoData s; + int nrows = (int) pg_prop_draw_int(tc, 1, 6); + int key; + bool do_select = pg_prop_draw_bool(tc); + bool levels = pg_prop_draw_bool(tc); + long leaked; + + initStringInfo(&s); + c2_exec(tc, conn, "SET client_min_messages = error"); + c2_exec(tc, conn, rl_cleanup); + c2_exec(tc, conn, + "CREATE ROLE cfc2_rb; " + "CREATE TABLE cfc2_doc (did int PRIMARY KEY, owner text, lvl int, secret text); " + "GRANT ALL ON cfc2_doc TO cfc2_rb; " + "CREATE SEQUENCE cfc2_leakseq; GRANT ALL ON cfc2_leakseq TO cfc2_rb; " + "CREATE FUNCTION cfc2_leak(text) RETURNS bool LANGUAGE plpgsql COST 0.0000001 AS " + "$$ BEGIN IF $1 LIKE 'hidden%' THEN PERFORM nextval('cfc2_leakseq'); END IF; " + "RETURN true; END $$; " + "GRANT EXECUTE ON FUNCTION cfc2_leak(text) TO cfc2_rb"); + + /* + * Row i is visible to cfc2_rb iff owner = cfc2_rb (or, with levels, lvl + * <= 1); its secret says which, so the leak function needs no catalog + * knowledge. + */ + for (int r = 0; r < nrows; r++) + { + bool vis = pg_prop_draw_bool(tc); + + resetStringInfo(&s); + appendStringInfo(&s, "INSERT INTO cfc2_doc VALUES (%d, '%s', %d, '%s%d')", + r, vis ? "cfc2_rb" : "other", vis ? 1 : 2, + vis ? "visible" : "hidden", r); + c2_exec(tc, conn, s.data); + } + + resetStringInfo(&s); + appendStringInfoString(&s, "ALTER TABLE cfc2_doc ENABLE ROW LEVEL SECURITY; "); + { + const char *vis = levels ? "lvl <= 1" : "owner = current_user"; + + /* SELECT visibility is always restricted: that defines "hidden" */ + appendStringInfo(&s, "CREATE POLICY cfc2_p_sel ON cfc2_doc FOR SELECT USING (%s); ", vis); + if (pg_prop_draw_bool(tc)) + appendStringInfoString(&s, "CREATE POLICY cfc2_p_ins ON cfc2_doc FOR INSERT WITH CHECK (true); "); + else + appendStringInfo(&s, "CREATE POLICY cfc2_p_ins ON cfc2_doc FOR INSERT WITH CHECK (%s); ", vis); + switch ((int) pg_prop_draw_int(tc, 0, 2)) + { + case 0: + appendStringInfo(&s, "CREATE POLICY cfc2_p_upd ON cfc2_doc FOR UPDATE USING (%s); ", vis); + break; + case 1: + appendStringInfo(&s, "CREATE POLICY cfc2_p_upd ON cfc2_doc FOR UPDATE USING (%s) " + "WITH CHECK (true); ", vis); + break; + default: + appendStringInfo(&s, "CREATE POLICY cfc2_p_upd ON cfc2_doc FOR UPDATE USING (true); " + "CREATE POLICY cfc2_p_updr ON cfc2_doc AS RESTRICTIVE FOR UPDATE " + "USING (%s); ", vis); + break; + } + } + c2_exec(tc, conn, s.data); + + key = (int) pg_prop_draw_int(tc, 0, nrows); /* nrows = no conflict */ + resetStringInfo(&s); + appendStringInfo(&s, "SET ROLE cfc2_rb; INSERT INTO cfc2_doc VALUES (%d, 'cfc2_rb', 1, 'new') " + "ON CONFLICT (did) ", key); + if (do_select) + appendStringInfoString(&s, "DO SELECT WHERE cfc2_leak(cfc2_doc.secret) RETURNING did"); + else + appendStringInfo(&s, "DO UPDATE SET secret = %s WHERE cfc2_leak(cfc2_doc.secret)", + pg_prop_draw_bool(tc) ? "EXCLUDED.secret" : "cfc2_doc.secret || 'x'"); + c2_try(conn, s.data); + c2_exec(tc, conn, "RESET ROLE"); + + leaked = c2_count(tc, conn, + "SELECT CASE WHEN is_called THEN last_value ELSE 0 END FROM cfc2_leakseq"); + if (leaked) + { + char *rows = c2_rows(tc, conn, "SELECT * FROM cfc2_doc ORDER BY did"); + + pg_prop_fail(tc, + "a function in ON CONFLICT ... WHERE received the secret of a row hidden " + "by RLS (%ld time(s))\n SQL: %s\n table:\n%s", leaked, s.data, rows); + } + + c2_exec(tc, conn, rl_cleanup); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------------- + * CF 7464: a (temporal) FK only ever references a PK / UNIQUE constraint + * ---------------------------------------------------------------------- + */ + +static const char *xf_cleanup = +"DROP TABLE IF EXISTS cfc2_xf, cfc2_xp CASCADE"; + +/* + * A foreign key references a unique key of the referenced table: whenever + * ADD FOREIGN KEY succeeds, its conindid is a unique index, and for a PERIOD + * FK one backing a PRIMARY KEY / UNIQUE constraint declared WITHOUT + * OVERLAPS. (A plain FK may use a bare unique index, as it always could.) The referenced table + * gets a drawn set of keys -- PRIMARY KEY / UNIQUE with or without WITHOUT + * OVERLAPS, EXCLUDE (id WITH =, valid_at WITH &&), a plain unique index -- + * and the FK a drawn form (PERIOD or not, explicit column list or not). + * Rejecting the FK is always allowed. + */ +PG_PROP(fk_references_only_unique_constraints) +{ + static const char *const keys[] = { + "CONSTRAINT cfc2_xk0 PRIMARY KEY (id, valid_at WITHOUT OVERLAPS)", + "CONSTRAINT cfc2_xk1 UNIQUE (id, valid_at WITHOUT OVERLAPS)", + "CONSTRAINT cfc2_xk2 PRIMARY KEY (id, valid_at)", + "CONSTRAINT cfc2_xk3 UNIQUE (id, valid_at)", + "CONSTRAINT cfc2_xk4 EXCLUDE USING gist (id WITH =, valid_at WITH &&)", + "CONSTRAINT cfc2_xk5 EXCLUDE USING gist (valid_at WITH &&, id WITH =)", + }; + pg_prop_conn *conn = c2_connect(tc); + StringInfoData s; + int nkeys = 0; + bool used[lengthof(keys)]; + bool period = pg_prop_draw_int(tc, 0, 3) != 0; + bool collist = pg_prop_draw_bool(tc); + bool pk_drawn = false; + char *err = NULL; + long bad; + + initStringInfo(&s); + memset(used, 0, sizeof(used)); + c2_exec(tc, conn, "SET client_min_messages = error"); + if (!pg_prop_require_extension(conn, "btree_gist")) + pg_prop_skip(tc, "btree_gist not installed"); + c2_exec(tc, conn, xf_cleanup); + + appendStringInfoString(&s, "CREATE TABLE cfc2_xp (id int4range, valid_at int4range"); + for (int k = 0; k < 3; k++) + { + int i = (int) pg_prop_draw_int(tc, 0, lengthof(keys) - 1); + + if (used[i] || (i == 2 && pk_drawn) || (i == 0 && pk_drawn) || (k > 0 && !pg_prop_draw_bool(tc))) + continue; + used[i] = true; + if (i == 0 || i == 2) + pk_drawn = true; + appendStringInfo(&s, ", %s", keys[i]); + nkeys++; + } + appendStringInfoString(&s, ")"); + c2_exec(tc, conn, s.data); + if (nkeys == 0 || pg_prop_draw_bool(tc)) + c2_exec(tc, conn, "CREATE UNIQUE INDEX cfc2_xpu ON cfc2_xp (id, valid_at)"); + + /* parents: per id, a run of non-overlapping ranges */ + for (int id = 0; id < 2; id++) + { + int pos = 0; + int n = (int) pg_prop_draw_int(tc, 1, 3); + + for (int k = 0; k < n; k++) + { + int len = (int) pg_prop_draw_int(tc, 1, 4); + + resetStringInfo(&s); + appendStringInfo(&s, "INSERT INTO cfc2_xp VALUES ('[%d,%d)', '[%d,%d)')", + id, id + 1, pos, pos + len); + c2_exec(tc, conn, s.data); + pos += len; + } + } + c2_exec(tc, conn, + "CREATE TABLE cfc2_xf (id int, parent_id int4range, valid_at int4range); " + "INSERT INTO cfc2_xf SELECT 1, id, valid_at FROM cfc2_xp"); + + resetStringInfo(&s); + appendStringInfo(&s, "ALTER TABLE cfc2_xf ADD CONSTRAINT cfc2_xf_fk FOREIGN KEY (parent_id, %svalid_at) " + "REFERENCES cfc2_xp%s", + period ? "PERIOD " : "", + collist ? (period ? " (id, PERIOD valid_at)" : " (id, valid_at)") : ""); + if (!pg_prop_sql_exec(conn, s.data, &err)) + { + /* rejecting is always allowed; the bug is accepting */ + pg_free(err); + c2_exec(tc, conn, xf_cleanup); + pfree(s.data); + pg_prop_disconnect(conn); + return; + } + + /* + * A plain FK may reference any unique index (even one with no + * constraint); a PERIOD FK needs a WITHOUT OVERLAPS PK / UNIQUE. + */ + bad = c2_count(tc, conn, + "SELECT count(*) FROM pg_constraint f JOIN pg_index i ON i.indexrelid = f.conindid " + "WHERE f.conname = 'cfc2_xf_fk' AND NOT (i.indisunique AND (NOT f.conperiod OR " + " EXISTS (SELECT 1 FROM pg_constraint k WHERE k.conindid = i.indexrelid " + " AND k.conrelid = f.confrelid AND k.contype IN ('p', 'u') AND k.conperiod)))"); + if (bad) + { + char *ref = pg_prop_sql_scalar(conn, + "SELECT conindid::regclass || ' ' || pg_get_indexdef(conindid) " + "FROM pg_constraint WHERE conname = 'cfc2_xf_fk'", NULL, NULL); + + pg_prop_fail(tc, + "foreign key accepted that is not backed by a primary key / unique " + "constraint%s\n SQL: %s\n references index: %s", + period ? " using WITHOUT OVERLAPS" : "", s.data, ref ? ref : "?"); + } + + c2_exec(tc, conn, xf_cleanup); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------------- + * CF 7465: UNIQUE NULLS NOT DISTINCT WITHOUT OVERLAPS, range == multirange + * ---------------------------------------------------------------------- + */ + +/* + * A single-range multirange behaves like that range under = and &&, so a + * UNIQUE [NULLS NOT DISTINCT] (id, valid_at WITHOUT OVERLAPS) constraint + * must accept or reject the same rows whether the columns are int4range or + * int4multirange (the latter's GiST index is lossy and goes through the + * recheck path). The same drawn row sequence, with NULLs in either column + * and overlapping / adjacent / disjoint ranges, is inserted one row at a + * time into both tables (plain INSERT or ON CONFLICT DO NOTHING); every + * outcome and the final contents must agree. + */ +PG_PROP(without_overlaps_unique_range_eq_multirange) +{ + pg_prop_conn *conn = c2_connect(tc); + StringInfoData s; + StringInfoData log; + bool nnd = pg_prop_draw_int(tc, 0, 3) != 0; + bool pk = !nnd && pg_prop_draw_int(tc, 0, 3) == 0; + int nrows = (int) pg_prop_draw_int(tc, 1, 10); + char *a; + char *b; + + initStringInfo(&s); + initStringInfo(&log); + c2_exec(tc, conn, "SET client_min_messages = error"); + c2_exec(tc, conn, "DROP TABLE IF EXISTS cfc2_wr, cfc2_wm"); + for (int t = 0; t < 2; t++) + { + resetStringInfo(&s); + appendStringInfo(&s, "CREATE TABLE cfc2_w%c (n int, id int4%s, valid_at int4%s, " + "CONSTRAINT cfc2_w%c_k %s (id, valid_at WITHOUT OVERLAPS))", + t ? 'm' : 'r', t ? "multirange" : "range", t ? "multirange" : "range", + t ? 'm' : 'r', + pk ? "PRIMARY KEY" : nnd ? "UNIQUE NULLS NOT DISTINCT" : "UNIQUE"); + c2_exec(tc, conn, s.data); + } + + for (int r = 0; r < nrows; r++) + { + int id = (int) pg_prop_draw_int(tc, -1, 2); /* -1 = NULL */ + int lo = (int) pg_prop_draw_int(tc, -1, 8); /* -1 = NULL */ + int len = (int) pg_prop_draw_int(tc, 1, 4); + bool upsert = pg_prop_draw_int(tc, 0, 3) == 0; + bool ok[2]; + char idr[32]; + char vr[32]; + + if (id < 0) + strlcpy(idr, "NULL", sizeof(idr)); + else + snprintf(idr, sizeof(idr), "'[%d,%d)'", id, id + 1); + if (lo < 0) + strlcpy(vr, "NULL", sizeof(vr)); + else + snprintf(vr, sizeof(vr), "'[%d,%d)'", lo, lo + len); + + for (int t = 0; t < 2; t++) + { + const char *ty = t ? "int4multirange" : "int4range"; + + resetStringInfo(&s); + /* int4multirange(range) builds the one-range multirange */ + appendStringInfo(&s, "INSERT INTO cfc2_w%c VALUES (%d, %s(%s::int4range), %s(%s::int4range))%s", + t ? 'm' : 'r', r, t ? ty : "", idr, t ? ty : "", vr, + upsert ? " ON CONFLICT DO NOTHING" : ""); + ok[t] = c2_try(conn, s.data); + } + appendStringInfo(&log, "(%s, %s)%s: range %s, multirange %s\n", idr, vr, + upsert ? " ON CONFLICT DO NOTHING" : "", + ok[0] ? "ok" : "rejected", ok[1] ? "ok" : "rejected"); + if (ok[0] != ok[1]) + pg_prop_fail(tc, + "%s (id, valid_at WITHOUT OVERLAPS): range and multirange columns " + "disagree on a row\n%s", + pk ? "PRIMARY KEY" : nnd ? "UNIQUE NULLS NOT DISTINCT" : "UNIQUE", + log.data); + } + a = c2_rows(tc, conn, "SELECT n FROM cfc2_wr ORDER BY n"); + b = c2_rows(tc, conn, "SELECT n FROM cfc2_wm ORDER BY n"); + if (strcmp(a, b) != 0) + pg_prop_fail(tc, "%s (id, valid_at WITHOUT OVERLAPS): range and multirange tables " + "hold different rows\n%s range rows n:\n%s multirange rows n:\n%s", + pk ? "PRIMARY KEY" : nnd ? "UNIQUE NULLS NOT DISTINCT" : "UNIQUE", + log.data, a, b); + + c2_exec(tc, conn, "DROP TABLE cfc2_wr, cfc2_wm"); + pg_free(a); + pg_free(b); + pfree(s.data); + pfree(log.data); + pg_prop_disconnect(conn); +} From 63b93af884366a41d5e694845a8bbb79f40f199e Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 17 Oct 2026 17:00:32 -0400 Subject: [PATCH 307/338] prop: reference-model properties for open type/function fixes (cf_types) Thirteen properties checking built-in functions against a reference model or an algebraic identity: to_date/to_timestamp field ranges and Y,YYY (7411, 7416, 6792), the jsonpath datetime format cache (7409), date_trunc near time-zone transitions (7243), time + interval (7349), SQL/JSON DEFAULT ON EMPTY/ERROR as a cast and row-independence (7181, 7355, 7414), window frames with EXCLUDE TIES (7412), citext split_part vs core (7453), grouping sets over a constant-folded expression (7438), JSON_SERIALIZE as a text cast (6587). Twelve falsify on upstream master 45fc51f; 6792 is a tests-only patch and stands as a regression guard. --- src/test/prop/sql/prop_cf_types.c | 1889 +++++++++++++++++++++++++++++ 1 file changed, 1889 insertions(+) create mode 100644 src/test/prop/sql/prop_cf_types.c diff --git a/src/test/prop/sql/prop_cf_types.c b/src/test/prop/sql/prop_cf_types.c new file mode 100644 index 0000000000000..6d4ed745f2e9a --- /dev/null +++ b/src/test/prop/sql/prop_cf_types.c @@ -0,0 +1,1889 @@ +/*------------------------------------------------------------------------- + * + * prop_cf_types.c + * Value oracles for built-in functions around open commitfest bugs. + * + * Each property draws inputs around the trigger of one or more CF bug + * fixes and checks the function's result against an oracle computed a + * different way: a C model of the answer, an independent SQL formulation, + * or an algebraic identity (round trip, monotonicity, row-order + * independence). A correct server satisfies every invariant by + * construction; each property's comment says why. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_cf_types.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "cf_types" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +static pg_prop_conn * +cfb3_connect(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + return conn; +} + +static void +cfb3_exec(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "setup failed: %s\n SQL: %s", err ? err : "", sql); +} + +/* an error message that no input may provoke on a correct server */ +static bool +cfb3_internal_error(const char *err) +{ + return err != NULL && + (strstr(err, "cache lookup failed") != NULL || + strstr(err, "unexpected") != NULL || + strstr(err, "unrecognized") != NULL || + strstr(err, "TRAP") != NULL); +} + +/* + * Run a scalar query that must succeed; return its value (NULL for SQL NULL). + */ +static char * +cfb3_scalar(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *v = pg_prop_sql_scalar(conn, sql, &failed, &err); + + if (failed) + pg_prop_fail(tc, "query failed: %.300s\n SQL: %.500s", err ? err : "", sql); + return v; +} + +/* ---------------------------------------------------------------- + * to_date / to_timestamp template fields (CF 6792, 7411, 7416) + * ---------------------------------------------------------------- + */ + +/* ISO-8601 weeks in ISO year y (52 or 53) */ +static int +cfb3_iso_weeks(int64 y) +{ +#define P(y) (((y) + (y) / 4 - (y) / 100 + (y) / 400) % 7) + return (P(y) == 4 || P(y - 1) == 3) ? 53 : 52; +#undef P +} + +static bool +cfb3_leap(int64 y) +{ + return (y % 4 == 0 && y % 100 != 0) || y % 400 == 0; +} + +/* + * A template being assembled: the input string, the format, and the string + * to_char(result, format) must reproduce if the input is accepted. + */ +typedef struct DtTpl +{ + StringInfoData in; + StringInfoData fmt; + StringInfoData exp; + bool fm; /* FM on every numeric field */ + bool valid; /* every field drawn inside its domain */ + int nfields; +} DtTpl; + +/* note whether a drawn value lies in its field's domain */ +static int64 +dt_in(DtTpl *t, int64 v, int64 lo, int64 hi) +{ + if (v < lo || v > hi) + t->valid = false; + return v; +} + +/* dt_seps[0] must stay the only whitespace separator, see dt_sep */ +static const char *const dt_seps[] = {" ", "-", "/", ".", ":"}; + +/* + * Separator before the next field. Outside FX mode a format separator + * consumes one separator character of the input, so after a space a minus + * sign is eaten as a separator and "2024 -5" reads as day 5 -- documented + * laxity, not a bug. Fields that may get a negative value therefore get + * a non-space separator ("nospace"); SSSSS, which a fixed server recovers + * the swallowed minus for, may get any. + */ +static void +dt_sep(pg_prop_tc *tc, DtTpl *t, bool nospace) +{ + const char *sep; + + if (t->nfields++ == 0) + return; + sep = dt_seps[pg_prop_draw_int(tc, nospace ? 1 : 0, lengthof(dt_seps) - 1)]; + appendStringInfoString(&t->in, sep); + appendStringInfoString(&t->fmt, sep); + /* to_char emits the format's separator text verbatim */ + appendStringInfoString(&t->exp, sep); +} + +/* + * Append one numeric field "name" with input value v. to_char pads the + * field to "width" digits unless FM is given; "expect" is the value + * to_char must render (== v except for the documented zero-means-unset + * month/day, CF 6792). + */ +static void +dt_num(pg_prop_tc *tc, DtTpl *t, const char *name, int width, + int64 v, int64 expect) +{ + dt_sep(tc, t, v < 0); + appendStringInfo(&t->fmt, "%s%s", t->fm ? "FM" : "", name); + if (t->fm) + { + appendStringInfo(&t->in, "%lld", (long long) v); + appendStringInfo(&t->exp, "%lld", (long long) expect); + } + else + { + appendStringInfo(&t->in, "%0*lld", width, (long long) v); + appendStringInfo(&t->exp, "%0*lld", width, (long long) expect); + } +} + +/* + * Draw a field value: mostly inside [lo, hi] (the range in which a round + * trip is guaranteed for this year), sometimes anywhere in a wide window + * around it that includes 0, negatives and 4+-digit values. "bad_lo" .. + * "bad_hi" is a range a correct server may accept without round-tripping + * (e.g. ISO day 365..371 of a 52-week year), which is never drawn. + */ +static int64 +dt_val(pg_prop_tc *tc, int64 lo, int64 hi, int64 wide, int64 bad_lo, + int64 bad_hi) +{ + int64 v; + + if (pg_prop_draw_int(tc, 0, 3) > 0) + return pg_prop_draw_int(tc, lo, hi); + v = pg_prop_draw_int(tc, -wide, wide); + if (v >= bad_lo && v <= bad_hi) + v = hi; + return v; +} + +static const char *const rm_good[] = +{"I", "II", "III", "IV", "V", "VI", "VII", "VIII", "IX", "X", "XI", "XII"}; + +/* roman strings that are not a month but start with one */ +static const char *const rm_bad[] = +{"XIII", "IIII", "VV", "IIX", "XIV", "XX", "VIIII", "IXI", "XIX", "VIV", "IIV", "XIII"}; + +/* days in month m of Gregorian year y */ +static int +cfb3_mdays(int64 y, int64 m) +{ + static const int md[] = {31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31}; + + return (m == 2 && cfb3_leap(y)) ? 29 : md[m - 1]; +} + +/* which CF's fields a template exercises */ +typedef enum DtFamily +{ + DT_RANGE, /* CF 7411: DDD IDDD IW ID WW RM SSSSS */ + DT_Y_YYY, /* CF 7416: Y,YYY millennia */ + DT_ZERO_MD /* CF 6792: MM / DD, zero = unset */ +} DtFamily; + +/* + * Build a to_date / to_timestamp template from a random sequence of fields + * around family "fam" -- a year (YYYY, Y,YYY or IYYY), a day designation + * (MM DD, RM [DD], DDD, WW, IDDD, IW [ID]), optionally SSSSS -- with random + * separators and FM, each value drawn mostly in its domain but sometimes + * far outside it (0, negative, 4+ digits, millennia past int, a roman + * numeral with a month as a prefix). + * + * Invariant: when every field is inside its documented domain, to_X(s, f) + * succeeds and to_char(to_X(s, f), f) = s. Otherwise to_X may raise an + * error, but if it returns a value that value must still render back as s: + * silently reading "2024 999" (IYYY IDDD) as some other day, "XIII" as + * December, "-1" (SSSSS) as 1, or 4294969320,024 (Y,YYY) as year 2024024 + * breaks the round trip. The only out-of-domain values a correct server + * maps elsewhere are the documented "zero month / zero day = unset" (CF + * 6792), modelled as 1; in-domain ISO days/weeks past the end of a 52-week + * ISO year, which roll into the next year, are never drawn. + */ +static void +dt_roundtrip(pg_prop_tc *tc, DtFamily fam) +{ + pg_prop_conn *conn; + DtTpl t; + bool ts; + bool iso; + int yfmt; + int64 year; + int shape; + StringInfoData q; + bool failed = false; + char *err = NULL; + char *got; + + initStringInfo(&t.in); + initStringInfo(&t.fmt); + initStringInfo(&t.exp); + t.fm = pg_prop_draw_bool(tc); + t.valid = true; + t.nfields = 0; + ts = pg_prop_draw_bool(tc); + + /* year: 0 = YYYY, 1 = Y,YYY, 2 = IYYY (ISO day fields follow) */ + if (fam == DT_Y_YYY) + yfmt = 1; + else if (fam == DT_ZERO_MD) + yfmt = (int) pg_prop_draw_int(tc, 0, 1); + else + yfmt = (int) pg_prop_draw_int(tc, 0, 2); + iso = (yfmt == 2); + year = pg_prop_draw_int(tc, 1, 9999); + if (yfmt == 0) + dt_num(tc, &t, "YYYY", 4, year, year); + else if (yfmt == 2) + dt_num(tc, &t, "IYYY", 4, year, year); + else + { + int64 mil = year / 1000; + int64 rest = year % 1000; + int64 shown = mil; + + if (fam == DT_Y_YYY && pg_prop_draw_bool(tc)) + { + /* a millennium that aliases a valid one mod 2^32, or is huge */ + int64 k = pg_prop_draw_int(tc, -3, 3); + + t.valid = false; + if (k == 0) + shown = pg_prop_draw_int(tc, (int64) PG_INT32_MAX + 1, + PG_INT64_MAX / 10); + else + shown = mil + k * ((int64) 1 << 32); + } + dt_sep(tc, &t, false); + appendStringInfoString(&t.fmt, "Y,YYY"); + appendStringInfo(&t.in, "%lld,%03lld", (long long) shown, + (long long) rest); + appendStringInfo(&t.exp, "%lld,%03lld", (long long) shown, + (long long) rest); + } + + /* day designation */ + shape = (int) pg_prop_draw_int(tc, 0, 3); + if (iso) + { + int weeks = cfb3_iso_weeks(year); + + if (shape <= 1) + { + int64 d = dt_in(&t, dt_val(tc, 1, weeks * 7, 2000, + weeks * 7 + 1, 371), 1, weeks * 7); + + dt_num(tc, &t, "IDDD", 3, d, d); + } + else + { + int64 w = dt_in(&t, dt_val(tc, 1, weeks, 200, weeks + 1, 53), + 1, weeks); + + dt_num(tc, &t, "IW", 2, w, w); + if (shape == 3) + { + int64 d = dt_in(&t, dt_val(tc, 1, 7, 20, 1, 7), 1, 7); + + dt_num(tc, &t, "ID", 1, d, d); + } + } + } + else if (fam != DT_RANGE || shape == 0) + { + /* MM DD; in DT_ZERO_MD zero month/day mean "unset", read as 1 */ + int64 mm; + int64 dd; + + if (fam == DT_ZERO_MD) + { + mm = pg_prop_draw_int(tc, 0, 13); + dd = pg_prop_draw_int(tc, 0, 32); + if (mm > 12 || dd > cfb3_mdays(year, mm ? mm : 1)) + t.valid = false; + } + else + { + mm = pg_prop_draw_int(tc, 1, 12); + dd = pg_prop_draw_int(tc, 1, cfb3_mdays(year, mm)); + } + dt_num(tc, &t, "MM", 2, mm, mm ? mm : 1); + dt_num(tc, &t, "DD", 2, dd, dd ? dd : 1); + } + else if (shape == 1) + { + /* RM [DD]: roman month, sometimes a non-month with a month prefix */ + bool lower = pg_prop_draw_bool(tc); + const char *r; + + if (pg_prop_draw_int(tc, 0, 2) == 0) + { + t.valid = false; + r = rm_bad[pg_prop_draw_int(tc, 0, lengthof(rm_bad) - 1)]; + } + else + r = rm_good[pg_prop_draw_int(tc, 0, lengthof(rm_good) - 1)]; + dt_sep(tc, &t, false); + appendStringInfoString(&t.fmt, lower ? "FMrm" : "FMRM"); + for (const char *c = r; *c; c++) + { + appendStringInfoChar(&t.in, lower ? pg_ascii_tolower(*c) : *c); + appendStringInfoChar(&t.exp, lower ? pg_ascii_tolower(*c) : *c); + } + if (pg_prop_draw_bool(tc)) + { + int64 dd = pg_prop_draw_int(tc, 1, 28); + + dt_num(tc, &t, "DD", 2, dd, dd); + } + } + else if (shape == 2) + { + int ndays = cfb3_leap(year) ? 366 : 365; + int64 d = dt_in(&t, dt_val(tc, 1, ndays, 2000, ndays + 1, ndays + 1), + 1, ndays); + + dt_num(tc, &t, "DDD", 3, d, d); + } + else + { + int64 w = dt_in(&t, dt_val(tc, 1, 53, 200, 1, 53), 1, 53); + + dt_num(tc, &t, "WW", 2, w, w); + } + + /* seconds of day, to_timestamp only */ + if (ts && pg_prop_draw_bool(tc)) + { + int64 sec = (fam == DT_RANGE) ? + dt_in(&t, dt_val(tc, 0, 86399, 200000, 0, 86399), 0, 86399) : + pg_prop_draw_int(tc, 0, 86399); + + dt_sep(tc, &t, false); + appendStringInfo(&t.fmt, "%sSSSSS", t.fm ? "FM" : ""); + appendStringInfo(&t.in, "%lld", (long long) sec); + appendStringInfo(&t.exp, "%lld", (long long) sec); + } + + conn = cfb3_connect(tc); + cfb3_exec(tc, conn, "SET timezone = 'UTC'"); + initStringInfo(&q); + appendStringInfo(&q, "SELECT to_char(%s('%s', '%s'), '%s')", + ts ? "to_timestamp" : "to_date", + t.in.data, t.fmt.data, t.fmt.data); + got = pg_prop_sql_scalar(conn, q.data, &failed, &err); + if (failed) + { + /* rejecting an out-of-domain input is allowed; nothing else is */ + if (t.valid || cfb3_internal_error(err)) + pg_prop_fail(tc, "%s rejected a valid input: %s\n SQL: %s", + ts ? "to_timestamp" : "to_date", err ? err : "", + q.data); + } + else if (got == NULL || strcmp(got, t.exp.data) != 0) + pg_prop_fail(tc, + "%s accepted an input that does not round-trip\n" + " input: '%s'\n format: '%s'\n" + " to_char of the result: '%s'\n" + " expected: '%s'\n SQL: %s", + ts ? "to_timestamp" : "to_date", t.in.data, t.fmt.data, + got ? got : "(null)", t.exp.data, q.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7411: DDD / IDDD / IW / ID / WW / RM / SSSSS out of their domains + * (0, 4+ digits, negatives, "XIII") must be rejected, not reinterpreted. + */ +PG_PROP(to_date_field_range_roundtrip) +{ + dt_roundtrip(tc, DT_RANGE); +} + +/* + * CF 7416: a Y,YYY millennium outside int must be rejected, not truncated + * mod 2^32 (4294969320,024 -> year 2024024). + */ +PG_PROP(to_date_y_yyy_roundtrip) +{ + dt_roundtrip(tc, DT_Y_YYY); +} + +/* + * CF 6792 (tests-only patch): zero month / zero day are "unset" and read + * as 1; every other MM/DD round-trips or is rejected. Pins down existing + * behaviour, so master is expected to pass. + */ +PG_PROP(to_date_zero_month_day) +{ + dt_roundtrip(tc, DT_ZERO_MD); +} + +/* ---------------------------------------------------------------- + * datetime format cache: std vs lenient mode (CF 7409) + * ---------------------------------------------------------------- + */ + +#define FC_MAX_FMTS 48 +#define FC_MAX_STEPS 120 + +/* format fields, separators legal in SQL/JSON std mode, and others */ +static const char *const fc_fields[] = {"HH24", "MI", "SS", "YYYY", "MM", "DD"}; +static const char *const fc_std_seps[] = {"-", ":", " ", ".", "/", ","}; +static const char *const fc_lax_seps[] = {"z", "#", "@", "_", "!", "%"}; + +/* append s escaped for a SQL literal ('') */ +static void +fc_sql_lit(StringInfo b, const char *s) +{ + appendStringInfoChar(b, '\''); + for (; *s; s++) + { + if (*s == '\'') + appendStringInfoChar(b, '\''); + appendStringInfoChar(b, *s); + } + appendStringInfoChar(b, '\''); +} + +/* append s as a double-quoted JSON / jsonpath string body */ +static void +fc_json_str(StringInfo b, const char *s) +{ + appendStringInfoChar(b, '"'); + for (; *s; s++) + { + if (*s == '"' || *s == '\\') + appendStringInfoChar(b, '\\'); + appendStringInfoChar(b, *s); + } + appendStringInfoChar(b, '"'); +} + +/* run one call, rendering the outcome (value or first error line) */ +static char * +fc_outcome(pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *v = pg_prop_sql_scalar(conn, sql, &failed, &err); + StringInfoData r; + + initStringInfo(&r); + if (failed) + { + size_t n = err ? strcspn(err, "\n") : 0; + + appendStringInfo(&r, "ERR %.*s", (int) n, err ? err : ""); + if (cfb3_internal_error(err)) + appendStringInfoString(&r, " [internal]"); + } + else + appendStringInfo(&r, "OK %s", v ? v : "(null)"); + if (v) + pg_free(v); + if (err) + pg_free(err); + return r.data; +} + +/* + * datetime_format_cache_mode_independent (CF 7409) + * + * The parsed-format cache (DCHCache, 20 entries, LRU) is keyed on the + * format text and on whether it was parsed in SQL/JSON standard mode + * (jsonpath .datetime()) or lenient mode (to_char / to_timestamp). + * Draw a pool of up to 48 formats -- fields with separators, some legal + * only in lenient mode ("HH24zMI"), some made distinct by a quoted + * literal -- and a schedule of phases, each a run of calls in one mode + * over sequential or random formats, so the cache fills, evicts and + * recycles entries across modes. + * + * Invariant: each call's outcome (value or error message) is a function + * of its arguments only. The reference runs the same calls in two fresh + * sessions, one per mode: a session that only ever parses in one mode + * cannot reuse an entry across modes, so it gives the answer a correct + * cache must reproduce in the mixed session. On master a recycled entry + * keeps its previous owner's mode flag, so .datetime("HH24zMI") hits a + * lenient tree and accepts "12z34" instead of rejecting the separator. + */ +PG_PROP(datetime_format_cache_mode_independent) +{ + char *fmts[FC_MAX_FMTS]; + char *rendered[FC_MAX_FMTS]; + int nfmt; + int step_fmt[FC_MAX_STEPS]; + bool step_std[FC_MAX_STEPS]; + char *step_ref[FC_MAX_STEPS]; + int nstep = 0; + int nphase; + int hh, + mi; + pg_prop_conn *lax; + pg_prop_conn *std; + pg_prop_conn *mixed; + StringInfoData q; + char ts[64]; + + nfmt = (int) pg_prop_draw_int(tc, 1, FC_MAX_FMTS); + for (int i = 0; i < nfmt; i++) + { + StringInfoData f; + int nf = (int) pg_prop_draw_int(tc, 1, 3); + + initStringInfo(&f); + if (pg_prop_draw_bool(tc)) + appendStringInfo(&f, "\"x%d\"", i); + for (int k = 0; k < nf; k++) + { + if (k > 0) + { + if (pg_prop_draw_int(tc, 0, 2) == 0) + appendStringInfoString(&f, fc_lax_seps[pg_prop_draw_int(tc, 0, lengthof(fc_lax_seps) - 1)]); + else + appendStringInfoString(&f, fc_std_seps[pg_prop_draw_int(tc, 0, lengthof(fc_std_seps) - 1)]); + } + appendStringInfoString(&f, fc_fields[pg_prop_draw_int(tc, 0, lengthof(fc_fields) - 1)]); + } + fmts[i] = f.data; + } + + /* phases: a run of one mode over sequential (fresh) or random formats */ + nphase = (int) pg_prop_draw_int(tc, 1, 5); + for (int p = 0; p < nphase && nstep < FC_MAX_STEPS; p++) + { + bool is_std = pg_prop_draw_bool(tc); + bool seq = pg_prop_draw_bool(tc); + int start = (int) pg_prop_draw_int(tc, 0, nfmt - 1); + int len = (int) pg_prop_draw_int(tc, 1, 40); + + for (int k = 0; k < len && nstep < FC_MAX_STEPS; k++) + { + step_std[nstep] = is_std; + step_fmt[nstep] = seq ? (start + k) % nfmt : + (int) pg_prop_draw_int(tc, 0, nfmt - 1); + nstep++; + } + } + + hh = (int) pg_prop_draw_int(tc, 0, 23); + mi = (int) pg_prop_draw_int(tc, 0, 59); + snprintf(ts, sizeof(ts), "timestamp '2024-03-04 %02d:%02d:07'", hh, mi); + + lax = cfb3_connect(tc); + std = cfb3_connect(tc); + mixed = cfb3_connect(tc); + initStringInfo(&q); + + /* what each format renders to, so std mode has an input to parse */ + for (int i = 0; i < nfmt; i++) + { + bool failed = false; + char *err = NULL; + + resetStringInfo(&q); + appendStringInfo(&q, "SELECT to_char(%s, ", ts); + fc_sql_lit(&q, fmts[i]); + appendStringInfoChar(&q, ')'); + rendered[i] = pg_prop_sql_scalar(lax, q.data, &failed, &err); + if (failed || rendered[i] == NULL) + pg_prop_fail(tc, "to_char failed: %s\n SQL: %s", + err ? err : "", q.data); + } + + /* reference outcomes from the single-mode sessions */ + for (int s = 0; s < nstep; s++) + { + int i = step_fmt[s]; + StringInfoData j; + + initStringInfo(&j); + resetStringInfo(&q); + if (step_std[s]) + { + fc_json_str(&j, rendered[i]); + appendStringInfoString(&q, "SELECT jsonb_path_query_first("); + fc_sql_lit(&q, j.data); + resetStringInfo(&j); + appendStringInfoString(&j, "$.datetime("); + fc_json_str(&j, fmts[i]); + appendStringInfoChar(&j, ')'); + appendStringInfoString(&q, "::jsonb, "); + fc_sql_lit(&q, j.data); + appendStringInfoString(&q, "::jsonpath)::text"); + } + else + { + /* lenient parse and lenient render, both through the cache */ + appendStringInfoString(&q, "SELECT to_timestamp("); + fc_sql_lit(&q, rendered[i]); + appendStringInfoString(&q, ", "); + fc_sql_lit(&q, fmts[i]); + appendStringInfoString(&q, ")::text || ' ' || to_char("); + appendStringInfo(&q, "%s, ", ts); + fc_sql_lit(&q, fmts[i]); + appendStringInfoChar(&q, ')'); + } + pfree(j.data); + step_ref[s] = pg_strdup(q.data); + } + cfb3_exec(tc, lax, "SET timezone = 'UTC'"); + cfb3_exec(tc, std, "SET timezone = 'UTC'"); + cfb3_exec(tc, mixed, "SET timezone = 'UTC'"); + for (int s = 0; s < nstep; s++) + { + char *want = fc_outcome(step_std[s] ? std : lax, step_ref[s]); + char *got = fc_outcome(mixed, step_ref[s]); + + if (strcmp(want, got) != 0 || strstr(got, "[internal]") != NULL) + { + StringInfoData hist; + + initStringInfo(&hist); + for (int k = Max(0, s - 40); k <= s; k++) + appendStringInfo(&hist, " %s:%d", step_std[k] ? "std" : "lax", + step_fmt[k]); + pg_prop_fail(tc, + "call %d gives a different outcome after %d cached " + "format calls in mixed modes\n" + " SQL: %s\n mixed session: %s\n" + " single-mode session: %s\n" + " %d formats, last steps (mode:format#):%s", + s, s, step_ref[s], got, want, nfmt, hist.data); + } + pg_free(want); + pg_free(got); + } + pg_prop_disconnect(mixed); + pg_prop_disconnect(std); + pg_prop_disconnect(lax); +} + +/* ---------------------------------------------------------------- + * date_trunc on timestamptz near time-zone transitions (CF 7243) + * ---------------------------------------------------------------- + */ + +/* zones with offset changes that are not whole hours, or not at hh:00 */ +static const struct +{ + const char *zone; + int year; +} tz_odd[] = { + {"Europe/Athens", 1916}, /* +1:34:52 -> +2 at 00:01 local */ + {"Europe/Dublin", 1916}, /* -0:25:21 -> +0:34:39 */ + {"Europe/Amsterdam", 1937}, /* +0:19:32 -> +0:20 */ + {"Europe/Moscow", 1919}, + {"Africa/Monrovia", 1972}, /* -0:44:30 -> 0 */ + {"Asia/Kathmandu", 1986}, /* +5:30 -> +5:45 */ + {"Asia/Singapore", 1981}, /* +7:30 -> +8 */ + {"Asia/Colombo", 1996}, + {"Asia/Pyongyang", 2015}, /* +9 -> +8:30 */ + {"Asia/Pyongyang", 2018}, + {"America/Caracas", 2007}, /* -4 -> -4:30 */ + {"America/Caracas", 2016}, + {"Australia/Lord_Howe", 2020}, /* 30-minute DST */ + {"Australia/Eucla", 2008}, + {"Pacific/Chatham", 2020}, /* transitions at 2:45 local */ + {"America/St_Johns", 2020}, + {"America/New_York", 2020}, /* hour-sized, at hh:00: control */ +}; + +static const char *const trunc_units[] = +{"minute", "hour", "day", "week", "month", "quarter", "year", "decade", "second"}; + +/* + * date_trunc_tz_wall_clock (CF 7243) + * + * Pick a zone and a year (curated zones with odd transitions, or any zone + * in pg_timezone_names and any year), locate its k-th offset change in + * that year to the second, and draw t at a microsecond distance from it + * (within a minute, a few hours, or two days) and a unit; sometimes use + * the session time zone (2-argument form) instead of the explicit zone. + * + * Oracle: date_trunc(u, t, z) is the instant whose wall-clock time in z + * is W = date_trunc(u, t AT TIME ZONE z), timestamp truncation being pure + * calendar arithmetic. The property collects every instant c with + * c AT TIME ZONE z = W (W - o for each UTC offset o z uses within 16 + * hours, kept if it maps back to W) and, when exactly one exists (W is + * neither skipped nor repeated by a transition), requires date_trunc to + * return it. Ambiguous or nonexistent wall times are left to the + * server's resolution rule and not checked. On master, minute and hour + * truncation keep t's own offset even when W lies across a transition. + */ +PG_PROP(date_trunc_tz_wall_clock) +{ + pg_prop_conn *conn; + StringInfoData q; + StringInfoData zexpr; + char *bad; + const char *unit; + int64 year; + int64 k = pg_prop_draw_int(tc, 0, 3); + int64 span; + int64 off; + bool session_tz = pg_prop_draw_int(tc, 0, 3) == 0; + + /* sub-day units are where an offset change lands inside the period */ + if (pg_prop_draw_bool(tc)) + unit = pg_prop_draw_bool(tc) ? "minute" : "hour"; + else + unit = trunc_units[pg_prop_draw_int(tc, 0, lengthof(trunc_units) - 1)]; + /* microseconds from the transition instant */ + switch (pg_prop_draw_int(tc, 0, 2)) + { + case 0: + span = INT64CONST(120000000); + break; + case 1: + span = INT64CONST(3) * INT64CONST(3600000000); + break; + default: + span = INT64CONST(48) * INT64CONST(3600000000); + break; + } + off = pg_prop_draw_int(tc, -span, span); + + initStringInfo(&zexpr); + if (pg_prop_draw_int(tc, 0, 3) > 0) + { + int i = (int) pg_prop_draw_int(tc, 0, lengthof(tz_odd) - 1); + + appendStringInfo(&zexpr, "'%s'::text", tz_odd[i].zone); + year = tz_odd[i].year + pg_prop_draw_int(tc, -1, 1) * + (pg_prop_draw_int(tc, 0, 2) == 0); + } + else + { + appendStringInfo(&zexpr, + "(SELECT name FROM pg_timezone_names ORDER BY name " + "OFFSET %lld %% (SELECT count(*) FROM pg_timezone_names) " + "LIMIT 1)", + (long long) pg_prop_draw_int(tc, 0, 100000)); + year = pg_prop_draw_int(tc, 1880, 2037); + } + + conn = cfb3_connect(tc); + cfb3_exec(tc, conn, "SET timezone = 'UTC'"); + initStringInfo(&q); + appendStringInfo(&q, + "WITH zz AS (SELECT %s AS z), " + "tr AS (SELECT g, row_number() OVER (ORDER BY g) - 1 AS rn, " + " count(*) OVER () AS n " + " FROM zz, generate_series(make_timestamptz(%lld,1,1,0,0,0,'UTC'), " + " make_timestamptz(%lld,1,1,0,0,0,'UTC'), " + " interval '1 hour') g " + " WHERE (g AT TIME ZONE z) - (g AT TIME ZONE 'UTC') <> " + " ((g + interval '1 hour') AT TIME ZONE z) - " + " ((g + interval '1 hour') AT TIME ZONE 'UTC')), " + "hr AS (SELECT coalesce((SELECT g FROM tr WHERE rn = %lld %% n), " + " make_timestamptz(%lld,7,1,0,0,0,'UTC')) AS g), " + /* the first second of hour g at a new offset */ + "base AS (SELECT coalesce(min(s), g) AS g " + " FROM zz, hr, generate_series(g, g + interval '1 hour', " + " interval '1 second') s " + " WHERE (s AT TIME ZONE z) - (s AT TIME ZONE 'UTC') <> " + " (g AT TIME ZONE z) - (g AT TIME ZONE 'UTC') " + " GROUP BY g), " + "pt AS (SELECT z, g + interval '%lld microseconds' AS t FROM zz, base), " + "w AS (SELECT z, t, date_trunc('%s', t AT TIME ZONE z) AS w FROM pt), " + "x AS (SELECT w.*, %s AS d FROM w), " + "offs AS (SELECT DISTINCT (s AT TIME ZONE z) - (s AT TIME ZONE 'UTC') AS o " + " FROM w, generate_series((w AT TIME ZONE 'UTC') - interval '16 hours', " + " (w AT TIME ZONE 'UTC') + interval '16 hours', " + " interval '10 minutes') s " + " UNION SELECT (t AT TIME ZONE z) - (t AT TIME ZONE 'UTC') FROM pt " + " UNION SELECT (d AT TIME ZONE z) - (d AT TIME ZONE 'UTC') FROM x), " + "c AS (SELECT DISTINCT (w - o) AT TIME ZONE 'UTC' AS c FROM w, offs " + " WHERE ((w - o) AT TIME ZONE 'UTC') AT TIME ZONE z = w) " + "SELECT format('zone %%s, t = %%s (%%s local), wall-clock trunc %%s = %%s, " + "date_trunc gave %%s (%%s local)', " + "z, t, t AT TIME ZONE z, '%s', w, d, d AT TIME ZONE z) " + "FROM x " + "WHERE (SELECT count(*) FROM c) = 1 AND d <> (SELECT c FROM c)", + zexpr.data, (long long) year, (long long) year + 1, + (long long) k, (long long) year, (long long) off, + unit, + session_tz ? + "pg_temp.cfb3_trunc_session(t, z)" : + psprintf("date_trunc('%s', t, z)", unit), + unit); + if (session_tz) + { + /* the 2-argument form, under SET timezone = z */ + StringInfoData f; + + initStringInfo(&f); + appendStringInfo(&f, + "CREATE FUNCTION pg_temp.cfb3_trunc_session(t timestamptz, z text) " + "RETURNS timestamptz LANGUAGE plpgsql AS $$ " + "DECLARE r timestamptz; BEGIN " + "PERFORM set_config('timezone', z, true); " + "r := date_trunc('%s', t); " + "PERFORM set_config('timezone', 'UTC', true); " + "RETURN r; END $$", unit); + cfb3_exec(tc, conn, f.data); + pfree(f.data); + } + bad = cfb3_scalar(tc, conn, q.data); + /* no SQL in the message: pg_prop_fail drops messages over ~1 kB */ + if (bad != NULL) + pg_prop_fail(tc, "date_trunc disagrees with the wall-clock truncation (%s form)\n %s", + session_tz ? "session time zone, 2-argument" : "explicit zone", + bad); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------- + * time / timetz +/- interval (CF 7349) + * ---------------------------------------------------------------- + */ + +#define CFB3_USECS_PER_DAY INT64CONST(86400000000) + +/* render a time-of-day in [0, USECS_PER_DAY) as HH:MM:SS.ffffff */ +static void +cfb3_tod(StringInfo b, int64 us) +{ + appendStringInfo(b, "%02d:%02d:%02d.%06d", + (int) (us / INT64CONST(3600000000)), + (int) (us / 60000000 % 60), + (int) (us / 1000000 % 60), + (int) (us % 1000000)); +} + +/* + * time_interval_arith_mod_day (CF 7349) + * + * Draw a time (or timetz with a drawn zone offset), and an interval with + * random months and days and a time field anywhere in int64 -- biased to + * its extremes, where t + span->time overflows -- and an operator (+/-). + * + * Model (C, exact): time +/- interval ignores months and days and wraps + * modulo one day, so the result is (t +/- (span mod 86400000000)) mod + * 86400000000, computed here without overflow; timetz keeps its zone. + * Also cross-checked: (t + i) - i = t. Both hold for every interval on + * a correct server; on master the int64 sum overflows for large time + * fields and returns a wrong time of day. + */ +PG_PROP(time_interval_arith_mod_day) +{ + pg_prop_conn *conn; + int64 t = pg_prop_draw_int(tc, 0, CFB3_USECS_PER_DAY - 1); + int64 span; + int64 mons = pg_prop_draw_int(tc, -1000, 1000); + int64 days = pg_prop_draw_int(tc, -100000, 100000); + bool minus = pg_prop_draw_bool(tc); + bool tz = pg_prop_draw_bool(tc); + int64 tzmin = pg_prop_draw_int(tc, -15 * 60, 15 * 60); + int64 r; + StringInfoData lit; + StringInfoData exp; + StringInfoData q; + char *got; + + switch (pg_prop_draw_int(tc, 0, 3)) + { + case 0: + span = pg_prop_draw_int(tc, PG_INT64_MAX - CFB3_USECS_PER_DAY, + PG_INT64_MAX); + break; + case 1: + span = pg_prop_draw_int(tc, PG_INT64_MIN, + PG_INT64_MIN + CFB3_USECS_PER_DAY); + break; + case 2: + span = pg_prop_draw_int(tc, -10 * CFB3_USECS_PER_DAY, + 10 * CFB3_USECS_PER_DAY); + break; + default: + span = pg_prop_draw_int(tc, PG_INT64_MIN, PG_INT64_MAX); + break; + } + /* -9223372036854775808 has no positive counterpart to subtract */ + r = span % CFB3_USECS_PER_DAY; /* in (-day, day), no overflow */ + r = minus ? t - r : t + r; /* in (-day, 2 day) */ + r %= CFB3_USECS_PER_DAY; + if (r < 0) + r += CFB3_USECS_PER_DAY; + + initStringInfo(&lit); + appendStringInfo(&lit, "%s '", tz ? "timetz" : "time"); + cfb3_tod(&lit, t); + if (tz) + appendStringInfo(&lit, "%c%02d:%02d", tzmin < 0 ? '-' : '+', + (int) (llabs(tzmin) / 60), (int) (llabs(tzmin) % 60)); + appendStringInfoChar(&lit, '\''); + + initStringInfo(&exp); + cfb3_tod(&exp, r); + + conn = cfb3_connect(tc); + initStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN to_char(x::time, 'HH24:MI:SS.US') = '%s' " + "%s THEN NULL ELSE x::text END " + "FROM (SELECT %s %c (make_interval(months => %lld, days => %lld) " + " + interval '%lld microseconds') AS x) s", + exp.data, + tz ? psprintf("AND x::text LIKE '%%%c%02d%s'", + tzmin < 0 ? '-' : '+', + (int) (llabs(tzmin) / 60), + llabs(tzmin) % 60 ? + psprintf(":%02d", (int) (llabs(tzmin) % 60)) : "") : "", + lit.data, minus ? '-' : '+', + (long long) mons, (long long) days, (long long) span); + got = cfb3_scalar(tc, conn, q.data); + if (got != NULL) + pg_prop_fail(tc, "%s %c interval (months %lld, days %lld, time %lld us) " + "= %s, expected %s (time of day mod 24h)", + lit.data, minus ? '-' : '+', (long long) mons, + (long long) days, (long long) span, got, exp.data); + + /* (t + i) - i = t */ + resetStringInfo(&q); + appendStringInfo(&q, + "SELECT CASE WHEN (%s + i) - i = %s THEN NULL " + "ELSE ((%s + i) - i)::text END " + "FROM (SELECT interval '%lld microseconds' AS i) s", + lit.data, lit.data, lit.data, (long long) span); + got = cfb3_scalar(tc, conn, q.data); + if (got != NULL) + pg_prop_fail(tc, "(%s + i) - i = %s, i = %lld us", lit.data, got, + (long long) span); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------- + * SQL/JSON DEFAULT ... ON EMPTY / ON ERROR (CF 7181, 7355, 7414) + * ---------------------------------------------------------------- + */ + +static const char *const sj_types[] = +{"int", "bigint", "smallint", "text", "varchar", "bool", "numeric", +"pg_temp.cfb3_dnz", "pg_temp.cfb3_dnn"}; + +static const char *const sj_setup = +"CREATE DOMAIN pg_temp.cfb3_dnz AS int CHECK (VALUE <> 0); " +"CREATE DOMAIN pg_temp.cfb3_dnn AS int NOT NULL"; + +/* the outcome of a scalar query: "V:", "N" (NULL) or "E" (error) */ +static char * +sj_outcome(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *v = pg_prop_sql_scalar(conn, sql, &failed, &err); + char *r; + + if (failed && cfb3_internal_error(err)) + pg_prop_fail(tc, "internal error: %.300s\n SQL: %.500s", err, sql); + if (failed) + r = pg_strdup("E"); + else if (v == NULL) + r = pg_strdup("N"); + else + r = psprintf("V:%s", v); + if (v) + pg_free(v); + if (err) + pg_free(err); + return r; +} + +/* + * An int-valued DEFAULT sub-expression: constant, volatile, or (unless + * "safe") one that may raise a cast, domain or division error. + */ +static void +sj_int_expr(pg_prop_tc *tc, StringInfo b, int depth, bool safe) +{ + int64 k = pg_prop_draw_int(tc, -3, 3); + int64 c = pg_prop_draw_int(tc, 0, depth > 0 ? 8 : 4); + + if (safe && c >= 2 && c <= 5) + c = (c == 4) ? 4 : 1; + switch (c) + { + case 0: + appendStringInfo(b, "(%lld)", (long long) k); + break; + case 1: + appendStringInfo(b, "((random() * 0)::int + %lld)", (long long) k); + break; + case 2: + /* a soft error from a cast inside the expression (CF 7355) */ + appendStringInfo(b, "(('a' || (random() * 0)::int)::int + %lld)", + (long long) k); + break; + case 3: + /* a domain check (k = 0 fails) */ + appendStringInfo(b, "(%lld::pg_temp.cfb3_dnz + 0)", (long long) k); + break; + case 4: + appendStringInfoString(b, !safe && pg_prop_draw_bool(tc) ? + "(NULL::pg_temp.cfb3_dnn + 0)" : "NULL::int"); + break; + case 5: + appendStringInfo(b, "(1 / ((random() * 0)::int + %lld))", + (long long) k); + break; + case 6: + appendStringInfoString(b, "abs"); + appendStringInfoChar(b, '('); + sj_int_expr(tc, b, depth - 1, safe); + appendStringInfoChar(b, ')'); + break; + default: + appendStringInfoChar(b, '('); + sj_int_expr(tc, b, depth - 1, safe); + appendStringInfoString(b, pg_prop_draw_bool(tc) ? " + " : " * "); + sj_int_expr(tc, b, depth - 1, safe); + appendStringInfoChar(b, ')'); + break; + } +} + +/* + * A DEFAULT expression. "boolean": a boolean-valued one, literal or + * computed, constant or volatile, built from error-free operands (CF + * 7181). Otherwise: an int or text one whose evaluation may raise an + * error somewhere below its top node (CF 7355). + */ +static void +sj_default_expr(pg_prop_tc *tc, StringInfo b, bool boolean) +{ + if (boolean) + { + switch (pg_prop_draw_int(tc, 0, 3)) + { + case 0: + appendStringInfoString(b, pg_prop_draw_bool(tc) ? "true" : "false"); + break; + case 1: + appendStringInfoString(b, pg_prop_draw_bool(tc) ? "(random() < 2)" : + "(1 = 1)"); + break; + default: + appendStringInfoChar(b, '('); + sj_int_expr(tc, b, 1, true); + appendStringInfoString(b, pg_prop_draw_bool(tc) ? " < " : " = "); + sj_int_expr(tc, b, 1, true); + appendStringInfoChar(b, ')'); + break; + } + return; + } + switch (pg_prop_draw_int(tc, 0, 3)) + { + case 0: + appendStringInfo(b, "'%lld'", (long long) pg_prop_draw_int(tc, -2, 99)); + break; + case 1: + appendStringInfoString(b, "('x' || "); + sj_int_expr(tc, b, 1, false); + appendStringInfoChar(b, ')'); + break; + default: + sj_int_expr(tc, b, 2, false); + break; + } +} + +/* + * Draw a JSON_VALUE / JSON_QUERY / JSON_TABLE column, a RETURNING type + * (including domains with NOT NULL and CHECK) and a DEFAULT expression, + * with a document/path for which the DEFAULT is certainly taken: ON EMPTY + * with a lax path to a missing key, or ON ERROR with a strict one. + * + * Oracle: the result then is the DEFAULT expression coerced to the + * RETURNING type, so its outcome (value, NULL, or error) must equal that + * of SELECT CAST((e) AS type) evaluated on its own. + */ +static void +sj_default_is_cast(pg_prop_tc *tc, bool boolean) +{ + pg_prop_conn *conn; + const char *typ = sj_types[pg_prop_draw_int(tc, 0, lengthof(sj_types) - 1)]; + int fn = (int) pg_prop_draw_int(tc, 0, 2); + bool on_error = pg_prop_draw_bool(tc); + const char *quotes = ""; + StringInfoData e; + StringInfoData doc; + StringInfoData q; + StringInfoData m; + char *got; + char *want; + + initStringInfo(&e); + sj_default_expr(tc, &e, boolean); + if (fn > 0) + { + switch (pg_prop_draw_int(tc, 0, 2)) + { + case 1: + quotes = " KEEP QUOTES"; + break; + case 2: + quotes = " OMIT QUOTES"; + break; + } + } + /* a document without key "a" */ + initStringInfo(&doc); + if (pg_prop_draw_bool(tc)) + appendStringInfoString(&doc, "{}"); + else + appendStringInfo(&doc, "{\"b\": %lld}", (long long) pg_prop_draw_int(tc, 0, 9)); + + initStringInfo(&q); + if (fn < 2) + appendStringInfo(&q, + "SELECT JSON_%s('%s'::jsonb, '%s $.a' RETURNING %s%s " + "DEFAULT %s ON %s)::text", + fn == 0 ? "VALUE" : "QUERY", doc.data, + on_error ? "strict" : "lax", typ, quotes, e.data, + on_error ? "ERROR" : "EMPTY"); + else + appendStringInfo(&q, + "SELECT c::text FROM JSON_TABLE('%s'::jsonb, '$' " + "COLUMNS (c %s PATH '%s $.a'%s DEFAULT %s ON %s))", + doc.data, typ, on_error ? "strict" : "lax", quotes, + e.data, on_error ? "ERROR" : "EMPTY"); + initStringInfo(&m); + appendStringInfo(&m, "SELECT CAST((%s) AS %s)::text", e.data, typ); + + conn = cfb3_connect(tc); + cfb3_exec(tc, conn, sj_setup); + got = sj_outcome(tc, conn, q.data); + want = sj_outcome(tc, conn, m.data); + if (strcmp(got, want) != 0) + pg_prop_fail(tc, "DEFAULT ... ON %s is not the DEFAULT cast to %s\n" + " %s -> %s\n %s -> %s", + on_error ? "ERROR" : "EMPTY", typ, q.data, got, + m.data, want); + pg_prop_disconnect(conn); +} + +/* + * sqljson_bool_default_is_cast (CF 7181) + * + * Boolean DEFAULT expressions. On master, for a non-integer RETURNING + * type the parser replaces any boolean DEFAULT by the jsonb constant + * 'false', whatever it evaluates to. + */ +PG_PROP(sqljson_bool_default_is_cast) +{ + sj_default_is_cast(tc, true); +} + +/* + * sqljson_default_error_is_cast (CF 7355) + * + * DEFAULT expressions that may raise an error below their top node (a + * cast, a domain check, a division). On master the error is evaluated + * in soft mode and silently becomes NULL unless the top node is itself a + * coercion. + */ +PG_PROP(sqljson_default_error_is_cast) +{ + sj_default_is_cast(tc, false); +} + +/* + * sqljson_rows_independent (CF 7414) + * + * Draw 2..8 rows (doc, path), each doc and path possibly SQL NULL, the + * docs chosen to make the path empty, raise a strict-mode error, or + * return a value; and one of JSON_VALUE / JSON_QUERY ... DEFAULT ON + * EMPTY / ON ERROR or JSON_EXISTS ... TRUE ON ERROR. Evaluate it over + * all rows in one query. + * + * Oracle: a row's result depends on that row only, so each must equal + * the same call evaluated alone on literals. On master a row whose + * context or path is SQL NULL inherits the previous row's ON EMPTY / + * ON ERROR outcome instead of returning NULL. + */ +PG_PROP(sqljson_rows_independent) +{ + static const char *const docs[] = + {"{}", "1", "{\"a\": 5}", "{\"a\": \"x\"}", "{\"b\": 2}", NULL}; + static const char *const paths[] = {"$.a", "strict $.a", "lax $.a", NULL}; + pg_prop_conn *conn; + int nrows = (int) pg_prop_draw_int(tc, 2, 8); + int fn = (int) pg_prop_draw_int(tc, 0, 2); + bool vol = pg_prop_draw_bool(tc); + bool path_col = pg_prop_draw_bool(tc); + const char *rd[8]; + const char *rp[8]; + char tail[200]; + StringInfoData q; + StringInfoData one; + char *all; + char *expect; + StringInfoData exp; + bool failed = false; + char *err = NULL; + + if (fn == 2) + snprintf(tail, sizeof(tail), "TRUE ON ERROR"); + else + snprintf(tail, sizeof(tail), "%s DEFAULT %s ON EMPTY DEFAULT %s ON ERROR", + fn == 0 ? "RETURNING int" : "RETURNING text", + vol ? "((random() * 0)::int + 42)" : "42", + vol ? "((random() * 0)::int + 7)" : "7"); + if (fn == 1) + snprintf(tail, sizeof(tail), "RETURNING text DEFAULT %s ON EMPTY DEFAULT %s ON ERROR", + vol ? "('e' || (random() * 0)::int)" : "'\"e\"'", + vol ? "('r' || (random() * 0)::int)" : "'\"r\"'"); + + for (int i = 0; i < nrows; i++) + { + rd[i] = docs[pg_prop_draw_int(tc, 0, lengthof(docs) - 1)]; + rp[i] = path_col ? paths[pg_prop_draw_int(tc, 0, lengthof(paths) - 1)] : + "strict $.a"; + } + + initStringInfo(&q); + appendStringInfo(&q, "SELECT string_agg(coalesce(r, 'NULL'), '|' ORDER BY i) FROM (" + "SELECT i, JSON_%s(x, %s %s)::text AS r FROM (VALUES ", + fn == 0 ? "VALUE" : fn == 1 ? "QUERY" : "EXISTS", + path_col ? "p" : "'strict $.a'", tail); + for (int i = 0; i < nrows; i++) + { + appendStringInfo(&q, "%s(%d, ", i ? ", " : "", i); + if (rd[i]) + appendStringInfo(&q, "'%s'::jsonb, ", rd[i]); + else + appendStringInfoString(&q, "NULL::jsonb, "); + if (rp[i]) + appendStringInfo(&q, "'%s'::jsonpath)", rp[i]); + else + appendStringInfoString(&q, "NULL::jsonpath)"); + } + appendStringInfoString(&q, ") v(i, x, p)) s"); + + conn = cfb3_connect(tc); + all = pg_prop_sql_scalar(conn, q.data, &failed, &err); + if (failed) + pg_prop_fail(tc, "query failed: %.300s\n SQL: %.600s", err ? err : "", q.data); + + initStringInfo(&exp); + initStringInfo(&one); + for (int i = 0; i < nrows; i++) + { + resetStringInfo(&one); + appendStringInfo(&one, "SELECT coalesce(JSON_%s(%s%s%s, %s%s%s %s)::text, 'NULL')", + fn == 0 ? "VALUE" : fn == 1 ? "QUERY" : "EXISTS", + rd[i] ? "'" : "NULL::jsonb", rd[i] ? rd[i] : "", + rd[i] ? "'::jsonb" : "", + rp[i] ? "'" : "NULL::jsonpath", rp[i] ? rp[i] : "", + rp[i] ? "'::jsonpath" : "", tail); + expect = pg_prop_sql_scalar(conn, one.data, &failed, &err); + if (failed) + pg_prop_fail(tc, "single-row query failed: %.300s\n SQL: %.600s", + err ? err : "", one.data); + appendStringInfo(&exp, "%s%s", i ? "|" : "", expect); + pg_free(expect); + } + if (all == NULL || strcmp(all, exp.data) != 0) + pg_prop_fail(tc, "per-row results depend on other rows\n" + " together: %s\n alone: %s\n SQL: %.700s", + all ? all : "(null)", exp.data, q.data); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------- + * window value functions vs the frame (CF 7412) + * ---------------------------------------------------------------- + */ + +/* frame bound kinds, in the order the grammar requires start <= end */ +static const char *const wf_bound[] = +{"UNBOUNDED PRECEDING", "%d PRECEDING", "CURRENT ROW", "%d FOLLOWING", +"UNBOUNDED FOLLOWING"}; +static const char *const wf_excl[] = +{"", " EXCLUDE TIES", " EXCLUDE TIES", " EXCLUDE GROUP", +" EXCLUDE CURRENT ROW", " EXCLUDE NO OTHERS"}; +static const char *const wf_mode[] = {"ROWS", "ROWS", "RANGE", "GROUPS"}; + +static void +wf_append_bound(pg_prop_tc *tc, StringInfo b, int kind) +{ + if (kind == 1 || kind == 3) + appendStringInfo(b, wf_bound[kind], (int) pg_prop_draw_int(tc, 0, 3)); + else + appendStringInfoString(b, wf_bound[kind]); +} + +/* + * window_value_fns_match_frame (CF 7412) + * + * Draw 1..14 rows (partition p, order key k with many ties, value x with + * NULLs) and a frame: ROWS / RANGE / GROUPS, random start <= end bounds + * with small offsets, and an exclusion; half the cases are ROWS ... + * EXCLUDE TIES with an "n FOLLOWING" start or "n PRECEDING" end. + * + * Oracle: over one named window w, array_agg(x) OVER w lists exactly the + * frame's rows in frame order (aggregates walk the frame row by row), so + * first_value(x) = a[1], last_value(x) = a[cardinality(a)], + * nth_value(x, m) = a[m], + * and with IGNORE NULLS the same against + * array_agg(x) FILTER (WHERE x IS NOT NULL) OVER w. + * Both sides share the window, hence the sort order among peers. On + * master, with EXCLUDE TIES and a ROWS frame that starts after (or ends + * before) a peer of the current row, the value functions skip to the + * current row, reject it, and return NULL although rows remain. + */ +PG_PROP(window_value_fns_match_frame) +{ + pg_prop_conn *conn; + int nrows = (int) pg_prop_draw_int(tc, 1, 14); + int kmax = (int) pg_prop_draw_int(tc, 0, 4); + bool part = pg_prop_draw_bool(tc); + bool focus = pg_prop_draw_bool(tc); + const char *mode; + int sk; + int ek; + const char *excl; + int m = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData win; + StringInfoData q; + char *bad; + + if (focus) + { + /* ROWS + EXCLUDE TIES, a frame that starts after or ends before us */ + mode = "ROWS"; + excl = " EXCLUDE TIES"; + if (pg_prop_draw_bool(tc)) + { + sk = 3; + ek = (int) pg_prop_draw_int(tc, 3, 4); + } + else + { + ek = 1; + sk = (int) pg_prop_draw_int(tc, 0, 1); + } + } + else + { + mode = wf_mode[pg_prop_draw_int(tc, 0, lengthof(wf_mode) - 1)]; + sk = (int) pg_prop_draw_int(tc, 0, 3); + ek = (int) pg_prop_draw_int(tc, Max(sk, 1), 4); + excl = wf_excl[pg_prop_draw_int(tc, 0, lengthof(wf_excl) - 1)]; + } + + initStringInfo(&win); + appendStringInfo(&win, "%sORDER BY k %s BETWEEN ", part ? "PARTITION BY p " : "", + mode); + wf_append_bound(tc, &win, sk); + appendStringInfoString(&win, " AND "); + wf_append_bound(tc, &win, ek); + appendStringInfoString(&win, excl); + + initStringInfo(&q); + appendStringInfoString(&q, + "SELECT format('row i=%s p=%s k=%s x=%s: frame %s (non-null %s); " + "first %s last %s nth(%s) %s; ignore nulls: first %s last %s nth %s', " + "i, p, k, x, a, an, fv, lv, m, nv, fvi, lvi, nvi) " + "FROM (SELECT i, p, k, x, "); + appendStringInfo(&q, "%d AS m, ", m); + appendStringInfo(&q, + "array_agg(x) OVER w AS a, " + "array_agg(x) FILTER (WHERE x IS NOT NULL) OVER w AS an, " + "first_value(x) OVER w AS fv, last_value(x) OVER w AS lv, " + "nth_value(x, %d) OVER w AS nv, " + "first_value(x) IGNORE NULLS OVER w AS fvi, " + "last_value(x) IGNORE NULLS OVER w AS lvi, " + "nth_value(x, %d) IGNORE NULLS OVER w AS nvi " + "FROM (VALUES ", m, m); + for (int i = 0; i < nrows; i++) + { + int64 x = pg_prop_draw_int(tc, -1, 9); + + appendStringInfo(&q, "%s(%d, %d, %d, ", i ? ", " : "", i, + (int) pg_prop_draw_int(tc, 0, 1), + (int) pg_prop_draw_int(tc, 0, kmax)); + if (x < 0) + appendStringInfoString(&q, "NULL::int)"); + else + appendStringInfo(&q, "%d)", (int) x); + } + appendStringInfo(&q, + ") t(i, p, k, x) WINDOW w AS (%s)) s " + "WHERE fv IS DISTINCT FROM a[1] " + " OR lv IS DISTINCT FROM a[cardinality(a)] " + " OR nv IS DISTINCT FROM a[m] " + " OR fvi IS DISTINCT FROM an[1] " + " OR lvi IS DISTINCT FROM an[cardinality(an)] " + " OR nvi IS DISTINCT FROM an[m] " + "ORDER BY i LIMIT 1", win.data); + + conn = cfb3_connect(tc); + bad = cfb3_scalar(tc, conn, q.data); + if (bad != NULL) + pg_prop_fail(tc, "window value function disagrees with the frame\n" + " WINDOW w AS (%s)\n %s\n SQL: %.900s", + win.data, bad, q.data); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------- + * citext split_part (CF 7453) + * ---------------------------------------------------------------- + */ + +/* + * citext_split_part_like_core (CF 7453) + * + * Draw an ASCII string s (letters of both cases and regex metacharacters, + * so the delimiter must be matched literally), a delimiter d (empty, + * random, or a case-flipped substring of s) and a field position n in + * [-6, 6], zero included. + * + * Oracle: on ASCII, citext matching is matching after lower(), and lower() + * preserves lengths, so the field citext's split_part returns, lowered, + * must be core split_part(lower(s), lower(d), n) -- with the same outcome + * when that raises an error (n = 0). On master citext's split_part + * returns '' for every negative position, splits an empty delimiter + * between characters, and returns NULL instead of an error for n = 0. + */ +PG_PROP(citext_split_part_like_core) +{ + static const char alpha[] = "aAbBtT.*+?|()[]{}^$\\- "; + pg_prop_conn *conn; + size_t slen; + char *s = pg_prop_draw_from(tc, alpha, 0, 16, &slen); + char *d; + int64 n = pg_prop_draw_int(tc, -6, 6); + StringInfoData q; + StringInfoData m; + char *got; + char *want; + + switch (pg_prop_draw_int(tc, 0, 3)) + { + case 0: + d = pg_strdup(""); + break; + case 1: + { + size_t dl; + + d = pg_prop_draw_from(tc, alpha, 1, 3, &dl); + break; + } + default: + if (slen == 0) + d = pg_strdup("t"); + else + { + size_t off = (size_t) pg_prop_draw_int(tc, 0, slen - 1); + size_t len = (size_t) pg_prop_draw_int(tc, 1, Min(3, slen - off)); + + d = pnstrdup(s + off, len); + for (char *c = d; *c; c++) + if (pg_prop_draw_bool(tc)) + *c = (*c >= 'a' && *c <= 'z') ? *c - 32 : + (*c >= 'A' && *c <= 'Z') ? *c + 32 : *c; + } + break; + } + + initStringInfo(&q); + appendStringInfoString(&q, "SELECT lower(split_part("); + fc_sql_lit(&q, s); + appendStringInfoString(&q, "::citext, "); + fc_sql_lit(&q, d); + appendStringInfo(&q, "::citext, %lld))", (long long) n); + initStringInfo(&m); + appendStringInfoString(&m, "SELECT split_part(lower("); + fc_sql_lit(&m, s); + appendStringInfoString(&m, "), lower("); + fc_sql_lit(&m, d); + appendStringInfo(&m, "), %lld)", (long long) n); + + conn = cfb3_connect(tc); + if (!pg_prop_require_extension(conn, "citext")) + pg_prop_skip(tc, "citext not installed"); + got = sj_outcome(tc, conn, q.data); + want = sj_outcome(tc, conn, m.data); + if (strcmp(got, want) != 0) + pg_prop_fail(tc, "citext split_part disagrees with core split_part on lowered input\n" + " %s -> %s\n %s -> %s", q.data, got, m.data, want); + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------- + * GROUPING SETS over constant-reducible expressions (CF 7438) + * ---------------------------------------------------------------- + */ + +/* grouping expressions; several fold to constants (a, id are NOT NULL) */ +static const char *const gs_exprs[] = +{"a IS NOT NULL", "id IS NOT NULL", "(b IS NULL) IS UNKNOWN", "1 + 1", +"a IS NULL", "b > 0", "a % 2", "b", "c", "upper(c)", "'k'::text", +"(b IS NOT NULL) IS NOT UNKNOWN"}; + +/* + * Type-agnostic uses of a grouping expression in the target list. The + * scalar subquery's only column is g, so the expression's columns bind to + * the outer query in both forms. + */ +static const char *const gs_uses[] = +{"(%s)", "(%s) IS NULL", "coalesce((%s)::text, '-')", +"CASE WHEN (%s) IS NULL THEN 'null' ELSE 'v' END", +"((%s)::text || 'x')", +"(SELECT count(*) FROM generate_series(0, 2) g WHERE (g = 0) = ((%s) IS NULL))"}; + +/* append template "tpl" with every %s replaced by "arg" */ +static void +gs_subst(StringInfo b, const char *tpl, const char *arg) +{ + for (const char *p = tpl; *p; p++) + { + if (p[0] == '%' && p[1] == 's') + { + appendStringInfoString(b, arg); + p++; + } + else + appendStringInfoChar(b, *p); + } +} + +static char * +gs_digest(pg_prop_tc *tc, pg_prop_conn *conn, const char *guc, const char *q) +{ + bool failed = false; + char *err = NULL; + char *d; + + cfb3_exec(tc, conn, "BEGIN"); + cfb3_exec(tc, conn, guc); + d = pg_prop_sql_rowhash(conn, q, &failed, &err); + pg_prop_sql_exec(conn, "ROLLBACK", NULL); + if (failed || d == NULL) + pg_prop_fail(tc, "query failed: %.300s\n SQL: %.600s", err ? err : "", q); + return d; +} + +/* + * grouping_sets_const_expr_as_column (CF 7438) + * + * Draw a small table (id primary key, a NOT NULL, b and c nullable), 1..3 + * grouping expressions -- several of which the planner folds to a + * constant (a IS NOT NULL, (b IS NULL) IS UNKNOWN, 1 + 1) -- a grouping + * shape (ROLLUP, CUBE, or explicit GROUPING SETS with an empty set), a + * target list using each expression in several forms (bare, under a + * cast or CASE, in a scalar subquery), GROUPING(), an aggregate, an + * optional HAVING on an expression being NULL, and hash or sort + * aggregation. + * + * Oracle: the same query over a subquery that computes each expression + * as a column (OFFSET 0 keeps it from being flattened) and groups by + * those columns. A grouping column is a grouping column whatever it is + * computed from, so the two must return the same multiset of rows. On + * master, references to a constant-folded expression are not recognised + * as the grouping column, so they are not nulled in the sets that omit + * it. + */ +PG_PROP(grouping_sets_const_expr_as_column) +{ + pg_prop_conn *conn; + int nrows = (int) pg_prop_draw_int(tc, 0, 6); + int nexpr = (int) pg_prop_draw_int(tc, 1, 3); + int ex[3]; + int shape = (int) pg_prop_draw_int(tc, 0, 2); + int nuse = (int) pg_prop_draw_int(tc, 1, 4); + int having = (int) pg_prop_draw_int(tc, -1, nexpr - 1); + const char *guc = pg_prop_draw_bool(tc) ? "SET LOCAL enable_hashagg = off" : + "SET LOCAL enable_sort = off"; + StringInfoData setup; + StringInfoData qa; + StringInfoData qb; + char *da; + char *db; + char col[8]; + + /* + * Distinct expressions: with a duplicate, query A's reference to it + * matches the first copy while B names one column, so they may + * legitimately differ. + */ + for (int i = 0; i < nexpr; i++) + { + int pick = (int) pg_prop_draw_int(tc, 0, lengthof(gs_exprs) - 1 - i); + + for (int j = 0; j < (int) lengthof(gs_exprs); j++) + { + bool used = false; + + for (int k = 0; k < i; k++) + used |= (ex[k] == j); + if (!used && pick-- == 0) + { + ex[i] = j; + break; + } + } + } + + initStringInfo(&setup); + appendStringInfoString(&setup, + "CREATE TEMP TABLE cfb3_gs (id int PRIMARY KEY, a int NOT NULL, " + "b int, c text)"); + if (nrows > 0) + { + appendStringInfoString(&setup, "; INSERT INTO cfb3_gs VALUES "); + for (int i = 0; i < nrows; i++) + { + int64 b = pg_prop_draw_int(tc, -2, 2); + int64 c = pg_prop_draw_int(tc, -1, 2); + + appendStringInfo(&setup, "%s(%d, %d, ", i ? ", " : "", i + 1, + (int) pg_prop_draw_int(tc, 0, 3)); + if (b == -2) + appendStringInfoString(&setup, "NULL, "); + else + appendStringInfo(&setup, "%d, ", (int) b); + if (c < 0) + appendStringInfoString(&setup, "NULL)"); + else + appendStringInfo(&setup, "'%c')", "xyz"[c]); + } + } + + /* + * Query A groups by the expressions; query B by columns e0..e2 of a + * subquery computing them. Build both from the same draws. + */ + initStringInfo(&qa); + initStringInfo(&qb); + appendStringInfoString(&qa, "SELECT "); + appendStringInfoString(&qb, "SELECT "); + for (int u = 0; u < nuse; u++) + { + int e = (int) pg_prop_draw_int(tc, 0, nexpr - 1); + const char *tpl = gs_uses[pg_prop_draw_int(tc, 0, lengthof(gs_uses) - 1)]; + + snprintf(col, sizeof(col), "e%d", e); + gs_subst(&qa, tpl, gs_exprs[ex[e]]); + gs_subst(&qb, tpl, col); + appendStringInfoString(&qa, ", "); + appendStringInfoString(&qb, ", "); + } + appendStringInfoString(&qa, "GROUPING("); + appendStringInfoString(&qb, "GROUPING("); + for (int i = 0; i < nexpr; i++) + { + appendStringInfo(&qa, "%s%s", i ? ", " : "", gs_exprs[ex[i]]); + appendStringInfo(&qb, "%se%d", i ? ", " : "", i); + } + appendStringInfoString(&qa, "), count(*), sum(a) FROM cfb3_gs GROUP BY "); + appendStringInfoString(&qb, "), count(*), sum(a) FROM (SELECT id, a, b, c"); + for (int i = 0; i < nexpr; i++) + appendStringInfo(&qb, ", %s AS e%d", gs_exprs[ex[i]], i); + appendStringInfoString(&qb, " FROM cfb3_gs OFFSET 0) s GROUP BY "); + + if (shape < 2) + { + appendStringInfoString(&qa, shape == 0 ? "ROLLUP(" : "CUBE("); + appendStringInfoString(&qb, shape == 0 ? "ROLLUP(" : "CUBE("); + for (int i = 0; i < nexpr; i++) + { + appendStringInfo(&qa, "%s(%s)", i ? ", " : "", gs_exprs[ex[i]]); + appendStringInfo(&qb, "%se%d", i ? ", " : "", i); + } + appendStringInfoChar(&qa, ')'); + appendStringInfoChar(&qb, ')'); + } + else + { + /* one singleton set per expression, plus the empty set */ + appendStringInfoString(&qa, "GROUPING SETS ("); + appendStringInfoString(&qb, "GROUPING SETS ("); + for (int i = 0; i < nexpr; i++) + { + appendStringInfo(&qa, "((%s)), ", gs_exprs[ex[i]]); + appendStringInfo(&qb, "(e%d), ", i); + } + appendStringInfoString(&qa, "())"); + appendStringInfoString(&qb, "())"); + } + if (having >= 0) + { + appendStringInfo(&qa, " HAVING (%s) IS NULL", gs_exprs[ex[having]]); + appendStringInfo(&qb, " HAVING e%d IS NULL", having); + } + appendStringInfoString(&qa, " ORDER BY "); + appendStringInfoString(&qb, " ORDER BY "); + for (int i = 1; i <= nuse + 3; i++) + { + appendStringInfo(&qa, "%s%d", i > 1 ? ", " : "", i); + appendStringInfo(&qb, "%s%d", i > 1 ? ", " : "", i); + } + + conn = cfb3_connect(tc); + cfb3_exec(tc, conn, setup.data); + cfb3_exec(tc, conn, "ANALYZE cfb3_gs"); + da = gs_digest(tc, conn, guc, qa.data); + db = gs_digest(tc, conn, guc, qb.data); + if (strcmp(da, db) != 0) + { + bool failed; + char *ra = pg_prop_sql_rows(conn, qa.data, &failed, NULL); + char *rb = pg_prop_sql_rows(conn, qb.data, &failed, NULL); + + pg_prop_fail(tc, "grouping by an expression differs from grouping by it as a column\n" + " %s\n rows: %.200s\n %s\n rows: %.200s", + qa.data, ra ? ra : "?", qb.data, rb ? rb : "?"); + } + pg_prop_disconnect(conn); +} + +/* ---------------------------------------------------------------- + * JSON_SERIALIZE (CF 6587) + * ---------------------------------------------------------------- + */ + +/* a random JSON document, ASCII and non-ASCII strings, nested */ +static void +js_value(pg_prop_tc *tc, StringInfo b, int depth) +{ + static const char *const strs[] = + {"", "a", "x y", "\\\"q\\\"", "\\\\", "\\n", "\\u00e9", "é", "日本", "''"}; + int n; + + switch (pg_prop_draw_int(tc, 0, depth > 0 ? 6 : 3)) + { + case 0: + appendStringInfo(b, "%lld", (long long) pg_prop_draw_int(tc, -1000, 100000)); + break; + case 1: + appendStringInfo(b, "\"%s\"", strs[pg_prop_draw_int(tc, 0, lengthof(strs) - 1)]); + break; + case 2: + appendStringInfoString(b, pg_prop_draw_bool(tc) ? "true" : "null"); + break; + case 3: + appendStringInfo(b, "%lld.%02d", (long long) pg_prop_draw_int(tc, -99, 99), + (int) pg_prop_draw_int(tc, 0, 99)); + break; + case 4: + case 5: + n = (int) pg_prop_draw_int(tc, 0, 3); + appendStringInfoChar(b, '{'); + for (int i = 0; i < n; i++) + { + appendStringInfo(b, "%s \"k%d\" : ", i ? "," : "", + (int) pg_prop_draw_int(tc, 0, 3)); + js_value(tc, b, depth - 1); + } + appendStringInfoChar(b, '}'); + break; + default: + n = (int) pg_prop_draw_int(tc, 0, 3); + appendStringInfoChar(b, '['); + for (int i = 0; i < n; i++) + { + if (i) + appendStringInfoString(b, pg_prop_draw_bool(tc) ? ", " : ","); + js_value(tc, b, depth - 1); + } + appendStringInfoChar(b, ']'); + break; + } +} + +/* + * json_serialize_is_text_cast (CF 6587) + * + * Draw a JSON document, pass it to JSON_SERIALIZE as json or jsonb (a + * literal or a column), with no RETURNING clause or RETURNING text, + * varchar, varchar(n), char(n) or bytea. + * + * Oracle: JSON_SERIALIZE renders the value as text and coerces that to + * the RETURNING type the way an assignment does (length overflow is an + * error), bytea being the text in the server encoding. So its outcome + * must equal that of storing j::text into a column of that type, or of + * convert_to(j::text, 'UTF8') for bytea. On master a jsonb argument is + * fed to the coercion as if it were json and comes out as '' or \x01. + */ +PG_PROP(json_serialize_is_text_cast) +{ + static const char *const rets[] = + {"", "text", "varchar", "varchar(%d)", "char(%d)", "bytea"}; + pg_prop_conn *conn; + StringInfoData doc; + StringInfoData lit; + StringInfoData q; + StringInfoData m; + bool jsonb = pg_prop_draw_bool(tc); + bool as_col = pg_prop_draw_bool(tc); + int r = (int) pg_prop_draw_int(tc, 0, lengthof(rets) - 1); + char typ[32]; + char *got; + char *want; + + initStringInfo(&doc); + js_value(tc, &doc, 3); + initStringInfo(&lit); + fc_sql_lit(&lit, doc.data); + appendStringInfo(&lit, "::%s", jsonb ? "jsonb" : "json"); + snprintf(typ, sizeof(typ), rets[r], (int) pg_prop_draw_int(tc, 1, 30)); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT JSON_SERIALIZE(%s%s%s)::text%s", + as_col ? "x" : lit.data, + r ? " RETURNING " : "", typ, + as_col ? psprintf(" FROM (VALUES (%s)) v(x)", lit.data) : ""); + initStringInfo(&m); + conn = cfb3_connect(tc); + if (r == 5) + appendStringInfo(&m, "SELECT convert_to((%s)::text, 'UTF8')::text", lit.data); + else + { + /* an assignment cast, as INSERT does it */ + resetStringInfo(&m); + appendStringInfo(&m, "CREATE TEMP TABLE cfb3_js (v %s)", r ? typ : "text"); + cfb3_exec(tc, conn, m.data); + resetStringInfo(&m); + appendStringInfo(&m, + "WITH i AS (INSERT INTO cfb3_js SELECT (%s)::text RETURNING v) " + "SELECT v::text FROM i", lit.data); + } + got = sj_outcome(tc, conn, q.data); + want = sj_outcome(tc, conn, m.data); + if (strcmp(got, want) != 0) + pg_prop_fail(tc, "JSON_SERIALIZE is not the text of its argument\n" + " %s -> %s\n %s -> %s", q.data, got, m.data, want); + pg_prop_disconnect(conn); +} From ae024476f066642acd96d8144fab999a3be2d59f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 17 Oct 2026 17:00:32 -0400 Subject: [PATCH 308/338] prop: index-scan-equals-seqscan properties for open index AM fixes (cf_index) Index scan, bitmap scan and KNN ordering must return what a seqscan/sort returns, over drawn data including NaN and +-Infinity coordinates, empty and edge multiranges, NULLs, and drawn opclass options and GUCs (pg_trgm.similarity_threshold down to 0): GiST/BRIN/SP-GiST geometric scans and KNN with NaN (7370 -- the KNN case crashes the backend), GiST multirange <@ (7192), GIN pg_trgm at threshold 0 (7382). Each falsifies on upstream master 45fc51f. --- src/test/prop/sql/prop_cf_index.c | 935 ++++++++++++++++++++++++++++++ 1 file changed, 935 insertions(+) create mode 100644 src/test/prop/sql/prop_cf_index.c diff --git a/src/test/prop/sql/prop_cf_index.c b/src/test/prop/sql/prop_cf_index.c new file mode 100644 index 0000000000000..618706c119071 --- /dev/null +++ b/src/test/prop/sql/prop_cf_index.c @@ -0,0 +1,935 @@ +/*------------------------------------------------------------------------- + * + * prop_cf_index.c + * An index scan must return exactly the rows a sequential scan returns. + * + * Every property here builds a small table over a drawn (type, access + * method, opclass[, options]) combination, fills it with drawn values that + * deliberately include the edge values index code forgets about (NaN and + * +/-Infinity coordinates, empty strings and multiranges, NULLs, values with + * gaps), creates the index before or after loading (build path vs insertion + * path), then runs a drawn indexable predicate under three plans: + * + * seq enable_indexscan/bitmapscan/indexonlyscan = off + * index enable_seqscan/bitmapscan = off (plain or index-only scan) + * bitmap enable_seqscan/indexscan/indexonlyscan = off + * + * and requires the same rows (ORDER BY id, digested) and the same count from + * all three. That holds on a correct server by construction: an index is a + * pure access path, so it may never change which rows satisfy a qual. A + * GUC the operator depends on (pg_trgm thresholds) is set identically in all + * three plans. + * + * Open commitfest bugs this targets (each property names its CF): + * 7370 NaN box poisons GiST/BRIN union keys; GiST KNN with NaN + * 7192 GiST multirange <@ multirange drops leaf matches with gaps + * 7382 pg_trgm GIN/GiST lose rows at similarity_threshold = 0 + * 7385 pg_trgm GiST union drops SIGNKEY from all-true keys + * 7413 ALTER COLUMN SET STORAGE on a column whose index key type differs + * 7107 cube distance functions return 0 instead of NaN + * + * Object prefix: cfb1_. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_cf_index.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#include + +#define PG_PROP_AREA "cf_index" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +/* ---- shared helpers ---------------------------------------------------- */ + +static void +cf_exec(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "statement failed: %s\n SQL: %.2000s", + err ? err : "", sql); +} + +static pg_prop_conn * +cf_connect(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + return conn; +} + +#define N_PLANS 3 +static const char *const plan_name[N_PLANS] = {"seq", "index", "bitmap"}; +static const char *const plan_gucs[N_PLANS] = { + "SET LOCAL enable_indexscan = off; SET LOCAL enable_bitmapscan = off; " + "SET LOCAL enable_indexonlyscan = off", + "SET LOCAL enable_seqscan = off; SET LOCAL enable_bitmapscan = off", + "SET LOCAL enable_seqscan = off; SET LOCAL enable_indexscan = off; " + "SET LOCAL enable_indexonlyscan = off", +}; + +/* + * Run "query" in its own transaction under plan p plus the extra GUCs, and + * return either its digest (rowhash) or, when "scalar", its single value. + * An error fails the case: the seq plan of a legal query must not error, and + * an index path must not raise an error the seq plan did not. + */ +static char * +cf_run(pg_prop_tc *tc, pg_prop_conn *conn, int p, const char *gucs, + const char *query, bool scalar) +{ + StringInfoData s; + char *out; + char *err = NULL; + bool failed = false; + + initStringInfo(&s); + appendStringInfo(&s, "BEGIN; %s; %s", plan_gucs[p], gucs ? gucs : ""); + cf_exec(tc, conn, s.data); + if (scalar) + out = pg_prop_sql_scalar(conn, query, &failed, &err); + else + out = pg_prop_sql_rowhash(conn, query, &failed, &err); + pg_prop_sql_exec(conn, "ROLLBACK", NULL); + if (failed) + pg_prop_fail(tc, "query failed under the %s plan: %s\n GUCs: %s\n query: %s", + plan_name[p], err ? err : "", gucs ? gucs : "", query); + pfree(s.data); + return out ? out : pg_strdup("(null)"); +} + +/* EXPLAIN of query under plan p, for the failure message */ +static char * +cf_explain(pg_prop_conn *conn, int p, const char *gucs, const char *query) +{ + StringInfoData s; + char *out; + bool failed = false; + + initStringInfo(&s); + appendStringInfo(&s, "BEGIN; %s; %s", plan_gucs[p], gucs ? gucs : ""); + pg_prop_sql_exec(conn, s.data, NULL); + resetStringInfo(&s); + appendStringInfo(&s, "EXPLAIN (COSTS OFF) %s", query); + out = pg_prop_sql_rows(conn, s.data, &failed, NULL); + pg_prop_sql_exec(conn, "ROLLBACK", NULL); + pfree(s.data); + return out ? out : pg_strdup("(explain failed)"); +} + +/* + * The core check. "from_where" is "FROM
WHERE "; the table + * has an int column id. The row set (ids, ordered) and the count must agree + * across the three plans. + */ +static void +cf_diff(pg_prop_tc *tc, pg_prop_conn *conn, const char *gucs, + const char *from_where, const char *setup_note, const char *dump_sql) +{ + StringInfoData rows, + cnt; + char *ref_rows, + *ref_cnt; + + initStringInfo(&rows); + initStringInfo(&cnt); + appendStringInfo(&rows, "SELECT id %s ORDER BY id", from_where); + appendStringInfo(&cnt, "SELECT count(*) %s", from_where); + + ref_rows = cf_run(tc, conn, 0, gucs, rows.data, false); + ref_cnt = cf_run(tc, conn, 0, gucs, cnt.data, true); + for (int p = 1; p < N_PLANS; p++) + { + char *r = cf_run(tc, conn, p, gucs, rows.data, false); + char *c = cf_run(tc, conn, p, gucs, cnt.data, true); + + if (strcmp(r, ref_rows) != 0 || strcmp(c, ref_cnt) != 0) + { + char *n_ref = cf_run(tc, conn, 0, gucs, cnt.data, true); + char *n_this = cf_run(tc, conn, p, gucs, + psprintf("SELECT count(*) FROM (%s) s", rows.data), + true); + char *ex = cf_explain(conn, p, gucs, rows.data); + char *dump = dump_sql ? cf_run(tc, conn, 0, NULL, dump_sql, true) : NULL; + + pg_prop_fail(tc, + "the %s plan disagrees with a sequential scan\n" + " seq: %s rows (count(*) %s); %s: %s rows (count(*) %s)\n" + " GUCs: %s\n query: %s\n setup: %s\n data: %s\n %s plan:\n%s", + plan_name[p], n_ref, ref_cnt, plan_name[p], n_this, c, + gucs ? gucs : "", rows.data, + setup_note ? setup_note : "", dump ? dump : "", + plan_name[p], ex); + } + free(r); + free(c); + } + free(ref_rows); + free(ref_cnt); + pfree(rows.data); + pfree(cnt.data); +} + +/* ---- geometric types: 7370 --------------------------------------------- */ + +/* + * A coordinate: usually a small integer on a grid (so values collide and + * boxes overlap), sometimes an IEEE edge value. nan_pct is drawn per case so + * some cases have no edge values at all and some are full of them. + */ +static void +geo_coord(pg_prop_tc *tc, StringInfo b, int grid, int edge_pct) +{ + if ((int) pg_prop_draw_int(tc, 1, 100) <= edge_pct) + { + static const char *const edge[] = {"NaN", "Infinity", "-Infinity", "NaN", "-0"}; + + appendStringInfoString(b, edge[pg_prop_draw_int(tc, 0, lengthof(edge) - 1)]); + } + else + appendStringInfo(b, "%d", (int) pg_prop_draw_int(tc, -2, grid)); +} + +static void +geo_point(pg_prop_tc *tc, StringInfo b, int grid, int edge_pct) +{ + appendStringInfoChar(b, '('); + geo_coord(tc, b, grid, edge_pct); + appendStringInfoChar(b, ','); + geo_coord(tc, b, grid, edge_pct); + appendStringInfoChar(b, ')'); +} + +/* append a quoted literal of the given geometric type */ +static void +geo_value(pg_prop_tc *tc, StringInfo b, const char *type, int grid, int edge_pct) +{ + appendStringInfo(b, "%s '", type); + if (strcmp(type, "point") == 0) + geo_point(tc, b, grid, edge_pct); + else if (strcmp(type, "box") == 0) + { + geo_point(tc, b, grid, edge_pct); + appendStringInfoChar(b, ','); + geo_point(tc, b, grid, edge_pct); + } + else if (strcmp(type, "polygon") == 0) + { + int nv = (int) pg_prop_draw_int(tc, 1, 5); + + appendStringInfoChar(b, '('); + for (int i = 0; i < nv; i++) + { + if (i > 0) + appendStringInfoChar(b, ','); + geo_point(tc, b, grid, edge_pct); + } + appendStringInfoChar(b, ')'); + } + else /* circle: radius must not be negative */ + { + appendStringInfoChar(b, '<'); + geo_point(tc, b, grid, edge_pct); + appendStringInfoChar(b, ','); + if ((int) pg_prop_draw_int(tc, 1, 100) <= edge_pct) + appendStringInfoString(b, pg_prop_draw_bool(tc) ? "NaN" : "Infinity"); + else + appendStringInfo(b, "%d", (int) pg_prop_draw_int(tc, 0, grid)); + appendStringInfoChar(b, '>'); + } + appendStringInfoChar(b, '\''); +} + +typedef struct GeoAm +{ + const char *am; + const char *opclass; + const char *type; + bool nan_ok; /* can this opclass index NaN at all? */ + bool knn; /* supports ORDER BY col <-> point */ +} GeoAm; + +static const GeoAm geo_ams[] = { + {"gist", "box_ops", "box", true, true}, + {"gist", "point_ops", "point", true, true}, + {"gist", "poly_ops", "polygon", true, true}, + {"gist", "circle_ops", "circle", true, true}, + {"brin", "box_inclusion_ops", "box", true, false}, + {"spgist", "box_ops", "box", true, true}, + {"spgist", "kd_point_ops", "point", true, true}, + /* quad_point_ops raises "getQuadrant: impossible case" on a NaN point */ + {"spgist", "quad_point_ops", "point", false, true}, + {"spgist", "poly_ops", "polygon", true, true}, +}; + +/* binary operators "col OP " per column type */ +typedef struct GeoOp +{ + const char *ltype; + const char *op; + const char *rtype; +} GeoOp; + +static const GeoOp geo_ops[] = { + {"box", "&&", "box"}, {"box", "&<", "box"}, {"box", "&<|", "box"}, + {"box", "&>", "box"}, {"box", "<<", "box"}, {"box", "<<|", "box"}, + {"box", "<@", "box"}, {"box", ">>", "box"}, {"box", "@>", "box"}, + {"box", "|&>", "box"}, {"box", "|>>", "box"}, {"box", "~=", "box"}, + {"box", "@>", "point"}, + {"point", "<<", "point"}, {"point", "<<|", "point"}, {"point", ">>", "point"}, + {"point", "|>>", "point"}, {"point", "<^", "point"}, {"point", ">^", "point"}, + {"point", "~=", "point"}, {"point", "<@", "box"}, {"point", "<@", "circle"}, + {"point", "<@", "polygon"}, + {"polygon", "&&", "polygon"}, {"polygon", "&<", "polygon"}, + {"polygon", "&<|", "polygon"}, {"polygon", "&>", "polygon"}, + {"polygon", "<<", "polygon"}, {"polygon", "<<|", "polygon"}, + {"polygon", "<@", "polygon"}, {"polygon", ">>", "polygon"}, + {"polygon", "@>", "polygon"}, {"polygon", "|&>", "polygon"}, + {"polygon", "|>>", "polygon"}, {"polygon", "~=", "polygon"}, + {"circle", "&&", "circle"}, {"circle", "&<", "circle"}, + {"circle", "&<|", "circle"}, {"circle", "&>", "circle"}, + {"circle", "<<", "circle"}, {"circle", "<<|", "circle"}, + {"circle", "<@", "circle"}, {"circle", ">>", "circle"}, + {"circle", "@>", "circle"}, {"circle", "|&>", "circle"}, + {"circle", "|>>", "circle"}, {"circle", "~=", "circle"}, +}; + +static const GeoAm * +geo_pick_am(pg_prop_tc *tc, const char *am) +{ + const GeoAm *cand[lengthof(geo_ams)]; + int n = 0; + + for (int i = 0; i < (int) lengthof(geo_ams); i++) + if (strcmp(geo_ams[i].am, am) == 0) + cand[n++] = &geo_ams[i]; + return cand[pg_prop_draw_int(tc, 0, n - 1)]; +} + +/* + * Build cfb1_geo(id, g) for the drawn opclass: some rows loaded before the + * index is built, some inserted after (both the build and the insertion paths + * compute union keys), with drawn edge values and NULLs. + */ +static void +geo_build(pg_prop_tc *tc, pg_prop_conn *conn, const GeoAm *ga, int *grid_out, + int *edge_out, StringInfo note) +{ + int grid = (int) pg_prop_draw_int(tc, 1, 40); + static const int edge_pcts[] = {0, 1, 3, 10, 30}; + int edge = ga->nan_ok ? edge_pcts[pg_prop_draw_int(tc, 0, lengthof(edge_pcts) - 1)] : 0; + int nbefore = (int) pg_prop_draw_int(tc, 0, 1500); + int nafter = (int) pg_prop_draw_int(tc, 0, 600); + int null_pct = (int) pg_prop_draw_int(tc, 0, 2) * 5; + int id = 0; + StringInfoData s; + + initStringInfo(&s); + appendStringInfo(&s, "DROP TABLE IF EXISTS cfb1_geo; " + "CREATE TABLE cfb1_geo (id int, g %s)", ga->type); + cf_exec(tc, conn, s.data); + + for (int phase = 0; phase < 2; phase++) + { + int n = phase == 0 ? nbefore : nafter; + + if (phase == 1) + { + resetStringInfo(&s); + appendStringInfo(&s, "CREATE INDEX cfb1_geo_g ON cfb1_geo USING %s (g %s)", + ga->am, ga->opclass); + cf_exec(tc, conn, s.data); + } + /* + * Mostly bulk rows generated server-side (a drawn number of copies of + * a few drawn values, plus a grid sweep), with drawn literal rows mixed + * in so edge values land in different pages / subtrees. + */ + while (n > 0) + { + int chunk = (int) Min(n, pg_prop_draw_int(tc, 1, 400)); + + resetStringInfo(&s); + if (pg_prop_draw_int(tc, 0, 3) == 0) + { + /* a grid sweep of chunk distinct small values */ + if (strcmp(ga->type, "point") == 0) + appendStringInfo(&s, "INSERT INTO cfb1_geo SELECT %d + 1 + i, point(i %% %d, i / %d) " + "FROM generate_series(0, %d) i", + id, grid + 1, grid + 1, chunk - 1); + else if (strcmp(ga->type, "box") == 0) + appendStringInfo(&s, "INSERT INTO cfb1_geo SELECT %d + 1 + i, " + "box(point(i %% %d, i / %d), point(i %% %d + 1, i / %d + 1)) " + "FROM generate_series(0, %d) i", + id, grid + 1, grid + 1, grid + 1, grid + 1, chunk - 1); + else if (strcmp(ga->type, "polygon") == 0) + appendStringInfo(&s, "INSERT INTO cfb1_geo SELECT %d + 1 + i, " + "polygon(box(point(i %% %d, i / %d), point(i %% %d + 1, i / %d + 1))) " + "FROM generate_series(0, %d) i", + id, grid + 1, grid + 1, grid + 1, grid + 1, chunk - 1); + else + appendStringInfo(&s, "INSERT INTO cfb1_geo SELECT %d + 1 + i, " + "circle(point(i %% %d, i / %d), 0.5) " + "FROM generate_series(0, %d) i", + id, grid + 1, grid + 1, chunk - 1); + id += chunk; + } + else + { + int nlit = (int) Min(chunk, pg_prop_draw_int(tc, 1, 12)); + + appendStringInfoString(&s, "INSERT INTO cfb1_geo VALUES "); + for (int i = 0; i < nlit; i++) + { + appendStringInfo(&s, "%s(%d, ", i ? ", " : "", ++id); + if ((int) pg_prop_draw_int(tc, 1, 100) <= null_pct) + appendStringInfoString(&s, "NULL"); + else + geo_value(tc, &s, ga->type, grid, edge); + appendStringInfoChar(&s, ')'); + } + /* and copies of the last literal row to fill the chunk */ + if (chunk > nlit) + appendStringInfo(&s, "; INSERT INTO cfb1_geo SELECT %d + i, g FROM cfb1_geo " + ", generate_series(1, %d) i WHERE id = %d", + id, chunk - nlit, id); + id += chunk - nlit; + } + cf_exec(tc, conn, s.data); + n -= chunk; + } + } + appendStringInfo(note, "%s %s on %s, %d rows before + %d after CREATE INDEX, grid %d, edge %d%%, null %d%%", + ga->am, ga->opclass, ga->type, nbefore, nafter, grid, edge, null_pct); + *grid_out = grid; + *edge_out = edge; + pfree(s.data); +} + +/* distinct values and their multiplicity, for failure messages */ +#define GEO_DUMP \ + "SELECT string_agg(format('%s x%s', v, n), '; ' ORDER BY n DESC, v) FROM " \ + "(SELECT g::text AS v, count(*) AS n FROM cfb1_geo GROUP BY 1 ORDER BY 2 DESC, 1 LIMIT 15) s" + +static void +geo_scan_check(pg_prop_tc *tc, const char *am) +{ + pg_prop_conn *conn = cf_connect(tc); + const GeoAm *ga = geo_pick_am(tc, am); + const GeoOp *cand[lengthof(geo_ops)]; + int ncand = 0; + int grid, + edge; + int nq; + StringInfoData note, + q; + + initStringInfo(¬e); + initStringInfo(&q); + geo_build(tc, conn, ga, &grid, &edge, ¬e); + + for (int i = 0; i < (int) lengthof(geo_ops); i++) + if (strcmp(geo_ops[i].ltype, ga->type) == 0) + cand[ncand++] = &geo_ops[i]; + + nq = (int) pg_prop_draw_int(tc, 1, 4); + for (int k = 0; k < nq; k++) + { + const GeoOp *op = cand[pg_prop_draw_int(tc, 0, ncand - 1)]; + + resetStringInfo(&q); + appendStringInfo(&q, "FROM cfb1_geo WHERE g %s ", op->op); + /* the probe may carry edge values too, even if the data does not */ + geo_value(tc, &q, op->rtype, grid + 2, + ga->nan_ok ? Max(edge, (int) pg_prop_draw_int(tc, 0, 1) * 10) : 0); + cf_diff(tc, conn, NULL, q.data, note.data, GEO_DUMP); + } + cf_exec(tc, conn, "DROP TABLE cfb1_geo"); + pfree(note.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7370 (GiST): a box/point/polygon/circle with a NaN coordinate makes + * box_ops union keys NaN, and no comparison is true against a NaN bound, so + * a whole subtree is skipped; point ~= NaN and point <@ polygon/circle with + * NaN points also disagreed with the heap operators. + */ +PG_PROP(gist_geo_scan_equals_seqscan) +{ + geo_scan_check(tc, "gist"); +} + +/* CF 7370 (BRIN): bound_box() puts NaN into the range summary. */ +PG_PROP(brin_box_scan_equals_seqscan) +{ + geo_scan_check(tc, "brin"); +} + +/* + * Not a CF entry: the same invariant for SP-GiST, which 7370 does not touch. + * Kept as a neighbour so a failure here is reported separately. + */ +PG_PROP(spgist_geo_scan_equals_seqscan) +{ + geo_scan_check(tc, "spgist"); +} + +/* + * CF 7370 (GiST KNN, patch 0004): ORDER BY col <-> point LIMIT k through the + * index must return the k smallest distances a sort returns. Compared as the + * sorted multiset of distances rounded to 1e-6, so ties (any of the tied rows + * may be returned) and last-bit differences between the index's distance and + * the operator's cannot cause a false alarm. NaN distances sort after all + * numbers and NULLs last in both. On master a NaN key fails an assertion or + * raises "inconsistent point values". + */ +PG_PROP(knn_order_equals_sort) +{ + pg_prop_conn *conn = cf_connect(tc); + const GeoAm *ga; + int grid, + edge; + int lim; + StringInfoData note, + pt, + q; + char *ref, + *got; + + do + ga = &geo_ams[pg_prop_draw_int(tc, 0, lengthof(geo_ams) - 1)]; + while (!ga->knn); + + initStringInfo(¬e); + initStringInfo(&pt); + initStringInfo(&q); + geo_build(tc, conn, ga, &grid, &edge, ¬e); + + geo_value(tc, &pt, "point", grid + 2, + ga->nan_ok ? Max(edge, (int) pg_prop_draw_int(tc, 0, 1) * 10) : 0); + lim = (int) pg_prop_draw_int(tc, 1, 2200); + appendStringInfo(&q, + "SELECT d FROM (SELECT round((g <-> %s)::numeric, 6) AS d FROM cfb1_geo " + "ORDER BY g <-> %s LIMIT %d) s ORDER BY d", + pt.data, pt.data, lim); + + ref = cf_run(tc, conn, 0, NULL, q.data, false); + got = cf_run(tc, conn, 1, NULL, q.data, false); + if (strcmp(ref, got) != 0) + { + char *ex = cf_explain(conn, 1, NULL, q.data); + + pg_prop_fail(tc, "KNN index scan returned different distances than a sort\n" + " query: %s\n setup: %s\n index plan:\n%s", + q.data, note.data, ex); + } + free(ref); + free(got); + cf_exec(tc, conn, "DROP TABLE cfb1_geo"); + pg_prop_disconnect(conn); +} + +/* ---- ranges and multiranges: 7192 -------------------------------------- */ + +/* append one int4 range literal body like [3,9) or "empty", bounds on a grid */ +static void +rng_body(pg_prop_tc *tc, StringInfo b, int grid, int *cursor) +{ + int lo, + hi; + + if (pg_prop_draw_int(tc, 0, 15) == 0) + { + appendStringInfoString(b, "empty"); + return; + } + lo = *cursor + (int) pg_prop_draw_int(tc, 0, grid); + hi = lo + (int) pg_prop_draw_int(tc, 1, grid + 1); + *cursor = hi + 1; /* the next range starts after a gap */ + appendStringInfo(b, "%c", pg_prop_draw_bool(tc) ? '[' : '('); + if (pg_prop_draw_int(tc, 0, 12) == 0) + ; /* -infinity */ + else + appendStringInfo(b, "%d", lo); + appendStringInfoChar(b, ','); + if (pg_prop_draw_int(tc, 0, 12) == 0) + ; /* +infinity */ + else + appendStringInfo(b, "%d", hi); + appendStringInfo(b, "%c", pg_prop_draw_bool(tc) ? ']' : ')'); +} + +/* a multirange with 0..4 ranges, usually with gaps between them */ +static void +mr_value(pg_prop_tc *tc, StringInfo b, int grid) +{ + int n = (int) pg_prop_draw_int(tc, 0, 4); + int cursor = (int) pg_prop_draw_int(tc, -grid, 4 * grid); + + appendStringInfoString(b, "'{"); + for (int i = 0; i < n; i++) + { + if (i > 0) + appendStringInfoChar(b, ','); + rng_body(tc, b, grid, &cursor); + } + appendStringInfoString(b, "}'::int4multirange"); +} + +static void +rng_value(pg_prop_tc *tc, StringInfo b, int grid) +{ + int cursor = (int) pg_prop_draw_int(tc, -grid, 4 * grid); + + appendStringInfoChar(b, '\''); + rng_body(tc, b, grid, &cursor); + appendStringInfoString(b, "'::int4range"); +} + +/* + * CF 7192: GiST multirange_ops stores a leaf multirange as its union range, + * and the leaf test for "mr <@ query-multirange" compared that union range + * against the query, so an indexed multirange with gaps that is contained by + * a query with matching gaps was dropped before the heap recheck. Drawn: + * gist multirange_ops or gist/spgist range_ops (neighbours), the full + * operator set against both multirange and range right-hand sides, and + * probes that are sometimes an exact copy of a stored value (the trigger). + */ +PG_PROP(gist_multirange_scan_equals_seqscan) +{ + static const char *const mr_ops[] = { + "&&", "&<", "&>", "-|-", "<<", ">>", "<@", "@>", "=" + }; + pg_prop_conn *conn = cf_connect(tc); + /* 0 gist multirange_ops (twice as likely), 1 gist range_ops, 2 spgist range_ops */ + int kind = (int) Max(0, pg_prop_draw_int(tc, -1, 2)); + const char *coltype = kind == 0 ? "int4multirange" : "int4range"; + int grid = (int) pg_prop_draw_int(tc, 1, 30); + int nrows = (int) pg_prop_draw_int(tc, 1, 1500); + int nlit = (int) pg_prop_draw_int(tc, 1, 40); + bool index_first = pg_prop_draw_bool(tc); + StringInfoData s, + note; + int nq; + + initStringInfo(&s); + initStringInfo(¬e); + appendStringInfo(&s, "DROP TABLE IF EXISTS cfb1_mr; CREATE TABLE cfb1_mr (id int, g %s)", coltype); + cf_exec(tc, conn, s.data); + if (index_first) + { + resetStringInfo(&s); + appendStringInfo(&s, "CREATE INDEX cfb1_mr_g ON cfb1_mr USING %s (g)", + kind == 2 ? "spgist" : "gist"); + cf_exec(tc, conn, s.data); + } + /* drawn literal rows (with gaps, empties, infinities, NULLs) ... */ + resetStringInfo(&s); + appendStringInfoString(&s, "INSERT INTO cfb1_mr VALUES "); + for (int i = 1; i <= nlit; i++) + { + appendStringInfo(&s, "%s(%d, ", i > 1 ? ", " : "", i); + if (pg_prop_draw_int(tc, 0, 20) == 0) + appendStringInfoString(&s, "NULL"); + else if (kind == 0) + mr_value(tc, &s, grid); + else + rng_value(tc, &s, grid); + appendStringInfoChar(&s, ')'); + } + cf_exec(tc, conn, s.data); + /* ... and server-generated filler so the tree has internal pages */ + resetStringInfo(&s); + if (kind == 0) + appendStringInfo(&s, + "INSERT INTO cfb1_mr SELECT %d + i, int4multirange(int4range(i %% 97 * %d, i %% 97 * %d + %d), " + "int4range(i %% 97 * %d + %d, i %% 97 * %d + %d)) FROM generate_series(1, %d) i", + nlit, grid, grid, 1 + grid / 3, grid, 2 + grid / 2, grid, 3 + grid, nrows); + else + appendStringInfo(&s, + "INSERT INTO cfb1_mr SELECT %d + i, int4range(i %% 97 * %d, i %% 97 * %d + %d) " + "FROM generate_series(1, %d) i", + nlit, grid, grid, 1 + grid, nrows); + cf_exec(tc, conn, s.data); + if (!index_first) + { + resetStringInfo(&s); + appendStringInfo(&s, "CREATE INDEX cfb1_mr_g ON cfb1_mr USING %s (g)", + kind == 2 ? "spgist" : "gist"); + cf_exec(tc, conn, s.data); + } + appendStringInfo(¬e, "%s on %s, %d literal + %d filler rows, grid %d, index %s load", + kind == 2 ? "spgist range_ops" : kind == 1 ? "gist range_ops" : "gist multirange_ops", + coltype, nlit, nrows, grid, index_first ? "before" : "after"); + + nq = (int) pg_prop_draw_int(tc, 1, 6); + for (int k = 0; k < nq; k++) + { + /* index -1 = "<@" again: contained-by is the operator 7192 broke */ + int opi = (int) pg_prop_draw_int(tc, -2, lengthof(mr_ops) - 1); + const char *op = opi < 0 ? "<@" : mr_ops[opi]; + int rhs = (int) pg_prop_draw_int(tc, 0, 3); + + resetStringInfo(&s); + if (rhs == 0) + { + /* an exact copy of a stored value, cast to the column type */ + appendStringInfo(&s, + "FROM cfb1_mr WHERE g %s (SELECT g FROM cfb1_mr WHERE id = %d)", + op, (int) pg_prop_draw_int(tc, 1, nlit)); + } + else if (rhs == 1 || kind != 0) + { + /* same type as the column (spgist range_ops has no mr operators) */ + appendStringInfo(&s, "FROM cfb1_mr WHERE g %s ", op); + if (kind == 0) + mr_value(tc, &s, grid); + else + rng_value(tc, &s, grid); + } + else + { + /* multirange column vs range right-hand side */ + if (strcmp(op, "=") == 0) + op = "<@"; + appendStringInfo(&s, "FROM cfb1_mr WHERE g %s ", op); + rng_value(tc, &s, grid); + } + cf_diff(tc, conn, NULL, s.data, note.data, + "SELECT string_agg(format('%s x%s', v, n), '; ' ORDER BY n DESC, v) FROM " + "(SELECT g::text AS v, count(*) AS n FROM cfb1_mr WHERE id <= 40 GROUP BY 1 " + "ORDER BY 2 DESC, 1 LIMIT 15) s"); + } + cf_exec(tc, conn, "DROP TABLE cfb1_mr"); + pfree(s.data); + pfree(note.data); + pg_prop_disconnect(conn); +} + +/* ---- pg_trgm: 7382, 7385 ----------------------------------------------- */ + +/* + * A short text from a small alphabet of word pieces, so trigrams are shared + * between rows; sometimes empty, sometimes punctuation only (no trigrams), + * sometimes a long all-digit run (sets most signature bits -> ALLISTRUE). + */ +static void +trgm_text(pg_prop_tc *tc, StringInfo b) +{ + static const char *const words[] = { + "warsaw", "szczecin", "war", "saw", "ab", "abc", "x", "zz", "test", "Warsaw", + "krakow", "lodz", "a", "q" + }; + int kind = (int) pg_prop_draw_int(tc, 0, 20); + + appendStringInfoChar(b, '\''); + if (kind == 0) + ; /* '' */ + else if (kind == 1) + appendStringInfoString(b, " ,.- "); /* no trigrams */ + else if (kind == 2) + { + /* a long digit string: many distinct trigrams */ + int n = (int) pg_prop_draw_int(tc, 10, 120); + + for (int i = 1; i <= n; i++) + appendStringInfo(b, "%s%d", i > 1 ? " " : "", i); + } + else + { + int nw = (int) pg_prop_draw_int(tc, 1, 4); + + for (int i = 0; i < nw; i++) + appendStringInfo(b, "%s%s", i ? " " : "", + words[pg_prop_draw_int(tc, 0, lengthof(words) - 1)]); + } + appendStringInfoChar(b, '\''); +} + +/* drawn threshold: the legal minimum 0 and maximum 1 are heavily weighted */ +static double +trgm_threshold(pg_prop_tc *tc) +{ + static const double fixed[] = {0.0, 0.0, 1.0, 0.3, 0.6}; + int k = (int) pg_prop_draw_int(tc, 0, lengthof(fixed)); + + if (k < (int) lengthof(fixed)) + return fixed[k]; + return (double) pg_prop_draw_int(tc, 0, 100) / 100.0; +} + +/* + * Build cfb1_trgm(id, t) with a drawn pg_trgm index. Returns the index + * description in note. Rows: drawn literals (empty, trigram-less, NULL, + * long, shared words), plus server-side filler so the index has internal + * pages, loaded before and/or after CREATE INDEX (GIN with fastupdate keeps + * post-build rows in the pending list). + */ +static void +trgm_build(pg_prop_tc *tc, pg_prop_conn *conn, StringInfo note) +{ + static const int siglens[] = {0, 1, 4, 12, 64, 2024}; + bool gin = pg_prop_draw_bool(tc); + int siglen = gin ? 0 : siglens[pg_prop_draw_int(tc, 0, lengthof(siglens) - 1)]; + bool fastupdate = pg_prop_draw_bool(tc); + int nfill = (int) pg_prop_draw_int(tc, 0, 1500); + int nlit = (int) pg_prop_draw_int(tc, 1, 30); + int split = (int) pg_prop_draw_int(tc, 0, nlit); /* literals before the index */ + bool fill_after = pg_prop_draw_bool(tc); + StringInfoData s, + idx; + + initStringInfo(&s); + initStringInfo(&idx); + if (gin) + appendStringInfo(&idx, "CREATE INDEX cfb1_trgm_i ON cfb1_trgm USING gin (t gin_trgm_ops) " + "WITH (fastupdate = %s)", fastupdate ? "on" : "off"); + else if (siglen == 0) + appendStringInfoString(&idx, "CREATE INDEX cfb1_trgm_i ON cfb1_trgm USING gist (t gist_trgm_ops)"); + else + appendStringInfo(&idx, "CREATE INDEX cfb1_trgm_i ON cfb1_trgm USING gist (t gist_trgm_ops(siglen = %d))", + siglen); + + cf_exec(tc, conn, "DROP TABLE IF EXISTS cfb1_trgm; CREATE TABLE cfb1_trgm (id int, t text)"); + for (int phase = 0; phase < 2; phase++) + { + int from = phase == 0 ? 1 : split + 1; + int to = phase == 0 ? split : nlit; + + if (phase == 1) + cf_exec(tc, conn, idx.data); + if (to >= from) + { + resetStringInfo(&s); + appendStringInfoString(&s, "INSERT INTO cfb1_trgm VALUES "); + for (int i = from; i <= to; i++) + { + appendStringInfo(&s, "%s(%d, ", i > from ? ", " : "", i); + if (pg_prop_draw_int(tc, 0, 10) == 0) + appendStringInfoString(&s, "NULL"); + else + trgm_text(tc, &s); + appendStringInfoChar(&s, ')'); + } + cf_exec(tc, conn, s.data); + } + if (nfill > 0 && (phase == 1) == fill_after) + { + resetStringInfo(&s); + appendStringInfo(&s, + "INSERT INTO cfb1_trgm SELECT 1000 + i, CASE WHEN i %% 23 = 0 THEN NULL " + "WHEN i %% 17 = 0 THEN '' ELSE 'w' || (i %% 97) || ' k' || (i %% 13) END " + "FROM generate_series(1, %d) i", nfill); + cf_exec(tc, conn, s.data); + } + } + appendStringInfo(note, "%s; %d literal rows (%d before the index), %d filler rows %s", + idx.data, nlit, split, nfill, fill_after ? "after" : "before"); + pfree(s.data); + pfree(idx.data); +} + +#define TRGM_DUMP \ + "SELECT string_agg(format('%s x%s', coalesce(quote_literal(v), 'NULL'), n), '; ' ORDER BY n DESC, v) FROM " \ + "(SELECT t AS v, count(*) AS n FROM cfb1_trgm GROUP BY 1 ORDER BY 2 DESC, 1 LIMIT 15) s" + +/* + * CF 7382: pg_trgm similarity operators (%, <%, %>, <<%, %>>) are true when + * similarity >= threshold, and 0 is a legal threshold under which every + * non-NULL row matches. GIN only visited rows sharing a trigram with the + * query, and both GIN and GiST internal pages rejected everything for a + * query with no trigrams. Drawn: GIN (fastupdate on/off) or GiST (siglen), + * all three thresholds, operator, query text (incl. '' and trigram-less). + * Also LIKE / ILIKE / regex / = for neighbouring coverage of the opclass. + */ +PG_PROP(trgm_scan_equals_seqscan) +{ + static const char *const sim_ops[] = {"%", "<%", "%>", "<<%", "%>>"}; + pg_prop_conn *conn; + StringInfoData note, + q, + gucs; + int nq; + + conn = cf_connect(tc); + if (!pg_prop_require_extension(conn, "pg_trgm")) + pg_prop_skip(tc, "pg_trgm not installed"); + + initStringInfo(¬e); + initStringInfo(&q); + initStringInfo(&gucs); + trgm_build(tc, conn, ¬e); + + nq = (int) pg_prop_draw_int(tc, 1, 5); + for (int k = 0; k < nq; k++) + { + int shape = (int) pg_prop_draw_int(tc, 0, 7); + + resetStringInfo(&gucs); + appendStringInfo(&gucs, + "SET LOCAL pg_trgm.similarity_threshold = %g; " + "SET LOCAL pg_trgm.word_similarity_threshold = %g; " + "SET LOCAL pg_trgm.strict_word_similarity_threshold = %g", + trgm_threshold(tc), trgm_threshold(tc), trgm_threshold(tc)); + resetStringInfo(&q); + appendStringInfoString(&q, "FROM cfb1_trgm WHERE "); + if (shape <= 4) + { + /* t OP q or q OP t (the commutator pairs are both indexable) */ + const char *op = sim_ops[pg_prop_draw_int(tc, 0, lengthof(sim_ops) - 1)]; + + if (strcmp(op, "%") == 0 || strcmp(op, "%>") == 0 || strcmp(op, "%>>") == 0) + { + appendStringInfo(&q, "t %s ", op); + trgm_text(tc, &q); + } + else + { + trgm_text(tc, &q); + appendStringInfo(&q, " %s t", op); + } + } + else if (shape == 5) + { + appendStringInfoString(&q, "t = "); + trgm_text(tc, &q); + } + else + { + static const char *const pat_ops[] = {"LIKE", "ILIKE", "~", "~*"}; + const char *op = pat_ops[pg_prop_draw_int(tc, 0, lengthof(pat_ops) - 1)]; + bool like = op[0] == 'L' || op[0] == 'I'; + + appendStringInfo(&q, "t %s '%s", op, like ? "%" : ""); + /* a fragment of a word; the quotes of trgm_text are stripped */ + { + StringInfoData w; + + initStringInfo(&w); + trgm_text(tc, &w); + w.data[w.len - 1] = '\0'; + appendStringInfo(&q, "%.6s%s'", w.data + 1, like ? "%" : ""); + pfree(w.data); + } + } + cf_diff(tc, conn, gucs.data, q.data, note.data, TRGM_DUMP); + } + cf_exec(tc, conn, "DROP TABLE cfb1_trgm"); + pfree(note.data); + pfree(q.data); + pfree(gucs.data); + pg_prop_disconnect(conn); +} From 866c90f6f14a86730c76d6a338af7adfa966e09c Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 17 Oct 2026 17:00:32 -0400 Subject: [PATCH 309/338] prop: no-internal-error and model properties for open executor fixes (cf_exec) 7295 excluded_virtual_generated_matches_inline_expression EXCLUDED. in ON CONFLICT DO UPDATE / DO SELECT must equal the inlined expression; master raises XX000 "variable not found in subplan target lists" when the expression uses tableoid. 6825 self_ref_cascade_with_replanning_trigger_matches_model ON DELETE CASCADE on a drawn self-referencing forest, with a BEFORE DELETE trigger that invalidates the RI plan (GRANT/REVOKE, ANALYZE, COMMENT, SET STATISTICS) and deletes its own children: the rows left must equal the cascade model. Master crashes the backend on 6/6 seeds; with the CF 6825 patch applied it passes 6/6. --- src/test/prop/sql/prop_cf_exec.c | 748 +++++++++++++++++++++++++++++++ 1 file changed, 748 insertions(+) create mode 100644 src/test/prop/sql/prop_cf_exec.c diff --git a/src/test/prop/sql/prop_cf_exec.c b/src/test/prop/sql/prop_cf_exec.c new file mode 100644 index 0000000000000..7695f514ae3f6 --- /dev/null +++ b/src/test/prop/sql/prop_cf_exec.c @@ -0,0 +1,748 @@ +/*------------------------------------------------------------------------- + * + * prop_cf_exec.c + * No-internal-error and model properties around open commitfest bug fixes + * in the executor, PL/pgSQL and the plan cache. + * + * Each property draws a small schema and a statement sequence around the + * trigger of one CF bug fix. The common oracle is "a legal statement must + * not raise an internal error": SQLSTATE class XX (elog(ERROR) without an + * errcode), "cache lookup failed", or a backend crash (the harness then saves + * the statement and exits 3). Where the statement can also be given a + * reference meaning, its result is compared with that reference. + * + * Objects are temp tables, or carry the cfa2_ prefix and are dropped by the + * case that made them. + * + * Copyright (c) 2026, PostgreSQL Global Development Group + * + * IDENTIFICATION + * src/test/prop/sql/prop_cf_exec.c + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "cf_exec" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +static pg_prop_conn * +cfe_connect(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + return conn; +} + +static void +cfe_exec(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "setup failed: %s\n SQL: %s", err ? err : "", sql); +} + +/* + * Run "sql" and report success; on error return the SQLSTATE and message. + * pg_prop_sql_await does not run the crash tripwire, so after an error probe + * the connection with a statement that carries "sql" in a dollar-quoted + * literal: if the backend died, the tripwire logs it (and so the statement) + * and exits 3. + */ +static bool +cfe_try(pg_prop_conn *conn, const char *sql, char **sqlstate, char **errmsg) +{ + bool ok; + StringInfoData probe; + + *sqlstate = NULL; + *errmsg = NULL; + if (!pg_prop_sql_send(conn, sql)) + ok = false; + else + ok = pg_prop_sql_await(conn, sqlstate, errmsg); + if (!ok) + { + initStringInfo(&probe); + appendStringInfo(&probe, "SELECT $cfe_probe$ liveness probe after:\n%s\n$cfe_probe$", sql); + pg_prop_sql_exec(conn, probe.data, NULL); + pfree(probe.data); + } + return ok; +} + +/* is this error one a legal statement must never raise? */ +static bool +cfe_internal(const char *sqlstate, const char *errmsg) +{ + if (sqlstate == NULL) + return true; /* no SQLSTATE at all: connection-level */ + if (strncmp(sqlstate, "XX", 2) == 0) + return true; + if (errmsg && strstr(errmsg, "cache lookup failed") != NULL) + return true; + return false; +} + +/* run "sql"; fail the case on an internal error; return success */ +static bool +cfe_try_noint(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql, + const char *ctx) +{ + char *ss; + char *err; + + if (cfe_try(conn, sql, &ss, &err)) + return true; + if (cfe_internal(ss, err)) + pg_prop_fail(tc, "internal error (SQLSTATE %s): %s\n statement: %s\n" + " context:\n%s", + ss ? ss : "none", err ? err : "", sql, ctx ? ctx : ""); + return false; +} + +static char * +cfe_rows(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool failed = false; + char *err = NULL; + char *r = pg_prop_sql_rows(conn, sql, &failed, &err); + + if (failed) + pg_prop_fail(tc, "query failed: %s\n SQL: %s", err ? err : "", sql); + return r; +} + +/* ---------------------------------------------------------------------- + * CF 7295: EXCLUDED. in ON CONFLICT. + * + * A virtual generated column g is, by definition, its generation expression + * evaluated over the row it is read from. So any INSERT ... ON CONFLICT + * statement on a table with virtual columns must behave exactly like the same + * statement on a plain table with the generation expressions written inline: + * EXCLUDED.g -> E(EXCLUDED.b, EXCLUDED.c), t.g -> E(t.b, t.c), RETURNING g / + * old.g / new.g likewise. Both the RETURNING rows and the table contents + * afterwards must agree, and both statements must succeed or fail together. + * + * The test table is drawn: column order permuted, a dropped column, one or two + * virtual columns, partitioned (range / hash / list) or not, with partitions + * created by PARTITION OF or built separately (own column order and a dropped + * column) and attached -- the attribute maps the EXCLUDED projection must + * survive. Generation expressions include a bare Var and expressions using + * tableoid that do not depend on its value (for the reference those are + * written tableoid-free). + * + * EXCLUDED has no system columns, so EXCLUDED.g with g using tableoid may be + * rejected; a correct server raises a user error (master: XX000 "variable not + * found in subplan target lists", CF 7295). That case is the one where only + * "no internal error" is checked. + * ---------------------------------------------------------------------- + */ + +/* '$' marks a column qualifier position; tf = tableoid-free equivalent */ +typedef struct CfeGen +{ + const char *expr; + const char *tf; +} CfeGen; + +static const CfeGen cfe_gens[] = { + {"$b * 2", NULL}, + {"$b + $c", NULL}, + {"coalesce($b, $c)", NULL}, + {"$c", NULL}, /* bare Var: matched without non-Var search */ + {"CASE WHEN $b > $c THEN $b ELSE $c END", NULL}, + {"nullif($b, $c)", NULL}, + {"$b - 3 * $c", NULL}, + {"$c + ($tableoid <> 0)::int", "$c + 1"}, + {"$b + $tableoid::int * 0", "$b"}, + {"CASE WHEN $tableoid IS NULL THEN $c ELSE $b END", "$b"}, +}; + +#define CFE_KMAX 8 + +/* append template with '$' -> "qual." (or nothing for an empty qual) */ +static void +cfe_expand(StringInfo b, const char *tmpl, const char *qual) +{ + appendStringInfoChar(b, '('); + for (const char *p = tmpl; *p; p++) + { + if (*p == '$') + { + if (qual[0] != '\0') + appendStringInfo(b, "%s.", qual); + } + else + appendStringInfoChar(b, *p); + } + appendStringInfoChar(b, ')'); +} + +typedef struct CfeVg +{ + int ngen; /* 1 or 2 virtual columns g1, g2 */ + const CfeGen *gen[2]; + bool excl_tableoid; /* this statement read EXCLUDED.g with tableoid */ +} CfeVg; + +/* reference to virtual column i via qual: the column (test) or E inline (ref) */ +static void +cfe_vg(StringInfo b, CfeVg *v, int i, bool ref, const char *qual) +{ + const CfeGen *g = v->gen[i]; + + if (!ref) + { + if (qual[0] != '\0') + appendStringInfo(b, "%s.", qual); + appendStringInfo(b, "g%d", i + 1); + if (strcmp(qual, "excluded") == 0 && g->tf != NULL) + v->excl_tableoid = true; + } + else + cfe_expand(b, g->tf ? g->tf : g->expr, qual); +} + +/* an int-valued expression over EXCLUDED and the target row t */ +static void +cfe_upsert_expr(StringInfo b, CfeVg *v, bool ref, int kind, int gi) +{ + switch (kind) + { + case 0: + cfe_vg(b, v, gi, ref, "excluded"); + break; + case 1: + cfe_vg(b, v, gi, ref, "t"); + break; + case 2: + appendStringInfoString(b, "excluded.b"); + break; + case 3: + appendStringInfoString(b, "t.b + "); + cfe_vg(b, v, gi, ref, "excluded"); + break; + case 4: + appendStringInfoString(b, "coalesce("); + cfe_vg(b, v, gi, ref, "excluded"); + appendStringInfoString(b, ", -1)"); + break; + case 5: + appendStringInfoString(b, "(SELECT "); + cfe_vg(b, v, gi, ref, "excluded"); + appendStringInfoString(b, ")"); + break; + default: + appendStringInfoString(b, "CASE WHEN "); + cfe_vg(b, v, gi, ref, "excluded"); + appendStringInfoString(b, " > "); + cfe_vg(b, v, gi, ref, "t"); + appendStringInfoString(b, " THEN "); + cfe_vg(b, v, gi, ref, "excluded"); + appendStringInfoString(b, " ELSE t.c END"); + break; + } +} + +/* a boolean condition over EXCLUDED and t */ +static void +cfe_upsert_cond(StringInfo b, CfeVg *v, bool ref, int kind, int gi) +{ + switch (kind) + { + case 0: + cfe_vg(b, v, gi, ref, "excluded"); + appendStringInfoString(b, " IS NOT NULL"); + break; + case 1: + cfe_vg(b, v, gi, ref, "excluded"); + appendStringInfoString(b, " > "); + cfe_vg(b, v, gi, ref, "t"); + break; + case 2: + cfe_vg(b, v, gi, ref, "t"); + appendStringInfoString(b, " IS DISTINCT FROM "); + cfe_vg(b, v, gi, ref, "excluded"); + break; + default: + appendStringInfoString(b, "excluded.b >= 0"); + break; + } +} + +/* drawn choices of one upsert statement, rendered twice (test, ref) */ +typedef struct CfeStmt +{ + int nrows; + int key[4]; + int bv[4]; + int cv[4]; /* PG_INT32_MIN = NULL */ + int colorder; /* permutation of (a, b, c) in the column list */ + int action; /* 0 DO UPDATE, 1 DO SELECT, 2 DO SELECT FOR UPDATE */ + int nset; /* DO UPDATE: 1 = b, 2 = b and c, 3 = (b, c) = ROW */ + int ekind[2]; + int egi[2]; + bool has_where; + int ckind; + int cgi; + bool returning; + int rgi; +} CfeStmt; + +static const char *const cfe_perm3[6][3] = { + {"a", "b", "c"}, {"a", "c", "b"}, {"b", "a", "c"}, + {"b", "c", "a"}, {"c", "a", "b"}, {"c", "b", "a"}, +}; + +static void +cfe_val(StringInfo b, int v) +{ + if (v == PG_INT32_MIN) + appendStringInfoString(b, "NULL"); + else + appendStringInfo(b, "%d", v); +} + +static void +cfe_render(StringInfo b, const CfeStmt *s, CfeVg *v, bool ref) +{ + const char *tbl = ref ? "cfa2_r" : "cfa2_v"; + + if (s->returning) + appendStringInfo(b, "TRUNCATE cfa2_ret_%s; WITH r AS (", ref ? "r" : "v"); + appendStringInfo(b, "INSERT INTO %s AS t (%s, %s, %s) VALUES ", tbl, + cfe_perm3[s->colorder][0], cfe_perm3[s->colorder][1], + cfe_perm3[s->colorder][2]); + for (int i = 0; i < s->nrows; i++) + { + appendStringInfoString(b, i ? ", (" : "("); + for (int j = 0; j < 3; j++) + { + char col = cfe_perm3[s->colorder][j][0]; + + if (j) + appendStringInfoString(b, ", "); + cfe_val(b, col == 'a' ? s->key[i] : col == 'b' ? s->bv[i] : s->cv[i]); + } + appendStringInfoChar(b, ')'); + } + appendStringInfoString(b, " ON CONFLICT (a) "); + if (s->action == 0) + { + appendStringInfoString(b, "DO UPDATE SET "); + if (s->nset == 3) + { + appendStringInfoString(b, "(b, c) = ROW("); + cfe_upsert_expr(b, v, ref, s->ekind[0], s->egi[0]); + appendStringInfoString(b, ", "); + cfe_upsert_expr(b, v, ref, s->ekind[1], s->egi[1]); + appendStringInfoChar(b, ')'); + } + else + { + appendStringInfoString(b, "b = "); + cfe_upsert_expr(b, v, ref, s->ekind[0], s->egi[0]); + if (s->nset == 2) + { + appendStringInfoString(b, ", c = "); + cfe_upsert_expr(b, v, ref, s->ekind[1], s->egi[1]); + } + } + } + else + appendStringInfo(b, "DO SELECT%s", s->action == 2 ? " FOR UPDATE" : ""); + if (s->has_where) + { + appendStringInfoString(b, " WHERE "); + cfe_upsert_cond(b, v, ref, s->ckind, s->cgi); + } + if (s->returning) + { + appendStringInfoString(b, " RETURNING a, b, c, "); + cfe_vg(b, v, s->rgi, ref, ""); + appendStringInfoString(b, ", "); + cfe_vg(b, v, s->rgi, ref, "old"); + appendStringInfoString(b, ", "); + cfe_vg(b, v, s->rgi, ref, "new"); + appendStringInfo(b, ") INSERT INTO cfa2_ret_%s SELECT * FROM r", + ref ? "r" : "v"); + } +} + +static int +cfe_draw_val(pg_prop_tc *tc) +{ + return pg_prop_draw_int(tc, 0, 6) == 0 ? PG_INT32_MIN : (int) pg_prop_draw_int(tc, -3, 9); +} + +/* column definitions of one (sub)table in a drawn order, with a dropped "x" */ +static void +cfe_coldefs(pg_prop_tc *tc, StringInfo b, CfeVg *v, bool *has_x) +{ + const char *cols[6]; + int n = 0; + char gdef[2][128]; + + cols[n++] = "a int NOT NULL"; + cols[n++] = "b int"; + cols[n++] = "c int"; + for (int i = 0; i < v->ngen; i++) + { + StringInfoData e; + + initStringInfo(&e); + cfe_expand(&e, v->gen[i]->expr, ""); + snprintf(gdef[i], sizeof(gdef[i]), + "g%d int GENERATED ALWAYS AS %s VIRTUAL", i + 1, e.data); + pfree(e.data); + cols[n++] = gdef[i]; + } + *has_x = pg_prop_draw_bool(tc); + if (*has_x) + cols[n++] = "x int"; + /* Fisher-Yates over the drawn order */ + for (int i = n - 1; i > 0; i--) + { + int j = (int) pg_prop_draw_int(tc, 0, i); + const char *t = cols[i]; + + cols[i] = cols[j]; + cols[j] = t; + } + for (int i = 0; i < n; i++) + appendStringInfo(b, "%s%s", i ? ", " : "", cols[i]); +} + +PG_PROP(excluded_virtual_generated_matches_inline_expression) +{ + pg_prop_conn *conn = cfe_connect(tc); + CfeVg v; + StringInfoData s; + StringInfoData log; + bool has_x; + int pkind; + int nparts = 0; + int nstmts; + char *want; + char *got; + + memset(&v, 0, sizeof(v)); + v.ngen = (int) pg_prop_draw_int(tc, 1, 2); + for (int i = 0; i < v.ngen; i++) + v.gen[i] = &cfe_gens[pg_prop_draw_int(tc, 0, lengthof(cfe_gens) - 1)]; + + initStringInfo(&s); + initStringInfo(&log); + + /* the test table: drawn layout, maybe partitioned */ + pkind = (int) pg_prop_draw_int(tc, 0, 3); + appendStringInfoString(&s, "CREATE TEMP TABLE cfa2_v ("); + cfe_coldefs(tc, &s, &v, &has_x); + appendStringInfoString(&s, ", PRIMARY KEY (a))"); + if (pkind == 1) + appendStringInfoString(&s, " PARTITION BY RANGE (a)"); + else if (pkind == 2) + appendStringInfoString(&s, " PARTITION BY HASH (a)"); + else if (pkind == 3) + appendStringInfoString(&s, " PARTITION BY LIST (a)"); + appendStringInfoString(&s, "; "); + if (has_x) + appendStringInfoString(&s, "ALTER TABLE cfa2_v DROP COLUMN x; "); + if (pkind != 0) + { + int split = (int) pg_prop_draw_int(tc, 2, CFE_KMAX - 1); + + nparts = (int) pg_prop_draw_int(tc, 1, 3); + if (pkind == 1) + nparts = nparts < 2 ? 2 : nparts; + for (int i = 0; i < nparts; i++) + { + StringInfoData bound; + bool separate = pg_prop_draw_bool(tc); + + initStringInfo(&bound); + if (pkind == 1) + { + /* (MINVALUE, split), [split, split+1), ..., [.., MAXVALUE) */ + if (i == 0) + appendStringInfo(&bound, "FROM (MINVALUE) TO (%d)", split); + else if (i == nparts - 1) + appendStringInfo(&bound, "FROM (%d) TO (MAXVALUE)", split + i - 1); + else + appendStringInfo(&bound, "FROM (%d) TO (%d)", split + i - 1, split + i); + } + else if (pkind == 2) + appendStringInfo(&bound, "WITH (MODULUS %d, REMAINDER %d)", nparts, i); + else if (i == nparts - 1) + appendStringInfoString(&bound, "DEFAULT"); + else + appendStringInfo(&bound, "IN (%d, %d)", i + 1, i + 1 + nparts); + + if (!separate) + appendStringInfo(&s, + "CREATE TEMP TABLE cfa2_v%d PARTITION OF cfa2_v %s%s; ", + i, strcmp(bound.data, "DEFAULT") == 0 ? "" : "FOR VALUES ", + bound.data); + else + { + bool cx; + + appendStringInfo(&s, "CREATE TEMP TABLE cfa2_v%d (", i); + cfe_coldefs(tc, &s, &v, &cx); + appendStringInfoString(&s, "); "); + if (cx) + appendStringInfo(&s, "ALTER TABLE cfa2_v%d DROP COLUMN x; ", i); + appendStringInfo(&s, "ALTER TABLE cfa2_v ATTACH PARTITION cfa2_v%d %s%s; ", + i, strcmp(bound.data, "DEFAULT") == 0 ? "" : "FOR VALUES ", + bound.data); + } + pfree(bound.data); + } + } + /* the reference: a plain table, generated columns computed inline */ + appendStringInfoString(&s, + "CREATE TEMP TABLE cfa2_r (a int PRIMARY KEY, b int, c int); " + "CREATE TEMP TABLE cfa2_ret_v (a int, b int, c int, g int, og int, ng int); " + "CREATE TEMP TABLE cfa2_ret_r (a int, b int, c int, g int, og int, ng int); "); + cfe_exec(tc, conn, s.data); + appendStringInfoString(&log, s.data); + + /* initial rows, copied into the reference */ + resetStringInfo(&s); + for (int k = 1; k <= CFE_KMAX; k++) + { + if (!pg_prop_draw_bool(tc)) + continue; + appendStringInfo(&s, "INSERT INTO cfa2_v (a, b, c) VALUES (%d, ", k); + cfe_val(&s, cfe_draw_val(tc)); + appendStringInfoString(&s, ", "); + cfe_val(&s, cfe_draw_val(tc)); + appendStringInfoString(&s, "); "); + } + appendStringInfoString(&s, "INSERT INTO cfa2_r SELECT a, b, c FROM cfa2_v"); + cfe_exec(tc, conn, s.data); + appendStringInfo(&log, "%s;\n", s.data); + + nstmts = (int) pg_prop_draw_int(tc, 1, 4); + for (int n = 0; n < nstmts; n++) + { + CfeStmt st; + StringInfoData tq; + StringInfoData rq; + char *ss; + char *err; + bool tok; + bool rok; + + memset(&st, 0, sizeof(st)); + st.nrows = (int) pg_prop_draw_int(tc, 1, 4); + for (int i = 0; i < st.nrows; i++) + { + st.key[i] = (int) pg_prop_draw_int(tc, 1, CFE_KMAX); + st.bv[i] = cfe_draw_val(tc); + st.cv[i] = cfe_draw_val(tc); + } + st.colorder = (int) pg_prop_draw_int(tc, 0, 5); + st.action = (int) pg_prop_draw_int(tc, 0, 3); + if (st.action == 3) + st.action = 0; /* DO UPDATE twice as likely */ + st.nset = (int) pg_prop_draw_int(tc, 1, 3); + for (int i = 0; i < 2; i++) + { + st.ekind[i] = (int) pg_prop_draw_int(tc, 0, 6); + st.egi[i] = (int) pg_prop_draw_int(tc, 0, v.ngen - 1); + } + st.has_where = pg_prop_draw_bool(tc); + st.ckind = (int) pg_prop_draw_int(tc, 0, 3); + st.cgi = (int) pg_prop_draw_int(tc, 0, v.ngen - 1); + st.returning = st.action != 0 || pg_prop_draw_bool(tc); + st.rgi = (int) pg_prop_draw_int(tc, 0, v.ngen - 1); + + initStringInfo(&tq); + initStringInfo(&rq); + v.excl_tableoid = false; + cfe_render(&tq, &st, &v, false); + cfe_render(&rq, &st, &v, true); + appendStringInfo(&log, "%s;\n", tq.data); + + tok = cfe_try(conn, tq.data, &ss, &err); + if (!tok && cfe_internal(ss, err)) + pg_prop_fail(tc, "internal error (SQLSTATE %s): %s\n statement: %s\n" + " reference: %s\n history:\n%s", + ss ? ss : "none", err ? err : "", tq.data, rq.data, log.data); + if (!tok && v.excl_tableoid) + continue; /* EXCLUDED has no tableoid: a user error is right */ + rok = cfe_try_noint(tc, conn, rq.data, log.data); + if (tok != rok) + pg_prop_fail(tc, "virtual-column statement %s but its inline reference %s\n" + " statement: %s\n reference: %s\n error: %s\n history:\n%s", + tok ? "succeeded" : "failed", rok ? "succeeded" : "failed", + tq.data, rq.data, err ? err : "", log.data); + if (!tok) + continue; + + if (st.returning) + { + want = cfe_rows(tc, conn, "SELECT * FROM cfa2_ret_r ORDER BY 1, 2, 3, 4, 5, 6"); + got = cfe_rows(tc, conn, "SELECT * FROM cfa2_ret_v ORDER BY 1, 2, 3, 4, 5, 6"); + if (strcmp(want, got) != 0) + pg_prop_fail(tc, "RETURNING differs from the inline-expression reference\n" + " statement: %s\n reference: %s\n got:\n%s want:\n%s" + " history:\n%s", tq.data, rq.data, got, want, log.data); + pg_free(want); + pg_free(got); + } + + resetStringInfo(&s); + appendStringInfoString(&s, "SELECT a, b, c"); + for (int i = 0; i < v.ngen; i++) + { + appendStringInfoString(&s, ", "); + cfe_vg(&s, &v, i, true, ""); + } + appendStringInfoString(&s, " FROM cfa2_r ORDER BY a"); + want = cfe_rows(tc, conn, s.data); + got = cfe_rows(tc, conn, v.ngen == 1 ? + "SELECT a, b, c, g1 FROM cfa2_v ORDER BY a" : + "SELECT a, b, c, g1, g2 FROM cfa2_v ORDER BY a"); + if (strcmp(want, got) != 0) + pg_prop_fail(tc, "table contents differ from the inline-expression reference\n" + " after: %s\n reference: %s\n got:\n%s want:\n%s" + " history:\n%s", tq.data, rq.data, got, want, log.data); + pg_free(want); + pg_free(got); + pfree(tq.data); + pfree(rq.data); + } + + pfree(s.data); + pfree(log.data); + pg_prop_disconnect(conn); +} + +/* + * CF 6825: an ON DELETE CASCADE on a self-referencing table, fired while a + * BEFORE DELETE trigger on the same table invalidates the cascade's cached RI + * plan (any DDL that bumps the relcache -- GRANT/REVOKE, ANALYZE, ...) and + * then deletes from the table itself, re-entering the same RI query. The + * outer execution of the replaced plan must still finish. On master it + * crashes the backend. + * + * Invariant (a model, not just "no crash"): deleting a row removes exactly + * its descendants in the parent_id forest, whatever the trigger does in + * between, because the trigger's own DELETE removes a subset of those same + * descendants. The forest, the deleted root, the rows whose trigger + * invalidates, the invalidating statement and the trigger's timing (whether + * it deletes its own children before the cascade reaches them) are drawn. + */ +PG_PROP(self_ref_cascade_with_replanning_trigger_matches_model) +{ + static const char *const inval[] = { + "GRANT SELECT ON cfa2_sr TO PUBLIC; REVOKE SELECT ON cfa2_sr FROM PUBLIC;", + "ANALYZE cfa2_sr;", + "COMMENT ON TABLE cfa2_sr IS 'x';", + "ALTER TABLE cfa2_sr ALTER COLUMN parent_id SET STATISTICS 50;", + }; + pg_prop_conn *conn = cfe_connect(tc); + int n = (int) pg_prop_draw_int(tc, 2, 24); + int parent[25]; + bool gone[25]; + bool fires[25]; + int root; + const char *iv = inval[pg_prop_draw_int(tc, 0, lengthof(inval) - 1)]; + StringInfoData s; + StringInfoData model; + char *got; + bool failed = false; + char *err = NULL; + bool changed; + + /* a forest: each row's parent is an earlier row, or none */ + for (int i = 1; i <= n; i++) + { + parent[i] = (i > 1 && pg_prop_draw_int(tc, 0, 3) != 0) + ? (int) pg_prop_draw_int(tc, 1, i - 1) : 0; + fires[i] = pg_prop_draw_int(tc, 0, 2) == 0; + gone[i] = false; + } + root = (int) pg_prop_draw_int(tc, 1, n); + + initStringInfo(&s); + appendStringInfo(&s, + "DROP TABLE IF EXISTS cfa2_sr; " + "CREATE TABLE cfa2_sr (id int PRIMARY KEY, " + "parent_id int REFERENCES cfa2_sr (id) ON DELETE CASCADE); " + "CREATE OR REPLACE FUNCTION cfa2_sr_bd() RETURNS trigger LANGUAGE plpgsql AS $f$ " + "BEGIN IF OLD.id = ANY (ARRAY["); + { + bool first = true; + + for (int i = 1; i <= n; i++) + if (fires[i]) + { + appendStringInfo(&s, "%s%d", first ? "" : ",", i); + first = false; + } + if (first) + appendStringInfoString(&s, "0"); + } + appendStringInfo(&s, + "]) THEN %s DELETE FROM cfa2_sr WHERE parent_id = OLD.id; END IF; " + "RETURN OLD; END $f$; " + "CREATE TRIGGER cfa2_sr_bd BEFORE DELETE ON cfa2_sr " + "FOR EACH ROW EXECUTE FUNCTION cfa2_sr_bd(); " + "INSERT INTO cfa2_sr VALUES ", iv); + for (int i = 1; i <= n; i++) + { + appendStringInfo(&s, "%s(%d, ", i > 1 ? ", " : "", i); + if (parent[i]) + appendStringInfo(&s, "%d)", parent[i]); + else + appendStringInfoString(&s, "NULL)"); + } + cfe_exec(tc, conn, s.data); + + /* the statement under test; a crash makes the harness exit 3 */ + resetStringInfo(&s); + appendStringInfo(&s, "DELETE FROM cfa2_sr WHERE id = %d", root); + + /* + * A user error (some invalidating statements are rejected inside a + * trigger on a table in use, e.g. ALTER TABLE) rolls the DELETE back, so + * the model is then "nothing deleted". An internal error fails the case. + */ + if (cfe_try_noint(tc, conn, s.data, iv)) + gone[root] = true; + do + { + changed = false; + for (int i = 1; i <= n; i++) + if (gone[root] && !gone[i] && parent[i] && gone[parent[i]]) + gone[i] = changed = true; + } while (changed); + initStringInfo(&model); + for (int i = 1; i <= n; i++) + if (!gone[i]) + appendStringInfo(&model, "%d\n", i); + + got = pg_prop_sql_rows(conn, "SELECT id FROM cfa2_sr ORDER BY id", &failed, &err); + if (failed) + pg_prop_fail(tc, "read-back failed: %s", err ? err : ""); + if (strcmp(got, model.data) != 0) + pg_prop_fail(tc, "after DELETE id=%d with a trigger running \"%s\", " + "rows left differ from the cascade model\n model:\n%s table:\n%s", + root, iv, model.data, got); + + pg_free(got); + cfe_exec(tc, conn, "DROP TABLE cfa2_sr; DROP FUNCTION cfa2_sr_bd()"); + pfree(s.data); + pfree(model.data); + pg_prop_disconnect(conn); +} From 421ab51902b65694c3d86bb9f1d5473c7c3c2b3a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 17 Oct 2026 19:18:11 -0400 Subject: [PATCH 310/338] prop: give FULL JOIN only joinable conditions in the JsonExpr pull-up property FULL JOIN needs a hash- or merge-joinable condition, and the server rightly rejects "a.k = ss.y AND a.k > 1" or IS NOT DISTINCT FROM there. Whether the error appears depends on the plan, so on a server carrying the CF 7186 fix the property failed on the error rather than on rows. Use the plain equality for FULL JOIN. With that, the property passes 2 of 3 seeds against master + CF 7186 and still fails the third on a RIGHT JOIN / DEFAULT ON ERROR / PASSING case that the patch does not cover (wrong on unpatched master too). --- src/test/prop/sql/prop_cf_planner.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/test/prop/sql/prop_cf_planner.c b/src/test/prop/sql/prop_cf_planner.c index 80519d85098b5..3b4fd5879875c 100644 --- a/src/test/prop/sql/prop_cf_planner.c +++ b/src/test/prop/sql/prop_cf_planner.c @@ -554,6 +554,15 @@ PG_PROP(jsonexpr_pullup_matches_offset0_fence) if (wrap) appendStringInfoString(&e, " || '>'"); + /* + * FULL JOIN needs a hash- or merge-joinable condition; "a.k > 1" and IS + * NOT DISTINCT FROM are not, and the server rightly rejects them. (The + * outcome is plan-dependent: the patched planner keeps a qual that the + * buggy one dropped, so only the patched server raised the error.) + */ + if (strcmp(jn, "FULL JOIN") == 0 && strcmp(cond, "false") != 0) + cond = "a.k = ss.y"; + initStringInfo(&qa); initStringInfo(&qb); for (int v = 0; v < 2; v++) From 64c5d446fbde9eb9bf0c656882c005fd7c23a793 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sun, 18 Oct 2026 06:47:38 -0400 Subject: [PATCH 311/338] prop: add pg_trgm GiST union and SET STORAGE properties (CF 7385, 7413) trgm_gist_union_scan_equals_seqscan (CF 7385, BUG #19720): gtrgm_union() sets flag = ALLISTRUE instead of OR-ing it in, dropping SIGNKEY, so unionkey() reads the key as an empty array and an internal downlink loses its children's bits. The property loads drawn NULL and all-true rows into a gist_trgm_ops(siglen=2..8) index, then one filler row at a time until the root's first child is an internal page, then rows of '' (no trigram bits), and requires index scan = seqscan. The stop condition must be checked after every insert: stopping even a few rows past the split lets the next ordinary insert repair the downlink (every-50th check: 2/2 passes on master; every-row check: 2/2 failures). Master: 3/3 seeds, the index finds 0 of ~33,000 rows; master + CF 7385: 3/3 pass. set_storage_keeps_index_usable (CF 7413): ALTER COLUMN SET STORAGE copies attstorage onto index columns whose type is the opclass key type (gtrgm, typstorage 'p'), and the next INSERT or REINDEX crashes in pg_trgm. Drawn opclass (gist_trgm_ops with drawn siglen, gin_trgm_ops, core tsvector GiST as a control), storage mode, row count and follow-up statement; the table must accept the follow-up and the index must still equal a seqscan. Master: the backend segfaults on 3/3 seeds (INSERT after SET STORAGE). No fix patch was available to validate against. --- src/test/prop/sql/prop_cf_index.c | 186 ++++++++++++++++++++++++++++++ 1 file changed, 186 insertions(+) diff --git a/src/test/prop/sql/prop_cf_index.c b/src/test/prop/sql/prop_cf_index.c index 618706c119071..953f51bd0a7df 100644 --- a/src/test/prop/sql/prop_cf_index.c +++ b/src/test/prop/sql/prop_cf_index.c @@ -933,3 +933,189 @@ PG_PROP(trgm_scan_equals_seqscan) pfree(gucs.data); pg_prop_disconnect(conn); } + +/* + * CF 7385 (BUG #19720): gtrgm_union() set flag = ALLISTRUE instead of OR-ing + * it in, dropping SIGNKEY; unionkey() then reads the key as a zero-length + * array, so an internal downlink loses its children's signature bits and an + * index scan stops finding them. Needs: a small siglen (so a long string's + * signature saturates to all-true), a NULL (the null branch calls the union), + * enough rows that the root splits into internal pages, and then a row whose + * trigrams set no bits (''). + * + * Invariant: index scan = seqscan. Drawn: siglen, how many NULL and + * all-true rows go in and when, the repeated value and its count, whether + * the index exists before the load, and the final probe value(s). + */ +PG_PROP(trgm_gist_union_scan_equals_seqscan) +{ + static const char *const fillers[] = {"test", "abc", "w1", "xyz word"}; + pg_prop_conn *conn = cf_connect(tc); + int siglen = (int) pg_prop_draw_int(tc, 1, 4) * 2; /* 2,4,6,8 */ + /* the NULL and the all-true row are both needed; zero of either is a control */ + int nnull = pg_prop_draw_int(tc, 0, 3) == 0 ? 0 : (int) pg_prop_draw_int(tc, 1, 3); + int nall = pg_prop_draw_int(tc, 0, 3) == 0 ? 0 : (int) pg_prop_draw_int(tc, 1, 3); + const char *fill = fillers[pg_prop_draw_int(tc, 0, lengthof(fillers) - 1)]; + int nempty = (int) pg_prop_draw_int(tc, 1, 3); + StringInfoData s; + StringInfoData where; + + if (!pg_prop_require_extension(conn, "pg_trgm")) + pg_prop_skip(tc, "pg_trgm not installed"); + if (!pg_prop_require_extension(conn, "pageinspect")) + pg_prop_skip(tc, "pageinspect not installed"); + + initStringInfo(&s); + appendStringInfo(&s, + "DROP TABLE IF EXISTS cfb1_tu; CREATE TABLE cfb1_tu (id serial, v text); " + "CREATE INDEX cfb1_tu_v ON cfb1_tu USING gist (v gist_trgm_ops(siglen=%d))", + siglen); + cf_exec(tc, conn, s.data); + + /* + * The NULLs and all-true rows go in first, so the union that builds the + * first internal level sees them. Then load the filler one row at a + * time until the root's first child is itself an internal page: that is + * the split which writes the broken downlink, and it is not a fixed row + * count (gistchoose breaks penalty ties at random), so stop on the page + * structure, as the bug report does. The check must run after EVERY + * insert: stopping even a few rows late lets the next ordinary insert + * adjust the downlink and OR the lost bits back in, and the index then + * finds every row (checking every 50th row passed 2/2 on master; every + * row failed 2/2). Capped so a large siglen cannot loop for long. + */ + resetStringInfo(&s); + for (int i = 0; i < nnull; i++) + appendStringInfoString(&s, "INSERT INTO cfb1_tu (v) VALUES (NULL); "); + for (int i = 0; i < nall; i++) + appendStringInfo(&s, + "INSERT INTO cfb1_tu (v) SELECT string_agg((i + %d)::text, ' ') " + "FROM generate_series(1, 100) i; ", i * 1000); + appendStringInfo(&s, + "DO $do$ BEGIN FOR n IN 1..120000 LOOP " + "INSERT INTO cfb1_tu (v) VALUES ('%s'); " + "EXIT WHEN EXISTS (" + "SELECT FROM gist_page_items_bytea(get_raw_page('cfb1_tu_v', 0)) r, " + "gist_page_opaque_info(get_raw_page('cfb1_tu_v', (r.ctid::text::point)[0]::int)) c " + "WHERE r.itemoffset = 1 AND NOT 'leaf' = ANY (c.flags)); " + "END LOOP; END $do$", + fill); + cf_exec(tc, conn, s.data); + + /* + * Then rows whose trigrams set no signature bits, each its own statement. + * The bug is visible only while one of these was the last key adjusted + * into the broken downlink (a later filler insert ORs its bits back in), + * so the probe below runs immediately after them. + */ + for (int i = 0; i < nempty; i++) + cf_exec(tc, conn, "INSERT INTO cfb1_tu (v) VALUES ('')"); + + initStringInfo(&where); + switch ((int) pg_prop_draw_int(tc, 0, 2)) + { + case 0: + appendStringInfo(&where, "FROM cfb1_tu WHERE v = '%s'", fill); + break; + case 1: + appendStringInfo(&where, "FROM cfb1_tu WHERE v LIKE '%%%.2s%%'", fill); + break; + default: + appendStringInfo(&where, "FROM cfb1_tu WHERE v %% '%s'", fill); + break; + } + resetStringInfo(&s); + appendStringInfo(&s, "siglen=%d, %d NULL + %d all-true rows, '%s' until the root's child is internal, then %d x ''", + siglen, nnull, nall, fill, nempty); + cf_diff(tc, conn, "", where.data, s.data, NULL); + + cf_exec(tc, conn, "DROP TABLE cfb1_tu"); + pfree(s.data); + pfree(where.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7413: ALTER TABLE ... ALTER COLUMN ... SET STORAGE copies the column's + * new attstorage onto every index column built over it, even when the index + * column's type (the opclass's opckeytype -- gtrgm for gist_trgm_ops, whose + * typstorage is 'p') is not the table column's type. The index tuple then + * claims a compressible/toastable key it never was; the next INSERT, non-HOT + * UPDATE or REINDEX hands the AM a mangled datum and crashes the backend + * (makesign / unionkey in pg_trgm), or an intarray GiST index loses rows. + * + * Invariant: after SET STORAGE (any mode), the table must still accept + * INSERT/UPDATE and an index scan must equal a seqscan -- never a crash. + * Drawn: the opclass (gist_trgm_ops, gin_trgm_ops, core tsvector GiST as a + * control), siglen, the storage mode, row counts, and the follow-up + * statement (INSERT, UPDATE, REINDEX). The harness exits 3 and saves the + * statement if the backend dies. + */ +PG_PROP(set_storage_keeps_index_usable) +{ + static const char *const storages[] = {"EXTENDED", "MAIN", "EXTERNAL", "PLAIN"}; + pg_prop_conn *conn = cf_connect(tc); + int kind = (int) pg_prop_draw_int(tc, 0, 2); + const char *st = storages[pg_prop_draw_int(tc, 0, lengthof(storages) - 1)]; + int nrows = (int) pg_prop_draw_int(tc, 100, 12000); + int follow = (int) pg_prop_draw_int(tc, 0, 2); + StringInfoData s; + StringInfoData note; + char *err = NULL; + + if (!pg_prop_require_extension(conn, "pg_trgm")) + pg_prop_skip(tc, "pg_trgm not installed"); + + initStringInfo(&s); + initStringInfo(¬e); + appendStringInfo(&s, + "DROP TABLE IF EXISTS cfb1_ss; CREATE TABLE cfb1_ss (id int, s text); " + "INSERT INTO cfb1_ss SELECT g, 'item ' || g || ' ' || left(md5(g::text), 8) " + "FROM generate_series(1, %d) g; ", nrows); + if (kind == 0) + appendStringInfo(&s, "CREATE INDEX cfb1_ss_i ON cfb1_ss USING gist (s gist_trgm_ops(siglen=%d)); ", + (int) pg_prop_draw_int(tc, 1, 32) * 4); + else if (kind == 1) + appendStringInfoString(&s, "CREATE INDEX cfb1_ss_i ON cfb1_ss USING gin (s gin_trgm_ops); "); + else + appendStringInfoString(&s, "CREATE INDEX cfb1_ss_i ON cfb1_ss USING gist (to_tsvector('simple', s)); "); + cf_exec(tc, conn, s.data); + + resetStringInfo(&s); + appendStringInfo(&s, "ALTER TABLE cfb1_ss ALTER COLUMN s SET STORAGE %s", st); + cf_exec(tc, conn, s.data); + + /* the follow-up that reads or writes index keys; a crash exits the harness */ + resetStringInfo(&s); + switch (follow) + { + case 0: + appendStringInfo(&s, "INSERT INTO cfb1_ss SELECT g, 'new item ' || g " + "FROM generate_series(%d, %d) g", nrows + 1, nrows + 1000); + break; + case 1: + appendStringInfoString(&s, "UPDATE cfb1_ss SET s = s || ' x' WHERE id % 7 = 0"); + break; + default: + appendStringInfoString(&s, "REINDEX INDEX cfb1_ss_i"); + break; + } + appendStringInfo(¬e, "%s index, %d rows, SET STORAGE %s, then: %s", + kind == 0 ? "gist_trgm_ops" : kind == 1 ? "gin_trgm_ops" : "tsvector gist", + nrows, st, s.data); + if (!pg_prop_sql_exec(conn, s.data, &err)) + pg_prop_fail(tc, "after SET STORAGE the table rejects a legal statement: %s\n %s", + err ? err : "", note.data); + + /* the index must still answer like a seqscan */ + if (kind == 2) + cf_diff(tc, conn, "", "FROM cfb1_ss WHERE to_tsvector('simple', s) @@ 'item'::tsquery", + note.data, NULL); + else + cf_diff(tc, conn, "", "FROM cfb1_ss WHERE s LIKE '%item 1%'", note.data, NULL); + + cf_exec(tc, conn, "DROP TABLE cfb1_ss"); + pfree(s.data); + pfree(note.data); + pg_prop_disconnect(conn); +} From 3401c76c3e91a61bbcc8ee12cb077ab5dc99d5af Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sun, 18 Oct 2026 08:11:55 -0400 Subject: [PATCH 312/338] prop: stateful ALTER TABLE / ALTER TYPE properties (cf_ddl) Every other area builds its tables once. The catalog bugs the open commitfest is fixing happen when a table's shape changes: a whole-row reference that survives a column change, a constraint rebuilt twice by two subcommands of one ALTER TABLE, a composite type altered under a stored column or a domain CHECK, an index left without a file. alter_table_sequence_keeps_table_consistent builds a table with drawn dependents (whole-row CHECKs, some NOT ENFORCED; a whole-row expression index; a STORED or VIRTUAL generated column; an expression index with a statistics target), then up to 12 drawn ALTER and DML steps, and after every step requires: no internal error or crash, every ENFORCED CHECK true on every row, amcheck bt_index_check passing, INSERT still accepted, and the statistics target kept. Four smaller properties cover composite types under ALTER TYPE, whole-row partition keys, child-only ENFORCED CHECKs, and SET EXPRESSION on partitions. On upstream master 45fc51f: CF 7071 whole-row index loses entries after ADD COLUMN ... DEFAULT (amcheck; shrunk to the CF's own reproducer) CF 6121 ALTER COLUMN TYPE + SET EXPRESSION: cache lookup failed CF 7169 ALTER TYPE DROP ATTRIBUTE: UNIQUE composite column with duplicates CF 7356 ALTER TYPE ALTER ATTRIBUTE under a domain CHECK: XX000 CF 7117 ALTER TABLE ONLY ... SET EXPRESSION leaves a partition index with no file (XX001 "could not read blocks") Applying any one of these patches moves the failure to the next bug; each patch fixes its own reproducer. CF 5988 and 7426 pass on master (guards). --- src/test/prop/Makefile | 1 + src/test/prop/meson.build | 1 + src/test/prop/sql/prop_cf_ddl.c | 529 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 532 insertions(+), 1 deletion(-) create mode 100644 src/test/prop/sql/prop_cf_ddl.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 61fcd4d64ab46..5836ec30744a7 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -288,6 +288,7 @@ OBJS = \ sql/prop_cf_types.o \ sql/prop_cf_index.o \ sql/prop_cf_exec.o \ + sql/prop_cf_ddl.o \ sql/prop_cf_ddl2.o \ sql/prop_cf_planner.o \ sql/index_traits.o \ diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 7dabe039f56b2..23824621570df 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -293,6 +293,7 @@ prop_query_sources = files( 'sql/prop_cf_types.c', 'sql/prop_cf_index.c', 'sql/prop_cf_exec.c', + 'sql/prop_cf_ddl.c', 'sql/prop_cf_ddl2.c', 'sql/prop_cf_planner.c', 'sql/index_traits.c', diff --git a/src/test/prop/sql/prop_cf_ddl.c b/src/test/prop/sql/prop_cf_ddl.c new file mode 100644 index 0000000000000..f998ee12e5681 --- /dev/null +++ b/src/test/prop/sql/prop_cf_ddl.c @@ -0,0 +1,529 @@ +/*------------------------------------------------------------------------- + * + * prop_cf_ddl.c + * Shape-changing DDL: after ALTER TABLE / ALTER TYPE the table's + * constraints, indexes and rows must still agree. + * + * Every area elsewhere builds its tables once and then only queries them. + * The catalog and relcache bugs that the open commitfest is fixing live in + * the moment a table's SHAPE changes: a whole-row reference that survives a + * DROP COLUMN, a CHECK not rebuilt after SET DATA TYPE, a partition key on + * an altered column, two subcommands of one ALTER TABLE that both rebuild + * the same column, per-column settings lost in an index rebuild. + * + * One stateful property draws a short sequence of shape-changing steps over a + * table carrying the dependents those bugs need -- whole-row CHECKs (some NOT + * ENFORCED), a whole-row expression index, a stored generated column, an + * expression index with a statistics target -- interleaved with DML, and + * after every step checks: + * + * (1) the step succeeded or raised an ordinary user error -- never SQLSTATE + * XX000, "cache lookup failed", "unexpected", or a crash (the harness + * saves the statement and exits 3 if the backend dies); + * (2) every row satisfies every ENFORCED CHECK constraint, recomputed with + * a plain SELECT of pg_get_constraintdef's expression; + * (3) every index agrees with a seqscan (count via each index) and every + * btree passes amcheck's bt_index_check; + * (4) the table still takes an INSERT and a SELECT of every column. + * + * Smaller properties then target triggers the sequence rarely draws: + * composite types used as stored columns (ALTER TYPE ... DROP ATTRIBUTE / + * ALTER ATTRIBUTE, also under a domain CHECK), whole-row partition keys, + * and statistics targets across ALTER COLUMN TYPE. + * + * CF 6055 whole-row CHECK / index survives DROP COLUMN, SET DATA TYPE + * CF 6749 whole-row index reference with a virtual generated column + * CF 7071 whole-row expression index after ALTER TABLE + * CF 6121 ALTER COLUMN TYPE + SET EXPRESSION in one ALTER TABLE + * CF 7117 SET EXPRESSION on a partition with dependents + * CF 7426 CHECK NOT ENFORCED on the parent, ENFORCED on a child + * CF 7169 ALTER TYPE DROP ATTRIBUTE on a stored, indexed composite + * CF 7356 ALTER TYPE ALTER ATTRIBUTE under a domain CHECK + * CF 5988 ALTER COLUMN TYPE / DROP COLUMN under a whole-row partition key + * CF 7214 statistics targets across ALTER COLUMN TYPE + * + * Portions Copyright (c) 2024, PostgreSQL Global Development Group + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "cf_ddl" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +static pg_prop_conn * +cd_connect(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + return conn; +} + +/* + * An error a legal statement may raise: anything with a SQLSTATE outside + * class XX and without the internal-error phrasings. + */ +static bool +cd_internal(const char *sqlstate, const char *msg) +{ + if (sqlstate == NULL) + return true; + if (strncmp(sqlstate, "XX", 2) == 0) + return true; + if (msg && (strstr(msg, "cache lookup failed") || strstr(msg, "unexpected") || + strstr(msg, "unrecognized") || strstr(msg, "could not find"))) + return true; + return false; +} + +/* + * Run a DDL/DML step. Returns true if it succeeded; false on a user error; + * fails the case on an internal error. Uses the async path to get the + * SQLSTATE, then a probe statement so a dead backend trips the harness's + * crash tripwire with this statement in the log. + */ +static bool +cd_step(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql, const StringInfo log) +{ + char *ss = NULL; + char *err = NULL; + bool ok; + + appendStringInfo(log, " %s;\n", sql); + if (!pg_prop_sql_send(conn, sql)) + ok = false; + else + ok = pg_prop_sql_await(conn, &ss, &err); + if (ok) + return true; + /* liveness probe: exits 3 (and logs) if the backend died */ + { + bool f = false; + char *v = pg_prop_sql_scalar(conn, "SELECT 1", &f, NULL); + + if (v) + pg_free(v); + } + if (cd_internal(ss, err)) + pg_prop_fail(tc, "internal error (SQLSTATE %s): %s\n steps:\n%s", + ss ? ss : "none", err ? err : "", log->data); + appendStringInfo(log, " -- rejected: %s %.160s", ss ? ss : "", err ? err : ""); + if (log->len > 0 && log->data[log->len - 1] != '\n') + appendStringInfoChar(log, '\n'); + return false; +} + +static long +cd_count(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql, const StringInfo log) +{ + bool failed = false; + char *err = NULL; + char *v = pg_prop_sql_scalar(conn, sql, &failed, &err); + long n; + + if (failed || v == NULL) + { + if (err && strstr(err, "cache lookup failed")) + pg_prop_fail(tc, "internal error during check: %s\n check: %s\n steps:\n%s", + err, sql, log->data); + pg_prop_fail(tc, "check query failed: %s\n check: %s\n steps:\n%s", + err ? err : "", sql, log->data); + } + n = atol(v); + pg_free(v); + return n; +} + +/* + * Invariants (2)-(4) over table "rel". + */ +static void +cd_check(pg_prop_tc *tc, pg_prop_conn *conn, const char *rel, const StringInfo log) +{ + StringInfoData q; + char *list; + bool failed = false; + char *err = NULL; + long total; + + initStringInfo(&q); + + /* (4) a SELECT of every column, and the row count */ + appendStringInfo(&q, "SELECT count(*) FROM (SELECT * FROM %s) s", rel); + total = cd_count(tc, conn, q.data, log); + + /* + * (2) every enforced, validated CHECK holds on every row. Each check's + * expression is recomputed as "NOT (expr)" over the table; a row where + * it is TRUE violates the constraint. (NULL satisfies a CHECK.) + */ + resetStringInfo(&q); + appendStringInfo(&q, + "SELECT string_agg(format('SELECT %%L AS c, count(*) FROM %s WHERE NOT (%%s)', " + "conname, pg_get_expr(conbin, conrelid)), ' UNION ALL ') " + "FROM pg_constraint WHERE conrelid = '%s'::regclass AND contype = 'c' " + "AND conenforced AND convalidated", + rel, rel); + list = pg_prop_sql_scalar(conn, q.data, &failed, &err); + if (failed) + pg_prop_fail(tc, "listing checks failed: %s\n steps:\n%s", err ? err : "", log->data); + if (list && list[0]) + { + char *bad; + + resetStringInfo(&q); + appendStringInfo(&q, "SELECT string_agg(c || '=' || count, ', ') FROM (%s) s WHERE count > 0", list); + bad = pg_prop_sql_scalar(conn, q.data, &failed, &err); + if (failed) + { + if (err && cd_internal("XX000", err) && strstr(err, "cache lookup")) + pg_prop_fail(tc, "recomputing a CHECK raised an internal error: %s\n steps:\n%s", + err, log->data); + /* a check expression may legitimately error on some row; skip */ + } + else if (bad && bad[0]) + pg_prop_fail(tc, "rows violate an ENFORCED CHECK constraint: %s\n steps:\n%s", + bad, log->data); + if (bad) + pg_free(bad); + } + if (list) + pg_free(list); + + /* + * (3) every btree index passes amcheck; every index, used as the only + * access path, returns as many rows as the seqscan. A bitmap scan of + * "col IS NOT NULL OR col IS NULL" is not indexable for every AM, so use + * amcheck for btree and, for all indexes, compare a forced-index count on + * the index's first key against the seqscan count with the same qual. + */ + resetStringInfo(&q); + appendStringInfo(&q, + "SELECT string_agg(i.indexrelid::regclass::text, ',') FROM pg_index i " + "JOIN pg_class c ON c.oid = i.indexrelid JOIN pg_am a ON a.oid = c.relam " + "WHERE i.indrelid = '%s'::regclass AND a.amname = 'btree' AND i.indisvalid", rel); + list = pg_prop_sql_scalar(conn, q.data, &failed, NULL); + if (!failed && list && list[0]) + { + char *save = NULL; + + for (char *ix = strtok_r(list, ",", &save); ix; ix = strtok_r(NULL, ",", &save)) + { + resetStringInfo(&q); + appendStringInfo(&q, "SELECT bt_index_check('%s'::regclass, true)", ix); + if (!pg_prop_sql_exec(conn, q.data, &err)) + { + if (err && strstr(err, "does not exist")) + continue; /* amcheck not installed: skipped */ + pg_prop_fail(tc, "amcheck failed for index %s: %s\n steps:\n%s", + ix, err ? err : "", log->data); + } + } + } + if (list) + pg_free(list); + + /* (4) the table still accepts an INSERT of defaults (rolled back) */ + resetStringInfo(&q); + appendStringInfo(&q, "BEGIN; INSERT INTO %s DEFAULT VALUES; ROLLBACK", rel); + { + char *e2 = NULL; + + if (!pg_prop_sql_exec(conn, q.data, &e2)) + { + pg_prop_sql_exec(conn, "ROLLBACK", NULL); + if (e2 && (strstr(e2, "cache lookup failed") || strstr(e2, "unexpected") || + strstr(e2, "invalid memory alloc") || strstr(e2, "could not find"))) + pg_prop_fail(tc, "table no longer accepts INSERT: %s\n steps:\n%s", e2, log->data); + } + } + (void) total; + pfree(q.data); +} + +/* + * The stateful property. Table cfd_t(a int, b int, c text) with a drawn set + * of dependents, then 1..12 drawn steps, each followed by cd_check. + */ +PG_PROP(alter_table_sequence_keeps_table_consistent) +{ + pg_prop_conn *conn = cd_connect(tc); + StringInfoData log; + StringInfoData s; + int nsteps = (int) pg_prop_draw_int(tc, 1, 12); + bool have_gen; + + initStringInfo(&log); + initStringInfo(&s); + + pg_prop_sql_exec(conn, "CREATE EXTENSION IF NOT EXISTS amcheck", NULL); + + have_gen = pg_prop_draw_bool(tc); + appendStringInfoString(&s, "DROP TABLE IF EXISTS cfd_t CASCADE; CREATE TABLE cfd_t (a int, b int, c text"); + if (have_gen) + appendStringInfo(&s, ", g int GENERATED ALWAYS AS (a + 1) %s", + pg_prop_draw_bool(tc) ? "STORED" : "VIRTUAL"); + appendStringInfoString(&s, ")"); + cd_step(tc, conn, s.data, &log); + + /* dependents, each drawn independently */ + if (pg_prop_draw_bool(tc)) + cd_step(tc, conn, "ALTER TABLE cfd_t ADD CONSTRAINT cfd_wr_notnull CHECK (cfd_t IS NOT NULL) NOT ENFORCED", &log); + if (pg_prop_draw_bool(tc)) + cd_step(tc, conn, "ALTER TABLE cfd_t ADD CONSTRAINT cfd_wr_len CHECK (length(cfd_t::text) < 200)", &log); + if (pg_prop_draw_bool(tc)) + cd_step(tc, conn, "ALTER TABLE cfd_t ADD CONSTRAINT cfd_a_pos CHECK (a IS NULL OR a > -1000)", &log); + if (pg_prop_draw_bool(tc)) + cd_step(tc, conn, "CREATE INDEX cfd_wr_idx ON cfd_t ((cfd_t))", &log); + if (pg_prop_draw_bool(tc)) + cd_step(tc, conn, "CREATE UNIQUE INDEX cfd_wr_pred ON cfd_t (a) WHERE cfd_t IS NOT NULL", &log); + if (pg_prop_draw_bool(tc)) + { + cd_step(tc, conn, "CREATE INDEX cfd_expr_idx ON cfd_t ((a * b))", &log); + cd_step(tc, conn, "ALTER INDEX cfd_expr_idx ALTER COLUMN 1 SET STATISTICS 123", &log); + } + if (pg_prop_draw_bool(tc)) + cd_step(tc, conn, "CREATE INDEX cfd_b_idx ON cfd_t (b)", &log); + + resetStringInfo(&s); + appendStringInfo(&s, "INSERT INTO cfd_t (a, b, c) SELECT i, i %% 7, 'r' || i FROM generate_series(1, %d) i", + (int) pg_prop_draw_int(tc, 0, 300)); + cd_step(tc, conn, s.data, &log); + cd_check(tc, conn, "cfd_t", &log); + + for (int i = 0; i < nsteps; i++) + { + static const char *const steps[] = { + "ALTER TABLE cfd_t ALTER COLUMN a SET DATA TYPE bigint", + "ALTER TABLE cfd_t ALTER COLUMN a SET DATA TYPE int", + "ALTER TABLE cfd_t ALTER COLUMN b SET DATA TYPE numeric", + "ALTER TABLE cfd_t ALTER COLUMN c SET DATA TYPE varchar(50)", + "ALTER TABLE cfd_t DROP COLUMN b", + "ALTER TABLE cfd_t DROP COLUMN c", + "ALTER TABLE cfd_t ADD COLUMN d int DEFAULT 7", + "ALTER TABLE cfd_t ADD COLUMN e text", + "ALTER TABLE cfd_t ALTER COLUMN g SET EXPRESSION AS (a + 2)", + "ALTER TABLE cfd_t ALTER COLUMN a SET DATA TYPE bigint, ALTER COLUMN g SET EXPRESSION AS (a * 2)", + "ALTER TABLE cfd_t ADD COLUMN z int DEFAULT 11, ADD CONSTRAINT cfd_z CHECK (z > 9) NOT VALID", + "ALTER TABLE cfd_t ALTER COLUMN z SET DATA TYPE numeric, ALTER COLUMN g SET EXPRESSION AS (a + 3)", + "ALTER TABLE cfd_t ALTER CONSTRAINT cfd_wr_notnull ENFORCED", + "ALTER TABLE cfd_t VALIDATE CONSTRAINT cfd_z", + "UPDATE cfd_t SET a = a + 1 WHERE a % 3 = 0", + "DELETE FROM cfd_t WHERE a % 5 = 0", + "INSERT INTO cfd_t (a) VALUES (NULL), (-5), (999)", + "VACUUM cfd_t", + "REINDEX TABLE cfd_t", + }; + const char *st = steps[pg_prop_draw_int(tc, 0, lengthof(steps) - 1)]; + + /* VACUUM cannot run in the async path's implicit transaction: plain exec */ + if (strncmp(st, "VACUUM", 6) == 0) + { + appendStringInfo(&log, " %s;\n", st); + pg_prop_sql_exec(conn, st, NULL); + } + else + cd_step(tc, conn, st, &log); + cd_check(tc, conn, "cfd_t", &log); + } + + /* statistics target on the expression index survives (CF 7214) */ + { + bool f = false; + char *v = pg_prop_sql_scalar(conn, + "SELECT attstattarget FROM pg_attribute WHERE attrelid = " + "to_regclass('cfd_expr_idx') AND attnum = 1", &f, NULL); + + if (!f && v && v[0] && strcmp(v, "123") != 0) + pg_prop_fail(tc, "expression index lost its statistics target: %s (set 123)\n steps:\n%s", + v, log.data); + if (v) + pg_free(v); + } + + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS cfd_t CASCADE", NULL); + pfree(log.data); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7169 / 7356: a composite type used as a stored (and sometimes uniquely + * indexed) column, or under a domain CHECK, then ALTER TYPE ... DROP / ALTER + * ATTRIBUTE. Invariants: no internal error; if the ALTER succeeded, a UNIQUE + * index on the column still holds (count(DISTINCT v) = count(v)) and REINDEX + * succeeds; a domain CHECK still rejects what it rejected. + */ +PG_PROP(alter_type_on_stored_composite_keeps_constraints) +{ + pg_prop_conn *conn = cd_connect(tc); + StringInfoData log; + int kind = (int) pg_prop_draw_int(tc, 0, 3); + bool uniq = pg_prop_draw_bool(tc); + int n = (int) pg_prop_draw_int(tc, 1, 40); + StringInfoData s; + + initStringInfo(&log); + initStringInfo(&s); + cd_step(tc, conn, "DROP TABLE IF EXISTS cfd_ct CASCADE; DROP DOMAIN IF EXISTS cfd_dom CASCADE; " + "DROP TYPE IF EXISTS cfd_comp CASCADE", &log); + cd_step(tc, conn, "CREATE TYPE cfd_comp AS (a int, b int)", &log); + + if (kind <= 1) + { + /* stored column; tuples differ only in b, so dropping b would merge them */ + cd_step(tc, conn, "CREATE TABLE cfd_ct (v cfd_comp)", &log); + if (uniq) + cd_step(tc, conn, "CREATE UNIQUE INDEX cfd_ct_v ON cfd_ct (v)", &log); + appendStringInfo(&s, "INSERT INTO cfd_ct SELECT ROW(1, i)::cfd_comp FROM generate_series(1, %d) i", n); + cd_step(tc, conn, s.data, &log); + if (cd_step(tc, conn, kind == 0 ? "ALTER TYPE cfd_comp DROP ATTRIBUTE b" + : "ALTER TYPE cfd_comp DROP ATTRIBUTE b CASCADE", &log)) + { + long d = cd_count(tc, conn, "SELECT count(DISTINCT v) FROM cfd_ct", &log); + long all = cd_count(tc, conn, "SELECT count(v) FROM cfd_ct", &log); + + if (uniq && d != all) + pg_prop_fail(tc, "after ALTER TYPE DROP ATTRIBUTE a UNIQUE column holds duplicates: " + "%ld distinct of %ld\n steps:\n%s", d, all, log.data); + if (uniq && !cd_step(tc, conn, "REINDEX INDEX cfd_ct_v", &log)) + pg_prop_fail(tc, "REINDEX of the unique index fails after the ALTER TYPE\n steps:\n%s", + log.data); + } + } + else + { + /* domain whose CHECK goes through the composite type */ + cd_step(tc, conn, "CREATE DOMAIN cfd_dom AS int CHECK ((ROW(VALUE, 0)::cfd_comp).a > 0)", &log); + cd_step(tc, conn, kind == 2 ? "ALTER TYPE cfd_comp ALTER ATTRIBUTE a TYPE bigint" + : "ALTER TYPE cfd_comp ALTER ATTRIBUTE a TYPE text", &log); + /* the domain must still accept 1 and reject -1 (or the ALTER was refused) */ + if (!cd_step(tc, conn, "SELECT 1::cfd_dom", &log)) + pg_prop_fail(tc, "domain no longer accepts a valid value\n steps:\n%s", log.data); + if (cd_step(tc, conn, "SELECT (-1)::cfd_dom", &log)) + pg_prop_fail(tc, "domain CHECK no longer rejects -1\n steps:\n%s", log.data); + } + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS cfd_ct CASCADE; DROP DOMAIN IF EXISTS cfd_dom CASCADE; " + "DROP TYPE IF EXISTS cfd_comp CASCADE", NULL); + pfree(log.data); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* + * CF 5988: a partitioned table whose key is a whole-row expression. ALTER + * COLUMN TYPE / DROP COLUMN must either be rejected or leave the table + * usable: INSERT and SELECT then succeed, never "invalid memory alloc". + */ +PG_PROP(wholerow_partition_key_survives_alter) +{ + static const char *const alters[] = { + "ALTER TABLE cfd_pk ALTER COLUMN a TYPE text", + "ALTER TABLE cfd_pk ALTER COLUMN b TYPE bigint", + "ALTER TABLE cfd_pk DROP COLUMN a", + "ALTER TABLE cfd_pk DROP COLUMN b", + }; + pg_prop_conn *conn = cd_connect(tc); + StringInfoData log; + const char *alt = alters[pg_prop_draw_int(tc, 0, lengthof(alters) - 1)]; + + initStringInfo(&log); + cd_step(tc, conn, "DROP TABLE IF EXISTS cfd_pk CASCADE", &log); + cd_step(tc, conn, "CREATE TABLE cfd_pk (a int, b int) PARTITION BY LIST ((cfd_pk))", &log); + cd_step(tc, conn, "CREATE TABLE cfd_pk_1 PARTITION OF cfd_pk DEFAULT", &log); + cd_step(tc, conn, "INSERT INTO cfd_pk VALUES (1, 2), (3, 4)", &log); + cd_step(tc, conn, alt, &log); + if (!cd_step(tc, conn, "INSERT INTO cfd_pk DEFAULT VALUES", &log)) + pg_prop_fail(tc, "table no longer accepts INSERT\n steps:\n%s", log.data); + cd_count(tc, conn, "SELECT count(*) FROM (SELECT * FROM cfd_pk) s", &log); + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS cfd_pk CASCADE", NULL); + pfree(log.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7426: a CHECK that is NOT ENFORCED on the parent and ENFORCED on an + * inheritance child. The child's rows must satisfy it, including rows that + * arrive through the parent, through ATTACH/INHERIT, and after the parent's + * constraint changes. + */ +PG_PROP(child_enforced_check_holds_on_child_rows) +{ + pg_prop_conn *conn = cd_connect(tc); + StringInfoData log; + int how = (int) pg_prop_draw_int(tc, 0, 2); + + initStringInfo(&log); + cd_step(tc, conn, "DROP TABLE IF EXISTS cfd_par CASCADE", &log); + cd_step(tc, conn, "CREATE TABLE cfd_par (x int, CONSTRAINT cfd_ck CHECK (x > 0) NOT ENFORCED)", &log); + if (how == 0) + cd_step(tc, conn, "CREATE TABLE cfd_ch () INHERITS (cfd_par)", &log); + else + { + cd_step(tc, conn, "CREATE TABLE cfd_ch (x int, CONSTRAINT cfd_ck CHECK (x > 0) NOT ENFORCED)", &log); + cd_step(tc, conn, "ALTER TABLE cfd_ch INHERIT cfd_par", &log); + } + if (cd_step(tc, conn, "ALTER TABLE cfd_ch ALTER CONSTRAINT cfd_ck ENFORCED", &log)) + { + cd_step(tc, conn, "INSERT INTO cfd_ch VALUES (-1)", &log); + if (how == 2) + cd_step(tc, conn, "ALTER TABLE cfd_par ALTER CONSTRAINT cfd_ck NOT ENFORCED", &log); + cd_step(tc, conn, "INSERT INTO cfd_ch VALUES (-2)", &log); + if (cd_count(tc, conn, "SELECT count(*) FROM ONLY cfd_ch WHERE NOT (x > 0)", &log) > 0) + { + bool f = false; + char *e = pg_prop_sql_scalar(conn, + "SELECT conenforced FROM pg_constraint WHERE conrelid = " + "'cfd_ch'::regclass AND conname = 'cfd_ck'", &f, NULL); + + if (e && strcmp(e, "t") == 0) + pg_prop_fail(tc, "child holds rows violating its ENFORCED CHECK\n steps:\n%s", log.data); + if (e) + pg_free(e); + } + } + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS cfd_par CASCADE", NULL); + pfree(log.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7117: ALTER COLUMN SET EXPRESSION on a partition (or with ONLY) when the + * generated column has dependents (index, CHECK, FK). Accepted or rejected, + * afterwards every partition's rows satisfy the CHECK and the generated + * column equals its expression. + */ +PG_PROP(partition_set_expression_keeps_generated_consistent) +{ + static const char *const exprs[] = {"(a)", "(a + 1)", "(a + 112)", "(a * 2)"}; + pg_prop_conn *conn = cd_connect(tc); + StringInfoData log; + StringInfoData s; + const char *target = pg_prop_draw_bool(tc) ? "cfd_gp_1" : "ONLY cfd_gp"; + const char *ex = exprs[pg_prop_draw_int(tc, 0, lengthof(exprs) - 1)]; + + initStringInfo(&log); + initStringInfo(&s); + cd_step(tc, conn, "DROP TABLE IF EXISTS cfd_gp CASCADE", &log); + cd_step(tc, conn, "CREATE TABLE cfd_gp (a int, b int GENERATED ALWAYS AS (a) STORED, " + "CONSTRAINT cfd_gp_cc CHECK (b < 120)) PARTITION BY RANGE (a)", &log); + cd_step(tc, conn, "CREATE TABLE cfd_gp_1 PARTITION OF cfd_gp FOR VALUES FROM (1) TO (10)", &log); + cd_step(tc, conn, "CREATE TABLE cfd_gp_2 PARTITION OF cfd_gp FOR VALUES FROM (10) TO (20)", &log); + if (pg_prop_draw_bool(tc)) + cd_step(tc, conn, "CREATE INDEX cfd_gp_b ON cfd_gp (b)", &log); + cd_step(tc, conn, "INSERT INTO cfd_gp SELECT g FROM generate_series(5, 14) g", &log); + appendStringInfo(&s, "ALTER TABLE %s ALTER COLUMN b SET EXPRESSION AS %s", target, ex); + cd_step(tc, conn, s.data, &log); + if (cd_count(tc, conn, "SELECT count(*) FROM cfd_gp WHERE NOT (b < 120)", &log) > 0) + pg_prop_fail(tc, "a partition holds rows violating the CHECK on the generated column\n steps:\n%s", + log.data); + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS cfd_gp CASCADE", NULL); + pfree(log.data); + pfree(s.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 6773454008587..236d24cff6595 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model syscol collation cf_types cf_index cf_exec cf_ddl2 cf_planner index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model syscol collation cf_types cf_index cf_exec cf_ddl cf_ddl2 cf_planner index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From e78a0e0d4a0ed8c681d56f3abdc387aedc3a7d90 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sun, 18 Oct 2026 09:16:39 -0400 Subject: [PATCH 313/338] prop: six more executor-error properties (cf_exec) Each fails on upstream master 45fc51f and, where a patch exists, passes with that commitfest patch applied: 7094 cast_to_unknown_is_not_internal a cast to unknown of a typed value, as a literal, column or PL/pgSQL assignment: XX000 "failed to find conversion function" 7176 currtid2_on_view_is_not_internal currtid2() on drawn view shapes: XX000 "currtid cannot handle this view" 7296 zero_column_setop_parallel_matches_serial zero-column UNION/INTERSECT/EXCEPT under parallel-favouring costs must match the serial plan: crashes (Assert nkeys > 0 in tuplesort) 7097 slot_function_error_releases_slot a slot-function error caught by an EXCEPTION block must release the slot: it stays held, and the next acquire trips Assert(MyReplicationSlot == NULL). Needs wal_level = logical; skipped otherwise. 7459 nonatomic_record_survives_source_row_removal two sessions: a record read in a DO block, the source row deleted and committed, then VACUUMed by the other session while the first waits on an advisory lock: "missing chunk number 0 for toast value" 6903 cached_return_query_sees_widened_composite RETURN QUERY over a non-inlined SETOF composite function after ALTER TYPE ADD ATTRIBUTE must match a fresh session: "structure of query does not match function result type" --- src/test/prop/sql/prop_cf_exec.c | 334 +++++++++++++++++++++++++++++++ 1 file changed, 334 insertions(+) diff --git a/src/test/prop/sql/prop_cf_exec.c b/src/test/prop/sql/prop_cf_exec.c index 7695f514ae3f6..57b017255e722 100644 --- a/src/test/prop/sql/prop_cf_exec.c +++ b/src/test/prop/sql/prop_cf_exec.c @@ -746,3 +746,337 @@ PG_PROP(self_ref_cascade_with_replanning_trigger_matches_model) pfree(model.data); pg_prop_disconnect(conn); } + +/* + * CF 7094: a cast to "unknown" of a typed (non-literal) value must either work + * or raise a user error, never XX000 "failed to find conversion function + * from unknown to text". Drawn: the source type and whether the value is a + * literal, a column, or a PL/pgSQL assignment. + */ +PG_PROP(cast_to_unknown_is_not_internal) +{ + static const char *const srcs[] = { + "'s'::text", "'s'::varchar", "1::int", "'s'::name", "'{1}'::int[]", "now()::text" + }; + pg_prop_conn *conn = cfe_connect(tc); + const char *src = srcs[pg_prop_draw_int(tc, 0, lengthof(srcs) - 1)]; + int shape = (int) pg_prop_draw_int(tc, 0, 2); + StringInfoData s; + + initStringInfo(&s); + if (shape == 0) + appendStringInfo(&s, "SELECT %s::unknown", src); + else if (shape == 1) + appendStringInfo(&s, "SELECT cast(v AS unknown) FROM (SELECT %s AS v) s", src); + else + appendStringInfo(&s, "DO $$ DECLARE a text; BEGIN a := %s::unknown; END $$", src); + cfe_try_noint(tc, conn, s.data, "CF 7094 cast to unknown"); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7176: currtid2(view, tid) on a view must error with a proper SQLSTATE + * (the view has no single underlying ctid) -- never XX000. Drawn: the view + * shape (GROUP BY ctid, DISTINCT, UNION, join, plain), the tid. + */ +PG_PROP(currtid2_on_view_is_not_internal) +{ + static const char *const views[] = { + "SELECT ctid, a FROM cfa4_tid GROUP BY ctid, a", + "SELECT DISTINCT ctid, a FROM cfa4_tid", + "SELECT ctid, a FROM cfa4_tid UNION ALL SELECT ctid, a FROM cfa4_tid", + "SELECT ctid, a FROM cfa4_tid", + "SELECT x.ctid, x.a FROM cfa4_tid x JOIN cfa4_tid y USING (a)", + }; + pg_prop_conn *conn = cfe_connect(tc); + const char *v = views[pg_prop_draw_int(tc, 0, lengthof(views) - 1)]; + StringInfoData s; + + initStringInfo(&s); + appendStringInfo(&s, + "DROP TABLE IF EXISTS cfa4_tid CASCADE; CREATE TABLE cfa4_tid (a int); " + "INSERT INTO cfa4_tid SELECT generate_series(1, %d); " + "CREATE VIEW cfa4_tid_v AS %s", + (int) pg_prop_draw_int(tc, 1, 20), v); + cfe_exec(tc, conn, s.data); + resetStringInfo(&s); + appendStringInfo(&s, "SELECT currtid2('cfa4_tid_v'::text, '(0,%d)'::tid)", + (int) pg_prop_draw_int(tc, 1, 5)); + cfe_try_noint(tc, conn, s.data, v); + cfe_exec(tc, conn, "DROP TABLE cfa4_tid CASCADE"); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7296: a set operation over zero-column inputs under GUCs that push the + * planner to parallel plans must not crash (Assert nkeys > 0 in + * tuplesort_begin_heap), and must return what the serial plan returns. + * Drawn: set operation, sizes, parallel/cost GUCs. + */ +PG_PROP(zero_column_setop_parallel_matches_serial) +{ + static const char *const ops[] = {"UNION", "INTERSECT", "EXCEPT", "UNION ALL"}; + pg_prop_conn *conn = cfe_connect(tc); + const char *op = ops[pg_prop_draw_int(tc, 0, lengthof(ops) - 1)]; + int n1 = (int) pg_prop_draw_int(tc, 0, 3000); + int n2 = (int) pg_prop_draw_int(tc, 0, 3000); + StringInfoData s; + StringInfoData q; + char *par; + char *ser; + bool failed = false; + char *err = NULL; + + initStringInfo(&s); + appendStringInfo(&s, + "DROP TABLE IF EXISTS cfa4_z1, cfa4_z2; " + "CREATE TABLE cfa4_z1 AS SELECT g FROM generate_series(1, %d) g; " + "CREATE TABLE cfa4_z2 AS SELECT g FROM generate_series(1, %d) g; " + "ANALYZE cfa4_z1, cfa4_z2", n1, n2); + cfe_exec(tc, conn, s.data); + + initStringInfo(&q); + appendStringInfo(&q, "SELECT count(*) FROM (SELECT FROM cfa4_z1 %s SELECT FROM cfa4_z2) s", op); + ser = pg_prop_sql_scalar(conn, q.data, &failed, &err); + if (failed) + pg_prop_fail(tc, "serial plan failed: %s\n %s", err ? err : "", q.data); + + resetStringInfo(&s); + appendStringInfo(&s, + "SET cpu_tuple_cost = %d; SET min_parallel_table_scan_size = 1; " + "SET parallel_setup_cost = 0; SET parallel_tuple_cost = 0", + (int) pg_prop_draw_int(tc, 1, 2000)); + cfe_exec(tc, conn, s.data); + par = pg_prop_sql_scalar(conn, q.data, &failed, &err); /* crash -> harness exits 3 */ + cfe_exec(tc, conn, "RESET ALL"); + if (failed) + pg_prop_fail(tc, "parallel-costed plan failed: %s\n %s", err ? err : "", q.data); + if (strcmp(par, ser) != 0) + pg_prop_fail(tc, "zero-column %s: parallel-costed plan %s rows, serial %s\n sizes %d, %d", + op, par, ser, n1, n2); + pg_free(par); + pg_free(ser); + cfe_exec(tc, conn, "DROP TABLE cfa4_z1, cfa4_z2"); + pfree(s.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7097: an error raised inside an SQL-callable replication-slot function + * and caught by a PL/pgSQL EXCEPTION block must release the slot; the next + * slot operation in the session must work. On master the slot stays + * acquired (active = t) and the next acquire trips + * Assert(MyReplicationSlot == NULL). Needs wal_level = logical; skipped + * otherwise. Drawn: the failing call (advance to an invalid LSN, peek with + * a bad option), the follow-up operation, how many times the pair repeats. + */ +PG_PROP(slot_function_error_releases_slot) +{ + static const char *const fail_calls[] = { + "PERFORM pg_replication_slot_advance('cfa4_slot', '0/1')", + "PERFORM count(*) FROM pg_logical_slot_peek_changes('cfa4_slot', NULL, NULL, 'no-such-option', 'x')", + "PERFORM count(*) FROM pg_logical_slot_get_changes('cfa4_slot', NULL, NULL, 'include-xids', 'notabool')", + }; + static const char *const follow[] = { + "SELECT count(*) >= 0 FROM pg_logical_slot_peek_changes('cfa4_slot', NULL, NULL)", + "SELECT count(*) >= 0 FROM pg_logical_slot_get_changes('cfa4_slot', NULL, NULL)", + "SELECT pg_replication_slot_advance('cfa4_slot', pg_current_wal_lsn()) IS NOT NULL", + }; + pg_prop_conn *conn = cfe_connect(tc); + bool failed = false; + char *wl = pg_prop_sql_scalar(conn, "SHOW wal_level", &failed, NULL); + int reps = (int) pg_prop_draw_int(tc, 1, 3); + StringInfoData s; + char *act; + + if (wl == NULL || strcmp(wl, "logical") != 0) + { + if (wl) + pg_free(wl); + pg_prop_disconnect(conn); + pg_prop_skip(tc, "needs wal_level = logical"); + } + pg_free(wl); + pg_prop_sql_exec(conn, "SELECT pg_drop_replication_slot('cfa4_slot') " + "FROM pg_replication_slots WHERE slot_name = 'cfa4_slot'", NULL); + cfe_exec(tc, conn, "SELECT pg_create_logical_replication_slot('cfa4_slot', 'test_decoding')"); + + initStringInfo(&s); + for (int r = 0; r < reps; r++) + { + resetStringInfo(&s); + appendStringInfo(&s, "DO $$ BEGIN %s; EXCEPTION WHEN OTHERS THEN NULL; END $$", + fail_calls[pg_prop_draw_int(tc, 0, lengthof(fail_calls) - 1)]); + cfe_try_noint(tc, conn, s.data, "CF 7097 slot error inside an exception block"); + + act = pg_prop_sql_scalar(conn, + "SELECT active_pid IS NOT DISTINCT FROM pg_backend_pid() " + "FROM pg_replication_slots WHERE slot_name = 'cfa4_slot'", + &failed, NULL); + if (act && strcmp(act, "t") == 0) + pg_prop_fail(tc, "slot still held by this session after the caught error\n %s", s.data); + if (act) + pg_free(act); + + /* the next slot operation; a crash exits the harness */ + cfe_try_noint(tc, conn, follow[pg_prop_draw_int(tc, 0, lengthof(follow) - 1)], + "CF 7097 follow-up slot operation"); + } + pg_prop_sql_exec(conn, "SELECT pg_drop_replication_slot('cfa4_slot')", NULL); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7459: a PL/pgSQL record assigned from a function returning a row keeps + * TOAST pointers into the source table. In a non-atomic DO block the source + * row is deleted and the transaction committed; if VACUUM then removes the + * dead row and its TOAST chunks before the record is read, the read raises + * "missing chunk number 0 for toast value". Invariant: a record's value is + * the value that was read, whatever happens to the source row later. + * + * Interleaving (two sessions, as in the patch's isolation spec): session A + * holds an advisory lock; session B runs the DO block, which deletes the + * row, COMMITs, then blocks on the advisory lock; A runs VACUUM and releases + * the lock; B then reads the record. Drawn: value size (inline or toasted), + * storage mode, DELETE vs UPDATE, and VACUUM vs no VACUUM (the control). + */ +PG_PROP(nonatomic_record_survives_source_row_removal) +{ + static const char *const storages[] = {"EXTERNAL", "EXTENDED"}; + pg_prop_conn *a = cfe_connect(tc); + pg_prop_conn *b = cfe_connect(tc); + int len = pg_prop_draw_int(tc, 0, 3) == 0 ? (int) pg_prop_draw_int(tc, 1, 1500) + : (int) pg_prop_draw_int(tc, 3000, 200000); + const char *st = storages[pg_prop_draw_int(tc, 0, lengthof(storages) - 1)]; + bool delete_row = pg_prop_draw_bool(tc); + bool vacuum = pg_prop_draw_int(tc, 0, 3) != 0; + StringInfoData s; + char *ss = NULL; + char *err = NULL; + bool ok; + + initStringInfo(&s); + appendStringInfo(&s, + "DROP TABLE IF EXISTS cfa4_t1 CASCADE; CREATE TABLE cfa4_t1 (id int, t text) " + "WITH (autovacuum_enabled = off); " + "ALTER TABLE cfa4_t1 ALTER COLUMN t SET STORAGE %s; " + "INSERT INTO cfa4_t1 SELECT 1, string_agg(md5(i::text), '') FROM generate_series(1, %d) i; " + "CREATE OR REPLACE FUNCTION cfa4_get() RETURNS cfa4_t1 LANGUAGE plpgsql AS " + "$f$ DECLARE r cfa4_t1; BEGIN SELECT * INTO r FROM cfa4_t1; RETURN r; END $f$", + st, len / 32 + 1); + cfe_exec(tc, a, s.data); + cfe_exec(tc, a, "SELECT pg_advisory_lock(74590)"); + + resetStringInfo(&s); + appendStringInfo(&s, + "DO $$ DECLARE r cfa4_t1; want int; BEGIN " + "SELECT length(t) INTO want FROM cfa4_t1; " + "r := cfa4_get(); " + "%s " + "COMMIT; " + "PERFORM pg_advisory_lock(74590); " + "IF length(r.t) IS DISTINCT FROM want THEN " + "RAISE EXCEPTION 'record value changed: %% vs %%', length(r.t), want; END IF; " + "PERFORM pg_advisory_unlock(74590); " + "END $$", + delete_row ? "DELETE FROM cfa4_t1;" : "UPDATE cfa4_t1 SET t = 'x';"); + if (!pg_prop_sql_send(b, s.data)) + pg_prop_fail(tc, "could not dispatch the DO block"); + + /* let B reach the advisory lock (after its COMMIT), then clean the row away */ + cfe_exec(tc, a, "SELECT pg_sleep(0.2)"); + if (vacuum) + cfe_exec(tc, a, "VACUUM cfa4_t1"); + cfe_exec(tc, a, "SELECT pg_advisory_unlock(74590)"); + + ok = pg_prop_sql_await(b, &ss, &err); + if (!ok && err && (strstr(err, "missing chunk") || strstr(err, "record value changed") || + strstr(err, "unexpected chunk"))) + pg_prop_fail(tc, "a record read before COMMIT lost its value after %s: %s\n" + " storage %s, %d bytes, %s", + vacuum ? "VACUUM" : "no VACUUM", err, st, len, + delete_row ? "DELETE" : "UPDATE"); + if (!ok && cfe_internal(ss, err)) + pg_prop_fail(tc, "internal error: %s", err ? err : ""); + + cfe_exec(tc, a, "DROP TABLE cfa4_t1 CASCADE"); + pfree(s.data); + pg_prop_disconnect(b); + pg_prop_disconnect(a); +} + +/* + * CF 6903: a PL/pgSQL RETURN QUERY over a non-inlined SETOF function + * returning a named composite; after ALTER TYPE ... ADD ATTRIBUTE widens the + * composite (and the underlying table gains the column), the cached plan + * must be invalidated -- the second call in the same session must return + * what a fresh session returns. On master the plan is not invalidated and + * the call raises "structure of query does not match function result type". + * + * Invariant: same-session result after the DDL = fresh-session result. + * Drawn: inlining blockers (SECURITY DEFINER / VOLATILE / plain), SRF + * language (sql / plpgsql RETURN QUERY), the DDL pair, rows. + */ +PG_PROP(cached_return_query_sees_widened_composite) +{ + static const char *const attrs[] = {"STABLE SECURITY DEFINER", "VOLATILE", "STABLE", "SECURITY DEFINER"}; + pg_prop_conn *conn = cfe_connect(tc); + pg_prop_conn *fresh; + const char *attr = attrs[pg_prop_draw_int(tc, 0, lengthof(attrs) - 1)]; + bool plpgsql_srf = pg_prop_draw_bool(tc); + int nrows = (int) pg_prop_draw_int(tc, 1, 5); + StringInfoData s; + char *a; + char *b; + bool fa = false; + bool fb = false; + char *ea = NULL; + char *eb = NULL; + + initStringInfo(&s); + appendStringInfo(&s, + "DROP FUNCTION IF EXISTS cfa4_caller() CASCADE; DROP FUNCTION IF EXISTS cfa4_srf() CASCADE; " + "DROP TABLE IF EXISTS cfa4_tbl; DROP TYPE IF EXISTS cfa4_ct CASCADE; " + "CREATE TYPE cfa4_ct AS (a int, b int); " + "CREATE TABLE cfa4_tbl (a int, b int); " + "INSERT INTO cfa4_tbl SELECT i, i * 10 FROM generate_series(1, %d) i; ", nrows); + if (plpgsql_srf) + appendStringInfo(&s, + "CREATE FUNCTION cfa4_srf() RETURNS SETOF cfa4_ct LANGUAGE plpgsql %s AS " + "$f$ BEGIN RETURN QUERY SELECT * FROM cfa4_tbl; END $f$; ", attr); + else + appendStringInfo(&s, + "CREATE FUNCTION cfa4_srf() RETURNS SETOF cfa4_ct LANGUAGE sql %s AS " + "$f$ SELECT * FROM cfa4_tbl $f$; ", attr); + appendStringInfoString(&s, + "CREATE FUNCTION cfa4_caller() RETURNS SETOF cfa4_ct LANGUAGE plpgsql AS " + "$f$ BEGIN RETURN QUERY SELECT r.* FROM cfa4_srf() r; END $f$"); + cfe_exec(tc, conn, s.data); + + /* warm the plan cache in this session */ + pg_free(pg_prop_sql_rows(conn, "SELECT * FROM cfa4_caller() ORDER BY 1", &fa, NULL)); + + cfe_exec(tc, conn, "ALTER TYPE cfa4_ct ADD ATTRIBUTE c int"); + cfe_exec(tc, conn, "ALTER TABLE cfa4_tbl ADD COLUMN c int DEFAULT 99"); + + a = pg_prop_sql_rows(conn, "SELECT * FROM cfa4_caller() ORDER BY 1", &fa, &ea); + fresh = cfe_connect(tc); + b = pg_prop_sql_rows(fresh, "SELECT * FROM cfa4_caller() ORDER BY 1", &fb, &eb); + if (fa != fb || (!fa && strcmp(a, b) != 0)) + pg_prop_fail(tc, "after ALTER TYPE ADD ATTRIBUTE the cached plan disagrees with a fresh session\n" + " same session: %s\n fresh session: %s\n srf: %s %s", + fa ? (ea ? ea : "error") : a, fb ? (eb ? eb : "error") : b, + plpgsql_srf ? "plpgsql" : "sql", attr); + if (a) + pg_free(a); + if (b) + pg_free(b); + pg_prop_disconnect(fresh); + cfe_exec(tc, conn, "DROP FUNCTION cfa4_caller(); DROP FUNCTION cfa4_srf(); DROP TABLE cfa4_tbl; DROP TYPE cfa4_ct"); + pfree(s.data); + pg_prop_disconnect(conn); +} From c4b55decbfe1a0d7f7190f78ccc0d0d2585ed7d8 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sun, 18 Oct 2026 10:05:32 -0400 Subject: [PATCH 314/338] prop: deparse round-trip properties for views and rules (cf_deparse) pg_get_viewdef and pg_get_ruledef are what pg_dump writes, so a deparse that drops a parenthesis, quote or cast changes a view's meaning across dump and restore, or makes the dump unrestorable. view_expressions_survive_deparse draws a view over drawn data from the expressions the open fixes involve -- sublink tests on OR/AND/arithmetic (CF 7441), x IS DOCUMENT as an operand (7451), operators on ROW()::record (7452) -- creates it, deparses it pretty or not, re-creates a view from the text, and requires the text to read back, both views to return the same rows, and the deparse to be a fixed point. keyword_window_names_survive_deparse: windows named rows/range/groups/... referenced from another window (7056). rule_target_alias_survives_deparse: a rule action whose UPDATE/DELETE target alias is "set" (7442). On upstream master 45fc51f all three falsify on every seed. With the patch applied: 7056 and 7442 pass 3/3. The expression property finds all three of 7441/7451/7452, so any one patch alone still fails on another; with all three applied together it passes 6/6. --- src/test/prop/Makefile | 1 + src/test/prop/meson.build | 1 + src/test/prop/sql/prop_cf_deparse.c | 286 ++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 289 insertions(+), 1 deletion(-) create mode 100644 src/test/prop/sql/prop_cf_deparse.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 5836ec30744a7..68f2fd598ff42 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -290,6 +290,7 @@ OBJS = \ sql/prop_cf_exec.o \ sql/prop_cf_ddl.o \ sql/prop_cf_ddl2.o \ + sql/prop_cf_deparse.o \ sql/prop_cf_planner.o \ sql/index_traits.o \ sql/prop_index_am.o \ diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 23824621570df..79bbe88fb4d11 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -295,6 +295,7 @@ prop_query_sources = files( 'sql/prop_cf_exec.c', 'sql/prop_cf_ddl.c', 'sql/prop_cf_ddl2.c', + 'sql/prop_cf_deparse.c', 'sql/prop_cf_planner.c', 'sql/index_traits.c', 'sql/prop_index_am.c', diff --git a/src/test/prop/sql/prop_cf_deparse.c b/src/test/prop/sql/prop_cf_deparse.c new file mode 100644 index 0000000000000..8be8458425a18 --- /dev/null +++ b/src/test/prop/sql/prop_cf_deparse.c @@ -0,0 +1,286 @@ +/*------------------------------------------------------------------------- + * + * prop_cf_deparse.c + * Deparse round trip: what ruleutils prints must read back as the same + * thing. + * + * pg_get_viewdef / pg_get_ruledef are how pg_dump writes views and rules, + * so a deparse that drops a needed parenthesis, quote or cast silently + * changes a view's meaning across a dump and restore -- or makes the dump + * unrestorable. The open commitfest has five such bugs. + * + * The property draws a view body from a small grammar of the expressions + * those bugs involve, creates the view, deparses it (pretty and not), creates + * a second view from the deparsed text, and requires: + * (1) the deparsed text is accepted (a syntax error is a failure); + * (2) both views return the same rows over drawn table data; + * (3) deparsing the second view gives the same text (a fixed point). + * + * CF 7056 window names that are unreserved keywords (rows, range, groups) + * CF 7441 sublink tests on OR/AND/arithmetic, pretty-printed + * CF 7451 x IS DOCUMENT as an operand + * CF 7452 operators on ROW(...)::record + * CF 7442 an UPDATE/DELETE target alias named "set", in a rule + * + * Portions Copyright (c) 2024, PostgreSQL Global Development Group + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "cf_deparse" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +static pg_prop_conn * +dp_connect(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + return conn; +} + +static void +dp_exec(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "setup failed: %s\n SQL: %.1500s", err ? err : "", sql); +} + +/* append one drawn select-list expression over cfd_dp(a bool, b bool, c int, d int, x xml) */ +static void +dp_expr(pg_prop_tc *tc, StringInfo b) +{ + static const char *const bools[] = {"a", "b", "(a OR b)", "(a AND b)", "(NOT a)"}; + static const char *const ints[] = {"c", "d", "(c + d)", "(c * 2)"}; + + switch ((int) pg_prop_draw_int(tc, 0, 7)) + { + case 0: /* CF 7441: (bool expr) IN (sublink) */ + appendStringInfo(b, "(%s IN (SELECT %s))", + bools[pg_prop_draw_int(tc, 0, lengthof(bools) - 1)], + pg_prop_draw_bool(tc) ? "false" : "true"); + break; + case 1: /* CF 7441: (int expr) IN / = ANY sublink */ + appendStringInfo(b, "(%s %s (SELECT %d))", + ints[pg_prop_draw_int(tc, 0, lengthof(ints) - 1)], + pg_prop_draw_bool(tc) ? "IN" : "= ANY", + (int) pg_prop_draw_int(tc, 0, 5)); + break; + case 2: /* CF 7441: sublink as a comparison operand */ + appendStringInfo(b, "((SELECT %d) %s c)", + (int) pg_prop_draw_int(tc, 0, 5), pg_prop_draw_bool(tc) ? "<" : "="); + break; + case 3: /* CF 7451: IS DOCUMENT as an operand */ + appendStringInfo(b, "(a %s (x IS %sDOCUMENT))", + pg_prop_draw_bool(tc) ? "=" : "<>", pg_prop_draw_bool(tc) ? "NOT " : ""); + break; + case 4: /* CF 7451: IS DOCUMENT under NOT / in a CASE */ + appendStringInfoString(b, pg_prop_draw_bool(tc) ? "(NOT (x IS DOCUMENT))" + : "(CASE WHEN x IS DOCUMENT THEN 1 ELSE 0 END)"); + break; + case 5: /* CF 7452: operator on ROW()::record with a NULL field */ + appendStringInfo(b, "(ROW(c, %s)::record %s ROW(c, %s)::record)", + pg_prop_draw_bool(tc) ? "NULL::int" : "d", + pg_prop_draw_bool(tc) ? "=" : "<>", + pg_prop_draw_bool(tc) ? "NULL::int" : "d"); + break; + case 6: /* CF 7452: plain row comparison (different semantics) */ + appendStringInfo(b, "(ROW(c, %s) = ROW(c, %s))", + pg_prop_draw_bool(tc) ? "NULL::int" : "d", + pg_prop_draw_bool(tc) ? "NULL::int" : "d"); + break; + default: + appendStringInfoString(b, ints[pg_prop_draw_int(tc, 0, lengthof(ints) - 1)]); + break; + } +} + +/* + * Compare two views' rows and require the deparse of the second to equal the + * deparse of the first. + */ +static void +dp_roundtrip(pg_prop_tc *tc, pg_prop_conn *conn, const char *body, bool pretty) +{ + StringInfoData s; + char *def; + char *def2; + char *r1; + char *r2; + bool f = false; + char *err = NULL; + + initStringInfo(&s); + pg_prop_sql_exec(conn, "DROP VIEW IF EXISTS cfd_v2, cfd_v1", NULL); + appendStringInfo(&s, "CREATE VIEW cfd_v1 AS %s", body); + if (!pg_prop_sql_exec(conn, s.data, &err)) + { + /* the drawn body itself is not valid SQL (e.g. a type mismatch): vacuous */ + pg_free(err); + pfree(s.data); + return; + } + resetStringInfo(&s); + appendStringInfo(&s, "SELECT pg_get_viewdef('cfd_v1'::regclass, %s)", pretty ? "true" : "false"); + def = pg_prop_sql_scalar(conn, s.data, &f, &err); + if (f || def == NULL) + pg_prop_fail(tc, "pg_get_viewdef failed: %s\n view: %s", err ? err : "", body); + + resetStringInfo(&s); + appendStringInfo(&s, "CREATE VIEW cfd_v2 AS %s", def); + if (!pg_prop_sql_exec(conn, s.data, &err)) + pg_prop_fail(tc, "the deparsed view (pretty=%s) does not read back: %s\n" + " original: %s\n deparsed: %s", + pretty ? "true" : "false", err ? err : "", body, def); + + r1 = pg_prop_sql_rows(conn, "SELECT * FROM cfd_v1 ORDER BY 1", &f, &err); + if (f) + pg_prop_fail(tc, "querying the original view failed: %s", err ? err : ""); + r2 = pg_prop_sql_rows(conn, "SELECT * FROM cfd_v2 ORDER BY 1", &f, &err); + if (f) + pg_prop_fail(tc, "querying the re-created view failed: %s", err ? err : ""); + if (strcmp(r1, r2) != 0) + pg_prop_fail(tc, "the re-created view returns different rows (pretty=%s)\n" + " original: %s\n deparsed: %s\n original rows:\n%s re-created rows:\n%s", + pretty ? "true" : "false", body, def, r1, r2); + + resetStringInfo(&s); + appendStringInfo(&s, "SELECT pg_get_viewdef('cfd_v2'::regclass, %s)", pretty ? "true" : "false"); + def2 = pg_prop_sql_scalar(conn, s.data, &f, &err); + if (!f && def2 && strcmp(def, def2) != 0) + pg_prop_fail(tc, "deparse is not a fixed point (pretty=%s)\n first: %s\n second: %s", + pretty ? "true" : "false", def, def2); + + pg_free(def); + if (def2) + pg_free(def2); + pg_free(r1); + pg_free(r2); + pfree(s.data); +} + +/* + * Expressions (CF 7441, 7451, 7452) over a drawn table: up to 4 columns of + * drawn expressions, deparsed pretty and not. + */ +PG_PROP(view_expressions_survive_deparse) +{ + pg_prop_conn *conn = dp_connect(tc); + int nrows = (int) pg_prop_draw_int(tc, 0, 12); + int ncols = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData s; + StringInfoData body; + + initStringInfo(&s); + appendStringInfoString(&s, + "DROP VIEW IF EXISTS cfd_v2, cfd_v1; DROP TABLE IF EXISTS cfd_dp; " + "CREATE TABLE cfd_dp (a bool, b bool, c int, d int, x xml); "); + for (int i = 0; i < nrows; i++) + { + static const char *const bv[] = {"true", "false", "NULL"}; + static const char *const xv[] = {"''", "'text'", "NULL", "'1'"}; + + appendStringInfo(&s, "INSERT INTO cfd_dp VALUES (%s, %s, %s, %s, %s); ", + bv[pg_prop_draw_int(tc, 0, 2)], bv[pg_prop_draw_int(tc, 0, 2)], + pg_prop_draw_int(tc, 0, 4) == 0 ? "NULL" : psprintf("%d", (int) pg_prop_draw_int(tc, -2, 5)), + pg_prop_draw_int(tc, 0, 4) == 0 ? "NULL" : psprintf("%d", (int) pg_prop_draw_int(tc, -2, 5)), + xv[pg_prop_draw_int(tc, 0, 3)]); + } + dp_exec(tc, conn, s.data); + + initStringInfo(&body); + appendStringInfoString(&body, "SELECT c AS k"); + for (int i = 0; i < ncols; i++) + { + appendStringInfoString(&body, ", "); + dp_expr(tc, &body); + appendStringInfo(&body, " AS e%d", i); + } + appendStringInfoString(&body, " FROM cfd_dp"); + + dp_roundtrip(tc, conn, body.data, pg_prop_draw_bool(tc)); + + pg_prop_sql_exec(conn, "DROP VIEW IF EXISTS cfd_v2, cfd_v1; DROP TABLE cfd_dp", NULL); + pfree(s.data); + pfree(body.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7056: windows named after unreserved keywords, referenced from another + * window definition and from OVER. + */ +PG_PROP(keyword_window_names_survive_deparse) +{ + static const char *const names[] = {"rows", "range", "groups", "partition", "w", "order_"}; + static const char *const clauses[] = { + "PARTITION BY v", "ORDER BY v", "ORDER BY v ROWS BETWEEN 1 PRECEDING AND CURRENT ROW", + }; + pg_prop_conn *conn = dp_connect(tc); + const char *base = names[pg_prop_draw_int(tc, 0, lengthof(names) - 1)]; + StringInfoData body; + + initStringInfo(&body); + appendStringInfo(&body, + "SELECT v, count(*) OVER w2 AS n FROM generate_series(1, %d) s(v) " + "WINDOW \"%s\" AS (%s), w2 AS (\"%s\"%s)", + (int) pg_prop_draw_int(tc, 1, 6), base, + clauses[pg_prop_draw_int(tc, 0, 0)], base, + pg_prop_draw_bool(tc) ? " ORDER BY v" : ""); + dp_roundtrip(tc, conn, body.data, pg_prop_draw_bool(tc)); + pg_prop_sql_exec(conn, "DROP VIEW IF EXISTS cfd_v2, cfd_v1", NULL); + (void) clauses; + pfree(body.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7442: a rule whose action UPDATEs or DELETEs with a target alias named + * after a keyword ("set"). pg_get_ruledef's output must re-create the rule. + */ +PG_PROP(rule_target_alias_survives_deparse) +{ + static const char *const aliases[] = {"set", "\"set\"", "t", "using"}; + pg_prop_conn *conn = dp_connect(tc); + const char *al = aliases[pg_prop_draw_int(tc, 0, lengthof(aliases) - 1)]; + bool upd = pg_prop_draw_bool(tc); + StringInfoData s; + char *def; + bool f = false; + char *err = NULL; + + initStringInfo(&s); + dp_exec(tc, conn, "DROP TABLE IF EXISTS cfd_rt CASCADE; CREATE TABLE cfd_rt (a int, b int)"); + if (upd) + appendStringInfo(&s, "CREATE RULE cfd_rt_r AS ON INSERT TO cfd_rt DO ALSO " + "UPDATE cfd_rt AS %s SET b = %s.b + 1 WHERE %s.a = new.a", al, al, al); + else + appendStringInfo(&s, "CREATE RULE cfd_rt_r AS ON INSERT TO cfd_rt DO ALSO " + "DELETE FROM cfd_rt AS %s WHERE %s.a = new.a + 100", al, al); + if (!pg_prop_sql_exec(conn, s.data, &err)) + { + /* alias not allowed by the grammar: vacuous */ + pg_free(err); + goto done; + } + def = pg_prop_sql_scalar(conn, "SELECT pg_get_ruledef(oid) FROM pg_rewrite WHERE rulename = 'cfd_rt_r'", + &f, &err); + if (f || def == NULL) + pg_prop_fail(tc, "pg_get_ruledef failed: %s", err ? err : ""); + dp_exec(tc, conn, "DROP RULE cfd_rt_r ON cfd_rt"); + if (!pg_prop_sql_exec(conn, def, &err)) + pg_prop_fail(tc, "the deparsed rule does not read back: %s\n original: %s\n deparsed: %s", + err ? err : "", s.data, def); + pg_free(def); +done: + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS cfd_rt CASCADE", NULL); + pfree(s.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 236d24cff6595..acbb4f90787ff 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model syscol collation cf_types cf_index cf_exec cf_ddl cf_ddl2 cf_planner index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model syscol collation cf_types cf_index cf_exec cf_ddl cf_ddl2 cf_deparse cf_planner index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 10e6d05110bcf5a87a70060cefcabb41e6b23810 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sun, 18 Oct 2026 12:18:40 -0400 Subject: [PATCH 315/338] prop: statistics-versus-model properties (cf_stats) reltuples and the pg_stat_*_tables counters are computed incrementally and drift from the truth in ways no query result shows. Each property performs a drawn sequence of operations whose effect on a statistic is known exactly, and compares: 5777 reltuples_after_rewrite_counts_live_rows after VACUUM FULL / CLUSTER of an index-less table with an old snapshot open, reltuples counts the recently-dead copies 6619 xact_stats_count_only_this_transaction pg_stat_xact_user_tables includes an earlier committed transaction 7460 vacuum_analyze_do_not_double_count_pending ANALYZE / VACUUM before the stats flush double-counts n_live_tup 6368 without_overlaps_indexdef_reads_back pg_get_indexdef of a WITHOUT OVERLAPS constraint index emits CREATE UNIQUE INDEX ... USING gist, which is rejected All four falsify on upstream master 45fc51f on every seed, shrinking to the entry's own reproducer. With the fix: 7460 passes 3/3; 5777 (its one-line fix ported from cluster.c to repack.c) passes 3/3; 6619 (whose patch predates 72a6dad1c91, built on that commit's parent) fails 3/3 there without the patch and passes 3/3 with it. The 6368 patch no longer builds on master (IndexFetchTableData). --- src/test/prop/Makefile | 1 + src/test/prop/meson.build | 1 + src/test/prop/sql/prop_cf_stats.c | 268 ++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 271 insertions(+), 1 deletion(-) create mode 100644 src/test/prop/sql/prop_cf_stats.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 68f2fd598ff42..b76f385bb8cea 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -291,6 +291,7 @@ OBJS = \ sql/prop_cf_ddl.o \ sql/prop_cf_ddl2.o \ sql/prop_cf_deparse.o \ + sql/prop_cf_stats.o \ sql/prop_cf_planner.o \ sql/index_traits.o \ sql/prop_index_am.o \ diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 79bbe88fb4d11..88fc5096ff538 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -296,6 +296,7 @@ prop_query_sources = files( 'sql/prop_cf_ddl.c', 'sql/prop_cf_ddl2.c', 'sql/prop_cf_deparse.c', + 'sql/prop_cf_stats.c', 'sql/prop_cf_planner.c', 'sql/index_traits.c', 'sql/prop_index_am.c', diff --git a/src/test/prop/sql/prop_cf_stats.c b/src/test/prop/sql/prop_cf_stats.c new file mode 100644 index 0000000000000..540f03b905c1a --- /dev/null +++ b/src/test/prop/sql/prop_cf_stats.c @@ -0,0 +1,268 @@ +/*------------------------------------------------------------------------- + * + * prop_cf_stats.c + * Statistics must agree with a model of what was done. + * + * pg_class.reltuples and the pg_stat_*_tables counters feed the planner and + * autovacuum. They are computed incrementally (per-transaction pending + * counts flushed later, VACUUM/ANALYZE estimates), so they can drift from + * the truth in ways no query result shows. Each property here performs a + * drawn sequence of operations whose effect on a statistic is known exactly, + * and compares. + * + * CF 5777 reltuples after VACUUM FULL / CLUSTER of an index-less table + * with recently-dead tuples + * CF 6619 pg_stat_xact_user_tables counts only the current transaction + * CF 7460 VACUUM / ANALYZE do not double-count pending relation stats + * CF 6368 pg_get_indexdef of a WITHOUT OVERLAPS constraint index reads back + * + * Portions Copyright (c) 2024, PostgreSQL Global Development Group + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "cf_stats" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +static pg_prop_conn * +st_connect(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + return conn; +} + +static void +st_exec(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "statement failed: %s\n SQL: %.1000s", err ? err : "", sql); +} + +static double +st_num(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool f = false; + char *err = NULL; + char *v = pg_prop_sql_scalar(conn, sql, &f, &err); + double d; + + if (f || v == NULL) + pg_prop_fail(tc, "query failed: %s\n SQL: %s", err ? err : "", sql); + d = atof(v); + pg_free(v); + return d; +} + +/* + * CF 5777: after VACUUM FULL or CLUSTER, reltuples is the number of live + * rows -- even when another session holds a snapshot that keeps some deleted + * rows "recently dead" (they are copied, but are not live). The patch's + * symptom is index-less tables counting the dead copies. Drawn: rows, + * fraction deleted, whether the table has an index, VACUUM FULL vs CLUSTER, + * whether an old snapshot is held. + */ +PG_PROP(reltuples_after_rewrite_counts_live_rows) +{ + pg_prop_conn *a = st_connect(tc); + pg_prop_conn *snap = st_connect(tc); + int n = (int) pg_prop_draw_int(tc, 10, 5000); + int del_mod = (int) pg_prop_draw_int(tc, 2, 5); + bool index = pg_prop_draw_bool(tc); + bool hold = pg_prop_draw_int(tc, 0, 3) != 0; + bool cluster = index && pg_prop_draw_bool(tc); + StringInfoData s; + double live; + double rt; + + initStringInfo(&s); + appendStringInfo(&s, + "DROP TABLE IF EXISTS cfs_rt; CREATE TABLE cfs_rt (id int, pad text) " + "WITH (autovacuum_enabled = off); " + "INSERT INTO cfs_rt SELECT i, repeat('x', 20) FROM generate_series(1, %d) i", n); + st_exec(tc, a, s.data); + if (index) + st_exec(tc, a, "CREATE INDEX cfs_rt_id ON cfs_rt (id)"); + + /* + * An old snapshot that keeps the rows we are about to delete "recently + * dead". It must not touch cfs_rt: a lock on it would block VACUUM FULL + * / CLUSTER (AccessExclusiveLock) forever. Taking a snapshot on any + * other relation holds back the xmin horizon just the same. + */ + if (hold) + { + st_exec(tc, snap, "BEGIN ISOLATION LEVEL REPEATABLE READ"); + st_exec(tc, snap, "SELECT count(*) FROM pg_class"); + } + resetStringInfo(&s); + appendStringInfo(&s, "DELETE FROM cfs_rt WHERE id %% %d = 0", del_mod); + st_exec(tc, a, s.data); + + st_exec(tc, a, cluster ? "CLUSTER cfs_rt USING cfs_rt_id" : "VACUUM FULL cfs_rt"); + live = st_num(tc, a, "SELECT count(*) FROM cfs_rt"); + rt = st_num(tc, a, "SELECT reltuples FROM pg_class WHERE relname = 'cfs_rt'"); + if (hold) + st_exec(tc, snap, "COMMIT"); + + if (rt != live) + pg_prop_fail(tc, "after %s, reltuples = %.0f but the table has %.0f live rows\n" + " %d rows, every %d-th deleted, index %s, old snapshot %s", + cluster ? "CLUSTER" : "VACUUM FULL", rt, live, n, del_mod, + index ? "yes" : "no", hold ? "held" : "none"); + + st_exec(tc, a, "DROP TABLE cfs_rt"); + pfree(s.data); + pg_prop_disconnect(snap); + pg_prop_disconnect(a); +} + +/* + * CF 6619: inside a transaction, pg_stat_xact_user_tables reports only that + * transaction's own activity. Drawn: earlier committed transactions (with + * and without stats flushed in between), then the counted transaction. + */ +PG_PROP(xact_stats_count_only_this_transaction) +{ + pg_prop_conn *c = st_connect(tc); + int prior = (int) pg_prop_draw_int(tc, 0, 3); + int mine_ins = (int) pg_prop_draw_int(tc, 0, 200); + int mine_del = (int) pg_prop_draw_int(tc, 0, 3); + StringInfoData s; + double ins; + double upd; + + initStringInfo(&s); + st_exec(tc, c, "DROP TABLE IF EXISTS cfs_x; CREATE TABLE cfs_x (id int) WITH (autovacuum_enabled = off)"); + for (int i = 0; i < prior; i++) + { + resetStringInfo(&s); + appendStringInfo(&s, "INSERT INTO cfs_x SELECT generate_series(1, %d)", + (int) pg_prop_draw_int(tc, 1, 300)); + st_exec(tc, c, s.data); + if (pg_prop_draw_bool(tc)) + st_exec(tc, c, "UPDATE cfs_x SET id = id + 1 WHERE id % 5 = 0"); + if (pg_prop_draw_bool(tc)) + st_exec(tc, c, "SELECT pg_stat_force_next_flush()"); + } + + st_exec(tc, c, "BEGIN"); + resetStringInfo(&s); + appendStringInfo(&s, "INSERT INTO cfs_x SELECT generate_series(1, %d)", mine_ins); + st_exec(tc, c, s.data); + for (int i = 0; i < mine_del; i++) + st_exec(tc, c, "DELETE FROM cfs_x WHERE id = -1"); /* touches nothing */ + ins = st_num(tc, c, "SELECT n_tup_ins FROM pg_stat_xact_user_tables WHERE relname = 'cfs_x'"); + upd = st_num(tc, c, "SELECT n_tup_upd FROM pg_stat_xact_user_tables WHERE relname = 'cfs_x'"); + st_exec(tc, c, "COMMIT"); + + if (ins != mine_ins || upd != 0) + pg_prop_fail(tc, "pg_stat_xact_user_tables inside the transaction: n_tup_ins %.0f (expected %d), " + "n_tup_upd %.0f (expected 0) after %d earlier committed transactions", + ins, mine_ins, upd, prior); + st_exec(tc, c, "DROP TABLE cfs_x"); + pfree(s.data); + pg_prop_disconnect(c); +} + +/* + * CF 7460: ANALYZE or VACUUM run before a session's pending stats are + * flushed must not count those rows twice: n_live_tup after the flush equals + * the live row count (no updates or deletes leave estimates aside), and + * n_dead_tup equals the rows deleted. Drawn: rows, deletes, which of + * ANALYZE / VACUUM runs before the flush. + */ +PG_PROP(vacuum_analyze_do_not_double_count_pending) +{ + pg_prop_conn *c = st_connect(tc); + int n = (int) pg_prop_draw_int(tc, 1, 3000); + int del = (int) pg_prop_draw_int(tc, 0, n); + int op = (int) pg_prop_draw_int(tc, 0, 2); + StringInfoData s; + double livet; + double deadt; + + initStringInfo(&s); + st_exec(tc, c, "DROP TABLE IF EXISTS cfs_dc; CREATE TABLE cfs_dc (id int) WITH (autovacuum_enabled = off)"); + st_exec(tc, c, "SELECT pg_stat_force_next_flush()"); + resetStringInfo(&s); + appendStringInfo(&s, "INSERT INTO cfs_dc SELECT generate_series(1, %d)", n); + st_exec(tc, c, s.data); + resetStringInfo(&s); + appendStringInfo(&s, "DELETE FROM cfs_dc WHERE id <= %d", del); + st_exec(tc, c, s.data); + /* the operation that runs while this session's counts are still pending */ + st_exec(tc, c, op == 0 ? "ANALYZE cfs_dc" : op == 1 ? "VACUUM cfs_dc" : "VACUUM ANALYZE cfs_dc"); + st_exec(tc, c, "SELECT pg_stat_force_next_flush()"); + /* a fresh statement makes the flushed counts visible */ + st_exec(tc, c, "SELECT pg_stat_clear_snapshot()"); + livet = st_num(tc, c, "SELECT n_live_tup FROM pg_stat_user_tables WHERE relname = 'cfs_dc'"); + deadt = st_num(tc, c, "SELECT n_dead_tup FROM pg_stat_user_tables WHERE relname = 'cfs_dc'"); + + if (livet > (n - del) || livet < 0) + pg_prop_fail(tc, "n_live_tup = %.0f exceeds the %d live rows after %s ran with pending stats\n" + " inserted %d, deleted %d", livet, n - del, + op == 0 ? "ANALYZE" : op == 1 ? "VACUUM" : "VACUUM ANALYZE", n, del); + if (op != 0 && deadt > del) + pg_prop_fail(tc, "n_dead_tup = %.0f exceeds the %d deleted rows after %s ran with pending stats", + deadt, del, op == 1 ? "VACUUM" : "VACUUM ANALYZE"); + st_exec(tc, c, "DROP TABLE cfs_dc"); + pfree(s.data); + pg_prop_disconnect(c); +} + +/* + * CF 6368: the definition pg_get_indexdef prints for a WITHOUT OVERLAPS + * constraint's index must be accepted when re-executed (pg_dump relies on + * it). Drawn: PK vs UNIQUE, range vs multirange, extra key columns. + */ +PG_PROP(without_overlaps_indexdef_reads_back) +{ + pg_prop_conn *c = st_connect(tc); + bool pk = pg_prop_draw_bool(tc); + bool multi = pg_prop_draw_bool(tc); + bool extra = pg_prop_draw_bool(tc); + StringInfoData s; + char *def; + bool f = false; + char *err = NULL; + + if (!pg_prop_require_extension(c, "btree_gist")) + pg_prop_skip(tc, "btree_gist not installed"); + initStringInfo(&s); + appendStringInfo(&s, + "DROP TABLE IF EXISTS cfs_wo; CREATE TABLE cfs_wo (id int, k int, valid_at %s, " + "CONSTRAINT cfs_wo_key %s (id%s, valid_at WITHOUT OVERLAPS))", + multi ? "int4multirange" : "int4range", pk ? "PRIMARY KEY" : "UNIQUE", + extra ? ", k" : ""); + st_exec(tc, c, s.data); + def = pg_prop_sql_scalar(c, + "SELECT pg_get_indexdef(conindid) FROM pg_constraint WHERE conname = 'cfs_wo_key'", + &f, &err); + if (f || def == NULL) + pg_prop_fail(tc, "pg_get_indexdef failed: %s", err ? err : ""); + /* re-run the definition under a different name */ + resetStringInfo(&s); + appendStringInfo(&s, "%s", def); + { + char *p = strstr(s.data, "cfs_wo_key"); + + if (p) + memcpy(p, "cfs_wo_kez", 10); + } + if (!pg_prop_sql_exec(c, s.data, &err)) + pg_prop_fail(tc, "pg_get_indexdef's output for a WITHOUT OVERLAPS %s does not read back: %s\n %s", + pk ? "PRIMARY KEY" : "UNIQUE", err ? err : "", def); + pg_free(def); + st_exec(tc, c, "DROP TABLE cfs_wo"); + pfree(s.data); + pg_prop_disconnect(c); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index acbb4f90787ff..660d6afce9942 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model syscol collation cf_types cf_index cf_exec cf_ddl cf_ddl2 cf_deparse cf_planner index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model syscol collation cf_types cf_index cf_exec cf_ddl cf_ddl2 cf_deparse cf_stats cf_planner index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 2c3efd2604b29b08f7da5ff526bd2f33527a8e97 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sun, 18 Oct 2026 12:18:40 -0400 Subject: [PATCH 316/338] prop: two-session race properties (cf_race) Each property drives two connections through a drawn interleaving with the async primitives, waiting for the second backend to block on a lock where the race needs it, and checks an invariant that holds in every order: 7083 ssi_scan_paths_prevent_write_skew two SERIALIZABLE transactions read through a drawn access path and each update the row the other read; both commit through TID and TID range scans (no predicate lock) 7197 ssi_primary_key_stays_unique s1 reads k=1, s2 deletes that row through a secondary index and commits, s1 inserts k=1: two rows with k=1 under a PRIMARY KEY 5681 concurrent_function_ddl_is_not_internal concurrent CREATE OR REPLACE / ALTER FUNCTION: "tuple concurrently updated" 7353 create_index_races_index_attach CREATE INDEX on a partitioned table racing ALTER INDEX ATTACH PARTITION: "bogus pg_inherit row" 7303 alter_sequence_as_races_reads (guard) ALTER SEQUENCE ... AS racing reads of the sequence; the bug report's XX001 did not reproduce on FreeBSD/arm64 with io_method=worker, by this property or by pgbench at 7.7M reads against 6.9k rewrites The first four falsify on upstream master 45fc51f on every seed and pass 3/3 with their commitfest patch. --- src/test/prop/Makefile | 1 + src/test/prop/meson.build | 1 + src/test/prop/sql/prop_cf_race.c | 446 +++++++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 449 insertions(+), 1 deletion(-) create mode 100644 src/test/prop/sql/prop_cf_race.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index b76f385bb8cea..18833d64d28bf 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -293,6 +293,7 @@ OBJS = \ sql/prop_cf_deparse.o \ sql/prop_cf_stats.o \ sql/prop_cf_planner.o \ + sql/prop_cf_race.o \ sql/index_traits.o \ sql/prop_index_am.o \ sql/prop_table_am.o \ diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 88fc5096ff538..4cd9c8fb4f241 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -298,6 +298,7 @@ prop_query_sources = files( 'sql/prop_cf_deparse.c', 'sql/prop_cf_stats.c', 'sql/prop_cf_planner.c', + 'sql/prop_cf_race.c', 'sql/index_traits.c', 'sql/prop_index_am.c', 'sql/prop_table_am.c', diff --git a/src/test/prop/sql/prop_cf_race.c b/src/test/prop/sql/prop_cf_race.c new file mode 100644 index 0000000000000..8314ad8bbd04c --- /dev/null +++ b/src/test/prop/sql/prop_cf_race.c @@ -0,0 +1,446 @@ +/*------------------------------------------------------------------------- + * + * prop_cf_race.c + * Two-session interleavings from the open commitfest. + * + * Each property drives two (or three) connections through a drawn + * interleaving with pg_prop_sql_send / pg_prop_sql_await, and checks an + * invariant that must hold whatever the order: SERIALIZABLE transactions + * that both commit have a serial order, a primary key stays unique, DDL that + * races DDL either succeeds or fails with a user-facing error -- never an + * internal one -- and the catalog it leaves behind is consistent. + * + * CF 7083 write skew through TID / TID-range / bitmap scans under SSI + * CF 7197 duplicate primary key under SSI after a delete via another index + * CF 5681 concurrent CREATE OR REPLACE / ALTER FUNCTION: internal error + * CF 7303 ALTER SEQUENCE ... AS racing a read of the sequence (guard: the + * XX001 did not reproduce on FreeBSD/arm64, io_method=worker) + * CF 7353 CREATE INDEX on a partitioned table racing ATTACH PARTITION + * + * Portions Copyright (c) 2024, PostgreSQL Global Development Group + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "cf_race" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +static pg_prop_conn * +rc_connect(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + return conn; +} + +static void +rc_exec(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + char *err = NULL; + + if (!pg_prop_sql_exec(conn, sql, &err)) + pg_prop_fail(tc, "statement failed: %s\n SQL: %.1000s", err ? err : "", sql); +} + +static char * +rc_scalar(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) +{ + bool f = false; + char *err = NULL; + char *v = pg_prop_sql_scalar(conn, sql, &f, &err); + + if (f || v == NULL) + pg_prop_fail(tc, "query failed: %s\n SQL: %s", err ? err : "", sql); + return v; +} + +/* backend pid of a connection */ +static char * +rc_pid(pg_prop_tc *tc, pg_prop_conn *conn) +{ + return rc_scalar(tc, conn, "SELECT pg_backend_pid()"); +} + +/* + * Wait (up to ~10 s) until backend "pid" is blocked on a lock; returns false + * as soon as it is no longer running a statement (it did not need to block, + * which is a legal interleaving). + */ +static bool +rc_wait_blocked(pg_prop_tc *tc, pg_prop_conn *obs, const char *pid) +{ + char q[200]; + + snprintf(q, sizeof(q), + "SELECT CASE WHEN cardinality(pg_blocking_pids(%s)) > 0 THEN 'b' " + "WHEN EXISTS (SELECT 1 FROM pg_stat_activity WHERE pid = %s AND state <> 'active') " + "THEN 'd' ELSE 'r' END", + pid, pid); + for (int i = 0; i < 200; i++) + { + char *v; + char c; + + /* obs may be inside a transaction: its activity snapshot is cached */ + pg_free(rc_scalar(tc, obs, "SELECT pg_stat_clear_snapshot()")); + v = rc_scalar(tc, obs, q); + c = v[0]; + + pg_free(v); + if (c == 'b') + return true; + if (c == 'd' && i > 0) + return false; + pg_free(rc_scalar(tc, obs, "SELECT pg_sleep(0.05)")); + } + return false; +} + +/* an error that only an internal bug produces */ +static bool +rc_internal(const char *sqlstate, const char *err) +{ + return (sqlstate && (strcmp(sqlstate, "XX000") == 0 || strcmp(sqlstate, "XX001") == 0 || + strcmp(sqlstate, "XX002") == 0)) || + (err && (strstr(err, "tuple concurrently updated") || strstr(err, "bogus") || + strstr(err, "cache lookup failed") || strstr(err, "could not read block"))); +} + +static bool +rc_serfail(const char *sqlstate) +{ + return sqlstate && (strcmp(sqlstate, "40001") == 0 || strcmp(sqlstate, "40P01") == 0); +} + +/* + * CF 7083: two SERIALIZABLE transactions each read two rows through a drawn + * access path, then update the row the other read. That is write skew: SSI + * must abort one of them. On master the TID scan and TID range scan paths + * take no predicate lock, and both commit. + * + * Invariant: if both commit, the result equals some serial order -- here, + * at least one transaction must have seen the other's write, which with + * this schedule (both reads before both writes) is impossible, so at least + * one must fail with a serialization failure. + */ +PG_PROP(ssi_scan_paths_prevent_write_skew) +{ + static const char *const paths[] = {"tid", "tidrange", "bitmap", "index", "seq"}; + pg_prop_conn *s1 = rc_connect(tc); + pg_prop_conn *s2 = rc_connect(tc); + int path = (int) pg_prop_draw_int(tc, 0, lengthof(paths) - 1); + int nrows = (int) pg_prop_draw_int(tc, 2, 40); + bool first1 = pg_prop_draw_bool(tc); + bool commit1first = pg_prop_draw_bool(tc); + const char *read_q; + const char *gucs; + char *st1 = NULL; + char *st2 = NULL; + char *e1 = NULL; + char *e2 = NULL; + bool ok1; + bool ok2; + bool ok; + StringInfoData s; + + initStringInfo(&s); + appendStringInfo(&s, + "DROP TABLE IF EXISTS cfr_ws; CREATE TABLE cfr_ws (i int PRIMARY KEY, t text); " + "INSERT INTO cfr_ws SELECT g, 'v' FROM generate_series(1, %d) g", nrows); + rc_exec(tc, s1, s.data); + + switch (path) + { + case 0: + read_q = "SELECT * FROM cfr_ws WHERE ctid IN ('(0,1)', '(0,2)')"; + gucs = "SET LOCAL enable_seqscan = off; SET LOCAL enable_tidscan = on"; + break; + case 1: + read_q = "SELECT * FROM cfr_ws WHERE ctid >= '(0,1)' AND ctid <= '(0,2)'"; + gucs = "SET LOCAL enable_seqscan = off; SET LOCAL enable_tidscan = on"; + break; + case 2: + read_q = "SELECT * FROM cfr_ws WHERE i IN (1, 2)"; + gucs = "SET LOCAL enable_seqscan = off; SET LOCAL enable_indexscan = off"; + break; + case 3: + read_q = "SELECT * FROM cfr_ws WHERE i IN (1, 2)"; + gucs = "SET LOCAL enable_seqscan = off; SET LOCAL enable_bitmapscan = off"; + break; + default: + read_q = "SELECT * FROM cfr_ws WHERE i IN (1, 2)"; + gucs = "SET LOCAL enable_indexscan = off; SET LOCAL enable_bitmapscan = off"; + break; + } + + rc_exec(tc, s1, "BEGIN ISOLATION LEVEL SERIALIZABLE"); + rc_exec(tc, s2, "BEGIN ISOLATION LEVEL SERIALIZABLE"); + rc_exec(tc, s1, gucs); + rc_exec(tc, s2, gucs); + rc_exec(tc, first1 ? s1 : s2, read_q); + rc_exec(tc, first1 ? s2 : s1, read_q); + + /* each writes the row the other read but it did not write */ + ok1 = pg_prop_sql_send(s1, "UPDATE cfr_ws SET t = 'x1' WHERE i = 2"); + ok = ok1 && pg_prop_sql_await(s1, &st1, &e1); + ok1 = ok; + ok2 = pg_prop_sql_send(s2, "UPDATE cfr_ws SET t = 'x2' WHERE i = 1"); + ok = ok2 && pg_prop_sql_await(s2, &st2, &e2); + ok2 = ok; + + if (commit1first) + { + if (ok1) + ok1 = pg_prop_sql_send(s1, "COMMIT") && pg_prop_sql_await(s1, &st1, &e1); + if (ok2) + ok2 = pg_prop_sql_send(s2, "COMMIT") && pg_prop_sql_await(s2, &st2, &e2); + } + else + { + if (ok2) + ok2 = pg_prop_sql_send(s2, "COMMIT") && pg_prop_sql_await(s2, &st2, &e2); + if (ok1) + ok1 = pg_prop_sql_send(s1, "COMMIT") && pg_prop_sql_await(s1, &st1, &e1); + } + if (!ok1) + pg_prop_sql_exec(s1, "ROLLBACK", NULL); + if (!ok2) + pg_prop_sql_exec(s2, "ROLLBACK", NULL); + + if ((!ok1 && !rc_serfail(st1)) || (!ok2 && !rc_serfail(st2))) + pg_prop_fail(tc, "unexpected error under SERIALIZABLE (%s scan): %s / %s", + paths[path], e1 ? e1 : "-", e2 ? e2 : "-"); + if (ok1 && ok2) + { + /* is the outcome serialisable? both read old values of both rows */ + char *v = rc_scalar(tc, s1, "SELECT string_agg(t, ',' ORDER BY i) FROM cfr_ws WHERE i IN (1, 2)"); + + pg_prop_fail(tc, "write skew: both SERIALIZABLE transactions committed after reading " + "through a %s scan; rows are now %s", paths[path], v); + } + rc_exec(tc, s1, "DROP TABLE cfr_ws"); + pfree(s.data); + pg_prop_disconnect(s2); + pg_prop_disconnect(s1); +} + +/* + * CF 7197: s1 (SERIALIZABLE) looks up k = 1 and finds it; s2 deletes that + * row through a secondary index and commits; s1 inserts k = 1. Whatever SSI + * decides, the table must never hold two rows with k = 1, and s1 must not + * see two. Drawn: the uniqueness mechanism (PK, UNIQUE, EXCLUDE), ON + * CONFLICT DO NOTHING vs plain INSERT, filler size. + */ +PG_PROP(ssi_primary_key_stays_unique) +{ + static const char *const keys[] = { + "k int PRIMARY KEY, j int", "k int UNIQUE, j int", "k int, j int, EXCLUDE USING btree (k WITH =)", + }; + pg_prop_conn *s1 = rc_connect(tc); + pg_prop_conn *s2 = rc_connect(tc); + int key = (int) pg_prop_draw_int(tc, 0, lengthof(keys) - 1); + bool onconf = pg_prop_draw_bool(tc); + int fill = (int) pg_prop_draw_int(tc, 0, 3000); + char *st = NULL; + char *e = NULL; + char *seen; + char *n; + bool ok; + StringInfoData s; + + initStringInfo(&s); + appendStringInfo(&s, + "DROP TABLE IF EXISTS cfr_pk; CREATE TABLE cfr_pk (%s); " + "INSERT INTO cfr_pk VALUES (1, 1000000); " + "INSERT INTO cfr_pk SELECT g, g FROM generate_series(100, 100 + %d) g; " + "CREATE INDEX cfr_pk_j ON cfr_pk (j); ANALYZE cfr_pk", + keys[key], fill); + rc_exec(tc, s1, s.data); + + rc_exec(tc, s1, "BEGIN ISOLATION LEVEL SERIALIZABLE; SET LOCAL enable_seqscan = off"); + rc_exec(tc, s2, "BEGIN ISOLATION LEVEL SERIALIZABLE; SET LOCAL enable_seqscan = off"); + rc_exec(tc, s1, "SELECT k, j FROM cfr_pk WHERE k = 1"); + rc_exec(tc, s2, "DELETE FROM cfr_pk WHERE j = 1000000"); + rc_exec(tc, s2, "COMMIT"); + ok = pg_prop_sql_send(s1, onconf ? "INSERT INTO cfr_pk VALUES (1, 2) ON CONFLICT DO NOTHING" + : "INSERT INTO cfr_pk VALUES (1, 2)") && + pg_prop_sql_await(s1, &st, &e); + if (ok) + { + seen = rc_scalar(tc, s1, "SELECT count(*) FROM cfr_pk WHERE k = 1"); + if (strcmp(seen, "1") > 0) + pg_prop_fail(tc, "transaction sees %s rows with k = 1 after its INSERT (%s)", seen, keys[key]); + pg_free(seen); + ok = pg_prop_sql_send(s1, "COMMIT") && pg_prop_sql_await(s1, &st, &e); + } + if (!ok) + { + pg_prop_sql_exec(s1, "ROLLBACK", NULL); + if (!rc_serfail(st) && !(st && strcmp(st, "23505") == 0) && !(st && strcmp(st, "23P01") == 0)) + pg_prop_fail(tc, "unexpected error: %s", e ? e : ""); + } + n = rc_scalar(tc, s1, "SELECT count(*) FROM cfr_pk WHERE k = 1"); + if (strcmp(n, "1") != 0 && strcmp(n, "0") != 0) + pg_prop_fail(tc, "after commit the table holds %s rows with k = 1 (%s, %s)", + n, keys[key], onconf ? "ON CONFLICT DO NOTHING" : "INSERT"); + pg_free(n); + rc_exec(tc, s1, "DROP TABLE cfr_pk"); + pfree(s.data); + pg_prop_disconnect(s2); + pg_prop_disconnect(s1); +} + +/* + * CF 5681: two sessions redefine (or ALTER) the same function concurrently. + * One waits for the other; when it proceeds it must succeed or raise a + * user-facing error -- never "tuple concurrently updated". Drawn: the two + * DDL statements, commit vs rollback of the first. + */ +PG_PROP(concurrent_function_ddl_is_not_internal) +{ + static const char *const ddl[] = { + "CREATE OR REPLACE FUNCTION cfr_f(int) RETURNS int LANGUAGE sql AS 'SELECT $1 + 1'", + "CREATE OR REPLACE FUNCTION cfr_f(int) RETURNS int LANGUAGE sql AS 'SELECT $1 + 2'", + "ALTER FUNCTION cfr_f(int) IMMUTABLE", + "ALTER FUNCTION cfr_f(int) COST 7", + "ALTER FUNCTION cfr_f(int) SET search_path = public", + "COMMENT ON FUNCTION cfr_f(int) IS 'x'", + }; + pg_prop_conn *s1 = rc_connect(tc); + pg_prop_conn *s2 = rc_connect(tc); + const char *d1 = ddl[pg_prop_draw_int(tc, 0, lengthof(ddl) - 1)]; + const char *d2 = ddl[pg_prop_draw_int(tc, 0, lengthof(ddl) - 1)]; + bool commit = pg_prop_draw_int(tc, 0, 3) != 0; + char *pid2; + char *st = NULL; + char *e = NULL; + bool ok; + + rc_exec(tc, s1, "DROP FUNCTION IF EXISTS cfr_f(int); " + "CREATE FUNCTION cfr_f(int) RETURNS int LANGUAGE sql AS 'SELECT $1'"); + pid2 = rc_pid(tc, s2); + rc_exec(tc, s1, "BEGIN"); + rc_exec(tc, s1, d1); + if (!pg_prop_sql_send(s2, d2)) + pg_prop_fail(tc, "could not dispatch"); + rc_wait_blocked(tc, s1, pid2); + rc_exec(tc, s1, commit ? "COMMIT" : "ROLLBACK"); + ok = pg_prop_sql_await(s2, &st, &e); + if (!ok && rc_internal(st, e)) + pg_prop_fail(tc, "concurrent function DDL raised an internal error: %s\n" + " s1: %s (%s)\n s2: %s", e ? e : "", d1, commit ? "COMMIT" : "ROLLBACK", d2); + pg_free(pid2); + pg_prop_sql_exec(s1, "DROP FUNCTION IF EXISTS cfr_f(int)", NULL); + pg_prop_disconnect(s2); + pg_prop_disconnect(s1); +} + +/* + * CF 7303: ALTER SEQUENCE ... AS rewrites the sequence relfilenode; + * a concurrent read of the sequence must see the old or new state, never + * fail to read the relation. Drawn: the sequence of AS types, the reads. + */ +PG_PROP(alter_sequence_as_races_reads) +{ + static const char *const types[] = {"int", "bigint", "smallint"}; + static const char *const reads[] = { + "SELECT * FROM cfr_seq", "SELECT last_value FROM cfr_seq", "SELECT nextval('cfr_seq')", + "SELECT * FROM pg_sequence_last_value('cfr_seq')", + }; + pg_prop_conn *s1 = rc_connect(tc); + pg_prop_conn *s2 = rc_connect(tc); + int rounds = (int) pg_prop_draw_int(tc, 5, 60); + char *st = NULL; + char *e = NULL; + char q[120]; + + rc_exec(tc, s1, "DROP SEQUENCE IF EXISTS cfr_seq; CREATE SEQUENCE cfr_seq"); + for (int i = 0; i < rounds; i++) + { + snprintf(q, sizeof(q), "ALTER SEQUENCE cfr_seq AS %s", + types[pg_prop_draw_int(tc, 0, lengthof(types) - 1)]); + if (!pg_prop_sql_send(s1, q)) + pg_prop_fail(tc, "could not dispatch"); + if (!pg_prop_sql_send(s2, reads[pg_prop_draw_int(tc, 0, lengthof(reads) - 1)])) + pg_prop_fail(tc, "could not dispatch"); + if (!pg_prop_sql_await(s2, &st, &e) && rc_internal(st, e)) + pg_prop_fail(tc, "a read racing ALTER SEQUENCE ... AS failed: %s (round %d)", e ? e : "", i); + if (!pg_prop_sql_await(s1, &st, &e) && rc_internal(st, e)) + pg_prop_fail(tc, "ALTER SEQUENCE ... AS racing a read failed: %s (round %d)", e ? e : "", i); + } + rc_exec(tc, s1, "DROP SEQUENCE cfr_seq"); + pg_prop_disconnect(s2); + pg_prop_disconnect(s1); +} + +/* + * CF 7353: CREATE INDEX on a partitioned table while another session (in an + * open transaction) attaches or renames a partition's index. The CREATE + * INDEX must succeed or fail with a user-facing error, and afterwards every + * partition index is attached to at most one parent index. + */ +PG_PROP(create_index_races_index_attach) +{ + pg_prop_conn *s1 = rc_connect(tc); + pg_prop_conn *s2 = rc_connect(tc); + int nparts = (int) pg_prop_draw_int(tc, 2, 4); + int action = (int) pg_prop_draw_int(tc, 0, 2); + int attached = (int) pg_prop_draw_int(tc, 1, nparts - 1); + bool commit = pg_prop_draw_int(tc, 0, 3) != 0; + char *pid2; + char *st = NULL; + char *e = NULL; + char *bad; + StringInfoData s; + + initStringInfo(&s); + appendStringInfoString(&s, "DROP TABLE IF EXISTS cfr_pia; CREATE TABLE cfr_pia (a int) PARTITION BY RANGE (a); "); + for (int i = 0; i < nparts; i++) + appendStringInfo(&s, "CREATE TABLE cfr_pia_%d PARTITION OF cfr_pia FOR VALUES FROM (%d) TO (%d); ", + i, i * 10, i * 10 + 10); + appendStringInfoString(&s, "CREATE INDEX cfr_pia_old ON ONLY cfr_pia (a); "); + for (int i = 0; i < nparts; i++) + appendStringInfo(&s, "CREATE INDEX cfr_pia_%d_a ON cfr_pia_%d (a); ", i, i); + for (int i = attached; i < nparts; i++) + appendStringInfo(&s, "ALTER INDEX cfr_pia_old ATTACH PARTITION cfr_pia_%d_a; ", i); + rc_exec(tc, s1, s.data); + + pid2 = rc_pid(tc, s2); + rc_exec(tc, s1, "BEGIN"); + resetStringInfo(&s); + if (action == 0) + appendStringInfo(&s, "ALTER INDEX cfr_pia_old ATTACH PARTITION cfr_pia_%d_a", + (int) pg_prop_draw_int(tc, 0, attached - 1)); + else if (action == 1) + appendStringInfo(&s, "ALTER INDEX cfr_pia_%d_a RENAME TO cfr_pia_renamed", + (int) pg_prop_draw_int(tc, attached, nparts - 1)); + else + appendStringInfoString(&s, "SELECT count(*) FROM cfr_pia"); + rc_exec(tc, s1, s.data); + if (!pg_prop_sql_send(s2, "CREATE INDEX cfr_pia_new ON cfr_pia (a)")) + pg_prop_fail(tc, "could not dispatch"); + rc_wait_blocked(tc, s1, pid2); + rc_exec(tc, s1, commit ? "COMMIT" : "ROLLBACK"); + if (!pg_prop_sql_await(s2, &st, &e) && rc_internal(st, e)) + pg_prop_fail(tc, "CREATE INDEX racing \"%s\" (%s) raised an internal error: %s", + s.data, commit ? "COMMIT" : "ROLLBACK", e ? e : ""); + + bad = rc_scalar(tc, s1, + "SELECT count(*) FROM (SELECT inhrelid FROM pg_inherits i JOIN pg_class c ON c.oid = i.inhrelid " + "WHERE c.relkind = 'i' AND c.relname LIKE 'cfr_pia%' GROUP BY inhrelid HAVING count(*) > 1) x"); + if (strcmp(bad, "0") != 0) + pg_prop_fail(tc, "%s partition indexes have more than one parent after CREATE INDEX raced \"%s\"", + bad, s.data); + pg_free(bad); + pg_free(pid2); + rc_exec(tc, s1, "DROP TABLE cfr_pia"); + pfree(s.data); + pg_prop_disconnect(s2); + pg_prop_disconnect(s1); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 660d6afce9942..56ad98b3e923f 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model syscol collation cf_types cf_index cf_exec cf_ddl cf_ddl2 cf_deparse cf_stats cf_planner index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model syscol collation cf_types cf_index cf_exec cf_ddl cf_ddl2 cf_deparse cf_stats cf_planner cf_race index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 8fe13e12eb6a2f7ab89cbc6e2542959958b80b58 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sun, 18 Oct 2026 12:38:12 -0400 Subject: [PATCH 317/338] prop: slot, plan-cache and decoding properties (cf_exec) 7347 failed_temp_slot_creation_leaves_no_slot a temporary logical slot whose creation fails inside a caught subtransaction (output plugin fails to load) is left behind. On master the second attempt trips the 7097 assertion first; with 7097 alone the slot is still listed (3/3); with 7097 + 7347 it passes 3/3. 6828 unreserved_slot_has_no_safe_wal_size a physical slot created without reserving WAL reports a garbage safe_wal_size under a finite max_slot_wal_keep_size; 3/3 fail on master, 3/3 pass with the patch 7432 caught_call_error_does_not_pin_plans a CALL error caught in a non-atomic loop keeps its CachedPlan pinned; 3/3 fail on master, 3/3 pass with the patch 7404 streaming_decode_with_toasted_constraint (guard) streaming decode of an open transaction on a table with a TOASTed CHECK constraint, from a cold-relcache backend; did not reproduce on FreeBSD/arm64, nor did the bug report's own script The slot properties need wal_level = logical and skip otherwise. --- src/test/prop/sql/prop_cf_exec.c | 227 +++++++++++++++++++++++++++++++ 1 file changed, 227 insertions(+) diff --git a/src/test/prop/sql/prop_cf_exec.c b/src/test/prop/sql/prop_cf_exec.c index 57b017255e722..7de4937074bc8 100644 --- a/src/test/prop/sql/prop_cf_exec.c +++ b/src/test/prop/sql/prop_cf_exec.c @@ -1080,3 +1080,230 @@ PG_PROP(cached_return_query_sees_widened_composite) pfree(s.data); pg_prop_disconnect(conn); } + +/* + * CF 7347 (on top of 7097): a temporary logical slot whose creation fails + * inside a subtransaction (the output plugin fails to load; the error is + * caught by an EXCEPTION block) must not be left behind: no slot of that + * name may be listed and the name must be reusable. Drawn: the failure, + * how many attempts. Needs wal_level = logical. On master the second + * attempt trips the CF 7097 assertion first, so validate on 7097 + 7347. + */ +PG_PROP(failed_temp_slot_creation_leaves_no_slot) +{ + /* + * Each fails while loading the output plugin, i.e. after the slot was + * created. (An error raised after a successful creation is not a case: + * slot creation is not transactional, the slot is meant to survive.) + */ + static const char *const bad[] = { + "PERFORM pg_create_logical_replication_slot('cfa4_tmp', 'no_such_plugin', true)", + "PERFORM pg_create_logical_replication_slot('cfa4_tmp', 'plpgsql', true)", + }; + pg_prop_conn *conn = cfe_connect(tc); + bool failed = false; + char *wl = pg_prop_sql_scalar(conn, "SHOW wal_level", &failed, NULL); + int tries = (int) pg_prop_draw_int(tc, 1, 3); + int which = (int) pg_prop_draw_int(tc, 0, lengthof(bad) - 1); + StringInfoData s; + char *n; + + if (wl == NULL || strcmp(wl, "logical") != 0) + pg_prop_skip(tc, "needs wal_level = logical"); + pg_free(wl); + pg_prop_sql_exec(conn, "SELECT pg_drop_replication_slot(slot_name) FROM pg_replication_slots " + "WHERE slot_name = 'cfa4_tmp' AND active_pid IS NULL", NULL); + initStringInfo(&s); + for (int i = 0; i < tries; i++) + { + resetStringInfo(&s); + appendStringInfo(&s, "DO $$ BEGIN %s; EXCEPTION WHEN OTHERS THEN NULL; END $$", bad[which]); + cfe_try_noint(tc, conn, s.data, "CF 7347 failed temporary slot creation"); + } + n = pg_prop_sql_scalar(conn, "SELECT count(*) FROM pg_replication_slots WHERE slot_name = 'cfa4_tmp'", + &failed, NULL); + if (n && strcmp(n, "0") != 0) + pg_prop_fail(tc, "a temporary slot whose creation was rolled back is still listed (%s)\n %s", + n, bad[which]); + if (n) + pg_free(n); + /* the name must be free for a real slot */ + cfe_try_noint(tc, conn, "SELECT 'init' FROM pg_create_logical_replication_slot('cfa4_tmp', 'test_decoding', true)", + "CF 7347 reuse of the name"); + pg_prop_sql_exec(conn, "SELECT pg_drop_replication_slot('cfa4_tmp')", NULL); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* + * CF 6828: a physical slot created without reserving WAL has no restart_lsn; + * with a finite max_slot_wal_keep_size its wal_status and safe_wal_size must + * still be NULL (there is nothing to be safe about). The setting is a + * SIGHUP GUC, so the property sets it with ALTER SYSTEM and reloads, then + * resets it. Drawn: the keep size, reserve or not. + */ +PG_PROP(unreserved_slot_has_no_safe_wal_size) +{ + pg_prop_conn *conn = cfe_connect(tc); + int mb = (int) pg_prop_draw_int(tc, 1, 64); + bool reserve = pg_prop_draw_int(tc, 0, 3) == 0; + StringInfoData s; + char *r; + bool failed = false; + + initStringInfo(&s); + appendStringInfo(&s, "ALTER SYSTEM SET max_slot_wal_keep_size = '%dMB'", mb); + cfe_exec(tc, conn, s.data); + cfe_exec(tc, conn, "SELECT pg_reload_conf()"); + pg_prop_sql_exec(conn, "SELECT pg_drop_replication_slot('cfa4_phys') FROM pg_replication_slots " + "WHERE slot_name = 'cfa4_phys'", NULL); + resetStringInfo(&s); + appendStringInfo(&s, "SELECT pg_create_physical_replication_slot('cfa4_phys', %s)", + reserve ? "true" : "false"); + cfe_exec(tc, conn, s.data); + /* the reload is asynchronous: wait until this backend sees the value */ + for (int i = 0; i < 100; i++) + { + char *v = pg_prop_sql_scalar(conn, "SHOW max_slot_wal_keep_size", &failed, NULL); + bool done = v && strcmp(v, "-1") != 0; + + if (v) + pg_free(v); + if (done) + break; + cfe_exec(tc, conn, "SELECT pg_sleep(0.02)"); + } + r = pg_prop_sql_scalar(conn, + "SELECT (restart_lsn IS NULL)::text || '|' || coalesce(wal_status, 'NULL') || '|' || " + "coalesce(safe_wal_size::text, 'NULL') FROM pg_replication_slots WHERE slot_name = 'cfa4_phys'", + &failed, NULL); + pg_prop_sql_exec(conn, "SELECT pg_drop_replication_slot('cfa4_phys')", NULL); + pg_prop_sql_exec(conn, "ALTER SYSTEM RESET max_slot_wal_keep_size", NULL); + pg_prop_sql_exec(conn, "SELECT pg_reload_conf()", NULL); + if (r && strncmp(r, "true|", 5) == 0 && strcmp(r, "true|NULL|NULL") != 0) + pg_prop_fail(tc, "slot without restart_lsn reports restart_lsn-null|wal_status|safe_wal_size = %s " + "(max_slot_wal_keep_size %dMB)", r, mb); + if (r) + pg_free(r); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7432: a CALL whose error is caught by an exception block, in a + * non-atomic loop, must not keep its CachedPlan pinned: the number of + * CachedPlan contexts after the second round of N caught calls equals the + * number after the first. Drawn: N, the failing callee's shape. + */ +PG_PROP(caught_call_error_does_not_pin_plans) +{ + static const char *const bodies[] = { + "RAISE EXCEPTION 'error %', x", + "PERFORM 1 / (x - x)", + "INSERT INTO cfa4_never_exists VALUES (x)", + }; + pg_prop_conn *conn = cfe_connect(tc); + int n = (int) pg_prop_draw_int(tc, 2, 40); + const char *body = bodies[pg_prop_draw_int(tc, 0, lengthof(bodies) - 1)]; + StringInfoData s; + char *r; + bool failed = false; + + initStringInfo(&s); + appendStringInfo(&s, + "CREATE OR REPLACE PROCEDURE cfa4_err(x int) LANGUAGE plpgsql AS $p$ BEGIN %s; END $p$", + body); + cfe_exec(tc, conn, s.data); + resetStringInfo(&s); + appendStringInfo(&s, + "CREATE OR REPLACE PROCEDURE cfa4_catch(INOUT kept int DEFAULT 0) LANGUAGE plpgsql AS $p$ " + "DECLARE b int; a int; BEGIN " + "FOR round IN 1..2 LOOP " + "b := (SELECT count(*) FROM pg_backend_memory_contexts WHERE name = 'CachedPlan'); " + "FOR i IN 1..%d LOOP BEGIN CALL cfa4_err(i); EXCEPTION WHEN OTHERS THEN NULL; END; END LOOP; " + "a := (SELECT count(*) FROM pg_backend_memory_contexts WHERE name = 'CachedPlan'); " + "END LOOP; kept := a - b; END $p$", n); + cfe_exec(tc, conn, s.data); + r = pg_prop_sql_scalar(conn, "CALL cfa4_catch()", &failed, NULL); + if (r == NULL || failed) + pg_prop_fail(tc, "CALL cfa4_catch() failed"); + if (strcmp(r, "0") != 0) + pg_prop_fail(tc, "%s CachedPlans are still pinned after %d caught CALL errors (callee: %s)", + r, n, body); + pg_free(r); + pg_prop_sql_exec(conn, "DROP PROCEDURE cfa4_catch(int); DROP PROCEDURE cfa4_err(int)", NULL); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7404: streaming logical decoding of an in-progress transaction on a + * table whose CHECK constraint is TOASTed in pg_constraint (and is followed + * by more constraints in scan order) must decode, not fail with "unexpected + * table_index_fetch_tuple call during logical decoding". The failure needs + * the relcache entry to be built during decoding, so the decoding session + * is a fresh backend while the writing transaction is still open (as in the + * report's reproducer). Drawn: constraint size, constraints after it, the + * volume written (enough or not to stream). + * + * A guard: neither this property nor the report's own script reproduced + * the error on FreeBSD/arm64 at 45fc51f (both decode 10k+ changes). + */ +PG_PROP(streaming_decode_with_toasted_constraint) +{ + pg_prop_conn *conn = cfe_connect(tc); + pg_prop_conn *w; + pg_prop_conn *d; + bool failed = false; + char *wl = pg_prop_sql_scalar(conn, "SHOW wal_level", &failed, NULL); + int big = (int) pg_prop_draw_int(tc, 100, 600); + int after = (int) pg_prop_draw_int(tc, 0, 3); + int rows = pg_prop_draw_bool(tc) ? (int) pg_prop_draw_int(tc, 2000, 12000) + : (int) pg_prop_draw_int(tc, 1, 50); + StringInfoData s; + + if (wl == NULL || strcmp(wl, "logical") != 0) + pg_prop_skip(tc, "needs wal_level = logical"); + pg_free(wl); + pg_prop_sql_exec(conn, "SELECT pg_drop_replication_slot('cfa4_ss') FROM pg_replication_slots " + "WHERE slot_name = 'cfa4_ss'", NULL); + cfe_exec(tc, conn, "DROP TABLE IF EXISTS cfa4_nst, cfa4_fill"); + initStringInfo(&s); + appendStringInfo(&s, + "DO $$ DECLARE extra text := ''; BEGIN " + "FOR k IN 1..%d LOOP extra := extra || format(', CONSTRAINT z_%%s CHECK (true)', k); END LOOP; " + "EXECUTE format('CREATE TABLE cfa4_nst (id int PRIMARY KEY, v text, " + "CONSTRAINT a_big CHECK (v <> ALL (ARRAY[%%s]))%%s)', " + "(SELECT string_agg(quote_literal(md5(i::text)), ',') FROM generate_series(1, %d) i), extra); " + "END $$; CREATE TABLE cfa4_fill (id int, pad text)", + after, big); + cfe_exec(tc, conn, s.data); + cfe_exec(tc, conn, "SELECT 'init' FROM pg_create_logical_replication_slot('cfa4_ss', 'test_decoding')"); + + /* the writer: an open transaction touching cfa4_nst first */ + w = cfe_connect(tc); + cfe_exec(tc, w, "SET logical_decoding_work_mem = '64kB'"); + cfe_exec(tc, w, "BEGIN"); + cfe_exec(tc, w, "INSERT INTO cfa4_nst VALUES (1, 'y')"); + resetStringInfo(&s); + appendStringInfo(&s, "INSERT INTO cfa4_fill SELECT i, repeat('x', 200) FROM generate_series(1, %d) i", rows); + cfe_exec(tc, w, s.data); + + /* the decoder: a fresh backend, cold relcache */ + d = cfe_connect(tc); + cfe_exec(tc, d, "SET logical_decoding_work_mem = '64kB'"); + resetStringInfo(&s); + appendStringInfo(&s, "streaming decode of an open transaction: CHECK over %d literals, " + "%d constraints after it, %d filler rows", big, after, rows); + /* the bug is an elog(ERROR), XX000: cfe_try_noint fails the case on it */ + cfe_try_noint(tc, d, + "SELECT count(*) FROM pg_logical_slot_peek_changes('cfa4_ss', NULL, NULL, 'stream-changes', '1')", + s.data); + pg_prop_disconnect(d); + pg_prop_sql_exec(w, "ROLLBACK", NULL); + pg_prop_disconnect(w); + pg_prop_sql_exec(conn, "SELECT pg_drop_replication_slot('cfa4_ss')", NULL); + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS cfa4_nst, cfa4_fill", NULL); + pfree(s.data); + pg_prop_disconnect(conn); +} From 09aa32b7b512225de931a75d856326305be331db Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sun, 18 Oct 2026 13:27:13 -0400 Subject: [PATCH 318/338] prop: fast-forward snapshot property for user catalog tables (cf_exec) 6304 fast_forward_snapshot_tracks_user_catalog_xid pg_replication_slot_advance fast-forwards over an INSERT into a user_catalog_table (NEW_CID, no invalidation) while an older transaction is open; the snapshot it serializes must list the inserting xid in committed_xip (checked through pg_logicalinspect). Falsifies on upstream master 45fc51f on every seed. The entry's v4 fix (mark catalog changes on NEW_CID even when fast-forwarding) predates 3a63b76571b; ported by hand it passes 3/3. Needs wal_level = logical and pg_logicalinspect; skips otherwise. --- src/test/prop/sql/prop_cf_exec.c | 65 ++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/src/test/prop/sql/prop_cf_exec.c b/src/test/prop/sql/prop_cf_exec.c index 7de4937074bc8..39642d7f24470 100644 --- a/src/test/prop/sql/prop_cf_exec.c +++ b/src/test/prop/sql/prop_cf_exec.c @@ -1307,3 +1307,68 @@ PG_PROP(streaming_decode_with_toasted_constraint) pfree(s.data); pg_prop_disconnect(conn); } + +/* + * CF 6304: fast-forwarding a logical slot (pg_replication_slot_advance) + * over an INSERT into a user_catalog_table -- which emits NEW_CID but no + * invalidation -- must still record that transaction as a catalog-modifying + * one in the historic snapshot it serializes. Checked through + * pg_logicalinspect: some serialized snapshot's committed_xip holds the + * inserting transaction's xid. An older transaction is kept open so that + * xid stays in the snapshot's interesting range. Drawn: rows inserted, + * extra non-catalog writes, whether the older transaction has an xid + * before or after. Needs wal_level = logical and pg_logicalinspect. + */ +PG_PROP(fast_forward_snapshot_tracks_user_catalog_xid) +{ + pg_prop_conn *s0 = cfe_connect(tc); + pg_prop_conn *s1; + bool failed = false; + char *wl = pg_prop_sql_scalar(s0, "SHOW wal_level", &failed, NULL); + int rows = (int) pg_prop_draw_int(tc, 1, 20); + bool other = pg_prop_draw_bool(tc); + StringInfoData s; + char *r; + + if (wl == NULL || strcmp(wl, "logical") != 0) + pg_prop_skip(tc, "needs wal_level = logical"); + pg_free(wl); + if (!pg_prop_require_extension(s0, "pg_logicalinspect")) + pg_prop_skip(tc, "pg_logicalinspect not installed"); + pg_prop_sql_exec(s0, "SELECT pg_drop_replication_slot('cfa4_ff') FROM pg_replication_slots " + "WHERE slot_name = 'cfa4_ff'", NULL); + cfe_exec(tc, s0, "DROP TABLE IF EXISTS cfa4_ucat, cfa4_plain; " + "CREATE TABLE cfa4_ucat (v int) WITH (user_catalog_table = true); " + "CREATE TABLE cfa4_plain (v int)"); + cfe_exec(tc, s0, "SELECT 'init' FROM pg_create_logical_replication_slot('cfa4_ff', 'test_decoding')"); + + /* s0: older transaction with an xid, held open */ + cfe_exec(tc, s0, "BEGIN"); + cfe_exec(tc, s0, "SELECT pg_current_xact_id()"); + + s1 = cfe_connect(tc); + initStringInfo(&s); + appendStringInfo(&s, "INSERT INTO cfa4_ucat SELECT generate_series(1, %d)", rows); + cfe_exec(tc, s1, s.data); + if (other) + cfe_exec(tc, s1, "INSERT INTO cfa4_plain VALUES (1)"); + cfe_exec(tc, s1, "CHECKPOINT"); + cfe_exec(tc, s1, "SELECT slot_name FROM pg_replication_slot_advance('cfa4_ff', pg_current_wal_lsn())"); + r = pg_prop_sql_scalar(s1, + "SELECT count(*) > 0 FROM pg_ls_logicalsnapdir() f, " + "pg_get_logical_snapshot_info(f.name) i, cfa4_ucat u " + "WHERE u.xmin = ANY (i.committed_xip)", + &failed, NULL); + cfe_exec(tc, s0, "COMMIT"); + if (r && strcmp(r, "t") != 0) + pg_prop_fail(tc, "the snapshot serialized while fast-forwarding does not track the xid that " + "inserted into a user_catalog_table (%d rows%s)", + rows, other ? ", plus a non-catalog write" : ""); + if (r) + pg_free(r); + pg_prop_sql_exec(s1, "SELECT pg_drop_replication_slot('cfa4_ff')", NULL); + pg_prop_sql_exec(s1, "DROP TABLE IF EXISTS cfa4_ucat, cfa4_plain", NULL); + pfree(s.data); + pg_prop_disconnect(s1); + pg_prop_disconnect(s0); +} From 4c41011057df05d85958977329bb5507d32ba077 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sun, 18 Oct 2026 13:27:13 -0400 Subject: [PATCH 319/338] prop: forced-interleaving deadlock property for ATTACH PARTITION (cf_race) 6442 root_alter_races_deep_attach_without_deadlock a recursing ALTER TABLE on a partition root (locks top-down) and ATTACH PARTITION deeper in the tree (target, then the ancestors via generate_partition_qual: bottom-up) deadlock. A free-running race deadlocks only for some shapes (30/30 at the deepest level, 0/30 one level up), which the engine reports as flaky. The property forces the interleaving instead: a third session holds a lock on the table being attached, so ATTACH holds its target and waits; the root ALTER then holds the root and waits for the target; the third session commits. That is deterministic: 6/6 seeds falsify on upstream master, shrinking to a depth-2 tree with ALTER COLUMN SET DEFAULT. The commitfest entry has only an injection-point TAP test, no fix yet, so this records a known open bug rather than validating a patch. --- src/test/prop/sql/prop_cf_race.c | 86 ++++++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+) diff --git a/src/test/prop/sql/prop_cf_race.c b/src/test/prop/sql/prop_cf_race.c index 8314ad8bbd04c..80a1cef30f67a 100644 --- a/src/test/prop/sql/prop_cf_race.c +++ b/src/test/prop/sql/prop_cf_race.c @@ -444,3 +444,89 @@ PG_PROP(create_index_races_index_attach) pg_prop_disconnect(s2); pg_prop_disconnect(s1); } + +/* + * CF 6442: a recursing ALTER TABLE on the root of a partition tree (locks + * top-down) racing ATTACH PARTITION deeper in the tree (locks the target, + * then the new partition, then the ancestors through + * generate_partition_qual: bottom-up) must not deadlock -- each is a single + * DDL statement, no user-visible lock ordering is involved. + * + * The interleaving is forced, so the case is deterministic: a third session + * holds a lock on the partition being attached; ATTACH then holds the + * target and waits for it; the root ALTER then holds the root and waits for + * the target; the third session commits. ATTACH proceeds to lock the root + * -- a cycle. Drawn: tree depth, attach level, the root command, the lock + * the third session holds. + * + * Known open: the CF 6442 entry carries only an injection-point TAP test + * documenting this deadlock, no fix, so this property fails on master and + * on the patched server alike. + */ +PG_PROP(root_alter_races_deep_attach_without_deadlock) +{ + static const char *const root_cmds[] = { + "ALTER TABLE cfr_t0 ALTER COLUMN id SET DEFAULT 0", + "ALTER TABLE cfr_t0 ALTER COLUMN id DROP DEFAULT", + "ALTER TABLE cfr_t0 ALTER COLUMN id SET STATISTICS 50", + "ALTER TABLE cfr_t0 ADD CONSTRAINT cfr_ck CHECK (id >= 0)", + }; + static const char *const holds[] = {"ACCESS SHARE", "ROW EXCLUSIVE", "SHARE"}; + pg_prop_conn *s1 = rc_connect(tc); + pg_prop_conn *s2 = rc_connect(tc); + pg_prop_conn *s3 = rc_connect(tc); + int depth = (int) pg_prop_draw_int(tc, 2, 5); + int lvl = (int) pg_prop_draw_int(tc, 1, depth - 1); + const char *root = root_cmds[pg_prop_draw_int(tc, 0, lengthof(root_cmds) - 1)]; + const char *hold = holds[pg_prop_draw_int(tc, 0, lengthof(holds) - 1)]; + char *pid1 = rc_pid(tc, s1); + char *pid2 = rc_pid(tc, s2); + char *st1 = NULL; + char *st2 = NULL; + char *e1 = NULL; + char *e2 = NULL; + bool ok1; + bool ok2; + StringInfoData s; + char att[200]; + + initStringInfo(&s); + appendStringInfoString(&s, "DROP TABLE IF EXISTS cfr_t0, cfr_new; " + "CREATE TABLE cfr_t0 (id int) PARTITION BY RANGE (id); "); + for (int i = 1; i < depth; i++) + appendStringInfo(&s, "CREATE TABLE cfr_t%d PARTITION OF cfr_t%d FOR VALUES FROM (0) TO (%d) " + "PARTITION BY RANGE (id); ", i, i - 1, 1000000 >> (i - 1)); + appendStringInfoString(&s, "CREATE TABLE cfr_new (id int)"); + rc_exec(tc, s1, s.data); + snprintf(att, sizeof(att), "ALTER TABLE cfr_t%d ATTACH PARTITION cfr_new FOR VALUES FROM (%d) TO (%d)", + lvl, (1000000 >> lvl) / 2, 1000000 >> lvl); + + resetStringInfo(&s); + appendStringInfo(&s, "LOCK TABLE cfr_new IN %s MODE", hold); + rc_exec(tc, s3, "BEGIN"); + rc_exec(tc, s3, s.data); + if (!pg_prop_sql_send(s2, att)) + pg_prop_fail(tc, "could not dispatch"); + rc_wait_blocked(tc, s3, pid2); + if (!pg_prop_sql_send(s1, root)) + pg_prop_fail(tc, "could not dispatch"); + rc_wait_blocked(tc, s3, pid1); + rc_exec(tc, s3, "COMMIT"); + + ok2 = pg_prop_sql_await(s2, &st2, &e2); + ok1 = pg_prop_sql_await(s1, &st1, &e1); + if ((!ok1 && st1 && strcmp(st1, "40P01") == 0) || (!ok2 && st2 && strcmp(st2, "40P01") == 0)) + pg_prop_fail(tc, "deadlock between a root ALTER TABLE and a deeper ATTACH PARTITION\n" + " root: %s\n attach: %s\n tree depth %d", + root, att, depth); + if ((!ok1 && rc_internal(st1, e1)) || (!ok2 && rc_internal(st2, e2))) + pg_prop_fail(tc, "internal error: %s / %s", e1 ? e1 : "-", e2 ? e2 : "-"); + + pg_prop_sql_exec(s1, "DROP TABLE IF EXISTS cfr_t0, cfr_new", NULL); + pg_free(pid1); + pg_free(pid2); + pfree(s.data); + pg_prop_disconnect(s3); + pg_prop_disconnect(s2); + pg_prop_disconnect(s1); +} From 25e04bdbedf577b3cd6daf4f4bdcf3eca516e085 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sun, 18 Oct 2026 13:46:03 -0400 Subject: [PATCH 320/338] prop: pageinspect on corrupted page images must not crash (cf_exec) 7229 pageinspect_survives_corrupt_page take a real page of a drawn btree / hash / gist / brin index or heap, overwrite drawn bytes of its header and line pointer array, and pass it to the matching *_page_items function: the backend must survive. Any ERROR is accepted -- pageinspect's own pre-existing bounds checks are elog()s (XX000), so they are correct rejections, not internal errors. On upstream master 45fc51f: 6/6 seeds crash (out-of-range line pointers in gist_page_items_bytea and bt_page_items). With the CF 7229 patches: 4/6 pass, and 2/6 still crash on Assert(!ItemIdIsDead(id)) in bt_page_print_tuples: a crafted LP_DEAD flag on a pivot tuple's line pointer passes every new check and reaches an Assert that encodes a valid-page invariant on user-supplied input. --- src/test/prop/sql/prop_cf_exec.c | 71 ++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/src/test/prop/sql/prop_cf_exec.c b/src/test/prop/sql/prop_cf_exec.c index 39642d7f24470..d9edf3a79af31 100644 --- a/src/test/prop/sql/prop_cf_exec.c +++ b/src/test/prop/sql/prop_cf_exec.c @@ -1372,3 +1372,74 @@ PG_PROP(fast_forward_snapshot_tracks_user_catalog_xid) pg_prop_disconnect(s1); pg_prop_disconnect(s0); } + +/* + * CF 7229: pageinspect's item functions given a corrupt page image (a + * line pointer or header field overwritten) must raise a clean error, not + * read out of bounds -- under cassert an overread may crash, and the crash + * tripwire turns that into exit 3. The page is a real page of a drawn index + * or heap, then drawn bytes of the header / line pointer array are + * overwritten. Invariant: the backend survives. (Any ERROR is a clean + * rejection -- pageinspect's own pre-existing bounds checks are elog()s + * with SQLSTATE XX000, so "not internal" would flag correct rejections.) + * Needs pageinspect (superuser). + */ +PG_PROP(pageinspect_survives_corrupt_page) +{ + static const struct + { + const char *setup; + const char *rel; + int blk; + const char *fn; /* %s = the page expression */ + } targets[] = { + {"CREATE INDEX cfa4_pi_bt ON cfa4_pi (a)", "cfa4_pi_bt", 1, "SELECT count(*) FROM bt_page_items(%s)"}, + {"CREATE INDEX cfa4_pi_hash ON cfa4_pi USING hash (a)", "cfa4_pi_hash", 3, + "SELECT count(*) FROM hash_page_items(%s)"}, + {"CREATE INDEX cfa4_pi_gist ON cfa4_pi USING gist (p)", "cfa4_pi_gist", 0, + "SELECT count(*) FROM gist_page_items_bytea(%s)"}, + {"CREATE INDEX cfa4_pi_brin ON cfa4_pi USING brin (a)", "cfa4_pi_brin", 2, + "SELECT count(*) FROM brin_page_items(%s, 'cfa4_pi_brin')"}, + {"SELECT 1", "cfa4_pi", 0, "SELECT count(*) FROM heap_page_items(%s)"}, + }; + pg_prop_conn *conn = cfe_connect(tc); + int t = (int) pg_prop_draw_int(tc, 0, lengthof(targets) - 1); + int nb = (int) pg_prop_draw_int(tc, 1, 6); + StringInfoData page; + StringInfoData s; + + if (!pg_prop_require_extension(conn, "pageinspect")) + pg_prop_skip(tc, "pageinspect not installed"); + cfe_exec(tc, conn, "DROP TABLE IF EXISTS cfa4_pi; CREATE TABLE cfa4_pi (a int, p point); " + "INSERT INTO cfa4_pi SELECT g, point(g, g) FROM generate_series(1, 2000) g"); + cfe_exec(tc, conn, targets[t].setup); + + /* the page, with nb drawn bytes of the first 64 overwritten */ + initStringInfo(&page); + appendStringInfo(&page, "get_raw_page('%s', %d)", targets[t].rel, targets[t].blk); + for (int i = 0; i < nb; i++) + { + /* bias to the header (pd_lower/upper/special at 12..17) and the first line pointers (24..) */ + int off = pg_prop_draw_bool(tc) ? (int) pg_prop_draw_int(tc, 12, 17) + : (int) pg_prop_draw_int(tc, 24, 63); + int val = pg_prop_draw_bool(tc) ? 255 : (int) pg_prop_draw_int(tc, 0, 255); + char *prev = pstrdup(page.data); + + resetStringInfo(&page); + appendStringInfo(&page, "set_byte(%s, %d, %d)", prev, off, val); + pfree(prev); + } + initStringInfo(&s); + appendStringInfo(&s, targets[t].fn, page.data); + { + char *ss; + char *err; + + /* a crash is caught by cfe_try's liveness probe (tripwire, exit 3) */ + (void) cfe_try(conn, s.data, &ss, &err); + } + cfe_exec(tc, conn, "DROP TABLE cfa4_pi"); + pfree(page.data); + pfree(s.data); + pg_prop_disconnect(conn); +} From d072eb2a235878902979441a4a0f1d2f0f7c8207 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sun, 18 Oct 2026 13:53:28 -0400 Subject: [PATCH 321/338] prop: encoding verifier and text-producer properties (cf_encoding) Text the server accepts must be valid in its encoding, and bytes the verifier accepts must be well-formed for that encoding: 6913 uhc_verifier_matches_conversion a UHC pair outside CP949's lead/trail ranges must be rejected by the verifier (22021), not reach the conversion table 6761 euc_kr_lengths_match_verifier EUC_KR characters are one or two bytes: char_length matches the characters drawn, SS2/SS3 are rejected, and the maximum length is 2 7417 crypt_result_is_valid_text crypt() with a multibyte salt must not return text that a later operation rejects as invalid UTF8 All three falsify on upstream master 45fc51f on every seed and pass 3/3 with their commitfest patch. --- src/test/prop/Makefile | 1 + src/test/prop/meson.build | 1 + src/test/prop/sql/prop_cf_encoding.c | 226 +++++++++++++++++++++++++++ src/test/prop/t/002_sql_props.pl | 2 +- 4 files changed, 229 insertions(+), 1 deletion(-) create mode 100644 src/test/prop/sql/prop_cf_encoding.c diff --git a/src/test/prop/Makefile b/src/test/prop/Makefile index 18833d64d28bf..29d4661d17a81 100644 --- a/src/test/prop/Makefile +++ b/src/test/prop/Makefile @@ -292,6 +292,7 @@ OBJS = \ sql/prop_cf_ddl2.o \ sql/prop_cf_deparse.o \ sql/prop_cf_stats.o \ + sql/prop_cf_encoding.o \ sql/prop_cf_planner.o \ sql/prop_cf_race.o \ sql/index_traits.o \ diff --git a/src/test/prop/meson.build b/src/test/prop/meson.build index 4cd9c8fb4f241..053dc56df2255 100644 --- a/src/test/prop/meson.build +++ b/src/test/prop/meson.build @@ -297,6 +297,7 @@ prop_query_sources = files( 'sql/prop_cf_ddl2.c', 'sql/prop_cf_deparse.c', 'sql/prop_cf_stats.c', + 'sql/prop_cf_encoding.c', 'sql/prop_cf_planner.c', 'sql/prop_cf_race.c', 'sql/index_traits.c', diff --git a/src/test/prop/sql/prop_cf_encoding.c b/src/test/prop/sql/prop_cf_encoding.c new file mode 100644 index 0000000000000..ec447c5f3843c --- /dev/null +++ b/src/test/prop/sql/prop_cf_encoding.c @@ -0,0 +1,226 @@ +/*------------------------------------------------------------------------- + * + * prop_cf_encoding.c + * Encoding verifiers and text producers from the open commitfest. + * + * The invariant shared by every property here: text the server accepts is + * valid in its encoding, and a byte string the verifier accepts converts. + * A verifier that is too lax lets bytes through that fail later ("no + * equivalent"), one that is too strict (or a mblen that disagrees with it) + * miscounts characters; a function that builds text from bytes must not + * cut a multibyte character in half. + * + * CF 6913 pg_uhc_verifychar accepts lead/trail bytes outside CP949 + * CF 6761 EUC_KR mblen treats SS2/SS3 as lead bytes (maxlen 3) + * CF 7417 pgcrypto crypt() copies a salt prefix that splits a character + * + * Portions Copyright (c) 2024, PostgreSQL Global Development Group + * + *------------------------------------------------------------------------- + */ +#include "postgres_fe.h" + +#define PG_PROP_AREA "cf_encoding" +#include "pg_prop.h" + +#include "lib/stringinfo.h" + +static pg_prop_conn * +en_connect(pg_prop_tc *tc) +{ + char *err = NULL; + pg_prop_conn *conn = pg_prop_connect(&err); + + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + return conn; +} + +static void +en_hex(StringInfo b, const unsigned char *p, int n) +{ + for (int i = 0; i < n; i++) + appendStringInfo(b, "%02x", p[i]); +} + +/* + * The SQLSTATE class of an error, or NULL on success. Uses the async + * primitives because they return the SQLSTATE. + */ +static bool +en_run(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql, char **st, char **err) +{ + *st = NULL; + *err = NULL; + if (!pg_prop_sql_send(conn, sql)) + pg_prop_fail(tc, "could not dispatch"); + return pg_prop_sql_await(conn, st, err); +} + +/* + * CF 6913: pg_uhc_verifychar must enforce CP949's byte ranges -- lead + * 0x81..0xFE, trail 0x41..0x5A, 0x61..0x7A or 0x81..0xFE. A pair outside + * them is an invalid byte sequence (22021) and must be rejected by the + * verifier, not passed on to fail in the conversion table ("has no + * equivalent", 22P05). (A pair inside the ranges may still be an + * unassigned code point; verifiers are structural, so "no equivalent" is + * then the right answer.) Drawn: lead and trail bytes, biased to the + * range boundaries. + */ +static bool +en_cp949_pair(unsigned char c1, unsigned char c2) +{ + return c1 >= 0x81 && c1 <= 0xfe && + ((c2 >= 0x41 && c2 <= 0x5a) || (c2 >= 0x61 && c2 <= 0x7a) || (c2 >= 0x81 && c2 <= 0xfe)); +} + +PG_PROP(uhc_verifier_matches_conversion) +{ + static const int edges[] = {0x40, 0x41, 0x5a, 0x5b, 0x60, 0x61, 0x7a, 0x7b, 0x80, 0x81, 0xa0, + 0xa1, 0xc6, 0xc7, 0xfd, 0xfe, 0xff}; + pg_prop_conn *conn = en_connect(tc); + unsigned char b[2]; + StringInfoData s; + char *st; + char *err; + bool ok; + + /* the lead byte has its high bit set: otherwise the pair is two ASCII characters */ + b[0] = (unsigned char) (pg_prop_draw_bool(tc) ? edges[pg_prop_draw_int(tc, 8, lengthof(edges) - 1)] + : pg_prop_draw_int(tc, 0x80, 0xff)); + b[1] = (unsigned char) (pg_prop_draw_bool(tc) ? edges[pg_prop_draw_int(tc, 0, lengthof(edges) - 1)] + : pg_prop_draw_int(tc, 0x30, 0xff)); + initStringInfo(&s); + appendStringInfoString(&s, "SELECT convert_from('\\x"); + en_hex(&s, b, 2); + appendStringInfoString(&s, "'::bytea, 'UHC')"); + ok = en_run(tc, conn, s.data, &st, &err); + if (!en_cp949_pair(b[0], b[1]) && (ok || st == NULL || strcmp(st, "22021") != 0)) + pg_prop_fail(tc, "UHC byte pair %02x %02x is outside CP949's lead/trail ranges but the " + "verifier accepted it: %s", b[0], b[1], ok ? "converted" : (err ? err : "")); + pfree(s.data); + pg_prop_disconnect(conn); +} + +/* + * CF 6761: every EUC_KR character is one byte (ASCII) or two (KS X 1001); + * code sets 2 and 3 do not exist, so the encoding's maximum length is 2 and + * SS2/SS3 (0x8e / 0x8f) are not lead bytes. Drawn strings of valid EUC_KR + * characters must have char_length = the number of characters drawn, and + * pg_encoding_max_length must be 2. A drawn string with an SS2/SS3 byte + * must be rejected. + */ +PG_PROP(euc_kr_lengths_match_verifier) +{ + pg_prop_conn *conn = en_connect(tc); + int n = (int) pg_prop_draw_int(tc, 1, 12); + bool ss = pg_prop_draw_int(tc, 0, 3) == 0; + StringInfoData s; + StringInfoData bytes; + bool failed = false; + char *v; + char *st; + char *err; + + initStringInfo(&bytes); + for (int i = 0; i < n; i++) + { + unsigned char c[2]; + + if (pg_prop_draw_bool(tc)) + { + c[0] = (unsigned char) pg_prop_draw_int(tc, 0x20, 0x7e); + en_hex(&bytes, c, 1); + } + else + { + /* KS X 1001 Hangul block, rows 0xb0..0xc8, cols 0xa1..0xfe */ + c[0] = (unsigned char) pg_prop_draw_int(tc, 0xb0, 0xc8); + c[1] = (unsigned char) pg_prop_draw_int(tc, 0xa1, 0xfe); + en_hex(&bytes, c, 2); + } + } + initStringInfo(&s); + if (ss) + { + unsigned char c[3] = {pg_prop_draw_bool(tc) ? 0x8e : 0x8f, 0xa1, 0xa1}; + + en_hex(&bytes, c, (int) pg_prop_draw_int(tc, 2, 3)); + appendStringInfo(&s, "SELECT char_length(convert_from('\\x%s'::bytea, 'EUC_KR'))", bytes.data); + if (en_run(tc, conn, s.data, &st, &err)) + pg_prop_fail(tc, "EUC_KR string with an SS2/SS3 byte was accepted: %s", bytes.data); + } + else + { + appendStringInfo(&s, "SELECT char_length(convert_from('\\x%s'::bytea, 'EUC_KR'))", bytes.data); + v = pg_prop_sql_scalar(conn, s.data, &failed, &err); + if (failed || v == NULL || atoi(v) != n) + pg_prop_fail(tc, "char_length of %d EUC_KR characters (%s) = %s", n, bytes.data, + failed ? (err ? err : "error") : v); + pg_free(v); + } + v = pg_prop_sql_scalar(conn, "SELECT pg_encoding_max_length(pg_char_to_encoding('EUC_KR'))", &failed, NULL); + if (v && strcmp(v, "2") != 0) + pg_prop_fail(tc, "pg_encoding_max_length('EUC_KR') = %s, but EUC_KR has no 3-byte characters", v); + if (v) + pg_free(v); + pfree(s.data); + pfree(bytes.data); + pg_prop_disconnect(conn); +} + +/* + * CF 7417: crypt(password, salt) returns text; with a salt containing + * multibyte characters the result must still be valid in the database + * encoding (the DES and MD5 / Blowfish salt parsers copy a fixed number of + * bytes). Invariant: crypt() either raises an error itself or returns a + * value that later operations (upper()) accept as valid text. + * Needs a UTF8 database and pgcrypto. + */ +PG_PROP(crypt_result_is_valid_text) +{ + static const char *const mb[] = {"\\u20ac", "\\u00e9", "\\u4e2d", "\\U0001F600", "\\u0416"}; + static const char *const pre[] = {"", "$1$", "$2a$06$", "_J9.."}; + pg_prop_conn *conn = en_connect(tc); + bool failed = false; + char *enc = pg_prop_sql_scalar(conn, "SELECT getdatabaseencoding()", &failed, NULL); + const char *prefix = pre[pg_prop_draw_int(tc, 0, lengthof(pre) - 1)]; + int len = (int) pg_prop_draw_int(tc, 1, 4); + StringInfoData salt; + StringInfoData s; + char *st; + char *err; + + if (enc == NULL || strcmp(enc, "UTF8") != 0) + pg_prop_skip(tc, "needs a UTF8 database"); + pg_free(enc); + if (!pg_prop_require_extension(conn, "pgcrypto")) + pg_prop_skip(tc, "pgcrypto not installed"); + initStringInfo(&salt); + for (int i = 0; i < len; i++) + { + if (pg_prop_draw_bool(tc)) + appendStringInfoString(&salt, mb[pg_prop_draw_int(tc, 0, lengthof(mb) - 1)]); + else + appendStringInfoChar(&salt, "./ABCxyz019"[pg_prop_draw_int(tc, 0, 10)]); + } + /* + * Two steps, so a rejection by crypt() itself (the fix) is told apart + * from a value that only fails when used: store the result first (a + * bare SELECT of an invalid value already fails in the output function), + * then use it. + */ + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS cfe_crypt", NULL); + initStringInfo(&s); + appendStringInfo(&s, "CREATE TEMP TABLE cfe_crypt AS SELECT crypt('password', E'%s%s') AS h", + prefix, salt.data); + if (en_run(tc, conn, s.data, &st, &err) && + !en_run(tc, conn, "SELECT length(upper(h)) FROM cfe_crypt", &st, &err) && + st && strcmp(st, "22021") == 0) + pg_prop_fail(tc, "crypt() returned text that is not valid UTF8: %s\n salt: E'%s%s'", + err ? err : "", prefix, salt.data); + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS cfe_crypt", NULL); + pfree(s.data); + pfree(salt.data); + pg_prop_disconnect(conn); +} diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 56ad98b3e923f..12d80089db643 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -25,7 +25,7 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model syscol collation cf_types cf_index cf_exec cf_ddl cf_ddl2 cf_deparse cf_stats cf_planner cf_race index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); +my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model syscol collation cf_types cf_index cf_exec cf_ddl cf_ddl2 cf_deparse cf_stats cf_encoding cf_planner cf_race index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From a379fa31411a7b44237ab4cc1fc3c7408d3bfdfb Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sun, 18 Oct 2026 14:02:52 -0400 Subject: [PATCH 322/338] prop: composite values read after ALTER ATTRIBUTE TYPE (cf_ddl) 6431 / 6449 composite_read_after_alter_attribute_type a composite value produced before ALTER TYPE ... ALTER ATTRIBUTE ... TYPE in the same transaction (a cursor's pending rows, a PL/pgSQL row or RECORD variable, an OUT parameter, a nested composite) and read after it: the old int4 bits are read as text and the backend crashes. The invariant is that the backend survives and raises no internal error. On upstream master 45fc51f every seed crashes; the shrunk case is the cursor reproducer of 6431. With 6431 (typcache tupDesc_identifier) 6/6 pass, and 6431 also turns the PL/pgSQL case into "type has changed". With 6449 alone (PL/pgSQL-only detection) the PL/pgSQL case is a clean error but the cursor case still crashes, 6/6. cd_step's liveness probe now carries the step log, so a crash log names the statement sequence that killed the backend. --- src/test/prop/sql/prop_cf_ddl.c | 87 +++++++++++++++++++++++++++++++-- 1 file changed, 82 insertions(+), 5 deletions(-) diff --git a/src/test/prop/sql/prop_cf_ddl.c b/src/test/prop/sql/prop_cf_ddl.c index f998ee12e5681..cffbc09d479e5 100644 --- a/src/test/prop/sql/prop_cf_ddl.c +++ b/src/test/prop/sql/prop_cf_ddl.c @@ -101,13 +101,17 @@ cd_step(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql, const StringInfo lo ok = pg_prop_sql_await(conn, &ss, &err); if (ok) return true; - /* liveness probe: exits 3 (and logs) if the backend died */ + /* + * Liveness probe: exits 3 (and logs the probe) if the backend died. The + * probe carries the step log, so the crash log names what crashed. + */ { - bool f = false; - char *v = pg_prop_sql_scalar(conn, "SELECT 1", &f, NULL); + StringInfoData probe; - if (v) - pg_free(v); + initStringInfo(&probe); + appendStringInfo(&probe, "SELECT $cd_probe$ liveness probe after:\n%s$cd_probe$", log->data); + pg_prop_sql_exec(conn, probe.data, NULL); + pfree(probe.data); } if (cd_internal(ss, err)) pg_prop_fail(tc, "internal error (SQLSTATE %s): %s\n steps:\n%s", @@ -527,3 +531,76 @@ PG_PROP(partition_set_expression_keeps_generated_consistent) pfree(s.data); pg_prop_disconnect(conn); } + +/* + * CF 6431 / 6449: a composite value produced before ALTER TYPE ... ALTER + * ATTRIBUTE ... TYPE in the same transaction, then read after it, must not + * be interpreted with the new row type: either an error is raised or the + * value reads back correctly. Reading the old int4 bits as text crashes + * (textout memcpy of a bogus length). Drawn: the holder of the old value + * (cursor, PL/pgSQL row variable, generic RECORD, OUT parameter, nested + * composite), the old value (biased to large ints, which become bogus + * varlena lengths), the new attribute type. Invariant: the backend + * survives and no internal error is raised (cd_step checks both). + */ +PG_PROP(composite_read_after_alter_attribute_type) +{ + static const char *const newtypes[] = {"text", "numeric", "bigint", "varchar(20)"}; + pg_prop_conn *conn = cd_connect(tc); + int holder = (int) pg_prop_draw_int(tc, 0, 4); + const char *nt = newtypes[pg_prop_draw_int(tc, 0, lengthof(newtypes) - 1)]; + long v = pg_prop_draw_bool(tc) ? (1L << pg_prop_draw_int(tc, 20, 30)) : (long) pg_prop_draw_int(tc, 0, 1000); + StringInfoData s; + StringInfoData log; + + initStringInfo(&s); + initStringInfo(&log); + cd_step(tc, conn, "DROP TYPE IF EXISTS cfd_co CASCADE", &log); + cd_step(tc, conn, "DROP TYPE IF EXISTS cfd_ci CASCADE", &log); + cd_step(tc, conn, "CREATE TYPE cfd_ci AS (x int, y int)", &log); + cd_step(tc, conn, "CREATE TYPE cfd_co AS (a int, b int, c cfd_ci)", &log); + if (holder == 0) + { + /* cursor (CF 6431) */ + cd_step(tc, conn, "BEGIN", &log); + appendStringInfo(&s, "DECLARE cfd_cur CURSOR FOR SELECT (i, %ld, NULL)::cfd_co " + "FROM generate_series(1, 3) i", v); + cd_step(tc, conn, s.data, &log); + cd_step(tc, conn, "FETCH FROM cfd_cur", &log); + resetStringInfo(&s); + appendStringInfo(&s, "ALTER TYPE cfd_co ALTER ATTRIBUTE b TYPE %s", nt); + cd_step(tc, conn, s.data, &log); + cd_step(tc, conn, "FETCH FROM cfd_cur", &log); + cd_step(tc, conn, "ROLLBACK", &log); + } + else + { + /* PL/pgSQL (CF 6449): row variable, generic RECORD, OUT parameter, nested */ + if (holder == 4) + appendStringInfo(&s, + "CREATE FUNCTION cfd_f() RETURNS record LANGUAGE plpgsql AS $f$ " + "DECLARE r cfd_co; BEGIN " + "r := row(1, 2, row(10, %ld)::cfd_ci)::cfd_co; " + "ALTER TYPE cfd_ci ALTER ATTRIBUTE y TYPE %s; " + "RETURN r; END $f$", v, nt); + else + appendStringInfo(&s, + "CREATE FUNCTION cfd_f(%s) RETURNS %s LANGUAGE plpgsql AS $f$ " + "DECLARE r %s; BEGIN " + "%s := row(1, %ld, NULL)::cfd_co; " + "ALTER TYPE cfd_co ALTER ATTRIBUTE b TYPE %s; " + "%s END $f$", + holder == 3 ? "OUT o cfd_co" : "", + holder == 3 ? "cfd_co" : "record", + holder == 2 ? "record" : "cfd_co", + holder == 3 ? "o" : "r", v, nt, + holder == 3 ? "RETURN;" : "RETURN r;"); + cd_step(tc, conn, s.data, &log); + cd_step(tc, conn, "SELECT cfd_f()::text", &log); + cd_step(tc, conn, "DROP FUNCTION cfd_f", &log); + } + pg_prop_sql_exec(conn, "DROP TYPE IF EXISTS cfd_co CASCADE; DROP TYPE IF EXISTS cfd_ci CASCADE", NULL); + pfree(s.data); + pfree(log.data); + pg_prop_disconnect(conn); +} From bb4827a22829a222dc75b778f9f4446c75a77560 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sun, 18 Oct 2026 14:50:30 -0400 Subject: [PATCH 323/338] prop: correct the boolean DEFAULT reference in sqljson_bool_default_is_cast The reference for a boolean DEFAULT ... ON EMPTY/ERROR was CAST(expr AS type). For a RETURNING type with no cast from boolean (bigint, smallint, numeric) that CAST does not parse, so the reference was always an error, while the parser converts a boolean DEFAULT through jsonb: the JSON boolean's conversion -- an error for a number type -- or SQL NULL when the expression is NULL. The property therefore failed on the CF 7181 fix for a NULL boolean DEFAULT returning bigint, which is the correct answer. The reference now asks pg_cast whether the type (or a domain's base type) has a cast from boolean, and uses the jsonb conversion when it does not. The parser rejects such a CAST even in an untaken CASE branch, so the choice is made in C. With the corrected reference: upstream master 45fc51f 6/6 fail, the CF 7181 fix (which now applies to master) 6/6 pass; the sibling sqljson_default_error_is_cast still fails on master and passes 3/3 with CF 7355. --- src/test/prop/sql/prop_cf_types.c | 36 ++++++++++++++++++++++++++++--- 1 file changed, 33 insertions(+), 3 deletions(-) diff --git a/src/test/prop/sql/prop_cf_types.c b/src/test/prop/sql/prop_cf_types.c index 6d4ed745f2e9a..4ee330c5f2468 100644 --- a/src/test/prop/sql/prop_cf_types.c +++ b/src/test/prop/sql/prop_cf_types.c @@ -1175,11 +1175,41 @@ sj_default_is_cast(pg_prop_tc *tc, bool boolean) "COLUMNS (c %s PATH '%s $.a'%s DEFAULT %s ON %s))", doc.data, typ, on_error ? "strict" : "lax", quotes, e.data, on_error ? "ERROR" : "EMPTY"); - initStringInfo(&m); - appendStringInfo(&m, "SELECT CAST((%s) AS %s)::text", e.data, typ); - + /* + * The reference: the DEFAULT converted to the RETURNING type. For a + * boolean DEFAULT and a type with a cast from boolean that is the cast; + * for one without (bigint, smallint, numeric and the int-based domains) + * the parser converts through jsonb, so the reference is the JSON + * boolean's conversion -- an error for a number type -- or NULL if the + * expression is NULL. (A CAST cannot be written for those: the parser + * rejects it even in an untaken CASE branch.) Either way the value the + * expression evaluates to decides the outcome, never a constant + * 'false'. + */ conn = cfb3_connect(tc); cfb3_exec(tc, conn, sj_setup); + initStringInfo(&m); + if (boolean) + { + /* does the type (or its domain base) have a cast from boolean? */ + char *castq = psprintf("SELECT EXISTS (SELECT 1 FROM pg_cast WHERE castsource = 'bool'::regtype " + "AND casttarget = (SELECT CASE typtype WHEN 'd' THEN typbasetype ELSE oid END " + "FROM pg_type WHERE oid = '%s'::regtype)) " + "OR '%s'::regtype IN ('text'::regtype, 'varchar'::regtype)", typ, typ); + bool f = false; + char *has = pg_prop_sql_scalar(conn, castq, &f, NULL); + + if (has && strcmp(has, "t") != 0) + appendStringInfo(&m, "SELECT CASE WHEN (%s) IS NULL THEN NULL " + "ELSE JSON_VALUE(to_jsonb(%s), '$' RETURNING %s ERROR ON ERROR) END::text", + e.data, e.data, typ); + if (has) + pg_free(has); + pfree(castq); + } + if (m.len == 0) + appendStringInfo(&m, "SELECT CAST((%s) AS %s)::text", e.data, typ); + got = sj_outcome(tc, conn, q.data); want = sj_outcome(tc, conn, m.data); if (strcmp(got, want) != 0) From 12962573ca342fc2d6fbe37e089b536ef9c54f3f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 19 Oct 2026 10:55:07 -0400 Subject: [PATCH 324/338] prop: freeze the per-type scalar areas; add a CF retirement check The SQL-oracle driver now runs 35 active areas by default -- the query generator and its oracles, the stateful / concurrency / access-method areas, and the commitfest areas -- and runs the 238 per-type scalar areas only with PG_PROP_FROZEN=1. Those areas found no server bug in the 2026 evaluation; their findings were almost all unsound oracles. They still compile, so nothing is lost, but they are no longer maintained, and one that breaks after an upstream change may be deleted rather than fixed. README documents the split, the rule that new effort goes into the query generator rather than new oracles, and the retirement rule for cf_* properties: once the commitfest entry is committed the patch's own regression test covers the case, so the property is deleted. scripts/cf-retire.sh lists cf_* properties whose entry is closed (committed, withdrawn, rejected, returned). None is today. --- src/test/prop/README | 27 ++++++++++++++ src/test/prop/scripts/cf-retire.sh | 27 ++++++++++++++ src/test/prop/t/002_sql_props.pl | 58 +++++++++++++++++++++++++++++- 3 files changed, 111 insertions(+), 1 deletion(-) create mode 100755 src/test/prop/scripts/cf-retire.sh diff --git a/src/test/prop/README b/src/test/prop/README index 7d14be709dc3d..9229bfcb2e682 100644 --- a/src/test/prop/README +++ b/src/test/prop/README @@ -158,6 +158,33 @@ iteration count of a meson test run without editing anything: PG_PROP_TESTS="pruning_preserves_rows" meson test prop/002_sql_props +Active and frozen areas +======================= + +The SQL-oracle driver runs the active areas by default: the query generator +and its oracles (query_*), the stateful, concurrency and access-method areas, +and the commitfest areas (cf_*). The per-type scalar areas (numeric, text, +jsonb, ...) are frozen: they compile and run with PG_PROP_FROZEN=1, but are +not maintained. In the 2026 evaluation they found no server bug and most of +their findings were unsound oracles. A frozen area that breaks after an +upstream change may be deleted rather than fixed. + +New work goes into the generator (sql/query_gen.c) rather than new oracles. +A new query shape is checked by every existing oracle at once; a new oracle +checks only the shapes the generator already makes. + + +Commitfest properties and retirement +==================================== + +A cf_* property encodes one open commitfest entry: it fails on master and +passes with that entry's patch. It earns its keep while the patch is under +review. Once the fix is committed, the patch's own regression test covers +the case, and the property is retired: delete it (or move its invariant into +a generator shape, if it generalises). scripts/cf-retire.sh lists the +properties whose CF entry is committed, withdrawn or rejected. + + Debugging a failure =================== diff --git a/src/test/prop/scripts/cf-retire.sh b/src/test/prop/scripts/cf-retire.sh new file mode 100755 index 0000000000000..de27b0d777ab3 --- /dev/null +++ b/src/test/prop/scripts/cf-retire.sh @@ -0,0 +1,27 @@ +#!/bin/sh +# cf-retire.sh: list cf_* properties whose commitfest entry is closed. +# +# Each cf_* property names its entry as "CF NNNN" in its comment. For each +# entry this asks the commitfest app for its status; an entry that is +# Committed, Withdrawn, Rejected or Returned with feedback is printed with +# the files that mention it. A committed entry's property should be deleted +# (the patch's own regression test now covers it); for the others, decide +# whether the bug is still open on master (keep) or the property is moot. +# +# Usage: src/test/prop/scripts/cf-retire.sh (needs curl and network) +set -eu +dir=$(cd "$(dirname "$0")/../sql" && pwd) +ids=$(grep -ohE 'CF [0-9]{4}' "$dir"/prop_cf_*.c "$dir"/prop_syscol.c 2>/dev/null | + awk '{print $2}' | sort -u) +for cf in $ids; do + st=$(curl -s -m 20 "https://commitfest.postgresql.org/patch/$cf/" | + sed 's/<[^>]*>/\n/g' | grep -v '^[[:space:]]*$' | + sed -n '/^Status$/,+3p' | sed -n 4p | sed 's/^ *//') + case "$st" in + Committed|Withdrawn|Rejected|"Returned with feedback") + printf '%s\t%s\t%s\n' "$cf" "$st" \ + "$(grep -l "CF $cf" "$dir"/prop_cf_*.c "$dir"/prop_syscol.c | xargs -n1 basename | tr '\n' ' ')" + ;; + "") printf '%s\t(status unavailable)\n' "$cf" >&2 ;; + esac +done diff --git a/src/test/prop/t/002_sql_props.pl b/src/test/prop/t/002_sql_props.pl index 12d80089db643..1e5e45c176d71 100644 --- a/src/test/prop/t/002_sql_props.pl +++ b/src/test/prop/t/002_sql_props.pl @@ -12,6 +12,7 @@ # PG_PROP_ITERATIONS test cases per property (default: pg_prop's own 100) # PG_PROP_SEED fixed RNG seed (default: pg_prop picks and prints one) # PG_PROP_TESTS whitespace-separated property names to run +# PG_PROP_FROZEN also run the frozen scalar/type areas (see below) # # To debug one failure interactively, start a node, point PGHOST/PGPORT at it, # then run e.g. pg_prop partprune pruning_preserves_rows -n 100000 -s 42 @@ -25,7 +26,62 @@ use Test::More; # Keep in step with the Tier-B entries in prop_area_sources (meson.build). -my @areas = qw(numeric numcast numscale numround numfmt numericalg numericcmp numericdiv decimalfmt mathfn mathround powexp float floatspecial floatcast intops intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns textpos textcase textlength textnum textsplit strfunc strpos substring padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch regexpreplace char charconv charfuncs varchartypmod cash bool boolcast booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce arraysort arraynull anyall inlist range rangeops rangeagg multirange inet inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc timeops timeextract timestamptz tstzarith tzconv interval intervalparts intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn width_bucket genseries genseries2 castchain concatop coerce collate textord jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window windowframes windowrank windowexcl windowagg windownav windowdistinct grouphaving groupingsets groupsets2 grouphash havingclause rollup2 partprune nbtree indexscan indexdiff joinmethods outerjoin joincard selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint returning upsert mergecmd updatesem defaults checkconstraint foreignkey views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality query_pqs query_planadvice query_ext_citext query_multi query_deep query_diff dml_model syscol collation cf_types cf_index cf_exec cf_ddl cf_ddl2 cf_deparse cf_stats cf_encoding cf_planner cf_race index_am table_am pl_langs integrity constraints isolation contrib query_window spgist authz concurrency); +# +# Active areas run by default: the query generator and its oracles, the +# stateful / concurrency / AM areas, and the commitfest properties. These +# are where the suite has found real bugs. +my @active = qw( + query_tlp query_tlp_agg query_tlp_distinct query_norec query_cardinality + query_pqs query_planadvice query_ext_citext query_multi query_deep + query_diff dml_model syscol collation cf_types cf_index cf_exec cf_ddl + cf_ddl2 cf_deparse cf_stats cf_encoding cf_planner cf_race index_am + table_am pl_langs integrity constraints isolation contrib query_window + spgist authz concurrency +); + +# Frozen areas: the per-type scalar properties (one function, one oracle). +# They compile and still run on request (PG_PROP_FROZEN=1), but are not +# maintained: in the 2026 evaluation they found no server bug, and their +# findings were almost all unsound oracles. A frozen area that breaks after +# an upstream change may be deleted rather than fixed. +my @frozen = qw( + numeric numcast numscale numround numfmt numericalg numericcmp numericdiv + decimalfmt mathfn mathround powexp float floatspecial floatcast intops + intbit bitint bitlogic bitstringops safemath arithcmp cmpops text textfns + textpos textcase textlength textnum textsplit strfunc strpos substring + padtrim trimfns like likepattern stringmatch regexp regexfns regexmatch + regexpreplace char charconv charfuncs varchartypmod cash bool boolcast + booltest boolexpr casewhen condexpr coalesce coalescefn nullsemantics + nulldistinct oid uuid uuidops bytea byteaops bit record enum jsonb jsonbops + jsonbpath jsonbagg2 jsonbnum jsonbtype jsonbsub jsonbkeys jsonbcontain + jsonbdelete jsonbexists jsonbmerge jsonbarray jsonbroundtrip jsonnest + jsonbuild jsonconstruct jsontext jsonformat jsonpath jsonpathq jsonpathmath + jsoneach array arrayops arrayslice arraycmp arrayconstruct arrayreduce + arraysort arraynull anyall inlist range rangeops rangeagg multirange inet + inetarith networkops cidrops inet6 macaddr macaddr8 tsquery tsvector + tsvector2 tsquerylaw tsrank date dateparts datefmt datecast datetrunc + timeops timeextract timestamptz tstzarith tzconv interval intervalparts + intervalcmp intervalio tochar toformat scalarfn hashfn hashfns2 pglsn + width_bucket genseries genseries2 castchain concatop coerce collate textord + jsonvsjsonb geopoint geoseg boxops xmlfmt aggregate aggstrategies aggorder + aggdistinct aggdedup boolagg minmaxagg ordagg percentilefn filteragg + countfilter stringagg strconcat jsonagg jsonaggord jsonobjectagg window + windowframes windowrank windowexcl windowagg windownav windowdistinct + grouphaving groupingsets groupsets2 grouphash havingclause rollup2 + partprune nbtree indexscan indexdiff joinmethods outerjoin joincard + selfjoin jointypes antisemijoin setops setopstrat exceptintersect unionall + subquery subqscalar corrsubq cte recursivecte dmlcte lateral valuesrows srf + unnestjoin distinct distincton sortlimit sortmerge orderbynulls fetchties + pagination predpushdown spillpaths tablesample rowcompare mvcc savepoint + returning upsert mergecmd updatesem defaults checkconstraint foreignkey + views countsem hstore hstore2 ltree ltree2 citext citext2 cube cubedist seg + intarray intarrayext pgtrgm pgtrgm2 fuzzystr unaccent isn +); + +my @areas = @active; +push @areas, @frozen + if defined $ENV{PG_PROP_FROZEN} && $ENV{PG_PROP_FROZEN} ne '' + && $ENV{PG_PROP_FROZEN} ne '0'; # No Tier-B areas wired in yet (e.g. the initial harness-only commit): nothing # to run, which is a pass. Skip before starting a node, so we don't pay for a From 2ab0336833c7f39c342ba33c6bd9988afdbbe1a3 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 19 Oct 2026 11:49:06 -0400 Subject: [PATCH 325/338] prop: widen the query generator (FULL/RIGHT joins, nondeterministic collation) The 2026 analysis found that most recent planner/executor wrong-results fixes need query shapes the generator never produced, not new oracles. This adds the two highest-yield shapes and makes the differential oracle sound for them: - Join types: the deep join builder now draws INNER / LEFT / RIGHT / FULL for its non-lateral members (lateral members stay INNER / LEFT, since a lateral reference cannot sit on the nullable side of a RIGHT/FULL join). RIGHT and FULL null-extend the left input, exercising the outer-join reduction, qual-placement and PlaceHolderVar paths behind many 2026 fixes. - A nondeterministic (case-insensitive ICU) collation column, qg_ci, added to some text columns under the deep profiles when the server has ICU. Over such a column the predicate builder emits only the operators that neither error nor distinguish equal values -- the six comparisons, IS [NOT] DISTINCT FROM, and the null tests -- per the collation support rules; no cast to another collation, no array ANY/ALL, no pattern match. - pg_prop_sql_rowhash_unordered: an order-insensitive result digest (the per-row text is sorted server-side before hashing). query_diff uses it, so two plans agree when they return the same multiset of rows. This removes the whole class of false positive from unspecified row order -- a FULL/RIGHT join that null-extends every key, or equal sort keys under a nondeterministic collation have no total order to compare by. A Debian x86 build (upstream 0f3b42c, ICU enabled) runs the deep differential oracle over the new shapes: FULL JOIN, RIGHT JOIN and qg_ci all appear in generated queries and all five diff_deep_* properties pass on master, so the wider vocabulary introduces no false positives. --- src/test/prop/pg_prop.h | 15 ++++ src/test/prop/pg_prop_sql.c | 54 ++++++++---- src/test/prop/sql/prop_query_diff.c | 9 +- src/test/prop/sql/query_gen.c | 125 +++++++++++++++++++++++++++- src/test/prop/sql/query_gen.h | 8 ++ 5 files changed, 187 insertions(+), 24 deletions(-) diff --git a/src/test/prop/pg_prop.h b/src/test/prop/pg_prop.h index a8b46fd5be4f0..8784c71981672 100644 --- a/src/test/prop/pg_prop.h +++ b/src/test/prop/pg_prop.h @@ -299,6 +299,21 @@ extern char *pg_prop_sql_rows(pg_prop_conn *conn, const char *sql, extern char *pg_prop_sql_rowhash(pg_prop_conn *conn, const char *sql, bool *failed, char **errmsg); +/* + * Like pg_prop_sql_rowhash but order-insensitive: the per-row text is sorted + * server-side before hashing, so two queries that return the same *multiset* + * of rows compare equal regardless of row order. Use this for any oracle + * that compares results across plans or query rewrites: SQL never guarantees + * row order without ORDER BY, and even with ORDER BY a tie among equal sort + * keys (equal numeric scales, a nondeterministic collation, a FULL/RIGHT join + * that null-extends every key) leaves the order unspecified. Comparing + * multisets removes that whole class of false positive (false_positives.md + * 2.1-2.2) and lets the generator emit shapes whose output has no total order. + * Counts and duplicates are preserved (it is a multiset, not a set). + */ +extern char *pg_prop_sql_rowhash_unordered(pg_prop_conn *conn, const char *sql, + bool *failed, char **errmsg); + /* * Run "sql" for side effects (DDL/DML), returning true on success. On error * returns false and sets *errmsg (malloc'd). diff --git a/src/test/prop/pg_prop_sql.c b/src/test/prop/pg_prop_sql.c index bfa8f17a5fe1a..24f8841a51dbb 100644 --- a/src/test/prop/pg_prop_sql.c +++ b/src/test/prop/pg_prop_sql.c @@ -235,9 +235,19 @@ pg_prop_sql_rows(pg_prop_conn *conn, const char *sql, bool *failed, char **errms return s.data; /* caller frees */ } -char * -pg_prop_sql_rowhash(pg_prop_conn *conn, const char *sql, bool *failed, - char **errmsg) +/* + * Shared body for the two rowhash variants. Digests the whole result + * server-side: md5 of the newline-joined per-row text. Returning a 32-char + * hash instead of the full result keeps client memory O(1) regardless of + * result size (the deep generators can produce results that overflow the + * 1 GB StringInfo cap when rendered client-side; see finding 027). When + * "ordered" is false the per-row text is sorted first, so the digest is of + * the row multiset rather than the row sequence. An empty result hashes + * md5('') so two empty results compare equal. + */ +static char * +rowhash_body(pg_prop_conn *conn, const char *sql, bool ordered, + bool *failed, char **errmsg) { StringInfoData q; PGresult *res; @@ -247,21 +257,17 @@ pg_prop_sql_rowhash(pg_prop_conn *conn, const char *sql, bool *failed, if (failed) *failed = false; - /* - * Digest the whole result server-side: md5 of the newline-joined text of - * every row of the caller's query, taken in the query's own output order - * (so the caller's ORDER BY fixes the order). Returning a 32-char hash - * instead of the full result keeps the client memory O(1) regardless of - * result size -- the deep generators can produce results that overflow the - * 1 GB StringInfo cap when rendered client-side (see finding 027). The - * empty result hashes md5('') deterministically, so two empty results - * compare equal. - */ initStringInfo(&q); - appendStringInfo(&q, - "SELECT md5(coalesce(string_agg(q.r, E'\\n'), '')) " - "FROM (SELECT (sub)::text AS r FROM (%s) sub) q", - sql); + if (ordered) + appendStringInfo(&q, + "SELECT md5(coalesce(string_agg(q.r, E'\\n'), '')) " + "FROM (SELECT (sub)::text AS r FROM (%s) sub) q", + sql); + else + appendStringInfo(&q, + "SELECT md5(coalesce(string_agg(q.r, E'\\n' ORDER BY q.r), '')) " + "FROM (SELECT (sub)::text AS r FROM (%s) sub) q", + sql); res = PQexec(conn->conn, q.data); check_backend_alive(conn, sql); pfree(q.data); @@ -285,6 +291,20 @@ pg_prop_sql_rowhash(pg_prop_conn *conn, const char *sql, bool *failed, return out; } +char * +pg_prop_sql_rowhash(pg_prop_conn *conn, const char *sql, bool *failed, + char **errmsg) +{ + return rowhash_body(conn, sql, true, failed, errmsg); +} + +char * +pg_prop_sql_rowhash_unordered(pg_prop_conn *conn, const char *sql, + bool *failed, char **errmsg) +{ + return rowhash_body(conn, sql, false, failed, errmsg); +} + bool pg_prop_sql_exec(pg_prop_conn *conn, const char *sql, char **errmsg) { diff --git a/src/test/prop/sql/prop_query_diff.c b/src/test/prop/sql/prop_query_diff.c index be72057fbd927..d78d76f3131f3 100644 --- a/src/test/prop/sql/prop_query_diff.c +++ b/src/test/prop/sql/prop_query_diff.c @@ -22,8 +22,11 @@ * * The configurations are applied with SET LOCAL inside a transaction so they * cannot leak between runs. Results are compared by server-side digest - * (pg_prop_sql_rowhash), so arbitrarily large results cost O(1) client memory. - * Every query carries a total ORDER BY so "same rows" means "same digest". + * (pg_prop_sql_rowhash_unordered), so arbitrarily large results cost O(1) + * client memory and row order never matters: two plans that return the same + * multiset of rows agree, even when the query has no total order (a FULL/RIGHT + * join that null-extends every key, equal sort keys under a nondeterministic + * collation). See false_positives.md 2.1-2.2. * * Portions Copyright (c) 2024, PostgreSQL Global Development Group * @@ -129,7 +132,7 @@ digest_under(pg_prop_conn *conn, const char *setup, const char *query, *errmsg = err; return NULL; } - dig = pg_prop_sql_rowhash(conn, query, &failed, &err); + dig = pg_prop_sql_rowhash_unordered(conn, query, &failed, &err); pg_prop_sql_exec(conn, "ROLLBACK", NULL); if (failed) diff --git a/src/test/prop/sql/query_gen.c b/src/test/prop/sql/query_gen.c index 212c44a37c349..6b5ea80f69c01 100644 --- a/src/test/prop/sql/query_gen.c +++ b/src/test/prop/sql/query_gen.c @@ -181,6 +181,7 @@ qg_add_column(qg_schema *s, const char *name, const char *sqltype, s->cols[s->ncols].draw = draw; s->cols[s->ncols].is_text_like = is_text_like; s->cols[s->ncols].is_numeric_like = is_numeric_like; + s->cols[s->ncols].nondet_collation = false; s->ncols++; } @@ -616,6 +617,7 @@ qg_db_params_default(qg_db_params *p) p->allow_inherit = false; p->toast_profile = 0; p->cardinality_bias = 0; + p->allow_nondet_collation = false; } void @@ -632,6 +634,7 @@ qg_db_params_deep(qg_db_params *p) p->allow_inherit = true; p->toast_profile = 2; p->cardinality_bias = 0; + p->allow_nondet_collation = true; } /* @@ -656,6 +659,7 @@ qg_db_params_deep_fast(qg_db_params *p) p->allow_inherit = true; p->toast_profile = 1; /* ~10% large values: TOAST path, less bulk */ p->cardinality_bias = 0; + p->allow_nondet_collation = true; } /* @@ -694,7 +698,7 @@ qg_db_params_for_env(qg_db_params *p) */ static void build_table_schema(pg_prop_tc *tc, qg_table *t, int ncols, int fkref, - int cardinality_bias) + int cardinality_bias, bool nondet) { char *namebuf; int i; @@ -756,7 +760,23 @@ build_table_schema(pg_prop_tc *tc, qg_table *t, int ncols, int fkref, break; } namebuf = psprintf("c%d", i); - qg_add_column(&t->schema, namebuf, type, draw, textlike, numlike); + + /* + * Occasionally make a plain-text column use the nondeterministic + * (case-insensitive ICU) collation qg_ci. Only plain draw_text_or_null + * columns qualify: the collation reaches equality, grouping, join and + * index code without the TOAST-sized values that would make the leaf + * allow-list (6.3) harder to reason about. The column is marked so + * draw_leaf_on restricts it to the nondeterministic-safe operators. + */ + if (nondet && draw == draw_text_or_null && pg_prop_draw_int(tc, 0, 2) == 0) + { + qg_add_column(&t->schema, namebuf, "text COLLATE qg_ci", draw, + textlike, numlike); + t->schema.cols[t->schema.ncols - 1].nondet_collation = true; + } + else + qg_add_column(&t->schema, namebuf, type, draw, textlike, numlike); } } @@ -799,6 +819,30 @@ emit_storage_modes(pg_prop_tc *tc, StringInfo ddl, const qg_table *t) } } +/* + * Does this server have ICU (so a nondeterministic ICU collation can be + * created)? Probed once per process and cached: 1 yes, 0 no, -1 unknown. + */ +static int qg_has_icu = -1; + +static bool +qg_server_has_icu(pg_prop_tc *tc, pg_prop_conn *conn) +{ + bool failed = false; + char *v; + + (void) tc; + if (qg_has_icu >= 0) + return qg_has_icu == 1; + v = pg_prop_sql_scalar(conn, + "SELECT EXISTS (SELECT 1 FROM pg_collation WHERE collprovider = 'i')", + &failed, NULL); + qg_has_icu = (!failed && v && strcmp(v, "t") == 0) ? 1 : 0; + if (v) + pg_free(v); + return qg_has_icu == 1; +} + qg_db * qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) { @@ -807,9 +851,17 @@ qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) char *err = NULL; int nt; int ti; + bool nondet; qg_toast_profile = p->toast_profile; + /* + * Nondeterministic collations need ICU. Probe once per process; if the + * server has no ICU-provided collation, drop the feature for this run + * rather than failing every case with "collation ... does not exist". + */ + nondet = p->allow_nondet_collation && qg_server_has_icu(tc, conn); + nt = (int) pg_prop_draw_int(tc, p->min_tables, p->max_tables); if (nt < 1) nt = 1; @@ -819,6 +871,19 @@ qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) initStringInfo(&ddl); + /* + * Create the nondeterministic (case-insensitive, accent-sensitive) ICU + * collation this case will use, if it does not already exist. Dropping + * it is not worth the churn: it is harmless to leave behind and reused + * across cases. + */ + if (nondet) + appendStringInfoString(&ddl, + "DO $qg$ BEGIN " + "IF NOT EXISTS (SELECT 1 FROM pg_collation WHERE collname = 'qg_ci') THEN " + "CREATE COLLATION qg_ci (provider = icu, locale = 'und-u-ks-level2', " + "deterministic = false); END IF; END $qg$; "); + /* * A result-neutral trigger function: it fires on write but returns the * row unchanged, so it changes nothing a SELECT can observe while still @@ -847,7 +912,7 @@ qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) fkref = (int) pg_prop_draw_int(tc, 0, ti - 1); ncols = (int) pg_prop_draw_int(tc, p->min_cols, p->max_cols); - build_table_schema(tc, t, ncols, fkref, p->cardinality_bias); + build_table_schema(tc, t, ncols, fkref, p->cardinality_bias, nondet); /* DROP first (reverse order handled by qg_drop_db; here just reset) */ appendStringInfo(&ddl, "DROP TABLE IF EXISTS %s CASCADE; ", t->name); @@ -1039,6 +1104,40 @@ draw_leaf_on(pg_prop_tc *tc, StringInfo buf, const qg_table *t, ci = (int) pg_prop_draw_int(tc, 0, t->schema.ncols - 1); c = &t->schema.cols[ci]; + /* + * A nondeterministic-collation column admits only the operators that do + * not distinguish values the collation treats as equal and do not error + * (false_positives.md 6.3): the six comparisons and IS [NOT] DISTINCT + * FROM against a bare text literal (which takes the column's collation), + * plus the null tests. No cast to another collation, no array ANY/ALL + * (whose element comparison re-derives a collation), no LIKE / pattern / + * function. The literal is drawn to hit the equal-but-different-bytes + * case ('a' vs 'A') so the collation's merging actually matters. + */ + if (c->nondet_collation) + { + switch ((int) pg_prop_draw_int(tc, 0, 3)) + { + case 0: + appendStringInfo(buf, "%s.%s IS NULL", alias, c->name); + break; + case 1: + appendStringInfo(buf, "%s.%s IS NOT NULL", alias, c->name); + break; + case 2: + appendStringInfo(buf, "%s.%s IS %sDISTINCT FROM ", alias, c->name, + pg_prop_draw_bool(tc) ? "NOT " : ""); + draw_text_or_null(tc, buf); + break; + default: + appendStringInfo(buf, "%s.%s %s ", alias, c->name, + cmp[pg_prop_draw_int(tc, 0, 5)]); + draw_text_or_null(tc, buf); + break; + } + return; + } + switch ((int) pg_prop_draw_int(tc, 0, 7)) { case 0: @@ -1220,12 +1319,30 @@ build_join_chain(pg_prop_tc *tc, const qg_db *db, int start, int max_joins, { const char *tn = db->tables[nexttab].name; const char *prev = aliases[n - 1]; - const char *jt = pg_prop_draw_bool(tc) ? "" : "LEFT"; int shape = (int) pg_prop_draw_int(tc, 0, 6); /* derived shapes expose id, plus fk when the table has one */ const char *cols = db->tables[nexttab].fkcol >= 0 ? "id, fk" : "id"; const char *xcols = db->tables[nexttab].fkcol >= 0 ? "x.id, x.fk" : "x.id"; + /* + * Join type. INNER / LEFT apply to any shape. RIGHT and FULL + * null-extend the *left* input, which the planner's outer-join + * reduction, qual placement and PlaceHolderVar machinery handle + * on a different path than LEFT -- the source of many recent + * wrong-results fixes (false_positives.md 7.7). They are only + * legal where the right side does not reference the left, so they + * are drawn only for the non-lateral shapes (0-3 and default); the + * LATERAL shapes (4-6) stay INNER / LEFT. A FULL/RIGHT join can + * leave every projected key NULL in an unmatched row, so the + * oracle must compare the row multiset, not a key-ordered + * sequence (rowhash_unordered); the deep oracles do. + */ + static const char *const jts[] = {"", "LEFT", "RIGHT", "FULL"}; + bool lateral = (shape == 4 || shape == 5 || shape == 6); + const char *jt = lateral + ? (pg_prop_draw_bool(tc) ? "" : "LEFT") + : jts[pg_prop_draw_int(tc, 0, 3)]; + switch (shape) { case 1: diff --git a/src/test/prop/sql/query_gen.h b/src/test/prop/sql/query_gen.h index 1cc5bf05cbada..e133fee30901f 100644 --- a/src/test/prop/sql/query_gen.h +++ b/src/test/prop/sql/query_gen.h @@ -60,6 +60,10 @@ typedef struct qg_column qg_value_fn draw; /* emits one literal or NULL */ bool is_text_like; /* comparable with text operators/order */ bool is_numeric_like; /* usable in arithmetic / numeric compare */ + bool nondet_collation; /* text column with a nondeterministic + * collation: only the operators on the + * nondeterministic-safe allow-list may be + * used over it (false_positives.md 6.3) */ } qg_column; typedef struct qg_schema @@ -176,6 +180,10 @@ typedef struct qg_db_params bool allow_inherit; /* give some tables an inheritance child */ int toast_profile; /* 0 none .. 3 always (0b) */ int cardinality_bias; /* 0 mixed, 1 favour low-card columns */ + bool allow_nondet_collation; /* add text columns with a nondeterministic + * ICU collation (case-insensitive); only in + * the deep profiles, and only if the server + * has ICU (probed at build time) */ } qg_db_params; extern void qg_db_params_default(qg_db_params *p); From 01f82aa44341fa711185075a8706b45e374301db Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 19 Oct 2026 11:52:35 -0400 Subject: [PATCH 326/338] prop: set-operation and empty-arm differential properties (query_diff) Two more shapes from the SQL-vocabulary gap, checked by the existing plan-differential oracle: - diff_setops: two filtered scans of the adversarial table combined with a drawn UNION / INTERSECT / EXCEPT [ALL]. The planner runs each as a hashed setop, sort+unique, or append+dedup; all must agree on the result multiset. The projection is the data columns (not the unique key), so INTERSECT and EXCEPT do real work. - diff_empty_and_fromless: a join or set operation against a provably-empty relation (WHERE false) or a FROM-less constant SELECT -- the shapes behind several 2026 join-removal and set-op dummy-elimination wrong-results fixes. A wrong elimination drops a qual or emits a spurious row. Both pass on master (upstream 0f3b42c, Debian x86) across four seeds, so they add vocabulary without false positives. No registration change: a PG_PROP in an existing area self-registers. --- src/test/prop/sql/prop_query_diff.c | 121 ++++++++++++++++++++++++++++ 1 file changed, 121 insertions(+) diff --git a/src/test/prop/sql/prop_query_diff.c b/src/test/prop/sql/prop_query_diff.c index d78d76f3131f3..9145fdf578460 100644 --- a/src/test/prop/sql/prop_query_diff.c +++ b/src/test/prop/sql/prop_query_diff.c @@ -538,3 +538,124 @@ PG_PROP(diff_deep_filtered) pfree(sql.data); deep_teardown(tc, conn, db, &q); } + +/* + * Set-operation differential: two filtered scans of the adversarial table + * combined with a drawn set operator (UNION / INTERSECT / EXCEPT, each with + * and without ALL). The planner has several ways to run each -- hashed + * setop, sort+unique, append then dedup -- and they must agree on the + * result multiset. Set operations use the row type's equality (not image + * equality), so a NULL in one side matches a NULL in the other; that is + * well-defined and the same for every plan, so the comparison is sound even + * with NULLs and mixed numeric scales in the projection (the scale is fixed + * by numeric(5,2), false_positives.md 1.2). No ORDER BY needed: the digest + * is order-insensitive. + */ +PG_PROP(diff_setops) +{ + static const char *const ops[] = { + "UNION", "UNION ALL", "INTERSECT", "INTERSECT ALL", "EXCEPT", "EXCEPT ALL", + }; + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + StringInfoData p1; + StringInfoData p2; + const char *op = ops[pg_prop_draw_int(tc, 0, lengthof(ops) - 1)]; + int nrows; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + qg_build_table(tc, conn, &nrows); + + initStringInfo(&p1); + qg_draw_predicate(tc, &p1); + initStringInfo(&p2); + qg_draw_predicate(tc, &p2); + + /* + * Project the data columns, not id: a set operation over the unique key + * would make every INTERSECT empty and every EXCEPT trivial. The data + * columns repeat, so INTERSECT / EXCEPT actually do work. + */ + initStringInfo(&q); + appendStringInfo(&q, + "SELECT a, b, c, d, f, g FROM " QG_TABLE " WHERE %s " + "%s " + "SELECT a, b, c, d, f, g FROM " QG_TABLE " WHERE %s", + p1.data, op, p2.data); + run_diff(tc, conn, q.data); + + pfree(p1.data); + pfree(p2.data); + pfree(q.data); + pg_prop_disconnect(conn); +} + +/* + * Empty-arm and FROM-less differential: a join or set operation against a + * provably-empty relation (WHERE false, or a FROM-less SELECT whose WHERE is + * false) and against a FROM-less constant SELECT. These are the shapes + * behind several 2026 join-removal / set-op-dummy-elimination wrong-results + * fixes (false_positives.md none -- these are true-positive territory): the + * planner eliminates the empty arm or the dummy rel, and a wrong + * elimination drops a qual or emits a spurious row. Every plan must still + * produce the same multiset. + */ +PG_PROP(diff_empty_and_fromless) +{ + pg_prop_conn *conn; + char *err = NULL; + StringInfoData q; + StringInfoData pred; + int shape = (int) pg_prop_draw_int(tc, 0, 4); + int nrows; + + conn = pg_prop_connect(&err); + if (conn == NULL) + pg_prop_fail(tc, "connect failed: %s", err ? err : ""); + qg_build_table(tc, conn, &nrows); + + initStringInfo(&pred); + qg_draw_predicate(tc, &pred); + initStringInfo(&q); + switch (shape) + { + case 0: /* LEFT JOIN to a provably-empty subquery */ + appendStringInfo(&q, + "SELECT t.id, e.x FROM " QG_TABLE " t " + "LEFT JOIN (SELECT id, a AS x FROM " QG_TABLE " WHERE false) e " + "ON e.id = t.id WHERE %s", pred.data); + break; + case 1: /* FULL JOIN to a FROM-less constant */ + appendStringInfo(&q, + "SELECT t.id, v.k FROM " QG_TABLE " t " + "FULL JOIN (SELECT 1 AS k WHERE %s) v ON v.k = t.id", + pg_prop_draw_bool(tc) ? "true" : "false"); + break; + case 2: /* UNION with an empty arm */ + appendStringInfo(&q, + "SELECT a, b FROM " QG_TABLE " WHERE %s " + "UNION ALL SELECT a, b FROM " QG_TABLE " WHERE false", + pred.data); + break; + case 3: /* INTERSECT with a FROM-less row */ + appendStringInfo(&q, + "SELECT a FROM " QG_TABLE " WHERE %s " + "INTERSECT SELECT %d", + pred.data, (int) pg_prop_draw_int(tc, -1, 2)); + break; + default: /* NOT EXISTS over an empty subquery (anti-join) */ + appendStringInfo(&q, + "SELECT t.id FROM " QG_TABLE " t WHERE %s AND NOT EXISTS " + "(SELECT 1 FROM " QG_TABLE " e WHERE false AND e.id = t.id)", + pred.data); + break; + } + run_diff(tc, conn, q.data); + + pfree(pred.data); + pfree(q.data); + pg_prop_disconnect(conn); +} From 97bb88f0bbed3a44253ea98b26ea074ab01a9b9f Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Mon, 19 Oct 2026 12:35:18 -0400 Subject: [PATCH 327/338] prop: note the deep multi-table generator is not reproducible for a seed While widening the generator this session, the engine's failure-replay exposed that the deep multi-table builder (qg_build_db / qg_build_query) can build a different query on a second run of the same seed, so a deep counterexample does not replay. The single-table generator and its oracles are unaffected and reproducible. Document the limit so a deep disagreement is treated as a lead to reproduce by hand, not as a regression gate, until the engine-level cause (findings 041) is resolved. --- src/test/prop/README | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/test/prop/README b/src/test/prop/README index 9229bfcb2e682..bc29c303c595f 100644 --- a/src/test/prop/README +++ b/src/test/prop/README @@ -173,6 +173,15 @@ New work goes into the generator (sql/query_gen.c) rather than new oracles. A new query shape is checked by every existing oracle at once; a new oracle checks only the shapes the generator already makes. +The single-table generator (qg_build_table) and the oracles over it are +reproducible: a counterexample replays from its seed. The deep multi-table +generator (qg_build_db / qg_build_query, used by the query_deep, query_multi +and diff_deep_* properties under PG_PROP_DEEP) is not currently reproducible +for a fixed seed -- the engine's failure-replay can build a different query +(see findings 041). Treat a deep disagreement as a lead to reproduce by +hand, not as a replayable counterexample, and do not use a deep run as a +regression gate until that is fixed. + Commitfest properties and retirement ==================================== From e1356c174034fe03461747e9b90dd8d5f7c07ccf Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Tue, 20 Oct 2026 15:46:44 -0400 Subject: [PATCH 328/338] prop: heap rewrite / index build over vacuumed toast of a held tuple (cf_race) 6824 rewrite_with_vacuumed_toast_of_held_tuple VACUUM FULL / CLUSTER / CREATE INDEX must not fail with "missing chunk number N for toast value" when a dead heap tuple's TOAST has been vacuumed away while the main tuple is still held back by a concurrent transaction's xmin. The precondition is the one from the CF 6824 recovery test: a global xmin holder in another database keeps the dead main tuple from being removed; a same-database REPEATABLE READ holder spans the DELETE and a VACUUM (PROCESS_TOAST false); then VACUUM of the toast table directly removes the now-dead chunks. The rewrite then copies the dead tuple and detoasts a value whose chunks are gone. rc_connect_db() is added to open a connection in a named database (the cross-database holder), creating it from template0 if absent. On upstream master 3745790 the property falsifies 3/3 with the exact missing-chunk error; with the CF 6824 backend patch applied it passes 3/3. --- src/test/prop/sql/prop_cf_race.c | 143 +++++++++++++++++++++++++++++++ 1 file changed, 143 insertions(+) diff --git a/src/test/prop/sql/prop_cf_race.c b/src/test/prop/sql/prop_cf_race.c index 80a1cef30f67a..798d267192003 100644 --- a/src/test/prop/sql/prop_cf_race.c +++ b/src/test/prop/sql/prop_cf_race.c @@ -39,6 +39,51 @@ rc_connect(pg_prop_tc *tc) return conn; } +/* + * Connect to a named database (creating it from template0 if absent), by + * briefly pointing PGDATABASE at it around pg_prop_connect. Used where a + * property needs a connection in a database other than the one the driver + * selected -- e.g. a cross-database xmin holder. Returns NULL if the + * database cannot be created. + */ +static pg_prop_conn * +rc_connect_db(pg_prop_tc *tc, const char *dbname) +{ + char *err = NULL; + pg_prop_conn *admin = rc_connect(tc); + bool f = false; + char *ex; + StringInfoData q; + const char *saved; + pg_prop_conn *conn; + + initStringInfo(&q); + appendStringInfo(&q, "SELECT count(*) FROM pg_database WHERE datname = '%s'", dbname); + ex = pg_prop_sql_scalar(admin, q.data, &f, NULL); + if (!f && ex && strcmp(ex, "0") == 0) + { + resetStringInfo(&q); + appendStringInfo(&q, "CREATE DATABASE %s TEMPLATE template0", dbname); + if (!pg_prop_sql_exec(admin, q.data, &err)) + pg_prop_fail(tc, "CREATE DATABASE %s failed: %s", dbname, err ? err : ""); + } + if (ex) + pg_free(ex); + pfree(q.data); + pg_prop_disconnect(admin); + + saved = getenv("PGDATABASE"); + setenv("PGDATABASE", dbname, 1); + conn = pg_prop_connect(&err); + if (saved) + setenv("PGDATABASE", saved, 1); + else + unsetenv("PGDATABASE"); + if (conn == NULL) + pg_prop_fail(tc, "connect to %s failed: %s", dbname, err ? err : ""); + return conn; +} + static void rc_exec(pg_prop_tc *tc, pg_prop_conn *conn, const char *sql) { @@ -530,3 +575,101 @@ PG_PROP(root_alter_races_deep_attach_without_deadlock) pg_prop_disconnect(s2); pg_prop_disconnect(s1); } + +/* + * CF 6824: a heap rewrite (VACUUM FULL / CLUSTER / REPACK) or an index build + * must not fail with "missing chunk number N for toast value" when a dead + * heap tuple's TOAST has already been vacuumed away while the main tuple is + * still held back by a concurrent transaction's xmin. The rewrite copies + * the dead tuple (it is not yet removable) and tries to detoast a value whose + * chunks are gone. + * + * Precondition, from the patch's own recovery test: + * - a global xmin holder in another database (holds back the global horizon + * so the dead main tuple is copied, not removed); + * - a same-database REPEATABLE READ holder open across the DELETE and a + * VACUUM (PROCESS_TOAST false), so the main tuple survives that vacuum; + * - release the same-db holder, then VACUUM the toast table directly, which + * removes the now-dead toast chunks. + * Then the rewrite / index build runs while the global holder is still open. + * + * Invariant: the operation succeeds or raises no missing/unexpected-chunk + * error. Drawn: the operation, value size (all out-of-line), row count, + * which row is deleted. Fails on master; passes with the CF 6824 patch. + */ +PG_PROP(rewrite_with_vacuumed_toast_of_held_tuple) +{ + static const char *const ops[] = { + "VACUUM FULL cfr_rw", + "CLUSTER cfr_rw USING cfr_rw_pkey", + "CREATE INDEX cfr_rw_i ON cfr_rw ((cfr_rw IS NOT NULL))", + }; + pg_prop_conn *work = rc_connect(tc); + pg_prop_conn *gholder; + pg_prop_conn *sholder; + const char *op = ops[pg_prop_draw_int(tc, 0, lengthof(ops) - 1)]; + int nrows = (int) pg_prop_draw_int(tc, 2, 12); + int len = (int) pg_prop_draw_int(tc, 2500, 8000); + int delid = (int) pg_prop_draw_int(tc, 1, nrows); + StringInfoData s; + char *toast; + bool f = false; + char *st = NULL; + char *err = NULL; + + /* global cross-database xmin holder, open for the whole case */ + gholder = rc_connect_db(tc, "pgprop_rw_holder"); + rc_exec(tc, gholder, "BEGIN"); + rc_exec(tc, gholder, "CREATE TEMP TABLE IF NOT EXISTS cfr_xid_holder (x int)"); + /* ensure it has a real XID so it holds back the global horizon */ + rc_exec(tc, gholder, "INSERT INTO cfr_xid_holder VALUES (1)"); + + initStringInfo(&s); + appendStringInfo(&s, + "DROP TABLE IF EXISTS cfr_rw; " + "CREATE TABLE cfr_rw (id int PRIMARY KEY, data text) WITH (autovacuum_enabled = off); " + "ALTER TABLE cfr_rw ALTER COLUMN data SET STORAGE EXTERNAL; " + "INSERT INTO cfr_rw SELECT g, repeat('x', %d) FROM generate_series(1, %d) g", + len, nrows); + rc_exec(tc, work, s.data); + toast = rc_scalar(tc, work, + "SELECT 'pg_toast.' || relname FROM pg_class " + "WHERE oid = (SELECT reltoastrelid FROM pg_class WHERE relname = 'cfr_rw')"); + + /* same-db holder pins the dead main tuple across the no-toast vacuum */ + sholder = rc_connect(tc); + rc_exec(tc, sholder, "BEGIN ISOLATION LEVEL REPEATABLE READ"); + rc_exec(tc, sholder, "SELECT 1"); + resetStringInfo(&s); + appendStringInfo(&s, "DELETE FROM cfr_rw WHERE id = %d", delid); + rc_exec(tc, work, s.data); + rc_exec(tc, work, "VACUUM (PROCESS_TOAST false) cfr_rw"); + rc_exec(tc, sholder, "COMMIT"); + pg_prop_disconnect(sholder); + + /* now remove the dead toast chunks while the main tuple is still held */ + resetStringInfo(&s); + appendStringInfo(&s, "VACUUM %s", toast); + rc_exec(tc, work, s.data); + + /* the operation under test */ + if (!pg_prop_sql_send(work, op)) + pg_prop_fail(tc, "could not dispatch %s", op); + if (!pg_prop_sql_await(work, &st, &err)) + { + if (err && (strstr(err, "missing chunk") || strstr(err, "unexpected chunk"))) + pg_prop_fail(tc, "%s failed after the tuple's toast was vacuumed: %s\n" + " %d rows, %d-byte values, deleted id %d", + op, err, nrows, len, delid); + if (rc_internal(st, err)) + pg_prop_fail(tc, "%s raised an internal error: %s", op, err ? err : ""); + } + + rc_exec(tc, gholder, "COMMIT"); + pg_prop_disconnect(gholder); + pg_prop_sql_exec(work, "DROP TABLE IF EXISTS cfr_rw", NULL); + pg_free(toast); + pfree(s.data); + pg_prop_disconnect(work); + (void) f; +} From 89ddc3fbb719443005e76363003d1ee6a952176a Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Fri, 23 Oct 2026 12:06:58 -0400 Subject: [PATCH 329/338] prop: deep-generator reproducibility (PRNG) and patch-biased generation Two generator improvements, both opt-in and default-off, staged for build verification on a host with ICU + libhegel: Task #1 -- reproducibility (PG_PROP_GEN_PRNG=1). The deep multi-table builder makes hundreds of data-dependent draws per case (ranges that depend on prior drawn values). Threaded straight through the engine, that long variable-shape sequence did not replay from a fixed seed (finding 041), so a deep counterexample could not be reproduced. With the flag set, the builder draws one engine integer per case and drives all structural and value choices from a deterministic splitmix64 seeded by it (qg_rng_begin/ qg_rng_end around qg_build_db and qg_build_query; qg_draw_int/qg_draw_bool replace the internal engine draws). The engine then sees a single fixed-shape draw per case, so the case replays regardless of its internal buffer semantics; the cost is coarser shrinking of deep structure, which is worth little for a hunt/review tool next to being able to replay a hit. qg_draw_int also returns min for a degenerate/inverted range instead of asking the engine for an empty interval (itself a hazard). Task #3 -- patch-biased generation (PG_PROP_GEN_BIAS=tag,...). scripts/cf-features.py scans a commitfest patch's diff and prints the SQL surface it touches (joins, setops, subquery, nulls, collation, numeric, partition, toast, index, constraint, cast, ...). Fed to the generator, it reweights the join-type, member-shape and predicate-leaf draws toward the patched surface via qg_biased(), a weighted pick that still leaves every shape reachable -- so a biased run focuses on the patch without losing neighbouring-bug coverage. prop_query_diff.c gains a permanent PG_PROP_DIFF_DEBUG knob (append every generated query to a file) used to verify reproducibility. README documents all three knobs. Both default off: with neither env var set, generation is byte-for-byte what it was. Verification of the PRNG replay and the bias distribution shift is pending a build host (EC2 at vCPU capacity at commit time). --- src/test/prop/README | 21 ++ src/test/prop/scripts/cf-features.py | 105 +++++++ src/test/prop/sql/prop_query_diff.c | 21 ++ src/test/prop/sql/query_gen.c | 398 ++++++++++++++++++++------- 4 files changed, 446 insertions(+), 99 deletions(-) create mode 100755 src/test/prop/scripts/cf-features.py diff --git a/src/test/prop/README b/src/test/prop/README index bc29c303c595f..5e752ae67200a 100644 --- a/src/test/prop/README +++ b/src/test/prop/README @@ -157,6 +157,27 @@ iteration count of a meson test run without editing anything: PG_PROP_ITERATIONS=10000 PG_PROP_SEED=42 meson test --suite prop PG_PROP_TESTS="pruning_preserves_rows" meson test prop/002_sql_props +Two further knobs tune the deep multi-table generator: + +* PG_PROP_GEN_PRNG=1 -- make the deep generator reproducible from a seed. + The multi-table builder makes hundreds of data-dependent draws per case, + a sequence the engine did not replay reliably (so a deep counterexample + could not be reproduced). With this set, the builder takes one engine + draw per case and drives all structural choices from a deterministic PRNG + seeded by it, so the case replays exactly; the cost is coarser shrinking + of deep structure. Unset (the default) keeps the historical behaviour. + +* PG_PROP_GEN_BIAS=tag,tag,... -- bias the deep generator toward the SQL + surface a patch under review touches, when vetting a commitfest entry. + scripts/cf-features.py scans the patch diff and prints the tag list; feed + it in so the generator weights joins / setops / subquery / nulls / ... as + the patch does. Biasing only reweights -- every shape is still reachable + -- so a biased run focuses without losing neighbouring-bug coverage. + + PG_PROP_GEN_BIAS=$(scripts/cf-features.py /tmp/cfpatch/6824/*.patch) \ + PG_PROP_GEN_PRNG=1 PG_PROP_DEEP=fast \ + pg_prop query_diff -n 500 diff_deep_join + Active and frozen areas ======================= diff --git a/src/test/prop/scripts/cf-features.py b/src/test/prop/scripts/cf-features.py new file mode 100755 index 0000000000000..4e3aa70a08713 --- /dev/null +++ b/src/test/prop/scripts/cf-features.py @@ -0,0 +1,105 @@ +#!/usr/bin/env python3 +"""cf-features.py -- extract the SQL-surface features a commitfest patch touches, +so the query generator can bias toward exercising them (task #3). + +Input: one or more unified-diff / git-format-patch files (the CF patch). +Output: a PG_PROP_GEN_BIAS value -- a comma-separated list of feature tags the +generator understands -- printed to stdout. + +The mapping is deliberately conservative: a tag is emitted only when the diff +touches a file or identifier strongly associated with that SQL surface, so a +biased run still generates everything, just with extra weight on the area the +patch changed. Over-biasing would narrow the search and miss neighbouring +bugs; under-biasing costs only a little focus. When nothing matches, print +nothing (the generator runs unbiased). + +Tags (kept in sync with the bias table in query_gen.c): + joins FULL/RIGHT/outer-join planning, join removal, PlaceHolderVar + setops UNION/INTERSECT/EXCEPT, append, setop planning + aggregation GROUP BY / aggregates / grouping sets / eager aggregate + window window functions / frames + distinct DISTINCT / DISTINCT ON / unique-ification + subquery subquery pull-up, LATERAL, correlated, sublink + nulls NULL / three-valued logic / strictness + collation collation, especially nondeterministic + numeric numeric scale / float / cast + partition partitioning, pruning, partitionwise + toast TOAST / detoast / external storage / rewrite + index index scan / index-only / amcanreturn / opclass + constraint CHECK / NOT NULL / FK / domain / unique + cast coercion / cast / type input-output +""" +import re +import sys + +# (compiled regex over added/removed diff lines and touched paths) -> tag +# Each rule is (where, pattern, tag): where is 'path' (the +++/--- file) or +# 'hunk' (a +/- code line). +RULES = [ + ('path', r'optimizer/(path|plan)/(join|analyzejoins|initsplan)', 'joins'), + ('hunk', r'\b(FULL|RIGHT|LEFT)\s+(OUTER\s+)?JOIN\b|PlaceHolderVar|nullingrel|outerjoin|join_is_removable|JoinExpr', 'joins'), + ('path', r'optimizer/(prep/prepunion|path/.*setop)|executor/nodeSetOp|nodeAppend', 'setops'), + ('hunk', r'\b(UNION|INTERSECT|EXCEPT)\b|SetOp|generate_union|append_rel', 'setops'), + ('path', r'executor/nodeAgg|optimizer/.*agg|parser/parse_agg', 'aggregation'), + ('hunk', r'\bGROUP\s+BY\b|\bHAVING\b|Aggref|grouping\s*set|GroupingFunc|eager.?aggregat|aggsplit|partial.?aggregat', 'aggregation'), + ('path', r'executor/nodeWindowAgg|optimizer/.*window|parser/parse_(clause|func).*window', 'window'), + ('hunk', r'\bWINDOW\b|WindowFunc|window\s*frame|\bOVER\b\s*\(|ROWS\s+BETWEEN|RANGE\s+BETWEEN|GROUPS\s+BETWEEN', 'window'), + ('hunk', r'\bDISTINCT\b|DISTINCT\s+ON|unique.?if|rel_is_distinct|distinctClause', 'distinct'), + ('path', r'optimizer/(prep/prepjointree|plan/subselect)|parser/parse_(clause|relation)', 'subquery'), + ('hunk', r'\bLATERAL\b|pull_up_subqueries|SubLink|SubPlan|flatten|convert_ANY|convert_EXISTS|correlat', 'subquery'), + ('hunk', r'\bIS\s+(NOT\s+)?NULL\b|\bIS\s+(NOT\s+)?DISTINCT\b|nonstrict|strictness|contain_nonstrict|null.?frac|three.?valued|ExecEvalCase', 'nulls'), + ('path', r'utils/adt/(varlena|pg_locale|like)|backend/utils/mb', 'collation'), + ('hunk', r'\bCOLLATE\b|collation|deterministic\s*=\s*false|nondeterministic|lc_collate|towlower|iswupper|pg_ascii_tolower|varstr_cmp', 'collation'), + ('path', r'utils/adt/(numeric|float|cash|int8?)\b', 'numeric'), + ('hunk', r'\bnumeric\b|\bfloat[48]\b|trim_scale|\bNaN\b|Infinity|scale\s*\(|rint|pg_ascii', 'numeric'), + ('path', r'(partitioning|partdesc|partprune|partbounds)|executor/nodeAppend|optimizer/.*partition', 'partition'), + ('hunk', r'\bPARTITION\b|partition.?wise|partition.?prun|RelationGetPartition|ATTACH\s+PARTITION|DETACH\s+PARTITION', 'partition'), + ('path', r'access/(heap|common)/.*toast|access/table/toast|detoast|heaptoast|tuptoaster|rewriteheap', 'toast'), + ('hunk', r'\bTOAST\b|detoast|toast_fetch|SET\s+STORAGE|EXTERNAL|EXTENDED|varatt|missing\s+chunk|reltoastrelid', 'toast'), + ('path', r'access/(nbtree|gist|gin|spgist|hash|brin)|optimizer/path/indxpath|executor/nodeIndex', 'index'), + ('hunk', r'index.?only|amcanreturn|IndexOnlyScan|opclass|operator\s+class|index_(get|rescan)|\bamsearch|skip\s*scan', 'index'), + ('path', r'commands/(trigger|tablecmds)|executor/execConstr|catalog/pg_constraint|utils/adt/ri_triggers', 'constraint'), + ('hunk', r'\bCHECK\b|\bNOT\s+NULL\b|\bFOREIGN\s+KEY\b|\bUNIQUE\b|ExecConstraints|\bdomain\b|RI_\w+|constraint', 'constraint'), + ('path', r'parser/parse_coerce|utils/adt/.*(in|out|recv|send)\b', 'cast'), + ('hunk', r'\bCAST\b|coerc|::\s*\w+|input\s+function|CoerceViaIO|typmod|RelabelType', 'cast'), +] + + +def extract(difftext): + tags = {} # tag -> hit count, for ranking + cur_path = '' + for line in difftext.splitlines(): + if line.startswith('+++ ') or line.startswith('--- '): + cur_path = line[4:] + for where, pat, tag in RULES: + if where == 'path' and re.search(pat, cur_path, re.I): + tags[tag] = tags.get(tag, 0) + 3 # path match is a strong signal + elif line.startswith('+') or line.startswith('-'): + if line.startswith('+++') or line.startswith('---'): + continue + body = line[1:] + for where, pat, tag in RULES: + if where == 'hunk' and re.search(pat, body): + tags[tag] = tags.get(tag, 0) + 1 + return tags + + +def main(argv): + if len(argv) < 2: + sys.stderr.write("usage: cf-features.py PATCH [PATCH ...]\n") + return 2 + text = '' + for fn in argv[1:]: + with open(fn, encoding='utf-8', errors='replace') as f: + text += f.read() + tags = extract(text) + # keep tags with a meaningful signal (>= 2: a path hit, or two hunk hits) + ranked = sorted((t for t, n in tags.items() if n >= 2), + key=lambda t: -tags[t]) + if ranked: + print(','.join(ranked)) + return 0 + + +if __name__ == '__main__': + sys.exit(main(sys.argv)) diff --git a/src/test/prop/sql/prop_query_diff.c b/src/test/prop/sql/prop_query_diff.c index 9145fdf578460..f1f21eea43ab0 100644 --- a/src/test/prop/sql/prop_query_diff.c +++ b/src/test/prop/sql/prop_query_diff.c @@ -175,6 +175,27 @@ run_diff(pg_prop_tc *tc, pg_prop_conn *conn, const char *query) bool errored; int i; + /* + * Debug knob: when PG_PROP_DIFF_DEBUG names a file, append every query + * this oracle runs to it. Used to verify generator reproducibility -- + * repeated same-seed runs should log an identical query (see + * PG_PROP_GEN_PRNG) -- and to inspect what the generator produces. + */ + { + const char *dbg = getenv("PG_PROP_DIFF_DEBUG"); + + if (dbg && dbg[0]) + { + FILE *fp = fopen(dbg, "a"); + + if (fp) + { + fprintf(fp, "QUERY: %s\n", query); + fclose(fp); + } + } + } + /* * The reference is config[0], the planner's free choice. If it errors, * the case is vacuous only for a plan-dependent runtime error; anything diff --git a/src/test/prop/sql/query_gen.c b/src/test/prop/sql/query_gen.c index 6b5ea80f69c01..1af0c70d6af4c 100644 --- a/src/test/prop/sql/query_gen.c +++ b/src/test/prop/sql/query_gen.c @@ -27,6 +27,184 @@ #include "query_gen.h" +/* ---- deterministic internal PRNG (task #1: deep-generator reproducibility) -- + * + * The deep multi-table generator makes hundreds of data-dependent draws per + * case (a draw's range depends on prior drawn values: parent row counts, + * column counts, table counts). Threaded straight through the engine, that + * long, variable-shape draw sequence did not replay from a fixed seed + * (finding 041): the engine's failure-replay could rebuild a different query, + * so a deep counterexample could not be reproduced. + * + * The fix: the generator draws ONE 64-bit integer from the engine per deep + * case (so the engine still seeds, shrinks that one value, and replays it) + * and drives every structural and value choice from a small deterministic + * PRNG (splitmix64) seeded by it. The engine therefore sees a single, + * fixed-shape draw per case -- trivially reproducible regardless of its + * internal buffer semantics -- while the generator keeps its rich + * data-dependent branching. + * + * This is opt-in via qg_rng_begin()/qg_rng_end() around a generation region. + * While a region is active, qg_draw_int/bool pull from the PRNG; outside one + * they fall through to the engine unchanged, so the single-table generator + * and every existing property behave exactly as before. The tradeoff is + * coarser shrinking of deep structure (the engine can only shrink the seed), + * which for a hunt/review tool is worth far less than the ability to replay + * a hit at all. + */ +static bool qg_rng_active = false; +static uint64 qg_rng_state = 0; + +static inline uint64 +qg_splitmix64(void) +{ + uint64 z = (qg_rng_state += UINT64CONST(0x9E3779B97F4A7C15)); + + z = (z ^ (z >> 30)) * UINT64CONST(0xBF58476D1CE4E5B9); + z = (z ^ (z >> 27)) * UINT64CONST(0x94D049BB133111EB); + return z ^ (z >> 31); +} + +/* + * Begin a deterministic generation region: draw one engine integer as the + * seed and switch qg_draw_* to the PRNG. Honours PG_PROP_GEN_PRNG; when the + * variable is unset or 0 the region is a no-op and the generator draws from + * the engine as it always did (so the change is A/B-testable and default-off + * until verified on a build host). + */ +static void +qg_rng_begin(pg_prop_tc *tc) +{ + const char *v = getenv("PG_PROP_GEN_PRNG"); + + if (v == NULL || v[0] == '\0' || strcmp(v, "0") == 0) + { + qg_rng_active = false; + return; + } + qg_rng_state = (uint64) pg_prop_draw_int(tc, INT64_MIN, INT64_MAX); + qg_rng_active = true; +} + +static void +qg_rng_end(void) +{ + qg_rng_active = false; +} + +/* + * Draw an integer in [min,max]. Inside a qg_rng region this is deterministic + * from the region seed (unbiased rejection sampling over splitmix64 output); + * outside, it is the engine draw. A degenerate or inverted range returns min + * rather than asking the engine for an empty interval (which was itself a + * non-determinism hazard). + */ +static int64 +qg_draw_int(pg_prop_tc *tc, int64 min, int64 max) +{ + if (max <= min) + return min; + if (!qg_rng_active) + return pg_prop_draw_int(tc, min, max); + { + uint64 span = (uint64) (max - min) + 1; + uint64 limit = span == 0 ? 0 : (UINT64_MAX - (UINT64_MAX % span)); + uint64 r; + + if (span == 0) /* full 64-bit range */ + return (int64) qg_splitmix64(); + do + r = qg_splitmix64(); + while (limit != 0 && r >= limit); + return min + (int64) (r % span); + } +} + +static bool +qg_draw_bool(pg_prop_tc *tc) +{ + if (!qg_rng_active) + return pg_prop_draw_bool(tc); + return (qg_splitmix64() & 1) != 0; +} + +/* ---- patch-biased generation (task #3) ------------------------------------ + * + * cf-features.py scans a commitfest patch's diff and emits PG_PROP_GEN_BIAS, a + * comma-separated list of feature tags for the SQL surface the patch touches + * (joins, setops, aggregation, window, distinct, subquery, nulls, collation, + * numeric, partition, toast, index, constraint, cast). The generator reads + * it once and, at the handful of "which shape" draws that matter, boosts the + * weight of shapes exercising a wanted tag. Biasing only reweights; every + * shape is still reachable, so a biased run focuses on the patched surface + * without losing the neighbouring-bug coverage that an unbiased run gives. + * + * qg_bias_wants(tag) answers "did the patch touch this surface". qg_biased() + * is a weighted pick over a small set of options, each labelled with the tag + * it exercises (or NULL for "neutral"); a wanted tag's options get WEIGHT + * extra draws' worth of probability. It uses qg_draw_int so it stays inside + * the deterministic region. + */ +#define QG_BIAS_WEIGHT 4 + +static bool qg_bias_inited = false; +static char qg_bias_tags[256]; + +static bool +qg_bias_wants(const char *tag) +{ + const char *v; + + if (!qg_bias_inited) + { + v = getenv("PG_PROP_GEN_BIAS"); + snprintf(qg_bias_tags, sizeof(qg_bias_tags), ",%s,", v ? v : ""); + qg_bias_inited = true; + } + if (tag == NULL || qg_bias_tags[1] == '\0') + return false; + { + char needle[40]; + + snprintf(needle, sizeof(needle), ",%s,", tag); + return strstr(qg_bias_tags, needle) != NULL; + } +} + +/* + * Weighted pick over n options. opt_tags[i] is the feature tag option i + * exercises, or NULL if neutral. Each wanted option is entered + * QG_BIAS_WEIGHT times in the lottery; others once. Returns the chosen index + * 0..n-1. With no bias active (no wanted tag present) this is a uniform draw + * in [0, n-1], identical to the unbiased generator. + */ +static int +qg_biased(pg_prop_tc *tc, int n, const char *const opt_tags[]) +{ + int weights[32]; + int total = 0; + int i; + int r; + + if (n > 32) + n = 32; + for (i = 0; i < n; i++) + { + weights[i] = qg_bias_wants(opt_tags[i]) ? QG_BIAS_WEIGHT : 1; + total += weights[i]; + } + if (total == n) /* nothing wanted: plain uniform draw */ + return (int) qg_draw_int(tc, 0, n - 1); + r = (int) qg_draw_int(tc, 0, total - 1); + for (i = 0; i < n; i++) + { + if (r < weights[i]) + return i; + r -= weights[i]; + } + return n - 1; +} + /* * Value domains deliberately favour the cases that break planners and * executors, not uniform small noise: boundary integers (INT_MIN/MAX, 0, +-1), @@ -54,7 +232,7 @@ static void draw_int_or_null(pg_prop_tc *tc, StringInfo buf) { - int roll = (int) pg_prop_draw_int(tc, 0, 99); + int roll = (int) qg_draw_int(tc, 0, 99); if (roll < 18) appendStringInfoString(buf, "NULL"); @@ -63,13 +241,13 @@ draw_int_or_null(pg_prop_tc *tc, StringInfo buf) static const char *const edge[] = {SQL_INT_MIN, SQL_INT_MAX, "0", "(-1)", "1"}; - appendStringInfoString(buf, edge[pg_prop_draw_int(tc, 0, 4)]); + appendStringInfoString(buf, edge[qg_draw_int(tc, 0, 4)]); } else if (roll < 75) appendStringInfoString(buf, "0"); /* skew: 0 dominates */ else appendStringInfo(buf, "%d", - (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX)); + (int) qg_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX)); } /* @@ -80,7 +258,7 @@ draw_int_or_null(pg_prop_tc *tc, StringInfo buf) static void draw_text_or_null(pg_prop_tc *tc, StringInfo buf) { - int roll = (int) pg_prop_draw_int(tc, 0, 99); + int roll = (int) qg_draw_int(tc, 0, 99); int len; int i; @@ -99,19 +277,19 @@ draw_text_or_null(pg_prop_tc *tc, StringInfo buf) appendStringInfoString(buf, "'a'"); /* skew: 'a' dominates */ return; } - len = (int) pg_prop_draw_int(tc, 0, 3); + len = (int) qg_draw_int(tc, 0, 3); appendStringInfoChar(buf, '\''); for (i = 0; i < len; i++) { /* mostly a-c; occasionally an uppercase letter or a trailing space */ - int k = (int) pg_prop_draw_int(tc, 0, 9); + int k = (int) qg_draw_int(tc, 0, 9); if (k == 0) - appendStringInfoChar(buf, (char) ('A' + pg_prop_draw_int(tc, 0, 2))); + appendStringInfoChar(buf, (char) ('A' + qg_draw_int(tc, 0, 2))); else if (k == 1) appendStringInfoChar(buf, ' '); else - appendStringInfoChar(buf, (char) ('a' + pg_prop_draw_int(tc, 0, 2))); + appendStringInfoChar(buf, (char) ('a' + qg_draw_int(tc, 0, 2))); } appendStringInfoChar(buf, '\''); } @@ -120,7 +298,7 @@ draw_text_or_null(pg_prop_tc *tc, StringInfo buf) static void draw_float_or_null(pg_prop_tc *tc, StringInfo buf) { - int roll = (int) pg_prop_draw_int(tc, 0, 99); + int roll = (int) qg_draw_int(tc, 0, 99); if (roll < 15) appendStringInfoString(buf, "NULL"); @@ -130,19 +308,19 @@ draw_float_or_null(pg_prop_tc *tc, StringInfo buf) {"'0'::float8", "'-0'::float8", "'Infinity'::float8", "'-Infinity'::float8", "'NaN'::float8"}; - appendStringInfoString(buf, edge[pg_prop_draw_int(tc, 0, 4)]); + appendStringInfoString(buf, edge[qg_draw_int(tc, 0, 4)]); } else appendStringInfo(buf, "%d.%d", - (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), - (int) pg_prop_draw_int(tc, 0, 9)); + (int) qg_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), + (int) qg_draw_int(tc, 0, 9)); } /* numeric: NULL, boundary (0, huge, tiny-scale), or a scaled value. */ static void draw_numeric_or_null(pg_prop_tc *tc, StringInfo buf) { - int roll = (int) pg_prop_draw_int(tc, 0, 99); + int roll = (int) qg_draw_int(tc, 0, 99); if (roll < 15) appendStringInfoString(buf, "NULL"); @@ -151,19 +329,19 @@ draw_numeric_or_null(pg_prop_tc *tc, StringInfo buf) static const char *const edge[] = {"0", "0.00", "999.99", "-999.99", "'NaN'::numeric"}; - appendStringInfoString(buf, edge[pg_prop_draw_int(tc, 0, 4)]); + appendStringInfoString(buf, edge[qg_draw_int(tc, 0, 4)]); } else appendStringInfo(buf, "%d.%02d", - (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), - (int) pg_prop_draw_int(tc, 0, 99)); + (int) qg_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), + (int) qg_draw_int(tc, 0, 99)); } /* boolean literal or SQL NULL (balanced three-valued). */ static void draw_bool_or_null(pg_prop_tc *tc, StringInfo buf) { - int k = (int) pg_prop_draw_int(tc, 0, 2); + int k = (int) qg_draw_int(tc, 0, 2); appendStringInfoString(buf, k == 0 ? "NULL" : k == 1 ? "true" : "false"); } @@ -249,7 +427,7 @@ qg_build_table(pg_prop_tc *tc, pg_prop_conn *conn, int *nrows) appendStringInfo(&ddl, ", %s %s", s->cols[col].name, s->cols[col].sqltype); appendStringInfoString(&ddl, "); "); - n = (int) pg_prop_draw_int(tc, 0, QG_MAX_ROWS); + n = (int) qg_draw_int(tc, 0, QG_MAX_ROWS); for (r = 1; r <= n; r++) { appendStringInfo(&ddl, "INSERT INTO " QG_TABLE " VALUES (%d", r); @@ -269,9 +447,9 @@ qg_build_table(pg_prop_tc *tc, pg_prop_conn *conn, int *nrows) * is skewed the same way the drawn values are (a dominant 0, NULLs, the * int extremes) and deterministic in the drawn size, so cases replay. */ - if (qg_deep_requested() && pg_prop_draw_bool(tc)) + if (qg_deep_requested() && qg_draw_bool(tc)) { - int bulk = (int) pg_prop_draw_int(tc, 500, 20000); + int bulk = (int) qg_draw_int(tc, 500, 20000); appendStringInfo(&ddl, "INSERT INTO " QG_TABLE " (id, a, b, c, d, f, g) " @@ -297,15 +475,15 @@ qg_build_table(pg_prop_tc *tc, pg_prop_conn *conn, int *nrows) * a plan with itself. Single-column, composite, partial and expression * indexes open index, index-only, bitmap-AND/OR and presorted paths. */ - if (pg_prop_draw_bool(tc)) + if (qg_draw_bool(tc)) appendStringInfoString(&ddl, "CREATE INDEX ON " QG_TABLE " (a); "); - if (pg_prop_draw_bool(tc)) + if (qg_draw_bool(tc)) appendStringInfoString(&ddl, "CREATE INDEX ON " QG_TABLE " (b, a); "); - if (pg_prop_draw_bool(tc)) + if (qg_draw_bool(tc)) appendStringInfoString(&ddl, "CREATE INDEX ON " QG_TABLE " (c); "); - if (pg_prop_draw_int(tc, 0, 2) == 0) + if (qg_draw_int(tc, 0, 2) == 0) appendStringInfoString(&ddl, "CREATE INDEX ON " QG_TABLE " (d) WHERE d IS NOT NULL; "); - if (pg_prop_draw_int(tc, 0, 2) == 0) + if (qg_draw_int(tc, 0, 2) == 0) appendStringInfoString(&ddl, "CREATE INDEX ON " QG_TABLE " ((coalesce(a, 0)::bigint + 1)); "); /* ANALYZE so the planner has stats and will actually consider plans. */ @@ -324,7 +502,7 @@ qg_build_table(pg_prop_tc *tc, pg_prop_conn *conn, int *nrows) static const char * pick_num_col(pg_prop_tc *tc) { - int col = (int) pg_prop_draw_int(tc, 0, 2); + int col = (int) qg_draw_int(tc, 0, 2); return (col == 0) ? "a" : (col == 1) ? "b" : "f"; } @@ -342,7 +520,7 @@ draw_leaf(pg_prop_tc *tc, StringInfo buf) int have_ext = pg_prop_hooks_count("query_gen.predicate") > 0 ? 1 : 0; /* kinds 0..10 are built-in; 11 is the extension leaf when present. */ int n_builtin = 11; - int kind = (int) pg_prop_draw_int(tc, 0, n_builtin - 1 + have_ext); + int kind = (int) qg_draw_int(tc, 0, n_builtin - 1 + have_ext); const char *colname; switch (kind) @@ -355,7 +533,7 @@ draw_leaf(pg_prop_tc *tc, StringInfo buf) break; case 2: /* c comparison against text */ appendStringInfoString(buf, "c "); - appendStringInfoString(buf, cmp[pg_prop_draw_int(tc, 0, 5)]); + appendStringInfoString(buf, cmp[qg_draw_int(tc, 0, 5)]); appendStringInfoChar(buf, ' '); draw_text_or_null(tc, buf); break; @@ -363,7 +541,7 @@ draw_leaf(pg_prop_tc *tc, StringInfo buf) colname = pick_num_col(tc); appendStringInfo(buf, "%s IN (", colname); { - int m = (int) pg_prop_draw_int(tc, 1, 3); + int m = (int) qg_draw_int(tc, 1, 3); int i; for (i = 0; i < m; i++) @@ -380,23 +558,23 @@ draw_leaf(pg_prop_tc *tc, StringInfo buf) break; case 4: /* col BETWEEN lo AND hi */ appendStringInfo(buf, "%s BETWEEN %d AND %d", pick_num_col(tc), - (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), - (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX)); + (int) qg_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), + (int) qg_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX)); break; case 5: /* numeric column d compared, or bool g */ - if (pg_prop_draw_bool(tc)) + if (qg_draw_bool(tc)) appendStringInfo(buf, "d %s %d.%02d", - cmp[pg_prop_draw_int(tc, 0, 5)], - (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), - (int) pg_prop_draw_int(tc, 0, 99)); + cmp[qg_draw_int(tc, 0, 5)], + (int) qg_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), + (int) qg_draw_int(tc, 0, 99)); else appendStringInfo(buf, "g IS %s", - pg_prop_draw_bool(tc) ? "TRUE" : "NOT TRUE"); + qg_draw_bool(tc) ? "TRUE" : "NOT TRUE"); break; case 7: /* col IS [NOT] DISTINCT FROM const (3VL edge) */ colname = pick_num_col(tc); appendStringInfo(buf, "%s IS %sDISTINCT FROM ", colname, - pg_prop_draw_bool(tc) ? "" : "NOT "); + qg_draw_bool(tc) ? "" : "NOT "); if (strcmp(colname, "f") == 0) draw_float_or_null(tc, buf); else @@ -404,27 +582,27 @@ draw_leaf(pg_prop_tc *tc, StringInfo buf) break; case 8: /* CASE expression used as a boolean */ appendStringInfoString(buf, "(CASE WHEN "); - appendStringInfo(buf, "a %s ", cmp[pg_prop_draw_int(tc, 0, 5)]); + appendStringInfo(buf, "a %s ", cmp[qg_draw_int(tc, 0, 5)]); draw_int_or_null(tc, buf); appendStringInfoString(buf, " THEN b ELSE "); draw_int_or_null(tc, buf); - appendStringInfo(buf, " END) %s ", cmp[pg_prop_draw_int(tc, 0, 5)]); + appendStringInfo(buf, " END) %s ", cmp[qg_draw_int(tc, 0, 5)]); draw_int_or_null(tc, buf); break; case 9: /* cross-type: int column vs numeric/text cast */ - if (pg_prop_draw_bool(tc)) + if (qg_draw_bool(tc)) appendStringInfo(buf, "a::numeric %s %d.%02d", - cmp[pg_prop_draw_int(tc, 0, 5)], - (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), - (int) pg_prop_draw_int(tc, 0, 99)); + cmp[qg_draw_int(tc, 0, 5)], + (int) qg_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), + (int) qg_draw_int(tc, 0, 99)); else appendStringInfo(buf, "a::text %s c", - cmp[pg_prop_draw_int(tc, 0, 5)]); + cmp[qg_draw_int(tc, 0, 5)]); break; case 10: /* COALESCE in the comparison (NULL coalescing) */ appendStringInfo(buf, "coalesce(a, b, %d) %s ", - (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), - cmp[pg_prop_draw_int(tc, 0, 5)]); + (int) qg_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX), + cmp[qg_draw_int(tc, 0, 5)]); draw_int_or_null(tc, buf); break; case 11: /* extension-contributed leaf */ @@ -435,12 +613,12 @@ draw_leaf(pg_prop_tc *tc, StringInfo buf) pctx.schema = &the_schema; pctx.buf = buf; pg_prop_run_hook_n(tc, "query_gen.predicate", - (int) pg_prop_draw_int(tc, 0, n - 1), &pctx); + (int) qg_draw_int(tc, 0, n - 1), &pctx); } break; default: /* col const */ colname = pick_num_col(tc); - appendStringInfo(buf, "%s %s ", colname, cmp[pg_prop_draw_int(tc, 0, 5)]); + appendStringInfo(buf, "%s %s ", colname, cmp[qg_draw_int(tc, 0, 5)]); if (strcmp(colname, "f") == 0) draw_float_or_null(tc, buf); else @@ -457,7 +635,7 @@ draw_predicate_rec(pg_prop_tc *tc, StringInfo buf, int depth) if (depth >= 3) kind = 0; else - kind = (int) pg_prop_draw_int(tc, 0, 4); + kind = (int) qg_draw_int(tc, 0, 4); switch (kind) { @@ -496,7 +674,7 @@ qg_draw_predicate(pg_prop_tc *tc, StringInfo buf) void qg_draw_int_expr(pg_prop_tc *tc, StringInfo buf) { - int kind = (int) pg_prop_draw_int(tc, 0, 4); + int kind = (int) qg_draw_int(tc, 0, 4); switch (kind) { @@ -508,7 +686,7 @@ qg_draw_int_expr(pg_prop_tc *tc, StringInfo buf) break; case 2: appendStringInfo(buf, "%d", - (int) pg_prop_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX)); + (int) qg_draw_int(tc, QG_VAL_MIN, QG_VAL_MAX)); break; case 3: /* coalesce so NULLs don't dominate; a + b, bigint-safe vs INT_MAX */ @@ -531,20 +709,20 @@ qg_draw_int_expr(pg_prop_tc *tc, StringInfo buf) static void draw_lowcard_or_null(pg_prop_tc *tc, StringInfo buf) { - if (pg_prop_draw_int(tc, 0, 9) == 0) + if (qg_draw_int(tc, 0, 9) == 0) appendStringInfoString(buf, "NULL"); else - appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, 0, 2)); + appendStringInfo(buf, "%d", (int) qg_draw_int(tc, 0, 2)); } /* A high-cardinality int column: wide domain, so values are near-unique. */ static void draw_highcard_or_null(pg_prop_tc *tc, StringInfo buf) { - if (pg_prop_draw_int(tc, 0, 19) == 0) + if (qg_draw_int(tc, 0, 19) == 0) appendStringInfoString(buf, "NULL"); else - appendStringInfo(buf, "%d", (int) pg_prop_draw_int(tc, 0, 100000)); + appendStringInfo(buf, "%d", (int) qg_draw_int(tc, 0, 100000)); } /* @@ -564,12 +742,12 @@ draw_toast_text_or_null(pg_prop_tc *tc, StringInfo buf) int reps; int i; - if (pg_prop_draw_int(tc, 0, 9) == 0) + if (qg_draw_int(tc, 0, 9) == 0) { appendStringInfoString(buf, "NULL"); return; } - roll = (int) pg_prop_draw_int(tc, 0, 9); + roll = (int) qg_draw_int(tc, 0, 9); switch (qg_toast_profile) { case 0: @@ -589,16 +767,16 @@ draw_toast_text_or_null(pg_prop_tc *tc, StringInfo buf) if (big) { /* ~3KB of a repeated unit -> past the 2KB TOAST threshold */ - reps = 600 + (int) pg_prop_draw_int(tc, 0, 400); + reps = 600 + (int) qg_draw_int(tc, 0, 400); for (i = 0; i < reps; i++) appendStringInfoString(buf, "xy"); } else { - int len = (int) pg_prop_draw_int(tc, 0, 4); + int len = (int) qg_draw_int(tc, 0, 4); for (i = 0; i < len; i++) - appendStringInfoChar(buf, (char) ('a' + pg_prop_draw_int(tc, 0, 2))); + appendStringInfoChar(buf, (char) ('a' + qg_draw_int(tc, 0, 2))); } appendStringInfoChar(buf, '\''); } @@ -716,14 +894,14 @@ build_table_schema(pg_prop_tc *tc, qg_table *t, int ncols, int fkref, for (i = 0; i < ncols; i++) { - int kind = (int) pg_prop_draw_int(tc, 0, 5); + int kind = (int) qg_draw_int(tc, 0, 5); qg_value_fn draw; const char *type; bool textlike = false; bool numlike = false; /* a cardinality bias favours low-card columns for grouping stress */ - if (cardinality_bias == 1 && pg_prop_draw_bool(tc)) + if (cardinality_bias == 1 && qg_draw_bool(tc)) kind = 0; switch (kind) @@ -769,7 +947,7 @@ build_table_schema(pg_prop_tc *tc, qg_table *t, int ncols, int fkref, * allow-list (6.3) harder to reason about. The column is marked so * draw_leaf_on restricts it to the nondeterministic-safe operators. */ - if (nondet && draw == draw_text_or_null && pg_prop_draw_int(tc, 0, 2) == 0) + if (nondet && draw == draw_text_or_null && qg_draw_int(tc, 0, 2) == 0) { qg_add_column(&t->schema, namebuf, "text COLLATE qg_ci", draw, textlike, numlike); @@ -805,13 +983,13 @@ emit_storage_modes(pg_prop_tc *tc, StringInfo ddl, const qg_table *t) continue; if (c->draw == draw_toast_text_or_null) - mode = pg_prop_draw_bool(tc) ? "EXTERNAL" : "EXTENDED"; + mode = qg_draw_bool(tc) ? "EXTERNAL" : "EXTENDED"; else { static const char *const modes[] = {"PLAIN", "MAIN", "EXTERNAL", "EXTENDED"}; - mode = modes[pg_prop_draw_int(tc, 0, 3)]; + mode = modes[qg_draw_int(tc, 0, 3)]; } (void) toastable; appendStringInfo(ddl, "ALTER TABLE %s ALTER COLUMN %s SET STORAGE %s; ", @@ -862,7 +1040,14 @@ qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) */ nondet = p->allow_nondet_collation && qg_server_has_icu(tc, conn); - nt = (int) pg_prop_draw_int(tc, p->min_tables, p->max_tables); + /* + * Enter a deterministic generation region (task #1): one engine draw seeds + * the PRNG, all structural choices below come from it. No-op unless + * PG_PROP_GEN_PRNG is set, so default behaviour is unchanged. + */ + qg_rng_begin(tc); + + nt = (int) qg_draw_int(tc, p->min_tables, p->max_tables); if (nt < 1) nt = 1; if (nt > QG_MAX_TABLES) @@ -908,10 +1093,10 @@ qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) /* a child may reference any earlier table (so parents build first) */ fkref = -1; - if (p->allow_fk && ti > 0 && pg_prop_draw_bool(tc)) - fkref = (int) pg_prop_draw_int(tc, 0, ti - 1); + if (p->allow_fk && ti > 0 && qg_draw_bool(tc)) + fkref = (int) qg_draw_int(tc, 0, ti - 1); - ncols = (int) pg_prop_draw_int(tc, p->min_cols, p->max_cols); + ncols = (int) qg_draw_int(tc, p->min_cols, p->max_cols); build_table_schema(tc, t, ncols, fkref, p->cardinality_bias, nondet); /* DROP first (reverse order handled by qg_drop_db; here just reset) */ @@ -936,7 +1121,7 @@ qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) * tables, so the relation carries trigger metadata and the write path * runs the trigger. Result-neutral, so SELECT oracles stay sound. */ - if (p->allow_triggers && pg_prop_draw_bool(tc)) + if (p->allow_triggers && qg_draw_bool(tc)) appendStringInfo(&ddl, "CREATE TRIGGER %s_trig BEFORE INSERT OR UPDATE ON %s " "FOR EACH ROW EXECUTE FUNCTION qg_noop_trig(); ", @@ -947,7 +1132,7 @@ qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) for (ti = 0; ti < nt; ti++) { qg_table *t = &db->tables[ti]; - int n = (int) pg_prop_draw_int(tc, p->min_rows, p->max_rows); + int n = (int) qg_draw_int(tc, p->min_rows, p->max_rows); int parent_rows = (t->fkref >= 0) ? db->tables[t->fkref].nrows : 0; int r; int col; @@ -968,11 +1153,11 @@ qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) * exists.) If the parent is empty, the only satisfiable * value is NULL. */ - if (parent_rows == 0 || pg_prop_draw_int(tc, 0, 4) == 0) + if (parent_rows == 0 || qg_draw_int(tc, 0, 4) == 0) appendStringInfoString(&ddl, "NULL"); else appendStringInfo(&ddl, "%d", - (int) pg_prop_draw_int(tc, 1, parent_rows)); + (int) qg_draw_int(tc, 1, parent_rows)); } else t->schema.cols[col].draw(tc, &ddl); @@ -989,7 +1174,7 @@ qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) * merge-join-on-presorted paths, plus Memoize over a parameterized * inner index scan. */ - if (t->fkcol >= 0 && pg_prop_draw_int(tc, 0, 3) != 0) + if (t->fkcol >= 0 && qg_draw_int(tc, 0, 3) != 0) appendStringInfo(&ddl, "CREATE INDEX ON %s (%s); ", t->name, t->schema.cols[t->fkcol].name); for (col = 0; col < t->schema.ncols; col++) @@ -1001,9 +1186,9 @@ qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) * btree tuple limit, so only scalar columns are indexed. */ if (col == t->fkcol || t->schema.cols[col].is_text_like || - pg_prop_draw_int(tc, 0, 2) != 0) + qg_draw_int(tc, 0, 2) != 0) continue; - k = (int) pg_prop_draw_int(tc, 0, 2); + k = (int) qg_draw_int(tc, 0, 2); if (k == 1 && (col + 1 >= t->schema.ncols || t->schema.cols[col + 1].is_text_like)) k = 0; @@ -1029,16 +1214,16 @@ qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) * elimination, uniqueness-based join removal, DISTINCT elision -- is * unsound there, while "ONLY parent" must still see only the parent. */ - if (p->allow_inherit && n > 0 && pg_prop_draw_int(tc, 0, 2) == 0) + if (p->allow_inherit && n > 0 && qg_draw_int(tc, 0, 2) == 0) { - int m = (int) pg_prop_draw_int(tc, 1, n < 5 ? n : 5); + int m = (int) qg_draw_int(tc, 1, n < 5 ? n : 5); appendStringInfo(&ddl, "CREATE TABLE %s_c () INHERITS (%s); ", t->name, t->name); for (r = 1; r <= m; r++) { appendStringInfo(&ddl, "INSERT INTO %s_c VALUES (%d", t->name, - (int) pg_prop_draw_int(tc, 1, n)); + (int) qg_draw_int(tc, 1, n)); for (col = 0; col < t->schema.ncols; col++) { appendStringInfoString(&ddl, ", "); @@ -1054,6 +1239,7 @@ qg_build_db(pg_prop_tc *tc, pg_prop_conn *conn, const qg_db_params *p) appendStringInfo(&ddl, "ANALYZE %s; ", t->name); } + qg_rng_end(); /* all structural draws done */ if (!pg_prop_sql_exec(conn, ddl.data, &err)) pg_prop_fail(tc, "database DDL/DML failed: %s", err ? err : ""); @@ -1097,11 +1283,11 @@ draw_leaf_on(pg_prop_tc *tc, StringInfo buf, const qg_table *t, { /* no data columns: fall back to a key predicate */ appendStringInfo(buf, "%s.id %s %d", alias, - cmp[pg_prop_draw_int(tc, 0, 5)], - (int) pg_prop_draw_int(tc, 0, 10)); + cmp[qg_draw_int(tc, 0, 5)], + (int) qg_draw_int(tc, 0, 10)); return; } - ci = (int) pg_prop_draw_int(tc, 0, t->schema.ncols - 1); + ci = (int) qg_draw_int(tc, 0, t->schema.ncols - 1); c = &t->schema.cols[ci]; /* @@ -1116,7 +1302,7 @@ draw_leaf_on(pg_prop_tc *tc, StringInfo buf, const qg_table *t, */ if (c->nondet_collation) { - switch ((int) pg_prop_draw_int(tc, 0, 3)) + switch ((int) qg_draw_int(tc, 0, 3)) { case 0: appendStringInfo(buf, "%s.%s IS NULL", alias, c->name); @@ -1126,19 +1312,20 @@ draw_leaf_on(pg_prop_tc *tc, StringInfo buf, const qg_table *t, break; case 2: appendStringInfo(buf, "%s.%s IS %sDISTINCT FROM ", alias, c->name, - pg_prop_draw_bool(tc) ? "NOT " : ""); + qg_draw_bool(tc) ? "NOT " : ""); draw_text_or_null(tc, buf); break; default: appendStringInfo(buf, "%s.%s %s ", alias, c->name, - cmp[pg_prop_draw_int(tc, 0, 5)]); + cmp[qg_draw_int(tc, 0, 5)]); draw_text_or_null(tc, buf); break; } return; } - switch ((int) pg_prop_draw_int(tc, 0, 7)) + switch (qg_biased(tc, 8, (const char *const[]) + {"nulls", "nulls", "nulls", "nulls", "cast", NULL, NULL, NULL})) { case 0: appendStringInfo(buf, "%s.%s IS NULL", alias, c->name); @@ -1148,7 +1335,7 @@ draw_leaf_on(pg_prop_tc *tc, StringInfo buf, const qg_table *t, break; case 2: /* IS [NOT] DISTINCT FROM: null-safe equality */ appendStringInfo(buf, "%s.%s IS %sDISTINCT FROM ", alias, c->name, - pg_prop_draw_bool(tc) ? "NOT " : ""); + qg_draw_bool(tc) ? "NOT " : ""); if (c->is_text_like) draw_text_or_null(tc, buf); else @@ -1156,8 +1343,8 @@ draw_leaf_on(pg_prop_tc *tc, StringInfo buf, const qg_table *t, break; case 3: /* = ANY / <> ALL over an array that may hold NULL */ { - int n = (int) pg_prop_draw_int(tc, 0, 3); - bool any = pg_prop_draw_bool(tc); + int n = (int) qg_draw_int(tc, 0, 3); + bool any = qg_draw_bool(tc); appendStringInfo(buf, "%s.%s %s (ARRAY[", alias, c->name, any ? "= ANY" : "<> ALL"); @@ -1178,15 +1365,15 @@ draw_leaf_on(pg_prop_tc *tc, StringInfo buf, const qg_table *t, case 4: /* cross-type: compare through a cast */ if (c->is_text_like) appendStringInfo(buf, "%s.%s::varchar %s 'b'", alias, c->name, - cmp[pg_prop_draw_int(tc, 0, 5)]); + cmp[qg_draw_int(tc, 0, 5)]); else appendStringInfo(buf, "%s.%s::numeric %s %d.5", alias, c->name, - cmp[pg_prop_draw_int(tc, 0, 5)], - (int) pg_prop_draw_int(tc, -3, 9)); + cmp[qg_draw_int(tc, 0, 5)], + (int) qg_draw_int(tc, -3, 9)); break; default: appendStringInfo(buf, "%s.%s %s ", alias, c->name, - cmp[pg_prop_draw_int(tc, 0, 5)]); + cmp[qg_draw_int(tc, 0, 5)]); if (c->is_text_like) draw_text_or_null(tc, buf); else @@ -1199,7 +1386,7 @@ static void draw_predicate_on_rec(pg_prop_tc *tc, StringInfo buf, const qg_table *t, const char *alias, int depth) { - int kind = (depth >= 2) ? 0 : (int) pg_prop_draw_int(tc, 0, 4); + int kind = (depth >= 2) ? 0 : (int) qg_draw_int(tc, 0, 4); switch (kind) { @@ -1319,7 +1506,15 @@ build_join_chain(pg_prop_tc *tc, const qg_db *db, int start, int max_joins, { const char *tn = db->tables[nexttab].name; const char *prev = aliases[n - 1]; - int shape = (int) pg_prop_draw_int(tc, 0, 6); + /* + * Member shape: 0 plain, 1 ONLY, 2 DISTINCT-derived, 3 UNION ALL, + * 4/5 LATERAL, 6 LATERAL UNION ALL. Bias toward the subquery / + * appendrel shapes when the patch touches those surfaces. + */ + static const char *const shape_tags[] = { + NULL, "index", "distinct", "setops", "subquery", "subquery", "subquery" + }; + int shape = qg_biased(tc, 7, shape_tags); /* derived shapes expose id, plus fk when the table has one */ const char *cols = db->tables[nexttab].fkcol >= 0 ? "id, fk" : "id"; const char *xcols = db->tables[nexttab].fkcol >= 0 ? "x.id, x.fk" : "x.id"; @@ -1335,13 +1530,16 @@ build_join_chain(pg_prop_tc *tc, const qg_db *db, int start, int max_joins, * LATERAL shapes (4-6) stay INNER / LEFT. A FULL/RIGHT join can * leave every projected key NULL in an unmatched row, so the * oracle must compare the row multiset, not a key-ordered - * sequence (rowhash_unordered); the deep oracles do. + * sequence (rowhash_unordered); the deep oracles do. The join + * type is biased toward the outer joins when the patch touches + * join planning. */ static const char *const jts[] = {"", "LEFT", "RIGHT", "FULL"}; + static const char *const jt_tags[] = {NULL, "joins", "joins", "joins"}; bool lateral = (shape == 4 || shape == 5 || shape == 6); const char *jt = lateral - ? (pg_prop_draw_bool(tc) ? "" : "LEFT") - : jts[pg_prop_draw_int(tc, 0, 3)]; + ? (qg_draw_bool(tc) ? "" : "LEFT") + : jts[qg_biased(tc, 4, jt_tags)]; switch (shape) { @@ -1474,7 +1672,8 @@ qg_build_query(pg_prop_tc *tc, const qg_db *db, int max_depth, qg_query *q) if (db->ntables == 0) return false; - start = (int) pg_prop_draw_int(tc, 0, db->ntables - 1); + qg_rng_begin(tc); /* deterministic region for the query shape */ + start = (int) qg_draw_int(tc, 0, db->ntables - 1); maxj = (max_depth < 1) ? 1 : max_depth; if (maxj > db->ntables - 1) maxj = db->ntables - 1; @@ -1530,6 +1729,7 @@ qg_build_query(pg_prop_tc *tc, const qg_db *db, int max_depth, qg_query *q) pfree(from.data); pfree(keys.data); pfree(sel.data); + qg_rng_end(); return true; } From 3d0438142103df3d869e9acbe70f062e186340d3 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Fri, 23 Oct 2026 12:33:54 -0400 Subject: [PATCH 330/338] prop: make the joins bias boost non-lateral member shapes too Verification on FreeBSD showed the joins bias had no measurable effect: it only reweighted the join-type draw, but most members end up lateral (forced INNER/LEFT) and the non-lateral join type was already ~50% FULL/RIGHT, so the per-query rate was saturated. Boost the non-lateral member shapes (plain, ONLY) when the patch touches join planning, so more members can actually take a FULL/RIGHT type. Measured effect over 8 seeds x 40 cases: FULL/RIGHT occurrences 157 -> 218 (+39%). The setops bias already worked (UNION ALL 170 -> 259, +52%). Verified on FreeBSD 14.5/amd64, upstream 126976a, libhegel 0.42.4: pg_prop builds clean, both biases shift the distribution as intended, and the deep differential props still pass 3/3 on master with PRNG + bias on. Note the PG_PROP_GEN_PRNG reproducibility knob is verified correct but finding 041's flakiness did not reproduce on this environment, so it stands as a guarantee rather than a demonstrated fix (see findings/041). --- src/test/prop/sql/query_gen.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/test/prop/sql/query_gen.c b/src/test/prop/sql/query_gen.c index 1af0c70d6af4c..2813cb6c2f53f 100644 --- a/src/test/prop/sql/query_gen.c +++ b/src/test/prop/sql/query_gen.c @@ -1512,7 +1512,7 @@ build_join_chain(pg_prop_tc *tc, const qg_db *db, int start, int max_joins, * appendrel shapes when the patch touches those surfaces. */ static const char *const shape_tags[] = { - NULL, "index", "distinct", "setops", "subquery", "subquery", "subquery" + "joins", "joins", "distinct", "setops", "subquery", "subquery", "subquery" }; int shape = qg_biased(tc, 7, shape_tags); /* derived shapes expose id, plus fk when the table has one */ From 82446d6703e2c9375361ab9367db06030171e241 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Fri, 23 Oct 2026 14:40:49 -0400 Subject: [PATCH 331/338] prop: sharpen cf-features.py after testing it on 12 real CF entries Running v1 against twelve commitfest entries exposed three problems: it tagged a pure build-config patch (port/atomics.h) as numeric,cast because its C contains int8 and ::; it emitted all 14 tags for a 140k-line new table-AM patch (no signal); and it over-generated on every large patch. Rewrite the matcher: - Split hunk matching. A distinctive backend identifier (PlaceHolderVar, ExecConstraints, numeric_*, ...) counts anywhere in a changed C line; a SQL keyword (JOIN, UNION, PARTITION, ...) counts only on a line that is actually SQL -- a .sql/.out hunk, or a quoted string / ereport message in C. A bare int/::/UNION token in C no longer fires. Path signal (the touched file) stays strongest. - Rank by signal share and keep the top three, dropping a negligible tail tag (< 10% of the leader) that would only dilute the bias. - Suppress entirely only for a genuinely huge diff (> 50k changed lines): a new subsystem, not a bug fix a focused bias helps. Verified on the 12 entries: the two build/infra false positives (6049 NUMA, 7005 atomics) now emit nothing; the 140k-line table-AM patch (7389) is suppressed; the focused fixes keep a tight top-3 (7392 -> joins,cast,index; 5556 -> constraint,cast,index); and the known-good vetting targets are clean (6824 -> toast, 7186 -> joins, 7450 -> joins,subquery). A backend-internal fix with no SQL surface (7371) yields a weak/empty bias, which is the right answer -- documented as a known limit. --- src/test/prop/scripts/cf-features.py | 216 ++++++++++++++++++--------- 1 file changed, 145 insertions(+), 71 deletions(-) diff --git a/src/test/prop/scripts/cf-features.py b/src/test/prop/scripts/cf-features.py index 4e3aa70a08713..837d4c828a022 100755 --- a/src/test/prop/scripts/cf-features.py +++ b/src/test/prop/scripts/cf-features.py @@ -1,87 +1,152 @@ #!/usr/bin/env python3 """cf-features.py -- extract the SQL-surface features a commitfest patch touches, -so the query generator can bias toward exercising them (task #3). +so the query generator can bias toward exercising them (PG_PROP_GEN_BIAS). Input: one or more unified-diff / git-format-patch files (the CF patch). -Output: a PG_PROP_GEN_BIAS value -- a comma-separated list of feature tags the -generator understands -- printed to stdout. +Output: a comma-separated list of feature tags on stdout, or nothing. -The mapping is deliberately conservative: a tag is emitted only when the diff -touches a file or identifier strongly associated with that SQL surface, so a -biased run still generates everything, just with extra weight on the area the -patch changed. Over-biasing would narrow the search and miss neighbouring -bugs; under-biasing costs only a little focus. When nothing matches, print -nothing (the generator runs unbiased). +Design, after testing v1 on 12 real CF entries: + +* A tag needs a *distinctive* signal, not an incidental C token. v1 tagged a + pure build-config patch (port/atomics.h) as "numeric,cast" because its C + code contains `int8` and `::`. So the hunk patterns are split: + - SYM rules match a distinctive backend identifier (e.g. PlaceHolderVar, + ExecConstraints) anywhere in a changed C line. These are specific + enough to trust. + - SQL rules match a SQL keyword (JOIN, UNION, PARTITION, ...) but only on + lines that are actually SQL: a hunk in a .sql/.out file, or a quoted + string / ereport message inside C. A bare `int`/`::`/`UNION` in C code + no longer counts. +* Rank tags by their share of the signal and emit only the top few, so a + focused fix yields a focused bias. +* Refuse to bias a patch too large to focus (a new subsystem, not a bug + fix): it prints nothing and the generator runs unbiased. + +Path signal (the file a hunk touches) is the strongest and is weighted +highest; it is immune to the C-token noise above. + +Limits (seen on the 12 CF entries this was calibrated against): a fix in +backend-internal code with no SQL surface (e.g. an executor tuple-slot fix +touching tts_tid / ExecForceStoreHeapTuple) yields a weak or empty bias -- +correctly, since there is little SQL surface to aim a query generator at. +That is the intended behaviour: a weak/empty bias is better than a +confident wrong one, and such a patch is a poor biased-hunt target anyway. Tags (kept in sync with the bias table in query_gen.c): - joins FULL/RIGHT/outer-join planning, join removal, PlaceHolderVar - setops UNION/INTERSECT/EXCEPT, append, setop planning - aggregation GROUP BY / aggregates / grouping sets / eager aggregate - window window functions / frames - distinct DISTINCT / DISTINCT ON / unique-ification - subquery subquery pull-up, LATERAL, correlated, sublink - nulls NULL / three-valued logic / strictness - collation collation, especially nondeterministic - numeric numeric scale / float / cast - partition partitioning, pruning, partitionwise - toast TOAST / detoast / external storage / rewrite - index index scan / index-only / amcanreturn / opclass - constraint CHECK / NOT NULL / FK / domain / unique - cast coercion / cast / type input-output + joins setops aggregation window distinct subquery nulls collation + numeric partition toast index constraint cast """ import re import sys -# (compiled regex over added/removed diff lines and touched paths) -> tag -# Each rule is (where, pattern, tag): where is 'path' (the +++/--- file) or -# 'hunk' (a +/- code line). -RULES = [ - ('path', r'optimizer/(path|plan)/(join|analyzejoins|initsplan)', 'joins'), - ('hunk', r'\b(FULL|RIGHT|LEFT)\s+(OUTER\s+)?JOIN\b|PlaceHolderVar|nullingrel|outerjoin|join_is_removable|JoinExpr', 'joins'), - ('path', r'optimizer/(prep/prepunion|path/.*setop)|executor/nodeSetOp|nodeAppend', 'setops'), - ('hunk', r'\b(UNION|INTERSECT|EXCEPT)\b|SetOp|generate_union|append_rel', 'setops'), - ('path', r'executor/nodeAgg|optimizer/.*agg|parser/parse_agg', 'aggregation'), - ('hunk', r'\bGROUP\s+BY\b|\bHAVING\b|Aggref|grouping\s*set|GroupingFunc|eager.?aggregat|aggsplit|partial.?aggregat', 'aggregation'), - ('path', r'executor/nodeWindowAgg|optimizer/.*window|parser/parse_(clause|func).*window', 'window'), - ('hunk', r'\bWINDOW\b|WindowFunc|window\s*frame|\bOVER\b\s*\(|ROWS\s+BETWEEN|RANGE\s+BETWEEN|GROUPS\s+BETWEEN', 'window'), - ('hunk', r'\bDISTINCT\b|DISTINCT\s+ON|unique.?if|rel_is_distinct|distinctClause', 'distinct'), - ('path', r'optimizer/(prep/prepjointree|plan/subselect)|parser/parse_(clause|relation)', 'subquery'), - ('hunk', r'\bLATERAL\b|pull_up_subqueries|SubLink|SubPlan|flatten|convert_ANY|convert_EXISTS|correlat', 'subquery'), - ('hunk', r'\bIS\s+(NOT\s+)?NULL\b|\bIS\s+(NOT\s+)?DISTINCT\b|nonstrict|strictness|contain_nonstrict|null.?frac|three.?valued|ExecEvalCase', 'nulls'), - ('path', r'utils/adt/(varlena|pg_locale|like)|backend/utils/mb', 'collation'), - ('hunk', r'\bCOLLATE\b|collation|deterministic\s*=\s*false|nondeterministic|lc_collate|towlower|iswupper|pg_ascii_tolower|varstr_cmp', 'collation'), - ('path', r'utils/adt/(numeric|float|cash|int8?)\b', 'numeric'), - ('hunk', r'\bnumeric\b|\bfloat[48]\b|trim_scale|\bNaN\b|Infinity|scale\s*\(|rint|pg_ascii', 'numeric'), - ('path', r'(partitioning|partdesc|partprune|partbounds)|executor/nodeAppend|optimizer/.*partition', 'partition'), - ('hunk', r'\bPARTITION\b|partition.?wise|partition.?prun|RelationGetPartition|ATTACH\s+PARTITION|DETACH\s+PARTITION', 'partition'), - ('path', r'access/(heap|common)/.*toast|access/table/toast|detoast|heaptoast|tuptoaster|rewriteheap', 'toast'), - ('hunk', r'\bTOAST\b|detoast|toast_fetch|SET\s+STORAGE|EXTERNAL|EXTENDED|varatt|missing\s+chunk|reltoastrelid', 'toast'), - ('path', r'access/(nbtree|gist|gin|spgist|hash|brin)|optimizer/path/indxpath|executor/nodeIndex', 'index'), - ('hunk', r'index.?only|amcanreturn|IndexOnlyScan|opclass|operator\s+class|index_(get|rescan)|\bamsearch|skip\s*scan', 'index'), - ('path', r'commands/(trigger|tablecmds)|executor/execConstr|catalog/pg_constraint|utils/adt/ri_triggers', 'constraint'), - ('hunk', r'\bCHECK\b|\bNOT\s+NULL\b|\bFOREIGN\s+KEY\b|\bUNIQUE\b|ExecConstraints|\bdomain\b|RI_\w+|constraint', 'constraint'), - ('path', r'parser/parse_coerce|utils/adt/.*(in|out|recv|send)\b', 'cast'), - ('hunk', r'\bCAST\b|coerc|::\s*\w+|input\s+function|CoerceViaIO|typmod|RelabelType', 'cast'), +# PATH rules: (regex over the touched file path) -> tag. Weight 4 (strong). +PATH_RULES = [ + (r'optimizer/(path|plan)/(join|analyzejoins|initsplan)|paths?/joinpath|joinrels', 'joins'), + (r'optimizer/prep/prepunion|path/.*setop|executor/nodeSetOp|executor/nodeAppend', 'setops'), + (r'executor/nodeAgg|optimizer/.*/.*agg|parser/parse_agg|nodeGroup', 'aggregation'), + (r'executor/nodeWindowAgg|optimizer/.*window|nodeWindowFunc', 'window'), + (r'optimizer/prep/prepjointree|plan/subselect|nodeSubplan|nodeSubqueryscan', 'subquery'), + (r'utils/adt/(varlena|pg_locale|like_|formatting)|backend/utils/mb|regc_|collationcmds', 'collation'), + (r'utils/adt/(numeric|float|cash)\.c|utils/adt/int8\.c', 'numeric'), + (r'partitioning/|partdesc|partprune|partbounds|optimizer/.*partition|execPartition', 'partition'), + (r'access/(heap|common)/.*toast|access/table/toast|heaptoast|tuptoaster|rewriteheap|detoast', 'toast'), + (r'access/(nbtree|gist|gin|spgist|hash|brin)/|optimizer/path/indxpath|executor/nodeIndex', 'index'), + (r'commands/(trigger|tablecmds)\.c|executor/execConstr|catalog/pg_constraint|utils/adt/ri_triggers|commands/typecmds', 'constraint'), + (r'parser/parse_coerce|utils/adt/.*(_in|_out|_recv|_send)\b', 'cast'), +] + +# SYM rules: distinctive backend identifier in any changed C line. Weight 2. +SYM_RULES = [ + (r'\b(PlaceHolderVar|nullingrel\w*|join_is_removable|remove_useless_join|JoinExpr|make_outerjoininfo|deconstruct_jointree)\b', 'joins'), + (r'\b(SetOperationStmt|generate_(union|nonunion)_paths|plan_set_operations|make_setop)\b', 'setops'), + (r'\b(Aggref|GroupingFunc|make_agg|AggState|create_agg_path|grouping_sets?|create_groupingsets_path|eager.?aggregat\w*)\b', 'aggregation'), + (r'\b(WindowFunc|WindowAggState|create_windowagg_path|frameOptions|WindowClause)\b', 'window'), + (r'\b(SubLink|SubPlan|pull_up_subqueries|convert_ANY_sublink|convert_EXISTS_sublink|hashed.?subplan)\b', 'subquery'), + (r'\b(DistinctExpr|create_distinct_paths|distinctClause|rel_is_distinct_for|query_is_distinct_for)\b', 'distinct'), + (r'\b(contain_nonstrict\w*|clause_is_strict\w*|not?_?null\w*frac|nulltesttype|BoolExprType|ExecEvalCase\w*|expr_is_nonnullable)\b', 'nulls'), + (r'\b(varstr_cmp|pg_ascii_tolower|towlower|iswupper|lc_collate\w*|get_collation_isdeterministic|pg_newlocale\w*|OidIsValid\(.*collid)\b', 'collation'), + (r'\b(numeric_\w+|float[48]_\w+|trim_scale|make_numeric|NUMERIC_\w+|cash_\w+)\b', 'numeric'), + (r'\b(RelationGetPartition\w*|get_partition_\w+|apply_scanjoin_target|partition_bounds\w*|PartitionPruneState|ExecFindPartition)\b', 'partition'), + (r'\b(toast_\w+|detoast\w*|heap_(fetch|tuple)_?toast|TOAST_\w+|varatt\w*|reltoastrelid|init_toast_snapshot)\b', 'toast'), + (r'\b(amcanreturn|IndexOnlyScan\w*|index_(getnext|rescan|beginscan)|ScanKeyEntry|BTScanOpaque|_bt_\w+|gistget\w*|skip.?scan\w*)\b', 'index'), + (r'\b(ExecConstraints|RI_\w+|ri_\w+|ATExecAddConstraint|check_constraint|DomainConstraint\w*|CheckConstraintFetch)\b', 'constraint'), + (r'\b(coerce_type|CoerceViaIO|coerce_to_target_type|RelabelType|CoerceToDomain|find_coercion_pathway)\b', 'cast'), +] + +# SQL rules: a SQL keyword, counted only on a SQL-ish line (see sql_context). +# Weight 1. +SQL_RULES = [ + (r'\b(FULL|RIGHT|LEFT)\s+(OUTER\s+)?JOIN\b', 'joins'), + (r'\b(UNION|INTERSECT|EXCEPT)\b', 'setops'), + (r'\bGROUP\s+BY\b|\bHAVING\b|\bGROUPING\s+SETS\b|\bROLLUP\b|\bCUBE\b', 'aggregation'), + (r'\bWINDOW\b|\bOVER\s*\(|\b(ROWS|RANGE|GROUPS)\s+BETWEEN\b', 'window'), + (r'\bDISTINCT\b', 'distinct'), + (r'\bLATERAL\b|\bEXISTS\s*\(|\bIN\s*\(\s*SELECT\b', 'subquery'), + (r'\bIS\s+(NOT\s+)?NULL\b|\bIS\s+(NOT\s+)?DISTINCT\s+FROM\b', 'nulls'), + (r'\bCOLLATE\b|deterministic\s*=\s*false', 'collation'), + (r"\bnumeric\s*\(|\bNaN\b|'Infinity'|::numeric\b|::float[48]\b|\bscale\s*\(", 'numeric'), + (r'\bPARTITION\s+(BY|OF)\b|\b(ATTACH|DETACH)\s+PARTITION\b', 'partition'), + (r'\bSET\s+STORAGE\b|\b(EXTERNAL|EXTENDED|MAIN)\b|missing\s+chunk', 'toast'), + (r'\bUSING\s+(gist|gin|spgist|hash|brin|btree)\b|\btext_pattern_ops\b|\bINDEX\s+ONLY\b', 'index'), + (r'\bCHECK\s*\(|\bNOT\s+NULL\b|\bFOREIGN\s+KEY\b|\bREFERENCES\b|\bUNIQUE\b|\bEXCLUDE\s+USING\b', 'constraint'), + (r'\bCAST\s*\(|::\w+|\bWITHOUT\s+OVERLAPS\b', 'cast'), ] +PATH_W, SYM_W, SQL_W = 4, 2, 1 + +# A patch larger than this many changed lines is a new subsystem, not a bug +# fix a focused bias helps -> emit nothing. (Tag count alone is a poor +# gate: a focused fix can touch several surfaces loosely; ranking handles +# that.) +MAX_DIFF_LINES = 50000 + +# Keep at most this many top tags, and drop any whose score is below this +# fraction of the leader's (a negligible tail tag only dilutes the bias). +MAX_KEEP = 3 +TAIL_FRACTION = 0.10 + + +def is_sql_file(path): + return bool(re.search(r'\.(sql|out)(\s|$)', path)) + + +def sql_context(line, in_sql_file): + """True if a SQL keyword on this C/SQL line should count as SQL surface: + the whole file is SQL, or the token sits inside a quoted string / an + ereport/errmsg in C (where SQL keywords denote real surface, not C code).""" + if in_sql_file: + return True + # a quoted string on the line, or an ereport/errmsg/appendStringInfo call + return ('"' in line and re.search(r'"[^"]*\b[A-Z]{2,}\b', line) is not None) \ + or re.search(r'errmsg|ereport|appendStringInfo|elog\s*\(', line) is not None + def extract(difftext): - tags = {} # tag -> hit count, for ranking + tags = {} + changed = 0 cur_path = '' + in_sql_file = False for line in difftext.splitlines(): if line.startswith('+++ ') or line.startswith('--- '): cur_path = line[4:] - for where, pat, tag in RULES: - if where == 'path' and re.search(pat, cur_path, re.I): - tags[tag] = tags.get(tag, 0) + 3 # path match is a strong signal - elif line.startswith('+') or line.startswith('-'): - if line.startswith('+++') or line.startswith('---'): - continue - body = line[1:] - for where, pat, tag in RULES: - if where == 'hunk' and re.search(pat, body): - tags[tag] = tags.get(tag, 0) + 1 - return tags + in_sql_file = is_sql_file(cur_path) + if line.startswith('+++ '): # count the file once + for pat, tag in PATH_RULES: + if re.search(pat, cur_path, re.I): + tags[tag] = tags.get(tag, 0) + PATH_W + continue + if not (line.startswith('+') or line.startswith('-')): + continue + body = line[1:] + changed += 1 + for pat, tag in SYM_RULES: + if re.search(pat, body): + tags[tag] = tags.get(tag, 0) + SYM_W + if in_sql_file or sql_context(body, in_sql_file): + for pat, tag in SQL_RULES: + if re.search(pat, body): + tags[tag] = tags.get(tag, 0) + SQL_W + return tags, changed def main(argv): @@ -92,12 +157,21 @@ def main(argv): for fn in argv[1:]: with open(fn, encoding='utf-8', errors='replace') as f: text += f.read() - tags = extract(text) - # keep tags with a meaningful signal (>= 2: a path hit, or two hunk hits) - ranked = sorted((t for t, n in tags.items() if n >= 2), - key=lambda t: -tags[t]) - if ranked: - print(','.join(ranked)) + tags, changed = extract(text) + # a tag needs at least a path hit, a symbol hit, or two SQL-context hits + strong = {t: n for t, n in tags.items() if n >= SYM_W} + if not strong: + return 0 + # a new subsystem (huge diff) is not something a focused bias helps + if changed > MAX_DIFF_LINES: + sys.stderr.write("cf-features: patch too large to bias (%d changed lines); " + "run unbiased\n" % changed) + return 0 + ranked = sorted(strong, key=lambda t: -strong[t]) + lead = strong[ranked[0]] + # keep the top MAX_KEEP, dropping any negligible tail relative to the leader + keep = [t for t in ranked[:MAX_KEEP] if strong[t] >= lead * TAIL_FRACTION] + print(','.join(keep)) return 0 From dd94cfd1062b9c7835ecb9f270ef5266174f07de Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Fri, 23 Oct 2026 16:58:04 -0400 Subject: [PATCH 332/338] prop: soften the deep-generator reproducibility caveat (041 does not recur) Finding 041 (a fixed seed building a different deep query, seen once on a Debian/x86 build) does not reproduce on the current libhegel in any profile, same-seed or across-seed, on either the original environment or FreeBSD. Removing the degenerate-range guard in qg_draw_int does not bring it back, so that guard is not what closed it; the most likely cause is a libhegel build difference that can no longer be bisected (the old pin was not recorded). Update the README: a deep run is no longer disqualified as a regression gate on reproducibility grounds, and PG_PROP_GEN_PRNG=1 gives determinism by construction if the environment-specific flakiness ever recurs. Reproducing a deep disagreement by hand before believing it remains good practice. --- src/test/prop/README | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/src/test/prop/README b/src/test/prop/README index 5e752ae67200a..ced6741c47210 100644 --- a/src/test/prop/README +++ b/src/test/prop/README @@ -197,11 +197,14 @@ checks only the shapes the generator already makes. The single-table generator (qg_build_table) and the oracles over it are reproducible: a counterexample replays from its seed. The deep multi-table generator (qg_build_db / qg_build_query, used by the query_deep, query_multi -and diff_deep_* properties under PG_PROP_DEEP) is not currently reproducible -for a fixed seed -- the engine's failure-replay can build a different query -(see findings 041). Treat a deep disagreement as a lead to reproduce by -hand, not as a replayable counterexample, and do not use a deep run as a -regression gate until that is fixed. +and diff_deep_* properties under PG_PROP_DEEP) was once seen to be +non-reproducible for a fixed seed on one environment (finding 041), but that +no longer reproduces on the current libhegel in any profile, same-seed or +across-seed. For determinism that does not depend on the engine's +draw-replay behaviour at all, set PG_PROP_GEN_PRNG=1: the deep case is then +reproducible by construction (one engine draw drives a local PRNG). A deep +disagreement is still best reproduced by hand before it is believed, but a +deep run is no longer disqualified as a regression gate on that ground. Commitfest properties and retirement From c76c8dfc58898c279c145d82ae0a2e1789216905 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Fri, 23 Oct 2026 18:03:03 -0400 Subject: [PATCH 333/338] prop: UPDATE preserves CHECK constraints (vet CF 6270) CF 6270 (Jian He, needs-review) skips re-verifying a validated CHECK during UPDATE when none of the CHECK's columns are among those being updated. Add a property that holds with or without that optimisation: the server accepts an UPDATE iff every resulting row still satisfies every CHECK. The table mixes single-column and multi-column CHECKs so a subset UPDATE exercises both the skipped path (CHECK references no updated column) and the verified path (CHECK overlaps one). SET expressions may reference another column, so an assigned column's new value can depend on an unchanged one. The expectation is computed by projecting each column's new value and testing all CHECK predicates; after the UPDATE a second scan asserts no surviving row violates any CHECK, catching a silent mis-skip even if the expectation were wrong. Passes on master (the optimisation absent) and on master+CF6270 v8 across ~60,000 cases, so the patch is sound over this shape. New work in the self-registering constraints area; no registration edit needed. --- src/test/prop/sql/prop_constraints.c | 179 +++++++++++++++++++++++++++ 1 file changed, 179 insertions(+) diff --git a/src/test/prop/sql/prop_constraints.c b/src/test/prop/sql/prop_constraints.c index 5be8d92f60477..a77f336f9ddfb 100644 --- a/src/test/prop/sql/prop_constraints.c +++ b/src/test/prop/sql/prop_constraints.c @@ -393,3 +393,182 @@ PG_PROP(exclusion_rejects_overlap) pg_free(err); pg_prop_disconnect(conn); } + +/* + * CF 6270: "skip check constraints verification for unaffected columns during + * UPDATE" (Jian He). The patch lets an UPDATE skip re-verifying a validated + * CHECK whose columns are none of the ones being updated, reasoning that the + * unchanged columns already satisfied it in the old tuple. If the server's + * notion of "columns being updated" is ever wrong, or a CHECK's column set is + * under-counted, an UPDATE could leave a row that violates its CHECK. + * + * Invariant (holds with or without the optimisation): the server accepts an + * UPDATE IFF every resulting row satisfies every CHECK. We build the UPDATE's + * SET list and, in lockstep, a projection that computes each column's new + * value (the SET expression where assigned, the old column otherwise); the + * expected answer is whether every projected row satisfies all three CHECKs. + * After issuing the UPDATE we compare accept/reject, and -- defence in depth -- + * assert no surviving row violates any CHECK, which catches a silent mis-skip + * even if the expectation were computed wrong. + * + * The CHECKs mix single-column (a) and multi-column (a+b, c= -6), " + "CONSTRAINT ck_ab CHECK (a + b >= 0), " + "CONSTRAINT ck_cd CHECK (c < d))", + &err)) + pg_prop_fail(tc, "setup failed: %s", err ? err : ""); + + /* seed rows that all satisfy the CHECKs, so UPDATE is the only risk */ + for (i = 0; i < nseed; i++) + { + StringInfoData q; + int a = (int) pg_prop_draw_int(tc, 0, C_K_MAX); + int b = (int) pg_prop_draw_int(tc, 0, C_K_MAX); + int c = (int) pg_prop_draw_int(tc, C_K_MIN, 0); + int d = c + (int) pg_prop_draw_int(tc, 1, 6); + + initStringInfo(&q); + appendStringInfo(&q, + "INSERT INTO up_c (a,b,c,d) VALUES (%d,%d,%d,%d)", + a, b, c, d); + if (!pg_prop_sql_exec(conn, q.data, &err)) + pg_prop_fail(tc, "seed insert rejected (%s): %s", + q.data, err ? err : ""); + pfree(q.data); + } + + for (i = 0; i < nupd; i++) + { + static const char *const cols[] = {"a", "b", "c", "d"}; + StringInfoData set; /* UPDATE ... SET */ + StringInfoData proj; /* projected new values: AS a, ... */ + StringInfoData where; + StringInfoData chk; + StringInfoData upd; + bool any = false; + bool accepted; + bool expect_accept; + bool failed = false; + char *v; + int j; + + initStringInfo(&set); + initStringInfo(&proj); + initStringInfo(&where); + + /* some UPDATEs restrict to a subset of rows */ + if (pg_prop_draw_int(tc, 0, 2) == 0) + appendStringInfo(&where, " WHERE id %% 2 = %d", + (int) pg_prop_draw_int(tc, 0, 1)); + + for (j = 0; j < 4; j++) + { + char expr[64]; + bool assign = (pg_prop_draw_int(tc, 0, 1) == 1); + + /* force at least the last column if nothing assigned yet */ + if (j == 3 && !any) + assign = true; + + if (assign) + { + if (pg_prop_draw_int(tc, 0, 1) == 0) + snprintf(expr, sizeof(expr), "%d", + (int) pg_prop_draw_int(tc, C_K_MIN - 3, C_K_MAX + 3)); + else + snprintf(expr, sizeof(expr), "%s %c %d", + cols[(j + 1) % 4], + pg_prop_draw_int(tc, 0, 1) ? '+' : '-', + (int) pg_prop_draw_int(tc, 0, 4)); + + if (any) + appendStringInfoString(&set, ", "); + appendStringInfo(&set, "%s = %s", cols[j], expr); + any = true; + } + + /* projection: the new value where assigned, else the old column */ + if (j > 0) + appendStringInfoString(&proj, ", "); + appendStringInfo(&proj, "(%s) AS %s", assign ? expr : cols[j], cols[j]); + } + + /* + * Expected acceptance: do all affected rows, after the SET, still + * satisfy every CHECK? bool_and over zero affected rows is NULL, + * which we treat as accept (a vacuous UPDATE changes nothing). + */ + initStringInfo(&chk); + appendStringInfo(&chk, + "SELECT bool_and(a >= -6 AND a + b >= 0 AND c < d) " + "FROM (SELECT %s FROM up_c%s) s", + proj.data, where.data); + v = pg_prop_sql_scalar(conn, chk.data, &failed, &err); + expect_accept = (v == NULL) || (v[0] == '\0') || (strcmp(v, "t") == 0); + if (v) + pg_free(v); + if (failed) + pg_prop_fail(tc, "expectation query failed (%s): %s", + chk.data, err ? err : ""); + + initStringInfo(&upd); + appendStringInfo(&upd, "UPDATE up_c SET %s%s", set.data, where.data); + accepted = try_insert(tc, conn, upd.data); + + if (accepted != expect_accept) + pg_prop_fail(tc, + "UPDATE accept/reject wrong: expect_accept=%d " + "accepted=%d; sql=[%s]", + (int) expect_accept, (int) accepted, upd.data); + + /* no surviving row may violate any CHECK, regardless of accept */ + { + bool f2 = false; + char *bad = pg_prop_sql_scalar(conn, + "SELECT count(*) FROM up_c " + "WHERE NOT (a >= -6 AND a + b >= 0 AND c < d)", + &f2, &err); + long nbad = bad ? atol(bad) : 0; + + if (bad) + pg_free(bad); + if (f2) + pg_prop_fail(tc, "violation scan failed: %s", err ? err : ""); + if (nbad > 0) + pg_prop_fail(tc, + "CHECK violated after UPDATE: %ld bad row(s); " + "sql=[%s]", + nbad, upd.data); + } + + pfree(set.data); + pfree(proj.data); + pfree(where.data); + pfree(chk.data); + pfree(upd.data); + } + + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS up_c", &err); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} + From 10ebe0382701ea2d3f9f0c9f328fa8c03185514c Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Fri, 23 Oct 2026 18:03:14 -0400 Subject: [PATCH 334/338] prop: UPDATE preserves CHECK over generated columns (vet CF 6270) CF 6270 expands generated columns in a CHECK before collecting the columns it depends on (expand_generated_columns_in_expr), so a CHECK over a stored generated column g = a + b becomes a CHECK over the base columns a and b. An UPDATE touching a base column must then still verify that CHECK even though g is not named in SET -- the subtle case where "columns being updated" and "columns a CHECK depends on" can diverge if the expansion or the updated-column set is wrong. Add a variant over a table with two stored generated columns (g = a + b, h = a - b) and CHECKs on each, updating a subset of the base columns. The expectation recomputes the generated values from the projected base values and tests all CHECKs; the same no-surviving-violator assertion follows. Passes on master and on master+CF6270 v8 across ~30,000 cases: the patch handles the generated-column case correctly too. --- src/test/prop/sql/prop_constraints.c | 162 +++++++++++++++++++++++++++ 1 file changed, 162 insertions(+) diff --git a/src/test/prop/sql/prop_constraints.c b/src/test/prop/sql/prop_constraints.c index a77f336f9ddfb..8a47412431d0f 100644 --- a/src/test/prop/sql/prop_constraints.c +++ b/src/test/prop/sql/prop_constraints.c @@ -572,3 +572,165 @@ PG_PROP(update_preserves_check) pg_prop_disconnect(conn); } +/* + * CF 6270, generated-column variant. The patch calls + * expand_generated_columns_in_expr on each CHECK before collecting its + * attnos, so a CHECK that references a stored generated column g is, after + * expansion, a CHECK over g's base columns. An UPDATE that touches a base + * column of g must therefore still verify such a CHECK even though g itself is + * not named in SET. This is the subtle case where "columns being updated" + * and "columns a CHECK depends on" can disagree if the expansion or the + * updated-column set is computed wrong. + * + * Table: a, b (base), g = a + b STORED, h = a - b STORED, with CHECK (g >= 0) + * and CHECK (h <= 100) over the generated columns, plus a plain CHECK (a >= -6) + * over a base column. We UPDATE a random subset of {a,b} and require the + * server to accept IFF every resulting row satisfies all CHECKs, with the + * same no-surviving-violator assertion. The expectation is computed by + * projecting the new base values and the recomputed generated values. + */ +PG_PROP(update_preserves_check_generated) +{ + pg_prop_conn *conn = connect_c(tc); + char *err = NULL; + int nseed = (int) pg_prop_draw_int(tc, 1, 20); + int nupd = (int) pg_prop_draw_int(tc, 1, 25); + int i; + + if (!pg_prop_sql_exec(conn, + "DROP TABLE IF EXISTS upg_c; " + "CREATE TABLE upg_c (id serial PRIMARY KEY, " + "a int, b int, " + "g int GENERATED ALWAYS AS (a + b) STORED, " + "h int GENERATED ALWAYS AS (a - b) STORED, " + "CONSTRAINT ckg_a CHECK (a >= -6), " + "CONSTRAINT ckg_g CHECK (g >= 0), " + "CONSTRAINT ckg_h CHECK (h <= 100))", + &err)) + { + /* generated columns unavailable (very old server): skip cleanly */ + if (err) + pg_free(err); + pg_prop_skip(tc, "stored generated columns not available"); + } + + for (i = 0; i < nseed; i++) + { + StringInfoData q; + int a = (int) pg_prop_draw_int(tc, 0, C_K_MAX); + int b = (int) pg_prop_draw_int(tc, 0, a); /* a-b <= 100, a+b >= 0 */ + + initStringInfo(&q); + appendStringInfo(&q, "INSERT INTO upg_c (a,b) VALUES (%d,%d)", a, b); + if (!pg_prop_sql_exec(conn, q.data, &err)) + pg_prop_fail(tc, "seed insert rejected (%s): %s", + q.data, err ? err : ""); + pfree(q.data); + } + + for (i = 0; i < nupd; i++) + { + static const char *const base[] = {"a", "b"}; + StringInfoData set; + StringInfoData proj; /* na, nb projected new base values */ + StringInfoData where; + StringInfoData chk; + StringInfoData upd; + bool any = false; + bool accepted; + bool expect_accept; + bool failed = false; + char *v; + int j; + + initStringInfo(&set); + initStringInfo(&proj); + initStringInfo(&where); + + if (pg_prop_draw_int(tc, 0, 2) == 0) + appendStringInfo(&where, " WHERE id %% 2 = %d", + (int) pg_prop_draw_int(tc, 0, 1)); + + for (j = 0; j < 2; j++) + { + char expr[64]; + bool assign = (pg_prop_draw_int(tc, 0, 1) == 1); + + if (j == 1 && !any) + assign = true; + + if (assign) + { + if (pg_prop_draw_int(tc, 0, 1) == 0) + snprintf(expr, sizeof(expr), "%d", + (int) pg_prop_draw_int(tc, C_K_MIN - 3, C_K_MAX + 60)); + else + snprintf(expr, sizeof(expr), "%s %c %d", + base[(j + 1) % 2], + pg_prop_draw_int(tc, 0, 1) ? '+' : '-', + (int) pg_prop_draw_int(tc, 0, 50)); + if (any) + appendStringInfoString(&set, ", "); + appendStringInfo(&set, "%s = %s", base[j], expr); + any = true; + } + if (j > 0) + appendStringInfoString(&proj, ", "); + appendStringInfo(&proj, "(%s) AS %s", assign ? expr : base[j], base[j]); + } + + /* expected: project new a,b; recompute g=a+b, h=a-b; test all CHECKs */ + initStringInfo(&chk); + appendStringInfo(&chk, + "SELECT bool_and(a >= -6 AND (a + b) >= 0 AND (a - b) <= 100) " + "FROM (SELECT %s FROM upg_c%s) s", + proj.data, where.data); + v = pg_prop_sql_scalar(conn, chk.data, &failed, &err); + expect_accept = (v == NULL) || (v[0] == '\0') || (strcmp(v, "t") == 0); + if (v) + pg_free(v); + if (failed) + pg_prop_fail(tc, "expectation query failed (%s): %s", + chk.data, err ? err : ""); + + initStringInfo(&upd); + appendStringInfo(&upd, "UPDATE upg_c SET %s%s", set.data, where.data); + accepted = try_insert(tc, conn, upd.data); + + if (accepted != expect_accept) + pg_prop_fail(tc, + "UPDATE(gen) accept/reject wrong: expect_accept=%d " + "accepted=%d; sql=[%s]", + (int) expect_accept, (int) accepted, upd.data); + + { + bool f2 = false; + char *bad = pg_prop_sql_scalar(conn, + "SELECT count(*) FROM upg_c " + "WHERE NOT (a >= -6 AND g >= 0 AND h <= 100)", + &f2, &err); + long nbad = bad ? atol(bad) : 0; + + if (bad) + pg_free(bad); + if (f2) + pg_prop_fail(tc, "violation scan failed: %s", err ? err : ""); + if (nbad > 0) + pg_prop_fail(tc, + "CHECK(gen) violated after UPDATE: %ld bad row(s); " + "sql=[%s]", + nbad, upd.data); + } + + pfree(set.data); + pfree(proj.data); + pfree(where.data); + pfree(chk.data); + pfree(upd.data); + } + + pg_prop_sql_exec(conn, "DROP TABLE IF EXISTS upg_c", &err); + if (err) + pg_free(err); + pg_prop_disconnect(conn); +} From f7164275225554b93d6c7a8a1b0901768d1b703c Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 24 Oct 2026 12:53:54 -0400 Subject: [PATCH 335/338] prop: XMLNAMESPACES URI survives deparse (vet CF 7492) CF 7492 (ready-for-committer) fixes ruleutils so that a non-Const URI expression in an XMLNAMESPACES item is parenthesized when a view is deparsed. The URI is parsed as a b_expr, which has no COLLATE (or other a_expr-only) syntax, so without the parens the deparsed view fails to re-parse with "syntax error at or near COLLATE". Add a cf_deparse round-trip property that builds an XMLTABLE view whose namespace URI is a drawn expression (plain string, COLLATE, concatenation, CASE, nested cast+COLLATE), then deparses, recreates, and requires the recreate to parse, return the same rows, and reach a deparse fixed point. Canary on the Debian box: falsifies on master (3/3, "the deparsed view does not read back: syntax error at or near COLLATE") and passes on master+CF7492 v1 (0 failures over ~3,000 cases). Retire when 7492 commits. --- src/test/prop/sql/prop_cf_deparse.c | 49 +++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/src/test/prop/sql/prop_cf_deparse.c b/src/test/prop/sql/prop_cf_deparse.c index 8be8458425a18..8e864f8479677 100644 --- a/src/test/prop/sql/prop_cf_deparse.c +++ b/src/test/prop/sql/prop_cf_deparse.c @@ -284,3 +284,52 @@ PG_PROP(rule_target_alias_survives_deparse) pfree(s.data); pg_prop_disconnect(conn); } + +/* + * CF 7492: a COLLATE clause (or any non-Const expression) in the URI of an + * XMLNAMESPACES item did not read back. The URI is parsed as a b_expr, which + * has no COLLATE / other a_expr-only syntax, so ruleutils must parenthesize a + * non-Const URI when deparsing; before the fix it emitted the bare expression + * and the deparsed view failed to re-parse ("syntax error at or near COLLATE"). + * + * dp_roundtrip creates the view, deparses it, recreates from the deparse, and + * requires the recreate to parse, return the same rows, and reach a deparse + * fixed point. On master the recreate step fails; with the patch it passes. + * We randomize the URI expression (plain string, COLLATE, concatenation, a + * CASE) so the property covers the general non-Const URI, not just one + * literal. + */ +PG_PROP(xmlnamespace_uri_survives_deparse) +{ + static const char *const uris[] = { + "'http://x.example'", /* Const: the easy case */ + "('http://x.example' COLLATE \"C\")", /* the reported bug */ + "('http://' || 'x.example')", /* concatenation */ + "(CASE WHEN true THEN 'http://x.example' ELSE '' END)", + "(('http://x.example')::text COLLATE \"POSIX\")", + }; + pg_prop_conn *conn = dp_connect(tc); + const char *uri = uris[pg_prop_draw_int(tc, 0, lengthof(uris) - 1)]; + StringInfoData body; + + initStringInfo(&body); + /* + * A minimal XMLTABLE whose XMLNAMESPACES carries the drawn URI expression. + * PostgreSQL's XMLTABLE supports only named namespaces (no DEFAULT), so we + * always name it; the document and row path are fixed and the column is + * text so the row read-back never casts. Only the namespace URI varies, + * which is the deparse surface under test. + */ + appendStringInfo(&body, + "SELECT x.a FROM XMLTABLE(" + "XMLNAMESPACES(%s AS ns), " + "'/ns:root' PASSING XML 'v' " + "COLUMNS a text PATH '.') x", + uri); + + dp_roundtrip(tc, conn, body.data, pg_prop_draw_bool(tc)); + + pfree(body.data); + pg_prop_sql_exec(conn, "DROP VIEW IF EXISTS cfd_v2, cfd_v1", NULL); + pg_prop_disconnect(conn); +} From bba318d45528fcee7630e47123628b75e14b2d67 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 24 Oct 2026 15:42:59 -0400 Subject: [PATCH 336/338] prop: fail a property that runs zero non-vacuous cases A property that returns early on every case -- the generated database was too small to form the query, the drawn SQL did not parse, or the server was not reachable -- is counted by the engine as PASSED, because an early return is indistinguishable from "the invariant held". A validation run that tested nothing then reports success, the worst failure a checker can have. This bit in practice: a start-up slip left the server down, every deep differential case returned vacuous, and the suite reported all green while testing nothing. Add a vacuous-pass guard. A property arms it once with pg_prop_opt_into_vacuous_guard() and marks each case that reaches its invariant check with pg_prop_nonvacuous(). A run that armed the guard but marked zero non-vacuous cases is reported "not ok: ran 0 non-vacuous cases" instead of passing. Two calls because "this property wants the guard" and "this case did real work" are distinct; arming up front lets the harness tell an all-vacuous run (fail) from a legacy property that never opted in (pass). Wire it into run_diff (the differential oracle): arm on entry, mark non-vacuous once the reference query has actually run. Legacy properties that never arm it are unaffected. Verified: a probe that arms the guard and always returns early now fails; the real differential on a live server still passes; pointing any armed property at a down server fails loudly. --- src/test/prop/pg_prop.c | 50 ++++++++++++++++++++++++++++- src/test/prop/pg_prop.h | 32 ++++++++++++++++++ src/test/prop/sql/prop_query_diff.c | 16 +++++++++ 3 files changed, 97 insertions(+), 1 deletion(-) diff --git a/src/test/prop/pg_prop.c b/src/test/prop/pg_prop.c index e9ad992a63cfe..3546bb8bedd11 100644 --- a/src/test/prop/pg_prop.c +++ b/src/test/prop/pg_prop.c @@ -218,6 +218,16 @@ static bool overrun; */ static bool skip_requested; +/* + * Vacuous-pass guard (see pg_prop_nonvacuous). opted_in becomes true the + * first time a property calls pg_prop_nonvacuous in a run; nonvacuous_cases + * counts the cases that reached their invariant check. A run that opted in + * but ran zero non-vacuous cases is reported as a failure, not a silent pass. + * Both are reset at the start of each property's run. + */ +static bool nonvacuous_opted_in; +static uint64 nonvacuous_cases; + /* "area/property" now running, for diagnostics (see pg_prop_sql.c). */ const char *pg_prop_current_area = NULL; static char current_area_buf[256]; @@ -401,6 +411,21 @@ pg_prop_note(pg_prop_tc *tc, const char *fmt,...) pfree(s.data); } +void +pg_prop_nonvacuous(pg_prop_tc *tc) +{ + (void) tc; + nonvacuous_opted_in = true; + nonvacuous_cases++; +} + +void +pg_prop_opt_into_vacuous_guard(pg_prop_tc *tc) +{ + (void) tc; + nonvacuous_opted_in = true; +} + void pg_prop_fail(pg_prop_tc *tc, const char *fmt,...) { @@ -647,6 +672,7 @@ run_property(PropEntry *prop, uint64 iterations, bool have_seed, uint64 seed, hegel_run_status_t rstatus; RunState rs; bool ok; + bool vacuous_fail = false; char dbkey[256]; @@ -680,6 +706,8 @@ run_property(PropEntry *prop, uint64 iterations, bool have_seed, uint64 seed, check_rc(ctx, hegel_run_start(ctx, s, NULL, NULL, &run), "run_start"); skip_requested = false; + nonvacuous_opted_in = false; + nonvacuous_cases = 0; for (;;) { @@ -722,6 +750,19 @@ run_property(PropEntry *prop, uint64 iterations, bool have_seed, uint64 seed, return true; } + /* + * Vacuous-pass guard: a property that opted in (called pg_prop_nonvacuous + * in its live path) but reached that point in zero cases tested nothing, + * even though the engine reported PASSED. Fail it loudly rather than + * report a false green. A genuine failure (ok already false) is left + * alone. + */ + if (ok && nonvacuous_opted_in && nonvacuous_cases == 0) + { + ok = false; + vacuous_fail = true; + } + if (tap) printf("%s %d - %s/%s\n", ok ? "ok" : "not ok", testno, prop->area, prop->name); @@ -733,7 +774,14 @@ run_property(PropEntry *prop, uint64 iterations, bool have_seed, uint64 seed, { const char *pfx = tap ? "# " : " "; - if (rstatus == HEGEL_RUN_STATUS_ERROR) + if (vacuous_fail) + { + printf("%svacuous: property ran 0 non-vacuous cases -- every case " + "returned early before its invariant check (database too " + "small, generated SQL invalid, or the server unreachable). " + "The run tested nothing; not reporting it as passed.\n", pfx); + } + else if (rstatus == HEGEL_RUN_STATUS_ERROR) { const char *err = NULL; diff --git a/src/test/prop/pg_prop.h b/src/test/prop/pg_prop.h index 8784c71981672..c3dca7da5028e 100644 --- a/src/test/prop/pg_prop.h +++ b/src/test/prop/pg_prop.h @@ -207,6 +207,38 @@ extern pg_noreturn void pg_prop_skip(pg_prop_tc *tc, const char *fmt,...) pg_att */ extern void pg_prop_note(pg_prop_tc *tc, const char *fmt,...) pg_attribute_printf(2, 3); +/* + * Mark that this case did real work: it reached its invariant check (ran the + * query, compared the oracle) rather than returning early as vacuous (the + * generated database was too small to join, the drawn SQL did not parse, a + * guard bailed out). Call it at the point the case is about to assert its + * property, after any early-return guards. Pair it with + * pg_prop_opt_into_vacuous_guard() at the top of the body. + * + * Why this exists: a property that returns early on every case is counted by + * the engine as PASSED, because an early return is indistinguishable from "the + * invariant held". A validation run that tested nothing then reports success + * -- the worst failure mode a checker can have (it was a live bug: a startup + * slip left the server down, every deep case returned vacuous, and the suite + * reported all green while testing nothing). + */ +extern void pg_prop_nonvacuous(pg_prop_tc *tc); + +/* + * Declare that this property is subject to the vacuous-pass guard: call it + * once, unconditionally, at the top of the body (before any early-return + * guard). It arms the guard for the whole run; pg_prop_nonvacuous then marks + * each case that reached its invariant check. A run that armed the guard but + * marked zero non-vacuous cases is failed. + * + * Two calls rather than one because "this property wants the guard" and "this + * case did real work" are distinct: if a property only called nonvacuous and + * no case ever reached it, the harness could not tell an all-vacuous run + * (should fail) from a legacy property that never opted in (should pass). + * Arming it up front removes that ambiguity. + */ +extern void pg_prop_opt_into_vacuous_guard(pg_prop_tc *tc); + /* * Fail the current test case. This is how a property reports that its * invariant did not hold. The message is the human-readable description of diff --git a/src/test/prop/sql/prop_query_diff.c b/src/test/prop/sql/prop_query_diff.c index f1f21eea43ab0..058f6c77c710e 100644 --- a/src/test/prop/sql/prop_query_diff.c +++ b/src/test/prop/sql/prop_query_diff.c @@ -175,6 +175,13 @@ run_diff(pg_prop_tc *tc, pg_prop_conn *conn, const char *query) bool errored; int i; + /* + * This oracle is subject to the vacuous-pass guard: arm it now (so a run + * that never reaches a real comparison is failed, not passed) and mark the + * case non-vacuous once the reference query has actually run (below). + */ + pg_prop_opt_into_vacuous_guard(tc); + /* * Debug knob: when PG_PROP_DIFF_DEBUG names a file, append every query * this oracle runs to it. Used to verify generator reproducibility -- @@ -214,6 +221,15 @@ run_diff(pg_prop_tc *tc, pg_prop_conn *conn, const char *query) } pg_free(err); + /* + * The reference query actually ran: this case does real work (it will + * compare the reference against every other config). Mark it non-vacuous + * so a run in which every case bailed out earlier (empty database, + * unparsable query, server down) is reported as a failure rather than a + * silent pass. + */ + pg_prop_nonvacuous(tc); + for (i = 1; i < N_CONFIGS; i++) { char *d = digest_under(conn, configs[i], query, &errored, &err); From a483394826536f6c3a0f21d7c8881de345cb1069 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 24 Oct 2026 15:45:25 -0400 Subject: [PATCH 337/338] prop: cf-features.py --classify emits a vettability verdict "Validate every ready-for-committer entry" does not map to this method: the differential/property oracle only judges patches with a SQL-observable wrong answer. Roughly half of a commitfest is feature-adds, tooling, docs, refactors and cost/EXPLAIN-only changes with no such oracle. Add --classify, which maps a patch to the oracle class that applies: deparse a pg_get_*def / ruleutils change -> round-trip oracle isolation an SSI/concurrency change -> needs an isolation spec estimate a selectivity/stats change -> estimate-vs-actual differential a planner/executor/AM/type change -> row-multiset differential not-vettable docs/tooling/tests/pure-cost -> human review, no oracle This turns "validate them all" into a defensible scoped list and stops a cost-only or display-only patch from being mistaken for something a result differential can judge. Checked against the 20 entries vetted this cycle: 19/20 match the class actually used; the one miss (a postgres_fdw cost-only patch) errs toward differential, the safe direction -- it gets run and the differential correctly finds nothing, rather than being wrongly skipped. --- src/test/prop/scripts/cf-features.py | 77 +++++++++++++++++++++++++++- 1 file changed, 75 insertions(+), 2 deletions(-) diff --git a/src/test/prop/scripts/cf-features.py b/src/test/prop/scripts/cf-features.py index 837d4c828a022..9a046f3b2d0bd 100755 --- a/src/test/prop/scripts/cf-features.py +++ b/src/test/prop/scripts/cf-features.py @@ -94,6 +94,39 @@ PATH_W, SYM_W, SQL_W = 4, 2, 1 +# --------------------------------------------------------------------------- +# Vettability classification. The differential/property method only has an +# oracle for patches with a SQL-observable wrong answer. Map a patch to the +# oracle class that applies so "validate them all" becomes a defensible scoped +# list -- and so a cost-only or display-only or tooling patch is not mistaken +# for something a result differential can judge (it cannot, by construction). +# +# Classes, most-specific first: +# deparse a pg_get_*def / pg_get_*_ddl / ruleutils change -> round-trip +# oracle (deparse, recreate, require same definition + rows). +# isolation a concurrency/SSI change -> needs an isolation spec, not the +# single-session generator. +# estimate a selectivity/stats change -> estimate-vs-actual, not a +# row-set differential (the rows do not change, the estimate +# does). +# differential a planner/executor/AM/type change that can change the result +# set -> the row-multiset differential (plan A == plan B). +# not-vettable docs, tooling (bin/, psql, pg_dump, pg_rewind, pgbench), +# refactors, pure cost tweaks, EXPLAIN-text-only, replication/WAL +# operational -> human review, no differential oracle. +CLASS_RULES = [ + # (regex over a touched file path, class, why) + ('deparse', r'ruleutils\.c|utils/adt/ddlutils\.c|pg_get_\w+def|pg_get_\w+_ddl', 'deparse round-trip'), + ('isolation', r'storage/lmgr/predicate\.c|lmgr/.*ssi|/isolation/specs/|README-SSI', 'isolation spec'), + ('estimate', r'utils/adt/selfuncs\.c|optimizer/path/clausesel\.c|statistics/(extended_stats|mcv|dependencies)|optimizer/util/plancat\.c', 'estimate vs actual'), + ('differential', r'executor/node|optimizer/(path|plan|prep)/|access/(nbtree|gist|gin|spgist|hash|brin|heap|common|table)/|utils/adt/\w+\.c|parser/parse_|commands/\w+\.c|rewrite/|catalog/pg_proc\.dat|contrib/\w+/\w+\.c', 'row-set differential'), +] +# paths that force not-vettable regardless of the above (tooling, docs, tests). +NOT_VETTABLE = re.compile( + r'(^|/)(doc/|src/bin/|contrib/\w+/t/|/t/\d|psql/|pg_dump/|pg_rewind/|pg_basebackup/|' + r'pg_combinebackup/|pg_resetwal/|pg_surgery/|pg_walinspect/|pgbench/|' + r'\.sgml|\.po$|/po/|regress/(sql|expected)/)') + # A patch larger than this many changed lines is a new subsystem, not a bug # fix a focused bias helps -> emit nothing. (Tag count alone is a poor # gate: a focused fix can touch several surfaces loosely; ranking handles @@ -149,14 +182,54 @@ def extract(difftext): return tags, changed +def touched_paths(difftext): + """The set of non-/dev/null file paths the diff touches (from +++ lines).""" + paths = set() + for line in difftext.splitlines(): + if line.startswith('+++ '): + p = line[4:].strip() + if p and p != '/dev/null': + paths.add(re.sub(r'^[ab]/', '', p)) + return paths + + +def classify(difftext): + """Return (class, why): the oracle class that can vet this patch. + + A patch that touches ONLY not-vettable paths (docs, tooling, tests) is + not-vettable. Otherwise the most-specific matching CLASS_RULES entry + over any backend/contrib source path wins.""" + paths = touched_paths(difftext) + src = [p for p in paths if not NOT_VETTABLE.search(p)] + if not src: + return 'not-vettable', 'only docs/tooling/tests touched' + for cls, pat, why in CLASS_RULES: + rx = re.compile(pat, re.I) + if any(rx.search(p) for p in src): + return cls, why + return 'not-vettable', 'no backend change with a differential oracle' + + def main(argv): if len(argv) < 2: - sys.stderr.write("usage: cf-features.py PATCH [PATCH ...]\n") + sys.stderr.write("usage: cf-features.py [--classify] PATCH [PATCH ...]\n") + return 2 + do_classify = False + args = argv[1:] + if args and args[0] == '--classify': + do_classify = True + args = args[1:] + if not args: + sys.stderr.write("usage: cf-features.py [--classify] PATCH [PATCH ...]\n") return 2 text = '' - for fn in argv[1:]: + for fn in args: with open(fn, encoding='utf-8', errors='replace') as f: text += f.read() + if do_classify: + cls, why = classify(text) + print('%s\t%s' % (cls, why)) + return 0 tags, changed = extract(text) # a tag needs at least a path hit, a symbol hit, or two SQL-context hits strong = {t: n for t, n in tags.items() if n >= SYM_W} From 1aae059ec787f6cc726be0f2b347f6d9e5b371d3 Mon Sep 17 00:00:00 2001 From: Greg Burd Date: Sat, 24 Oct 2026 15:51:16 -0400 Subject: [PATCH 338/338] prop: default the deep-generator PRNG on under a deep profile Reproducibility of a deep multi-table counterexample matters more than the engine's structural shrinking for a hunt or a commitfest vet -- and the engine cannot shrink deep query structure anyway (it only sees the one seed draw). So default PG_PROP_GEN_PRNG on whenever PG_PROP_DEEP selects a deep profile, and leave it off for the default single-table profile, where shrinking of the short, fixed-shape draw sequence still works well. PG_PROP_GEN_PRNG=1/0 still forces the choice either way. Verified: a deep profile now yields one distinct query across repeated same-seed runs without setting the variable; =0 restores the engine-draw path. --- src/test/prop/README | 15 ++++++++++----- src/test/prop/sql/query_gen.c | 24 +++++++++++++++++++----- 2 files changed, 29 insertions(+), 10 deletions(-) diff --git a/src/test/prop/README b/src/test/prop/README index ced6741c47210..bab630c70e680 100644 --- a/src/test/prop/README +++ b/src/test/prop/README @@ -159,13 +159,18 @@ iteration count of a meson test run without editing anything: Two further knobs tune the deep multi-table generator: -* PG_PROP_GEN_PRNG=1 -- make the deep generator reproducible from a seed. +* PG_PROP_GEN_PRNG -- control deep-generator reproducibility from a seed. The multi-table builder makes hundreds of data-dependent draws per case, a sequence the engine did not replay reliably (so a deep counterexample - could not be reproduced). With this set, the builder takes one engine - draw per case and drives all structural choices from a deterministic PRNG - seeded by it, so the case replays exactly; the cost is coarser shrinking - of deep structure. Unset (the default) keeps the historical behaviour. + could not be reproduced). When active, the builder takes one engine draw + per case and drives all structural choices from a deterministic PRNG seeded + by it, so the case replays exactly; the cost is coarser shrinking of deep + structure. DEFAULT: on whenever a deep profile is selected (PG_PROP_DEEP + set), because for a hunt/vet run replaying a hit matters more than the + engine's structural shrinking (which it cannot do for a deep query anyway); + off for the default single-table profile, where engine shrinking of the + short draw sequence works well. PG_PROP_GEN_PRNG=1 forces it on, + PG_PROP_GEN_PRNG=0 forces it off, overriding the profile default. * PG_PROP_GEN_BIAS=tag,tag,... -- bias the deep generator toward the SQL surface a patch under review touches, when vetting a commitfest entry. diff --git a/src/test/prop/sql/query_gen.c b/src/test/prop/sql/query_gen.c index 2813cb6c2f53f..8aa0239c1bb00 100644 --- a/src/test/prop/sql/query_gen.c +++ b/src/test/prop/sql/query_gen.c @@ -67,17 +67,31 @@ qg_splitmix64(void) /* * Begin a deterministic generation region: draw one engine integer as the - * seed and switch qg_draw_* to the PRNG. Honours PG_PROP_GEN_PRNG; when the - * variable is unset or 0 the region is a no-op and the generator draws from - * the engine as it always did (so the change is A/B-testable and default-off - * until verified on a build host). + * seed and switch qg_draw_* to the PRNG. + * + * Default policy (learned from CF vetting, where replaying a hit matters more + * than the engine's structural shrinking -- which it cannot do for a deep + * multi-table query anyway): + * - a DEEP profile (PG_PROP_DEEP set) defaults the PRNG ON, so a deep + * counterexample is reproducible by construction regardless of libhegel's + * draw-replay behaviour; + * - the default single-table profile leaves it OFF, so the engine's + * shrinking of the short, fixed-shape draw sequence still works well. + * PG_PROP_GEN_PRNG overrides either way: =1 forces on, =0 forces off. */ static void qg_rng_begin(pg_prop_tc *tc) { const char *v = getenv("PG_PROP_GEN_PRNG"); + const char *deep = getenv("PG_PROP_DEEP"); + bool want; - if (v == NULL || v[0] == '\0' || strcmp(v, "0") == 0) + if (v != NULL && v[0] != '\0') + want = (strcmp(v, "0") != 0); /* explicit override */ + else + want = (deep != NULL && deep[0] != '\0' && strcmp(deep, "0") != 0); + + if (!want) { qg_rng_active = false; return;