Skip to content

feat: add a polars regex comparator option to eq/ne reindex filters - #232

Merged
SkyeAv merged 1 commit into
fix/docs-taxon-filtering-linkfrom
feat/reindex-regex-config
Oct 21, 2026
Merged

SkyeAv merged 1 commit into
fix/docs-taxon-filtering-linkfrom
feat/reindex-regex-config

Conversation

@SkyeAv

@SkyeAv SkyeAv commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner

feat: add a polars regex comparator option to eq/ne reindex filters

Adds an opt-in regex flag to source.reindex entries so an eq/ne filter can match its comparator as a polars-compatible pattern instead of a literal value. One anchored pattern then drops a whole missing-value family (N/A, n/a, --, not reported) instead of one literal filter per spelling.

Config surface

  • New field: Reindex.regex in src/tablassert/models.py, defaulting to false, so a config that never names it validates, dumps, and filters exactly as before.
  • Operator gate: regex: true with lt/le/gt/ge is rejected at config time with the new comparison-regex-requires-eq-ne code in src/tablassert/errors.py; ordering comparisons cast the column to Float64, so a pattern there would be silently ignored.
  • Pattern gate: the comparator must compile in the polars engine and must not be able to match an empty cell (^, (), .*, x*); both raise regex-bad-pattern, the code NodeEncoding.exclude_regex and Regex.pattern already use. An every-cell pattern under ne would silently drop the whole table. A whitespace-only comparator is rejected for the same reason, while literal ne keeps its empty-comparator idiom (test_reindex_valid).

Testing

  • uv run pytest tests/test_models.py -k reindex -n 0 --no-cov -q -> 19 passed (operator mismatch, bad pattern, every-cell pattern parametrized over ^/()/.*/x*, empty and whitespace-only, non-str comparator under regex, default round-trip).
  • uv run pyright -> 0 errors.
  • uv run ruff check . and uv run ruff format --check . -> clean.

@SkyeAv
SkyeAv added this pull request to stack #235 October 21, 2026 21:09
@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: e5f735aa-8cd6-4b8d-9761-95954a894e80
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

source.reindex entries gain regex (default false). With regex true, an eq/ne
entry treats comparator as a polars-compatible pattern searched in the cell
(eq keeps the matches, ne keeps the rest), so one anchored pattern drops a
whole missing-value family instead of one literal filter per spelling.

Guards, all at config time: regex on lt/le/gt/ge is rejected
(comparison-regex-requires-eq-ne, new code in the TablassertErrorCodes
Literal); a pattern the polars engine cannot compile, or an empty or
whitespace-only pattern, is rejected (regex-bad-pattern, the code exclude_regex
and Regex.pattern already use) because an empty pattern matches every cell and
an ne filter would then silently empty the table. The str-comparator rule for
eq/ne is unchanged, and regex false keeps the literal comparison, so shipped
configs validate and dump exactly as before.
@SkyeAv
SkyeAv force-pushed the feat/reindex-regex-config branch from 835bc02 to 6f3fcfa Compare October 21, 2026 21:43
@SkyeAv
SkyeAv merged commit caf6d39 into main Oct 21, 2026
5 of 11 checks passed
SkyeAv added a commit that referenced this pull request Oct 22, 2026
Cut 22.2.0 and bump the package version in pyproject.toml,
CITATION.cff, and uv.lock.

Minor: the --qc study stage fails the build on self-referential
edges via a new self-loop-edges assertion (#229); eq/ne reindex
filters can match cells with a Polars regex behind a default-off
regex key (#232, #233, #234); source.reindex can filter rows by
identifier membership with join/antijoin semi/anti hash joins
(#236). Docs: the Taxon Filtering link in the table config
reference now matches its section (#228, #231).

Changelog:
- Versioned the Unreleased section as 22.2.0 - 2026-10-22.
- Added PR links to the three Added entries (#229; #232, #233,
  #234; #236).
- Added the missing Fixed entry for the Taxon Filtering docs link
  (#228, #231).

Docs: none needed here; the shipped docs changes are covered by
the changelog entries above.

Testing:
- make check -> ruff, ruff format, pyright, pytest, cargo test,
  cargo clippy -D warnings: all green.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant