Repository navigation
feat: add a polars regex comparator option to eq/ne reindex filters - #232
Merged
Merged
Conversation
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configuration
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This was referenced Oct 21, 2026
source.reindex entries gain regex (default false). With regex true, an eq/ne entry treats comparator as a polars-compatible pattern searched in the cell (eq keeps the matches, ne keeps the rest), so one anchored pattern drops a whole missing-value family instead of one literal filter per spelling. Guards, all at config time: regex on lt/le/gt/ge is rejected (comparison-regex-requires-eq-ne, new code in the TablassertErrorCodes Literal); a pattern the polars engine cannot compile, or an empty or whitespace-only pattern, is rejected (regex-bad-pattern, the code exclude_regex and Regex.pattern already use) because an empty pattern matches every cell and an ne filter would then silently empty the table. The str-comparator rule for eq/ne is unchanged, and regex false keeps the literal comparison, so shipped configs validate and dump exactly as before.
SkyeAv
force-pushed
the
feat/reindex-regex-config
branch
from
October 21, 2026 21:43
835bc02 to
6f3fcfa
Compare
SkyeAv
added a commit
that referenced
this pull request
Oct 22, 2026
Cut 22.2.0 and bump the package version in pyproject.toml, CITATION.cff, and uv.lock. Minor: the --qc study stage fails the build on self-referential edges via a new self-loop-edges assertion (#229); eq/ne reindex filters can match cells with a Polars regex behind a default-off regex key (#232, #233, #234); source.reindex can filter rows by identifier membership with join/antijoin semi/anti hash joins (#236). Docs: the Taxon Filtering link in the table config reference now matches its section (#228, #231). Changelog: - Versioned the Unreleased section as 22.2.0 - 2026-10-22. - Added PR links to the three Added entries (#229; #232, #233, #234; #236). - Added the missing Fixed entry for the Taxon Filtering docs link (#228, #231). Docs: none needed here; the shipped docs changes are covered by the changelog entries above. Testing: - make check -> ruff, ruff format, pyright, pytest, cargo test, cargo clippy -D warnings: all green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: add a polars regex comparator option to eq/ne reindex filters
Adds an opt-in
regexflag tosource.reindexentries so aneq/nefilter can match itscomparatoras a polars-compatible pattern instead of a literal value. One anchored pattern then drops a whole missing-value family (N/A,n/a,--,not reported) instead of one literal filter per spelling.Config surface
Reindex.regexinsrc/tablassert/models.py, defaulting tofalse, so a config that never names it validates, dumps, and filters exactly as before.regex: truewithlt/le/gt/geis rejected at config time with the newcomparison-regex-requires-eq-necode insrc/tablassert/errors.py; ordering comparisons cast the column toFloat64, so a pattern there would be silently ignored.comparatormust compile in the polars engine and must not be able to match an empty cell (^,(),.*,x*); both raiseregex-bad-pattern, the codeNodeEncoding.exclude_regexandRegex.patternalready use. An every-cell pattern undernewould silently drop the whole table. A whitespace-only comparator is rejected for the same reason, while literalnekeeps its empty-comparator idiom (test_reindex_valid).Testing
uv run pytest tests/test_models.py -k reindex -n 0 --no-cov -q->19 passed(operator mismatch, bad pattern, every-cell pattern parametrized over^/()/.*/x*, empty and whitespace-only, non-str comparator under regex, default round-trip).uv run pyright->0 errors.uv run ruff check .anduv run ruff format --check .-> clean.