[core] Optimize BTree negative predicate complement - #8945
Conversation
| } | ||
|
|
||
| @Override | ||
| public boolean supportsRangeComplement() { |
There was a problem hiding this comment.
Can we just implement methods in this class?
There was a problem hiding this comment.
Do you mean overriding visitIsNotNull, visitNotEqual, and visitNotIn directly in LazyFilteredBTreeReader?
The challenge is that LazyFilteredBTreeReader only has the index files and local row IDs, but it does not know the complete row-id range of the shard. A correct complement requires this full range, which is currently only available in OffsetGlobalIndexReader as [offset, to].
Implementing the complement directly in LazyFilteredBTreeReader would either require reading all BTree files to reconstruct the full row-id set, which defeats this optimization, or passing the row range through GlobalIndexer#createReader.
That is why the current implementation computes the complement in OffsetGlobalIndexReader and lets LazyFilteredBTreeReader opt in through supportsRangeComplement().
| } | ||
|
|
||
| @Override | ||
| public boolean supportsRangeComplement() { |
There was a problem hiding this comment.
Bitmap also supports this?
There was a problem hiding this comment.
Yes, technically Bitmap can also use the range-complement path.
However, the benefit is generally smaller than for BTree. Bitmap already stores nonNullRows as a bitmap, making per-file negative evaluation cheap, while BTree must scan all non-null key entries and row IDs in each selected file.
2 participants
Purpose
close #8944
Tests