perf(vindex): execute compatible queries through native batch search - #646
Conversation
f2403ec to
70fb052
Compare
5037330 to
7769355
Compare
7769355 to
414853f
Compare
JingsongLi
left a comment
JingsongLi
left a comment
There was a problem hiding this comment.
I found three issues in the new native-batch and range-read path.
| })?; | ||
| let source = VindexFileReader::new( | ||
| Arc::new(file_reader), | ||
| current_tokio_runtime_handle()?, |
There was a problem hiding this comment.
[P2] Preserve the non-Tokio batch fallback
This now unconditionally requires a Tokio handle, but before this change the vector_searches.len() > 1 branch read the file into a Cursor and therefore worked when polled by async-std, smol, or a plain futures executor. A public batch with two or more queries will now fail before searching with Vector index range reader requires a Tokio runtime.
Please retain the whole-file Cursor fallback when Handle::try_current() fails, or make the range bridge executor-agnostic, and add a non-Tokio batch regression test. The existing test only covers the pre-existing single-query limitation.
| source: Some(Box::new(e)), | ||
| })?, | ||
| None => reader | ||
| .search_batch(&queries, indices.len(), params) |
There was a problem hiding this comment.
[P2] Bound the native batch by its full working set
Every compatible group is flattened and passed to one native batch call. In paimon-vindex-core 0.3.0, this allocates a query_count * nlist f32 centroid-product matrix, plus query copies, per-query heaps, and query_count * top_k result arrays; L2 IVF-PQ also retains per-query m * ksub tables. Posting-list reads are byte-bounded, but this query-side scratch is not.
For example, 10,000 queries with nlist=4096 consume about 164 MB per active index-file job for the centroid matrix alone. With the default 32 concurrent shard jobs, those matrices alone can exceed 5 GB. The previous scalar loop bounded this scratch to one query at a time. Please chunk compatible groups using a working-set budget derived from at least nlist, dimension, top_k, and shard concurrency, then restore results to their original positions.
| source: None, | ||
| } | ||
| })?; | ||
| let source = VindexFileReader::new( |
There was a problem hiding this comment.
[P2] Apply global-index.thread-num to aggregate range I/O
Each admitted shard now creates an independent VindexFileReader, and each reader owns a private 32-permit range-read semaphore. The outer scheduler already admits up to global-index.thread-num shard jobs (default 32), so native multi-range reads can drive up to 32 * 32 = 1024 concurrent FileRead::read calls. This defeats the option's documented per-operation global-index I/O fan-out limit; in particular, setting it to 1 no longer reproduces strictly sequential I/O. The removed batch path issued one whole-file read per admitted shard.
Please share one operation-wide semaphore across all readers, or otherwise divide the range-read budget across admitted shard jobs.
2 participants
Purpose
This is PR 2/3 of a stacked vector-search performance series and builds on #645.
PR #645 introduced positional range reads while deliberately keeping multi-query
global-index vindex searches on the whole-file buffered path. This PR replaces
that fallback with the native batch-search APIs from
paimon-vindex-core, whileretaining positional range reads.
Brief change log
top_k,nprobe, and row-id filters.streaming instead of imposing a Rust-side query-count cap.
scalar range reads.
PK-vector ANN segments remain on their existing buffered path until #647. The
reader validation and optimization hooks introduced here intentionally prepare
the lifecycle consumed by #647.
Tests
cargo test -p paimon vindex --lib— 123 passedcargo test -p paimon vector_search_builder --lib— 77 passedcargo test -p paimon --lib— 2096 passed, 1 ignoredcargo fmt --checkgit diff --checkCoverage verifies mixed-group ordering, scalar equivalence, filtered native
batch execution, range reuse, avoidance of full-file reads, reuse across the
former 16-query boundary, and graceful failure outside a Tokio runtime.
API and Format
No public API or storage-format changes. Existing batch result ordering is
preserved.
Documentation
No user-facing documentation changes are required.