fix(indexing): release write lock during reindex Finalizing phase

[ManSio]

Root cause

IndexProjectRunner.run() held db_manager.begin_write() (the global _write_lock RLock) for the ENTIRE reindex job, including the heavy LanceDB optimize() / create_index() calls in _safe_ivf_index(). Running those operations under the global lock deadlocked the 'Finalizing' phase (both processes at 0% CPU, job never completed, ETA grew forever). The reindexing flag was therefore never cleared, permanently blocking concurrent search.

Fix

• _safe_ivf_index() now releases the global write lock via _suspend_write_lock() around optimize()/create_index(). The reindex guard is_reindexing() (independent of this lock) still blocks concurrent search, so it is safe.
• create_index is bounded by a timeout + non-blocking executor shutdown, so a stalled LanceDB index build can no longer hang the reindex job forever.

Regression guard

ests/test_reindex_finalizing_deadlock.py:

• asserts optimize/create_index run with the write lock RELEASED (reproduces the deadlock root cause if it regresses);
• asserts a hung create_index does not block the Finalizing phase past the timeout.

Test result

python -m pytest tests/test_reindex_finalizing_deadlock.py -q → 2 passed. Existing ests/test_reindex_responsive.py (9) and ests/test_index_runner_deadlock.py (4) still pass.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 6410274c-7bb4-4c52-bd66-cab990ec8691

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}