sift

A code RAG (retrieval-augmented generation) CLI. Point it at a codebase, it chunks the code semantically with tree-sitter, embeds and stores it in Postgres/pgvector, and answers questions about it grounded in the actual retrieved source — every claim cites a file:line range, not a guess.

Contents

• How it works
• Setup
• Usage
• Configuration
• Supported languages
• Project structure
• Development
• Known limitations

How it works

Indexing (sift index <path>):

walk_repo → chunk_file → embed (NVIDIA NIM) → Postgres (pgvector)

• Every file (respecting .gitignore and common vendor/build directories) is parsed with tree-sitter into function/method/class-level chunks — semantically meaningful units, not arbitrary line windows. Files in unsupported languages still get indexed via a generic line-window fallback; only binaries are skipped.
• Each chunk also carries its enclosing class's header (decorators, annotations, base classes) as context, so a method chunk pulled in isolation still shows e.g. its Spring @RequestMapping or Python base class — without that, an LLM has no way to reconstruct a full route path from a bare method.
• Chunks are embedded (NVIDIA NIM nvidia/llama-nemotron-embed-1b-v2, 2048-dim) and stored in Postgres with an HNSW index for fast cosine search.
• Re-running sift index is incremental by default: it diffs the working tree against the last-indexed git commit and only re-processes what changed. --force does a full re-index.

Querying (sift ask / the REPL):

hybrid_search (vector + keyword, RRF-merged) → dynamic cutoff → streamed answer

• Hybrid retrieval: a vector search (pgvector cosine similarity) and a keyword search (Postgres full-text + trigram fuzzy symbol matching) run in parallel, then get merged via Reciprocal Rank Fusion — a chunk that ranks well on either signal surfaces, not just chunks that happen to be semantically close.
• Dynamic result count: instead of a fixed "top 10," the number of sources is decided in two stages — how many chunks are relevant enough (score-based, no fixed ceiling), then how many of those fit a context token budget. A narrow, specific query naturally pulls in fewer sources than a broad one; a handful of huge functions and a page of one-liners both end up sized correctly.
• Grounded generation: retrieved chunks + the question go to an NVIDIA NIM chat model (swappable via /model), instructed to answer only from what it was given and cite file:line for every claim, or say "not found in indexed code."

Setup

Requires Docker (for Postgres/pgvector) and an NVIDIA NIM API key.

git clone https://github.com/barathsuresh/sift.git
cd sift
docker compose up -d          # starts Postgres with pgvector on :5432
pip install -e .

Run the migrations against the running Postgres instance (migrations/*.sql, applied in order) — or point DATABASE_URL at any Postgres 16+ instance with the vector and pg_trgm extensions available.

Log in with your NVIDIA API key:

sift login              # prompts, input hidden
# or
sift login --api-key nvapi-...

The key is saved to ~/.sift_config.json (mode 600) so you don't need to re-export it every session. NVIDIA_API_KEY in the environment or a .env file always takes precedence over the saved key.

Usage

Index a repo, then ask questions — either as one-shot commands or in the REPL.

sift index /path/to/repo --name myrepo
sift ask "how does the retry logic work?" --repo myrepo

REPL

Running sift with no arguments drops into a chat-style REPL:

Command	Does
/index <path> [name]	Ingest a repo (incremental by default; --force for a full re-index). Sets it as the active repo.
/repo <name>	Switch to (or show) the active repo. Persisted — the next session resumes here.
/model [id]	Show or switch the chat model. Tab-completes live from the NIM model catalog.
/k [n|auto]	Pin the number of retrieved sources, or return to the dynamic score-based cutoff.
/show <n>	Pretty-print the nth source from the last answer, syntax-highlighted.
/login, /logout, /whoami	Manage the saved NVIDIA API key.
/db-url [url|clear]	Show, set, or clear the saved DATABASE_URL.
/clear	Clear conversation memory (follow-up context) for the active repo.
/help	Show this list.
/exit, /quit, q	Leave.

Anything else is treated as a question against the active repo. Answers stream in as they're generated, with full Markdown/syntax-highlighting once each paragraph or code block completes.

CLI commands

sift login|logout|whoami, sift db-url [url] [--clear], sift index <path> [--name] [--url] [--force], sift ask <query> --repo <name> [-k n]. Run sift --help or sift <command> --help for full flag details.

Configuration

All persistent settings live in ~/.sift_config.json (mode 600): API key, chat model, DATABASE_URL, and the active repo. Every setting resolves with the same precedence: environment variable / .env > saved config > hardcoded default. This means a .env in the working directory always wins, so you can override any saved setting per-project without touching the global config.

cp .env.example .env   # NVIDIA_API_KEY, DATABASE_URL, NVIDIA_CHAT_MODEL

Supported languages

Semantic (function/class-level) chunking via tree-sitter:

Python · JavaScript · TypeScript · Go · Java · Ruby · C · C++ · C# · Rust · PHP · Kotlin · Scala

Any other text file (Markdown, YAML, JSON, plain config, ...) is still indexed via a generic line-window fallback — only genuine binary files are skipped.

Project structure

sift/
├── cli.py         Typer app + REPL (prompt_toolkit + rich)
├── ingest.py       repo walking, gitignore/vendor filtering, incremental git diff
├── chunker.py      tree-sitter based semantic code chunking
├── db.py           Postgres storage/retrieval, config persistence, advisory locks
├── embedder.py      NVIDIA NIM embedding generation
├── retriever.py     hybrid (vector + keyword) search, RRF, dynamic cutoff
├── generator.py     NVIDIA NIM chat generation, streaming, retry/backoff
└── ratelimit.py      shared rate limiter across embedding + chat calls

migrations/           schema (Postgres + pgvector)
tests/                pytest suite — chunker, retriever, ingest, db, generator,
                       embedder, ratelimit
scripts/               standalone debugging helpers (dump_chunks.py, ...)

Development

pip install -e ".[dev]"
pytest tests/

The test suite runs against a real local Postgres (the docker compose instance) for db.py/ingest.py integration tests — no mocking of SQL or schema behavior — and mocks only the NVIDIA API client for generator.py/embedder.py, so retry/backoff logic is exercised for real, just without a network call. All tests are fast (~1s total) and clean up any rows they create.

Known limitations

• Requires Postgres + pgvector and a NIM API key — not a zero-config "point at any repo" tool without that infra running.
• No connection pooling; each operation opens a fresh Postgres connection. Fine for CLI usage, adds latency under very heavy REPL use.
• Non-git codebases always do a full re-index (no commit to diff against), correct but not incremental.
• Retrieval quality depends on the NIM embedding/chat models available on your account; results vary by model choice (/model).