[gkennos]

Summary

Adds read-only serving of prebuilt DuckDB vocabulary/graph files and separate DuckDBVSS embedding stores through groundworkers[duckdb].

Configuration and setup support the two-file topology, reject missing or colocated files, and prevent maintenance writes against served assets. Vocabulary lookup, normalized search, ancestry, relationships and embedding retrieval remain available; PostgreSQL full-text search is reported as unavailable on DuckDB.

Graph preparation now uses omop-graph’s packaged classification defaults. Removes Groundworkers’ obsolete CSV copies while preserving explicit custom-directory overrides.

Requires oa-configurator ≥2.1, omop-alchemy ≥3, omop-graph ≥4 and omop-emb ≥3.1. Includes relationship-traversal hardening and integrates backend guidance into the existing documentation.

Production asset building, manifests and deployment validation remain separate follow-on work.

Closes #16

Checklist

• Applied exactly one label (breaking, feature, fix, dependencies, or chore)
• Tests pass locally (uv run pytest -q)
• Lint passes (uv run ruff check .)

[Copilot]

🟡 Changes recommended

Outstanding defects can over-fetch graph data, discard existing DuckDB schema defaults, accept unusable embedding-store paths, and incorrectly display supported DuckDB stores as unsupported.

4 open issues

• Recognize supported DuckDB stores in the embedding presenter · New
• Preserve existing DuckDB schema defaults when updating configuration · New
• Require the separate DuckDB connection to reference an existing file · New
• Fetch only a truncation probe after a seed exhausts its edge budget · New

What changed in this PR

Adds read-only DuckDB support for prebuilt vocabulary/graph files and separate embedding stores. Graph preparation now uses omop-graph’s packaged classification defaults, addressing #16.

Changes:

• Adds DuckDB dependencies, configuration, read-only connection policies, and maintenance guards.
• Supports DuckDB normalized search and strengthens bounded relationship traversal.
• Adds runtime and packaging tests, setup coverage, and deployment documentation.

File	Description
uv.lock	Locks upgraded OMOP packages and DuckDB dependencies.
tests/​unit/​test_vector_store_setup.py	Tests detection of colocated DuckDB file aliases.
tests/​unit/​test_relationship_edges.py	Tests filtered, ordered, bounded relationship queries.
tests/​unit/​test_predicate_mapping_classification.py	Checks classification data from omop-graph.
tests/​unit/​test_omop_emb_adapter.py	Uses approximate floating-point assertions.
tests/​unit/​test_normalized_index.py	Makes test schema bindings explicit.
tests/​unit/​test_migration_boundaries.py	Updates dependency and extra assertions.
tests/​unit/​test_mapping_service.py	Tests traversal budgets, metadata errors, and cache reuse.
tests/​unit/​test_mapping_context.py	Updates delegated argument expectations.
tests/​unit/​test_graph_maintenance.py	Tests upstream defaults and read-only guards.
tests/​unit/​test_graph_maintenance_wizard.py	Tests upstream classification choices and incomplete configuration.
tests/​unit/​test_configuration_provider.py	Adds DuckDB configuration round-trip coverage.
tests/​test_bootstrap_config.py	Updates mocks for runtime engine creation.
tests/​support/​wheel_smoke.py	Adds installed-wheel DuckDB checks.
tests/​integration/​test_omop_duckdb_runtime.py	Tests file-backed serving, validation, and normalization.
src/​groundworkers/​tui/​wizards/​graph_maintenance.py	Offers upstream defaults and disables DuckDB preparation.
src/​groundworkers/​services/​vocab.py	Adds DuckDB search handling and full-text availability reporting.
src/​groundworkers/​services/​relationship_neighborhood.py	Introduces per-seed bounded traversal.
src/​groundworkers/​services/​mapping.py	Integrates traversal budgets and strict metadata checks.
src/​groundworkers/​services/​graph.py	Exposes bounded relationship queries and strict lookups.
src/​groundworkers/​plugins.py	Explicitly forwards shared-ancestor options.
src/​groundworkers/​config/​README	Removes obsolete bundled-data explanation.
src/​groundworkers/​config/​predicate_classification.csv	Removes the local classification copy.
src/​groundworkers/​bootstrap.py	Validates embedding files and uses runtime connection policies.
src/​groundworkers/​base/​sql.py	Adds DuckDB-compatible normalization expressions.
src/​groundworkers/​base/​server.py	Simplifies tool registration and signature assignment.
src/​groundworkers/​base/​database.py	Adds DuckDB file validation and read-only engine creation.
src/​groundworkers/​application/​setup/​performance_maintenance.py	Rejects non-PostgreSQL index maintenance earlier.
src/​groundworkers/​application/​setup/​graph_maintenance.py	Uses upstream defaults and guards read-only preparation.
src/​groundworkers/​application/​setup/​embedding_setup.py	Refuses DuckDB store initialization.
src/​groundworkers/​application/​setup/​embedding_population.py	Validates coverage inputs and blocks DuckDB population.
src/​groundworkers/​application/​setup/​databases.py	Opens DuckDB readiness connections read-only.
src/​groundworkers/​application/​setup/​configuration_provider.py	Adds DuckDB setup choices and topology validation.
src/​groundworkers/​adapters/​omop_graph.py	Adds bounded active outbound relationship queries.
pyproject.toml	Adds the DuckDB extra and raises dependency requirements.
mkdocs.yaml	Adds ancestry documentation to navigation.
docs/​usage/​installation.md	Documents DuckDB installation and offline prerequisites.
docs/​usage/​configuration.md	Explains file topology and backend limitations.
docs/​tools/​mapping.md	Links to the runtime ancestry contract.
docs/​from-scratch.md	Documents prepared-file setup and separate maintenance.
docs/​development/​ancestry-mcp-design.md	Removes the superseded ancestry design document.
docs/​adapters/​omop_emb.md	Documents DuckDB storage and startup validation.
docs/​adapters/​ancestry.md	Documents ancestry semantics and traversal limits.
config/​groundworkers.duckdb.example.toml	Provides a two-file DuckDB configuration example.
.github/​workflows/​ci.yml	Installs VSS and adds DuckDB wheel smoke checks.

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Hard-linked embedding paths bypass file-separation checks, allowing initialization to open a writer against the CDM itself, and existing DuckDB stores are incorrectly displayed as unconfigured.

4 open issues

• Prevent embedding initialization from opening a hard-linked CDM · New
• Populate DuckDB path and existence in setup configuration · New
• Require the separate DuckDB connection to reference an existing file
• Recognize supported DuckDB stores in the embedding presenter

1 previously missed issue

In code that hasn't changed since last review

Reject hard links to the CDM in setup validation

src/​groundworkers/​application/​setup/​configuration_provider.py:1308

This setup check also accepts a hard link to the CDM: the resolved filenames differ even though Path.samefile() returns true. The wizard can therefore apply a configuration that violates the separate-file requirement. Check physical file identity for existing paths, while keeping path equality for new targets.

2 resolved since last review

• Fetch only a truncation probe after a seed exhausts its edge budget
• Preserve existing DuckDB schema defaults when updating configuration

_{Review effort: Balanced}