Localized Document Knowledge Base and Visual Q&A Skill (local-doc-kb)

1. Overview

The Localized Document Knowledge Base and Visual Q&A Skill (local-doc-kb) is a modular agent skill that ingests local document collections (.pdf, .md, .txt, and .docx), extracts hierarchical text sections alongside embedded diagrams and specification tables, builds a local hybrid index (combining 384-dimensional vector embeddings in SQLite/NumPy and a document-induced Knowledge Graph), and answers questions strictly from the uploaded documents. Every query also generates a standalone HTML verification interface (dashboard.html) titled Document Knowledge Explorer.

Users can point the skill at any local folder or document file - such as a household appliance manual, a water heater installation guide, a craft or furniture assembly booklet, or a technical maintenance handbook - or upload a single file directly through the browser interface. A set of sample documents is included in assets/sample_docs/ (field_transceiver_vx900_manual.pdf, infusion_pump_sop_v4.md, telemetry_gateway_spec.txt, and pneumatic_valve_maintenance.docx) to build and query a sample Knowledge Base immediately.

Core Capabilities

• Interactive Single-File Upload and Progress Telemetry: Supports initializing an empty dashboard (dashboard.html) where users can drag and drop or browse for a single file, watch real-time indexing progress ("I am working on it"), and submit questions through the Prompt Edit Box.
• Single-Pass Ingestion with Checksum Caching: Computes SHA-256 hashes for all input files and records them in assets/storage_manifest.json, skipping redundant parsing when source files have not changed.
• Domain-Agnostic Knowledge Graph Construction: Analyzes the uploaded document's headings, grammatical noun-phrase slots, prepositional relationships (PART_OF, CONNECTS_TO, USES), measurements (SPECIFIES), and step sequences (REQUIRES) to build a self-contained Knowledge Graph tailored to the uploaded topic without relying on hardcoded domain dictionaries.
• Universal Retrieval and Grounded LLM Synthesis: Combines morphological stem normalization, corpus-calibrated BM25 IDF scoring, same-sentence proximity, and Table of Contents page-pointer resolution with grounded LLM and discourse synthesis (scripts/llm_client.py) to generate readable step-by-step answers strictly from the document.
• Visual Verification Dashboard (dashboard.html): Produces a standalone HTML file showing the grounded answer, the rendered source page with a bounding-box highlight over the cited passage, an interactive zoomable diagrams carousel (+, -, Reset, drag-to-pan), and an interactive 2-hop Knowledge Graph viewer with zoom, drag-and-pan physics, node inspection, and light/dark theme switching.
• Local Execution: Runs on the local filesystem using SQLite, NumPy, and JSON files without requiring external database services.

---

2. Project Directory and File Structure

All paths in the project are organized relative to the project root directory:

.
|-- LICENSE
|-- README.md
|-- SKILL.md
|-- dashboard.html
|-- prd.md
|-- requirements.txt
|-- assets/
|   |-- dashboard_template.html
|   |-- skill_workflow_overview.png
|   |-- storage_manifest.json
|   `-- sample_docs/
|       |-- field_transceiver_vx900_manual.pdf
|       |-- infusion_pump_sop_v4.md
|       |-- pneumatic_valve_maintenance.docx
|       `-- telemetry_gateway_spec.txt
|-- documents/
|   `-- prd.md
|-- references/
|   |-- kg_schema.json
|   |-- query_rubric.md
|   `-- vector_index_config.json
|-- scripts/
|   |-- __init__.py
|   |-- build_knowledge_graph.py
|   |-- generate_sample_docs.py
|   |-- index_vectors.py
|   |-- ingest_documents.py
|   |-- llm_client.py
|   |-- query_engine.py
|   |-- render_dashboard.py
|   `-- serve_dashboard.py
|-- tests/
|   |-- __init__.py
|   |-- test_smoke.py
|   |-- test_system.py
|   `-- test_unit.py
`-- .kb_store/
    |-- last_query_result.json
    `-- field_transceiver_vx_900/
        |-- chunks.json
        |-- document_profile.json
        |-- knowledge_graph.json
        |-- vectors.db
        |-- vectors.npy
        `-- extracted_media/

---

3. Non-Technical User Guide

You do not need programming experience to use this skill with an AI assistant. Think of the skill as a private reader for your manuals: you give it one or more documents from your computer (or upload a single file directly inside the dashboard.html interface), it reads and maps out every part, measurement, and instruction inside those pages, and it answers your questions using only what is written in your document.

How It Works in Plain Language

1. Provide or Upload Your Document: Either point the assistant at a document folder or launch the empty UI Dashboard (dashboard.html) and upload a single file (such as a .pdf, .docx, .md, or .txt) directly in the browser. While the Knowledge Base is being built, the dashboard displays an active "I am working on it" progress bar.
2. Ask Everyday Questions via the Prompt Edit Box: Once your uploaded document finishes processing, the dashboard reveals the Prompt Edit Box where you can type any question about steps, measurements, parts, or safety rules.
3. Verify on the Visual Dashboard (dashboard.html): Open dashboard.html in any standard web browser by double-clicking the file or visiting http://127.0.0.1:8765. You will see:
   • Panel 1 (Top-Left): The direct, easy-to-read answer to your question and clickable citation chips showing the exact document name, section, and page number.
   • Panel 2 (Bottom-Left): A picture of the actual page from your manual with a highlighted box around the exact paragraph where the answer came from.
   • Panel 3 (Top-Right): Any diagrams, charts, or tables from that section of your manual, complete with + (Zoom In), - (Zoom Out), Reset, mouse-wheel zoom, and click-and-drag panning so you can inspect small labels clearly.
   • Panel 4 (Bottom-Right): An interactive map of how the parts, steps, and measurements connect. You can click any node to read its details, drag nodes around, use the + and - buttons to zoom, click Reset to center the map, or click the [Theme: Light/Dark] button in the top-right corner to switch between dark and light backgrounds.

Real-World Everyday Examples

Example A: Installing a Home Water Heater

Suppose you downloaded a 40-gallon residential water heater installation manual (water_heater_guide.pdf) and want to check the plumbing and safety steps before starting work.

• What you type to the assistant:

  Please build a Knowledge Base named "Home Water Heater Guide" from my file manuals/water_heater_guide.pdf. Then tell me: how do I connect the Dielectric Union to the Cold Water Dip Tube, and what must I check before turning on the upper heating element?
  

• What the skill does:
  1. Reads manuals/water_heater_guide.pdf and builds a self-contained Knowledge Graph of its plumbing parts (Dielectric Union, Cold Water Dip Tube, Magnesium Anode Rod), ratings (40 gallons, 150 psi), and safety rules.
  2. Synthesizes the step-by-step instructions directly from your manual (Page 1, Section 1.2) - for example, threading the Dielectric Union onto the Cold Water Dip Tube with PTFE tape and ensuring the tank is completely filled with water before energizing the upper heating element.
  3. Saves dashboard.html so you can see the highlighted paragraph and plumbing diagram on your screen.

Example B: Building a Scale Dollhouse Kit

Suppose you have an instruction booklet for a wooden dollhouse kit (dollhouse_assembly_manual.md) and want to know how to prepare the roof shingles and which glue to avoid on the windows.

• What you type to the assistant:

  Read my file guides/dollhouse_assembly_manual.md into a Knowledge Base called "Dollhouse Kit Manual". How long should I soak the Cedar Shingles before tiling, how do I attach the Dormer Gable to Roof Panel B, and what glue warning is listed for the window panes?
  

• What the skill does:
  1. Maps the kit's parts (Dormer Gable, Roof Panel B, Cedar Shingles, Porch Balustrade) and measurements (15 minutes, 30 minutes, 1:12 scale).
  2. Answers directly from the booklet: soak the Cedar Shingles in warm water for 15 minutes, glue and clamp the Dormer Gable to Roof Panel B using wood adhesive for 30 minutes, and do not apply cyanoacrylate glue to the clear acrylic window panes.
  3. If you ask a question that is not in the booklet (such as "Where can I buy extra miniature furniture online?"), the skill will never guess from external sources - it will reply: I cannot answer this question based on the provided document(s).

---

4. Technical Setup and Command-Line Execution

4.1 System Requirements

• Operating System: Linux, macOS, or Windows.
• Python Version: Python 3.10 or higher.
• Disk Storage: Local read/write access within the project directory for .kb_store/ and assets/storage_manifest.json.

4.2 Virtual Environment Installation

Run the following commands from the project root directory:

python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install --upgrade pip
python3 -m pip install -r requirements.txt

4.3 CLI Pipeline Commands

Interactive Option: Build an Empty Dashboard and Launch Single-File Upload Server

python3 scripts/render_dashboard.py --empty --output-html dashboard.html
python3 scripts/serve_dashboard.py --port 8765 --output-html dashboard.html

Step 1: Generate or Refresh Sample Documents (Optional)

python3 scripts/generate_sample_docs.py

Step 2: Ingest Documents into a Local Knowledge Base

python3 scripts/ingest_documents.py \
  --input assets/sample_docs/ \
  --kb-name "Field Transceiver VX-900" \
  --kb-id "field_transceiver_vx_900"

Step 3: Build the Document Knowledge Graph and Dense Vector Index

python3 scripts/build_knowledge_graph.py --kb-id "field_transceiver_vx_900"
python3 scripts/index_vectors.py --kb-id "field_transceiver_vx_900"

(Optional Agent Ontology Handshake): To export a document digest (corpus_digest.json) for inspection and merge an ontology JSON file (agent_ontology.json):

python3 scripts/build_knowledge_graph.py --kb-id "field_transceiver_vx_900" --prepare-digest
python3 scripts/build_knowledge_graph.py --kb-id "field_transceiver_vx_900" --agent-ontology ".kb_store/field_transceiver_vx_900/agent_ontology.json"

Step 4: Execute a Grounded Query and Render dashboard.html

python3 scripts/query_engine.py \
  --query "How do I store a repeater frequency into memory channel 5?" \
  --kb-id "field_transceiver_vx_900" \
  --output-json .kb_store/last_query_result.json

python3 scripts/render_dashboard.py \
  --query-result-json .kb_store/last_query_result.json \
  --output-html dashboard.html

Step 5: Verify Out-of-Domain Abstention

python3 scripts/query_engine.py \
  --query "What is the orbital period of Jupiter?" \
  --kb-id "field_transceiver_vx_900"

This returns "abstained": true, "grounding_status": "ABSTAINED", and "answer": "I cannot answer this question based on the provided document(s).".

---

5. Automated Test Suite

Run the unit, smoke, and end-to-end system tests from the project root directory:

pytest -v

• tests/test_smoke.py: Verifies project directory structure, CLI --help exit codes, relative paths, and character encoding standards.
• tests/test_unit.py: Verifies SHA-256 checksums, hierarchical block splitting, 384-D vector normalization, scanned PDF fallback, automotive manual induction, and dollhouse/water heater manual Knowledge Graph induction and dashboard rendering.
• tests/test_system.py: Verifies end-to-end ingestion, caching, grounded retrieval, Table of Contents pointer resolution, cross-domain zero-synonym retrieval, abstention enforcement, interactive single-file upload, and multi-KB disambiguation.

---

6. License

This project is licensed under the Apache License, Version 2.0. See the LICENSE file in the project root directory for the full license text and terms.