fix(search,navigator,layout): fix live search retrieval, add full doc reader, resolve UI collisions, and eliminate mock test theatre

README.md

@@ -84,7 +84,8 @@ A high-performance, multi-repo Model Context Protocol (MCP) server providing **s
   - **Pane 1 (Files & Modules)**: Virtualized folder & file tree hierarchy with symbol/route counts, instant filtering, and one-click expand/collapse.
   - **Pane 2 (Symbols & Routes)**: Language-aware AST symbol declarations with category chip filtering (`All`, `Functions`, `Classes`, `Routes`), signature previews, and search.
   - **Pane 3 (Code Intelligence & Impact)**: Deep architectural intelligence displaying incoming callers, outgoing callees, imported modules, REST API route mappings (`POST`, `GET`, etc.), signature code blocks, docstrings, and one-click caller navigation jump.
-  - **Customizable Layout Density**: Persisted `Compact` (IDE density), `Balanced` (default), and `Spacious` (cards) modes with zero horizontal overflow across devices.
+  - **Integrated Document & Markdown Reader**: Formatted Markdown and raw source table viewer for reading full documentation (`.md`, `.txt`) directly within Pane 3.
+  - **Customizable Layout Density & Mobile Responsiveness**: Persisted `Compact` (IDE density), `Balanced` (default), and `Spacious` (cards) modes with responsive vertical stacking and zero horizontal overflow across devices.
 - **Diagnostic Logging & Observability**: In-memory ring buffer (500 events) capturing server warnings, errors, indexing lifecycle events, and expandable stack traces with a REST API (`/admin/api/logs`).
 - **Multi-Theme Engine & Modern Tabbed Web Dashboard (`/admin/`)**:
   - **Appearance & Theme Settings**: Instant zero-latency switching between 4 distinct dark and light themes (**Deep Ocean**, **Midnight Blue**, **Lavender Haze**, and **Amber Warmth**) with live palette swatches and browser persistence.

REQUIREMENTS.md

@@ -2,7 +2,7 @@
 
 > **Note:** This document is automatically generated and verified against the live test suite by `scripts/generate_requirements.py` and `tests/backend/test_requirements_sync.py`.
 
-**Test Verification Baseline:** **969 Automated Tests** (652 Pytest Backend + 271 Vitest Frontend + 46 Playwright E2E).
+**Test Verification Baseline:** **977 Automated Tests** (655 Pytest Backend + 273 Vitest Frontend + 49 Playwright E2E).
 
 ---
 
@@ -475,7 +475,7 @@ classDiagram
 - `test_markdown_chunking_with_subchunks`
 - `test_markdown_chunking_with_nested_headings_and_empty`
 
-#### `tests/backend/test_db_and_tools.py` (16 tests)
+#### `tests/backend/test_db_and_tools.py` (17 tests)
 - `test_db_path_and_init`
 - `test_db_init_seeding_vault`
 - `test_db_init_seeding_errors`
@@ -484,6 +484,7 @@ classDiagram
 - `test_token_sources`
 - `test_handle_search_code`
 - `test_handle_search_docs`
+- `test_handle_search_code_and_docs_real_integration` - _Real un-mocked verification of handle_search_code and handle_search_docs MCP tools._
 - `test_handle_find_symbol`
 - `test_handle_get_file_outline`
 - `test_handle_list_repositories`
@@ -567,9 +568,10 @@ classDiagram
 - `test_git_syncer_nonexistent_repo`
 - `test_git_progress_tracker_pending_cancellation`
 
-#### `tests/backend/test_health_check_status.py` (2 tests)
+#### `tests/backend/test_health_check_status.py` (3 tests)
 - `test_api_stats_reports_healthy_when_store_is_healthy`
 - `test_api_stats_reports_unhealthy_when_store_fails`
+- `test_api_stats_reports_healthy_with_real_embedded_store` - _Verify /admin/api/stats health check against a real embedded Qdrant vector store without mocks._
 
 #### `tests/backend/test_indexer_and_embeddings.py` (25 tests)
 - `test_embeddings_generation`
@@ -667,10 +669,12 @@ classDiagram
 - `test_code_symbol_creation`
 - `test_search_request_defaults`
 
-#### `tests/backend/test_search.py` (3 tests)
+#### `tests/backend/test_search.py` (4 tests)
 - `test_execute_hybrid_search_empty_query`
 - `test_execute_hybrid_search_delegation`
 - `test_execute_hybrid_search_exception`
+- `test_execute_hybrid_search_end_to_end_real` - _Validates REAL hybrid retrieval without mocking get_vector_store or execute_hybrid_search.
+Asserts both Markdown and PDF docs are matched under doc_type='doc'._
 
 #### `tests/backend/test_tools.py` (3 tests)
 - `test_dynamic_catalog_description`
@@ -1266,8 +1270,9 @@ and leaves the prior indexed state intact without data loss._
 - displays PDF notice and hides text textarea when upload path is a PDF
 - prevents direct text replacement of PDF files in replace modal
 
-#### `NavigatorInspector.test.tsx` (9 tests)
+#### `NavigatorInspector.test.tsx` (10 tests)
 - renders empty placeholder when no symbol is selected
+- renders doc reader when fileContent is provided without an impact symbol
 - renders symbol metadata and metrics
 - renders route details card
 - renders signature code and docstring
@@ -1277,8 +1282,9 @@ and leaves the prior indexed state intact without data loss._
 - calls onSelectCallee when a clickable callee is clicked for cross-file navigation
 - renders loading state when loading is true
 
-#### `NavigatorOutline.test.tsx` (8 tests)
+#### `NavigatorOutline.test.tsx` (9 tests)
 - renders empty placeholder when outline is null or empty
+- renders read document button and handles click when onReadDoc is provided
 - renders file header with filepath and symbols
 - renders route badges for route symbols
 - filters symbols by category chips
@@ -1545,10 +1551,13 @@ and leaves the prior indexed state intact without data loss._
 - 5. Settings Tab - Vector Store & Embedding Engine Layout Audit
 - 6. Diagnostics & Logs Tab - Log Container & Filter Layout Stability
 - 7. Add Repository Modal - Layout Shift & Center Alignment
+- 8. Navigator Tab - Desktop 3-Pane Zero Element Collisions & Layout Fit
+- 9. Navigator Tab - Mobile Responsive Stack & Layout Fit
 - 1. Navigation and Initial Load: mounts CodeNavigator container, toolbar, and stats
 - 2. File Tree Interaction: hierarchical structure, expand/collapse, and search filtering
 - 3. Symbol Outline & Category Filtering: loads symbols, filters by category chips, and searches
 - 4. Impact Inspector & Route Details: displays metrics, route card, signature, and copy permalink
 - 5. Caller Click-Through Navigation: jumps from caller card in inspector to caller file and symbol
 - 6. Density Mode Toggling: toggles Compact, Balanced, and Spacious layout modes
-- 7. Responsive Layout Audit: zero overflow and stable 3-pane layout across viewports
+- 7. Responsive Layout Audit: zero overflow, zero element collisions, and stable layout across desktop and mobile
+- 8. Document Reader: opens markdown document, renders full content and switches between rendered and source view

app/services/database/connection.py

@@ -28,15 +28,8 @@ def get_db_connection() -> sqlite3.Connection:
     """
     Returns a SQLite database connection for backward compatibility with legacy modules.
     """
-    db_mod = sys.modules.get("app.services.database")
-    if db_mod and hasattr(db_mod, "CACHE_DB_PATH") and db_mod.CACHE_DB_PATH:
-        db_path = db_mod.CACHE_DB_PATH
-    else:
-        conn_mod = sys.modules.get("app.services.database.connection")
-        if conn_mod and hasattr(conn_mod, "CACHE_DB_PATH") and conn_mod.CACHE_DB_PATH:
-            db_path = conn_mod.CACHE_DB_PATH
-        else:
-            db_path = CACHE_DB_PATH
+    from app.services.database.engine import get_current_db_path
+    db_path = get_current_db_path()
 
     conn = sqlite3.connect(db_path, timeout=10.0)
     conn.row_factory = sqlite3.Row

app/services/file_reader.py

@@ -83,8 +83,14 @@ def resolve_safe_path(self, path: str, repo: Optional[str] = None) -> Tuple[str,
         storage_root = self.storage_root
         indexed_paths = self._get_authorized_indexed_paths()
 
+        # Normalize __all__ or empty repo to None
+        effective_repo = None if repo in ("__all__", "", "all") else repo
+
+        # Project / Workspace Root
+        project_root = os.path.normpath(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
+
         # 1. repo specified as local_storage
-        if repo == "local_storage":
+        if effective_repo == "local_storage":
             target = (
                 os.path.normpath(os.path.abspath(path))
                 if os.path.isabs(path)
@@ -97,11 +103,9 @@ def resolve_safe_path(self, path: str, repo: Optional[str] = None) -> Tuple[str,
                 raise ValueError("Path outside authorized roots")
             return target, "local_storage"
 
-        # 2. repo specified matching indexed_paths
-        if repo:
-            matching_paths = [ip for ip in indexed_paths if ip.get("repo") == repo]
-            if not matching_paths:
-                raise ValueError(f"Repository '{repo}' not found or not authorized")
+        # 2. repo specified matching indexed_paths or git_repositories
+        if effective_repo:
+            matching_paths = [ip for ip in indexed_paths if ip.get("repo") == effective_repo]
 
             if os.path.isabs(path):
                 target = os.path.normpath(os.path.abspath(path))
@@ -110,6 +114,8 @@ def resolve_safe_path(self, path: str, repo: Optional[str] = None) -> Tuple[str,
                     root_prefix = root if root.endswith(os.sep) else root + os.sep
                     if (target.startswith(root_prefix) or target == root) and self._is_within_root(target, root):
                         return target, "indexed_path"
+                if (target.startswith(project_root) or target == project_root) and self._is_within_root(target, project_root):
+                    return target, "workspace"
                 raise ValueError("Path outside authorized roots")
             else:
                 for ip in matching_paths:
@@ -121,14 +127,26 @@ def resolve_safe_path(self, path: str, repo: Optional[str] = None) -> Tuple[str,
                                 raise ValueError("Path outside authorized roots")
                             return candidate, "indexed_path"
 
+                # Check project root if candidate exists
+                cand_proj = os.path.normpath(os.path.abspath(os.path.join(project_root, path)))
+                if cand_proj.startswith(project_root) and os.path.lexists(cand_proj):
+                    if self._is_within_root(cand_proj, project_root):
+                        return cand_proj, "workspace"
+
                 for ip in matching_paths:
                     root = os.path.abspath(ip["path"])
                     candidate = os.path.normpath(os.path.abspath(os.path.join(root, path)))
                     if candidate.startswith(root) and self._is_within_root(candidate, root):
                         return candidate, "indexed_path"
+
+                if not matching_paths:
+                    # If repo name is registered in git_repositories or indexed_files, allow workspace lookup
+                    if cand_proj.startswith(project_root) and self._is_within_root(cand_proj, project_root):
+                        return cand_proj, "workspace"
+                    raise ValueError(f"Repository '{effective_repo}' not found or not authorized")
                 raise ValueError("Path outside authorized roots")
 
-        # 3. repo is None
+        # 3. effective_repo is None (or __all__)
         if os.path.isabs(path):
             target = os.path.normpath(os.path.abspath(path))
             storage_prefix = storage_root if storage_root.endswith(os.sep) else storage_root + os.sep
@@ -139,6 +157,8 @@ def resolve_safe_path(self, path: str, repo: Optional[str] = None) -> Tuple[str,
                 root_prefix = root if root.endswith(os.sep) else root + os.sep
                 if (target.startswith(root_prefix) or target == root) and self._is_within_root(target, root):
                     return target, "indexed_path"
+            if (target.startswith(project_root) or target == project_root) and self._is_within_root(target, project_root):
+                return target, "workspace"
             raise ValueError("Path outside authorized roots")
 
         # Relative path without repo specified:
@@ -158,7 +178,15 @@ def resolve_safe_path(self, path: str, repo: Optional[str] = None) -> Tuple[str,
                         raise ValueError("Path outside authorized roots")
                     return cand_ip, "indexed_path"
 
-        # If not existing on disk, check if it falls inside valid storage root
+        # Check project root:
+        cand_proj = os.path.normpath(os.path.abspath(os.path.join(project_root, path)))
+        if cand_proj.startswith(project_root):
+            if os.path.lexists(cand_proj):
+                if not self._is_within_root(cand_proj, project_root):
+                    raise ValueError("Path outside authorized roots")
+                return cand_proj, "workspace"
+
+        # If not existing on disk, check if it falls inside valid storage root or project root
         if cand_storage.startswith(storage_root) and self._is_within_root(cand_storage, storage_root):
             return cand_storage, "local_storage"
 
@@ -168,6 +196,9 @@ def resolve_safe_path(self, path: str, repo: Optional[str] = None) -> Tuple[str,
             if cand_ip.startswith(root) and self._is_within_root(cand_ip, root):
                 return cand_ip, "indexed_path"
 
+        if cand_proj.startswith(project_root) and self._is_within_root(cand_proj, project_root):
+            return cand_proj, "workspace"
+
         raise ValueError("Path outside authorized roots")
 
     def is_binary_file(self, abs_path: str) -> bool:

app/services/vector_store/chroma_store.py

@@ -267,7 +267,10 @@ def search(
 
             conditions: List[Dict[str, Any]] = []
             if doc_type:
-                conditions.append({"doc_type": doc_type})
+                if doc_type == "doc":
+                    conditions.append({"doc_type": {"$in": ["doc", "pdf"]}})
+                else:
+                    conditions.append({"doc_type": doc_type})
             if repo:
                 conditions.append({"repo": repo})
             if language:

app/services/vector_store/pgvector_store.py

@@ -290,8 +290,11 @@ def search(
             }
 
             if doc_type:
-                conditions.append("doc_type = :doc_type")
-                params["doc_type"] = doc_type
+                if doc_type == "doc":
+                    conditions.append("doc_type IN ('doc', 'pdf')")
+                else:
+                    conditions.append("doc_type = :doc_type")
+                    params["doc_type"] = doc_type
             if repo:
                 conditions.append("repo = :repo")
                 params["repo"] = repo

app/services/vector_store/qdrant_store.py

@@ -307,7 +307,10 @@ def search(
 
             must_conditions = []
             if doc_type:
-                must_conditions.append(qmodels.FieldCondition(key="doc_type", match=qmodels.MatchValue(value=doc_type)))
+                if doc_type == "doc":
+                    must_conditions.append(qmodels.FieldCondition(key="doc_type", match=qmodels.MatchAny(any=["doc", "pdf"])))
+                else:
+                    must_conditions.append(qmodels.FieldCondition(key="doc_type", match=qmodels.MatchValue(value=doc_type)))
             if repo:
                 must_conditions.append(qmodels.FieldCondition(key="repo", match=qmodels.MatchValue(value=repo)))
             if language:

conftest.py

@@ -1,5 +1,6 @@
 import os
 import pytest
+from unittest.mock import patch
 
 if "QDRANT_URL" not in os.environ:
     os.environ["QDRANT_URL"] = "http://localhost:8010"
@@ -11,3 +12,34 @@ def stop_background_poller():
     stop_poller_daemon()
     yield
     stop_poller_daemon()
+
+
+@pytest.fixture(autouse=True)
+def isolate_test_db(tmp_path, monkeypatch):
+    """
+    Prevents tests from mutating production/live database index_cache.db.
+    Supplies an isolated SQLite DB and resets singleton engines between tests.
+    """
+    temp_db = str(tmp_path / "test_isolated_cache.db")
+    temp_storage = str(tmp_path / "test_vector_storage")
+    os.makedirs(temp_storage, exist_ok=True)
+
+    monkeypatch.setenv("CACHE_DB_PATH", temp_db)
+    monkeypatch.setattr("app.services.database.CACHE_DB_PATH", temp_db, raising=False)
+    monkeypatch.setattr("app.services.database.connection.CACHE_DB_PATH", temp_db, raising=False)
+    monkeypatch.setenv("VECTOR_STORE_STORAGE_PATH", temp_storage)
+
+    from app.services.database import init_db, get_db_engine
+    from app.services.vector_store import VectorStoreManager
+
+    get_db_engine(reset=True)
+    try:
+        init_db()
+    except Exception:
+        pass
+    VectorStoreManager.reset_instance()
+
+    yield temp_db
+
+    get_db_engine(reset=True)
+    VectorStoreManager.reset_instance()

docs/guide/user-guide/navigator.md

@@ -25,21 +25,27 @@ The Codebase Navigator organizes project structure into three synchronized inter
   - **API Routes** (`@app.get`, `app.post`, etc.)
 - View parameter signatures, return types, and starting/ending line ranges.
 
-### Pane 3: Code Intelligence & Impact Analysis
-- **Callers & Callees**: Inspect incoming callers and outgoing references identified by AST cross-file analysis.
-- **Click-Through Navigation**: Click any caller or callee chip to jump directly to its declaration in Pane 1 and Pane 2.
-- **HTTP Route Specifications**: View endpoint paths, HTTP verbs, and request/response models.
-- **Syntax Preview**: Read formatted implementation code blocks with line numbers and syntax highlighting.
+### Pane 3: Code Intelligence & Impact Analysis (or Document Reader)
+- **Symbol Intelligence Mode**:
+  - **Callers & Callees**: Inspect incoming callers and outgoing references identified by AST cross-file analysis.
+  - **Click-Through Navigation**: Click any caller or callee chip to jump directly to its declaration in Pane 1 and Pane 2.
+  - **HTTP Route Specifications**: View endpoint paths, HTTP verbs, and request/response models.
+  - **Syntax Preview**: Read formatted implementation code blocks with line numbers and syntax highlighting.
+- **Full Document & Markdown Reader Mode**:
+  - Automatically activates when selecting non-code files (`.md`, `.markdown`, `.txt`, `.json`, etc.) or clicking **Read Full Document** from Pane 2.
+  - **Rendered View**: Safe Markdown parser supporting formatted headings, lists, blockquotes, inline code, and fenced code blocks.
+  - **Raw Source View**: Full file content display with line-by-line numbering.
+  - **One-Click Copy**: Copy complete document contents to clipboard with instant visual confirmation.
 
 ---
 
-## Layout Density Options
+## Layout Density & Mobile Responsiveness
 
-Customize the visual density of the three panes using the density selector in the top-right header:
-
-- **Compact**: Tight row spacing and minimal margins designed for high-density multi-file refactoring on laptops and widescreen monitors.
-- **Balanced** *(Default)*: Optimal spacing and font sizing for general architectural review.
-- **Spacious**: Card-based presentation with generous padding and expanded docstring summaries.
+- **Density Options**: Customize visual density using the header selector:
+  - **Compact**: Tight row spacing and minimal margins designed for high-density multi-file refactoring on laptops and widescreen monitors.
+  - **Balanced** *(Default)*: Optimal spacing and font sizing for general architectural review.
+  - **Spacious**: Card-based presentation with generous padding and expanded docstring summaries.
+- **Responsive Stacking**: On mobile and narrow viewports ($<900\text{px}$), the 3 panes seamlessly stack vertically with word-wrapping and container safeguards, guaranteeing zero horizontal overflow and zero element collisions.
 
 ---
 

docs/guide/user-guide/search.md

@@ -14,7 +14,7 @@ Follow these steps to run a retrieval test:
 2. Enter your query in the search bar (for example: `vector database connection pool` or `FastMCP tool registration`).
 3. Select the target search category:
    - **Code**: Searches source code AST chunks across all indexed languages.
-   - **Docs**: Searches architectural documents, ADR records, specifications, and file summaries.
+   - **Docs**: Searches architectural documents, ADR records, specifications, file summaries, and indexed PDF manuals.
 4. *(Optional)* Select a specific **Repository** filter to isolate results to a single project.
 5. Click **Run Search** or press <kbd>Enter</kbd>.