Skip to content

Improve MCP source navigation, scoped context and compiler evidence - #29

Merged
TNYCL merged 9 commits into
mainfrom
codex/source-context-mcp
Oct 4, 2026
Merged

TNYCL merged 9 commits into
mainfrom
codex/source-context-mcp

Conversation

@TNYCL

@TNYCL TNYCL commented Oct 4, 2026 •

Copy link
Copy Markdown
Member

Problem and resulting behavior

Coding agents were receiving bulky metadata or unrelated context before reaching the relevant source. MCP now defaults to one readable Source response containing redacted source passages or precise local locators, version pins, match signals, and bounded continuation information. Existing Full/Compact modes remain available. Retrieval and source selection do not call a generative model; semantic query embeddings retain the configured provider.

Agents can use Knowell to discover ownership and relationships, then use local search and file reads. The repository ships an adaptive skills/knowell skill for this workflow; installation and use do not require every read to pass through MCP.

Implementation

  • Apply project, path-prefix and language scope before candidate limits, expansion and packing. Keep focus hints as preferences. Locator searches skip source hydration and body packing; scoped metadata caches are separate from graph snapshots.
  • Preserve small-file and grouped symbol labels in immutable, version-bound syntax evidence. Old syntax policies trigger a normal T1 refresh without rewriting unchanged chunks or embeddings. Correct repeated tree T0 resumes so unchanged content can reach T1.
  • Resolve Rust lexical call sites and scoped imports/aliases conservatively. Optional graph navigation expands strong resolved edges, bounds ambiguous candidate leaves and reports omissions. It does not represent heuristic evidence as compiler completeness.
  • Bound inspection facets and occurrence/edge paging, distinguish imports from calls and tests, reject stale or shadowed source IDs, and report same-line symbol ambiguity. Personal contexts do not borrow base compiler evidence.
  • Add explicit, opt-in SCIP artifact imports with revision/build-input attestation, bounded and hostile-input validation, immutable generation admission and transactional coverage/evidence replacement. No compiler is launched automatically.
  • Add migration 16 for analysis origin and coverage. Runtime and update-publication admission both require schema 16. Existing databases need explicit maintenance with old writers stopped; unchanged older indexes also need their normal syntax-policy refresh.
  • Preserve Claude Code discovery/initialize compatibility, a single Source text channel and legacy output modes. Keep optional handles for historical or incomplete source retrieval.
  • Fix full-suite merge-queue regressions: canonical path identity on macOS, bounded Windows backup ACL subprocess handling, and concurrent child-output draining with an appropriate prepare-only timeout in update lifecycle tests.

Independent local research, benchmarks and machine configuration are excluded from this PR. No new external package is introduced.

Validation

Passed locally through the shared build lock:

  • Workspace formatting and cargo clippy --workspace --all-targets --locked -- -D warnings.
  • Graph, MCP, setup, store, index, engine, evaluation and CLI suites, plus targeted follow-up regressions for scope, immutable labels, tree resume, Rust imports, navigation, compiler-import rollback and historical/same-line source identity.
  • Real-binary MCP and update lifecycle tests, connect path/hook routing and Windows backup ACL regressions.
  • Strict database checks against isolated PostgreSQL 18.6 instances: SCRAM bad-password rejection without secret leakage, pgvector 0.8.7 operation, and a server where pgvector is genuinely unavailable. The latter covers store migration, engine lexical/graph/memory/catalogue behavior, CLI initialization and hybrid rejection.
  • All 10 update-asset publication tests, including runtime/migration/release schema consistency.
  • Dependency-free skill frontmatter/reference/UI checks. The official skill validator was unavailable locally because PyYAML is absent.

The existing managed-PostgreSQL download test remains ignored. Full CI run 37201274825 passed on final commit aac0a861f09a2027100a1327fd1586b4f031611c: full Linux, macOS and Windows workspace suites, Linux partitions, Windows process checks, PostgreSQL 18 storage tests, MSRV, documentation, panel, clippy, distribution/tooling checks, and the required gate. The synthetic small/seed-42 grep/BM25/hybrid evaluation passed its existing baseline gate. This was a manual full-lane run using the same full-suite jobs as the merge queue; the PR has not been enqueued or merged automatically.

Practical limits

Source/body budgets are estimates, not model-tokenizer guarantees. Unscoped metadata and full graph snapshots still scale with a generation; caches are count-bounded rather than byte-bounded. Rust syntax evidence is partial, and imported compiler evidence is tied to caller-attested artifacts rather than proof of compiler authenticity or complete runtime behavior. These changes do not establish billion-line capacity, agent-quality gains or measured latency/token savings.

No new live Claude coding session, paid Gemini benchmark, user database migration or main merge is performed in this revision. Those operational checks remain a separate next step.

@TNYCL
TNYCL marked this pull request as ready for review October 4, 2026 09:30
@TNYCL
TNYCL added this pull request to the merge queue Oct 4, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 4, 2026
@TNYCL
TNYCL added this pull request to the merge queue Oct 4, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 4, 2026
@TNYCL
TNYCL added this pull request to the merge queue Oct 4, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 4, 2026
@TNYCL TNYCL changed the title Return bounded source context by default over MCP Improve MCP source navigation, scoped context and compiler evidence Oct 4, 2026
@TNYCL
TNYCL added this pull request to the merge queue Oct 4, 2026
Merged via the queue into main with commit 2629e69 Oct 4, 2026
42 checks passed
@TNYCL
TNYCL deleted the codex/source-context-mcp branch October 4, 2026 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant