Repository navigation
1.1.0: plain-language questions (ripgrep holdout 0.042 → 0.500), stdout fix, cold start - #127
Merged
Merged
Conversation
ParsaVictor
changed the title
ParsaVictor
marked this pull request as ready for review
October 1, 2026 08:35
…re any run Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…guesses Plain-language questions (no identifier) were resolved one word at a time and landed on whatever symbol shared a word. New set tests/third_party/concept (14 q on this repo): recall 0.286 -> 0.679. - F89: "How does this tool ..." made `tool` a strong identifier anchor; a lowercase word the prompt never marks as code is retagged stem: - F90: neuromesh-graph/src/file_rank.rs - BM25F over path, defined names, comments and body, Snowball-stemmed (rust-stemmers); new comment_index in the graph (snapshot v4); W3 seeds the best file and up to two runners-up within 70%, and drops guesses far below it - F91: general software thesaurus (thesaurus.txt, weight 0.35), with no topic of the ripgrep holdout in it - NM_RANK_PROBE=1 in the gold harness shows where the ranker puts gold Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ose halves drop with the compound; word! is code - file_rank index keyed on (generation, file count): learning bumped the node revision every query and rebuilt the index (CI 250 ms gate) - guess pruning scores against every file, tests included - compound-stem pair drops stem: halves too (F89 retag) - `trainstep!` marks code (Julia/Rust macros): holdout-lang back to 0.589 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t retagged (predict / predict_proba); clippy holdout-ml back to 0.478. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…big files seeded by their matching definitions Cold start, fresh clone of this repo: first query 3.56 s -> 2.65 s (includes the index build), warm queries 0.1-0.4 s. - spawn_live_sync marks the gate Indexing before the scan thread runs: the gate said Ready over an empty graph and the first question got "no_seed_resolved" - wait_for_index answers from a complete index only (finished scan or loaded snapshot; reset on clear), waits up to 120 s (NEUROMESH_INDEX_WAIT_SECS), never from a graph still filling - reindex_incremental marks Ready before the optional embeddings refresh - rank index built after ingest and snapshot load, not on the first question (CI 250 ms gate) - a ranked file over 8k tokens is seeded by up to three definitions whose name/doc match the question, not whole - NM_TIMING=1: index stages, rank build, seeds/select, MCP stages - benchmark-holdout.sh: `concept` and `concept-holdout` sets - ripgrep holdout run once: recall 0.042 -> 0.500 (F92-F94 open) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…acket not smaller); changelog 1.1.0, reply to upstream, benchmark-fast.sh, roadmap H-M Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…enchmark-fast.sh fix Nine sets in 78 s warm with benchmark-fast.sh (was ~40 min), same numbers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ParsaVictor
force-pushed
the
concept/gold-and-fixes
branch
from
October 1, 2026 08:55
25ee1be to
4b3debe
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Plain-language questions (Yoosef's report). Draft: holdout (ripgrep, gold locked in 97f106b) and latency still to come. Findings F88–F91 in docs/planning/stage5-findings.fa.md.
🤖 Generated with Claude Code