diff --git a/CLAUDE.md b/CLAUDE.md index 734123e..e4062ec 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -93,10 +93,10 @@ LinkedIn: linkedin.com/in/abhinaysai-kamineni ## Current Phase -**Version:** **V1 closed**. **Cortex V2 — Phase 0 complete**; P0 Must Have is next. -**Status:** Spec + gap report on `main`: `docs/CORTEX_V2.md`, `docs/CURRENT_STATE.md`, `docs/V1_RELEASE.md` -**Next:** V2 P0 starting at #68 (Claim/Evidence data foundations). Do not skip to Reliability Gate before Evidence + Temporal. -**Target:** Evidence Graph + Temporal Truth + Reliability Gate + CortexBench baseline +**Version:** **V1 closed**. **Cortex V2 scaffold on `main`** (PRs #82–#91). +**Status:** Spec + state: `docs/CORTEX_V2.md`, `docs/CURRENT_STATE.md`, `docs/V1_RELEASE.md` +**Next:** Deepen V2 hot path (Evidence/Firewall/Gate/coverage on write + `/query` + MCP inject) and prove the signature BLOCK + explain demo (`CORTEX_V2.md` §19). Then #61 demo API origin, #65 thread extraction. +**Target:** Demonstrable memory control plane (not scaffold-only APIs) --- @@ -113,7 +113,7 @@ LinkedIn: linkedin.com/in/abhinaysai-kamineni | 6 | React dashboard + knowledge graph explorer | Week 3-4 | ✅ Done | | 7 | Live demo URL + README polish | Week 4 | ✅ Done | | **V1** | Organizational memory MVP (phases 0–7) | — | ✅ **Closed** | -| V2 | Memory control plane (see `docs/CORTEX_V2.md`) | — | ⏳ Phase 0 next | +| V2 | Memory control plane (see `docs/CORTEX_V2.md`) | — | ✅ Scaffold on main; ⏳ hot-path deepening | | ~~8–10~~ | Coverage / outcomes / elicitation / federation | — | Folded into V2 | --- diff --git a/DECISIONS.md b/DECISIONS.md index 30b296f..d18bc9d 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -23,13 +23,21 @@ Agent picks up OPEN instructions at session start, executes, marks DONE. ## ACTIVE INSTRUCTIONS +### 2026-09-22 — V2 hygiene closed; deepen hot path next +Priority: HIGH +Status: OPEN +Detail: +- V2 scaffold PRs #82–#91 are on `main`. Docs (`CURRENT_STATE.md`, CLAUDE.md, README) reflect scaffold ≠ finished control plane. +- Close completed GitHub issues when permissions allow: #33, #36, #67–#77 (implementation landed). Keep #78 open until DoD / signature demo. +- **Next build work:** wire Evidence/Firewall/Gate/coverage into write + `/query` + MCP inject; prove CORTEX_V2 §19 BLOCK+explain demo. Then #61, #65, #63, #64. + ### 2026-09-18 — Close V1 then V2 Phase 0 only Priority: HIGH -Status: DONE — V1 closed (#79); Phase 0 merged (#80) (2026-09-18) +Status: DONE — V1 closed (#79); Phase 0 merged (#80); V2 scaffold #82–#91 on main (2026-09-22) Detail: - V1 = phases 0–7. Coverage/outcomes/meetings are V2, not V1. - Phase 0 delivered: `docs/CORTEX_V2.md` + `docs/CURRENT_STATE.md`. -- Next allowed work: V2 P0 starting at #68 (Claim/Evidence). Do not skip to Reliability Gate before Evidence + Temporal. +- P0–P2 scaffold implemented via #82–#91. Remaining: hot-path deepening (see 2026-09-22 instruction). ### 2026-06-10 — LLM-backed CMVK verifiers (production) Priority: HIGH diff --git a/README.md b/README.md index f703ed8..b936ee0 100644 --- a/README.md +++ b/README.md @@ -101,7 +101,7 @@ When any agent touches the payments service, Cortex enriches its context automat | **RBAC** | Graph-level access control — contractors don't see salary decisions | | **GDPR erasure** | Cascade delete with audit trail; query cache invalidated per workspace | -**V1 scope note:** Coverage scoring, outcome linking, and meeting connectors are **not shipped in V1** — they are tracked for Cortex V2. See [docs/V1_RELEASE.md](docs/V1_RELEASE.md), [docs/CORTEX_V2.md](docs/CORTEX_V2.md), and [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md). +**V1 / V2 honesty:** V1 = phases 0–7 (see [docs/V1_RELEASE.md](docs/V1_RELEASE.md)). Cortex V2 **modules and APIs are on `main`** (Evidence Graph behind flag, Temporal Truth, Reliability Gate, Firewall helpers, procedure models, outcome ledger, gaps/abstention, CortexBench baseline, MCP `cortex_evaluate` / `cortex_explain_memory`) — treat them as **scaffold**, not a finished control plane. Meeting connectors and full hot-path wiring remain open. Details: [docs/CORTEX_V2.md](docs/CORTEX_V2.md), [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md). --- @@ -389,7 +389,7 @@ cortex/ | Phase 6 | React dashboard (Ask, memory map, guide, agent inject) | ✅ Shipped | | Phase 7 | Live demo URL + README polish | ✅ Done (wire `CORTEX_API_ORIGIN` for API-backed search) | | **V1** | Phases 0–7 — organizational memory MVP | ✅ **Closed** — [docs/V1_RELEASE.md](docs/V1_RELEASE.md) | -| V2 | Memory control plane | ⏳ Phase 0 audit — [docs/CORTEX_V2.md](docs/CORTEX_V2.md) · [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) · [#78](https://github.com/askmy-stack/cortex/issues/78) | +| V2 | Memory control plane | ✅ Scaffold on `main` (#82–#91); ⏳ deepen hot path + signature demo — [docs/CORTEX_V2.md](docs/CORTEX_V2.md) · [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) · [#78](https://github.com/askmy-stack/cortex/issues/78) | | ~~Phase 8–10~~ | Coverage / outcomes / elicitation / federation | Folded into **Cortex V2** roadmap | **CI:** GitHub Actions runs `pytest` + seed dry-run on push/PR ([`.github/workflows/ci.yml`](.github/workflows/ci.yml)). diff --git a/SESSIONS.md b/SESSIONS.md index ee78609..a187186 100644 --- a/SESSIONS.md +++ b/SESSIONS.md @@ -649,3 +649,29 @@ 1. Merge Phase 0 PR; mark #67 done 2. Then start V2 P0 #68 (Claim/Evidence data foundations) only 3. Do not jump to Reliability Gate before Evidence + Temporal + +--- + +## Session — 2026-09-22 — V2 close-the-loop hygiene +**Duration:** ~20m +**Phase:** V2 docs/issue hygiene (no feature code) + +### Built +- Refreshed `docs/CURRENT_STATE.md` for post-#82–#91 reality (scaffold vs deepening) +- Updated `CLAUDE.md` Current Phase + build table +- Updated README V1/V2 honesty note + roadmap row +- ACTIVE instruction: deepen hot path next; keep epic #78 until DoD +- Attempted close of completed issues #33, #36, #67–#77 + +### State at end +- V2 modules remain on `main`; docs no longer claim Phase 0 / #68 as next +- Remaining product work: hot-path wiring, signature demo, #61/#65/#63/#64 + +### Decisions made +- Document V2 as scaffold-on-main; do not claim control-plane DoD until §19 demo works + +### Next session starts with +1. Merge this hygiene PR +2. Wire Evidence/Firewall/Gate/coverage into write + `/query` + MCP inject +3. Prove CORTEX_V2 §19 BLOCK + explain scenario + diff --git a/docs/CURRENT_STATE.md b/docs/CURRENT_STATE.md index b1f1220..527d48a 100644 --- a/docs/CURRENT_STATE.md +++ b/docs/CURRENT_STATE.md @@ -1,19 +1,21 @@ -# Cortex CURRENT_STATE — V2 Phase 0 Audit +# Cortex CURRENT_STATE — Post V2 Scaffold Land -**Date:** 2026-09-18 +**Date:** 2026-09-22 **Baseline:** Cortex **V1 closed** (phases 0–7) — see [V1_RELEASE.md](V1_RELEASE.md) **Spec:** [CORTEX_V2.md](CORTEX_V2.md) -**Tracking:** GitHub [#67](https://github.com/askmy-stack/cortex/issues/67) · epic [#78](https://github.com/askmy-stack/cortex/issues/78) +**Tracking:** Epic [#78](https://github.com/askmy-stack/cortex/issues/78) (close when hot-path deepening + signature demo land) -> Phase 0 deliverable only. No Claim/Evidence migrations or Reliability Gate code in this document’s accompanying PR. +> Phase 0 audit (2026-09-18) delivered the gap report. PRs **#82–#91** then landed V2 modules on `main`. This document reflects **code on main as of 2026-09-22**, not the pre-implementation audit. --- ## 1. Executive summary -Cortex V1 is a working decision-capture → Kafka → extract → score → Neo4j → query/inject/MCP pipeline with RBAC, GDPR, contradictions, decay, and a React dashboard. It is **not** yet a memory control plane: there is no Evidence Graph, no Reliability Gate (`ACT|VERIFY|ASK|ESCALATE|BLOCK`), no temporal state-at-time API, no procedure memory, and no CortexBench. +Cortex V1 remains a working decision-capture → Kafka → extract → score → Neo4j → query/inject/MCP pipeline with RBAC, GDPR, contradictions, decay, and a React dashboard. -V2 should **extend** existing modules (`scoring/`, `intelligence/`, `graph/`, `memory/`, `api/`, `mcp/`) rather than replace Kafka, Neo4j, or the dashboard. +**V2 scaffold is on `main`:** Evidence Graph (flagged), Temporal Truth, Reliability Gate + MCP `cortex_evaluate` / `cortex_explain_memory`, CortexBench baseline, Memory Firewall helpers, Procedure models, Outcome ledger API, Abstention/gaps API. + +**Not done yet:** full hot-path wiring (write pipeline / `/query` / MCP inject), procedure extractor + versioning, outcome→incident linker, coverage on query responses, signature BLOCK demo, and live demo API origin (#61). Treat V2 as **landed scaffold + APIs**, not a complete memory control plane. --- @@ -27,110 +29,97 @@ V2 should **extend** existing modules (`scoring/`, `intelligence/`, `graph/`, `m | Trust scoring | **Full** | `scoring/trust_scorer.py` | Bayesian; hardcoded source priors (#42) | | CMVK | **Full** | `scoring/cmvk.py`, `cmvk_llm.py` | Heuristic / openai / ollama | | Write pipeline gate | **Full** | `scoring/write_pipeline.py` | importance → CMVK → trust | -| Quarantine audit | **Partial** | `memory/quarantine.py` | Write rejects only; no retrieval firewall | +| Quarantine audit | **Partial** | `memory/quarantine.py` | Write rejects; firewall helpers exist but not fully unified | | Neo4j Decision graph | **Full** | `graph/writer.py`, `graph/query.py` | MERGE idempotent | | RBAC (graph) | **Full** | `graph/rbac.py`, V002/V009 | Read-path filter | | GDPR erase | **Full** | `graph/gdpr.py`, `api/gdpr.py` | + Qdrant/Timescale purge | -| Redis query cache | **Full** | `api/memory.py`, `memory/cache_epoch.py` | Epoch bump on GDPR + **writes** (V1 closeout) | +| Redis query cache | **Full** | `api/memory.py`, `memory/cache_epoch.py` | Epoch bump on GDPR + writes (V1 closeout / #33) | | Semantic (Qdrant) | **Partial** | `memory/semantic.py` | Flag `CORTEX_SEMANTIC_ENABLED` | | Episodic (Timescale) | **Partial** | `memory/episodic.py` | Optional append / GDPR purge | | Contradiction detector | **Full** | `intelligence/contradiction_detector.py` | + API resolve | -| Decay engine | **Full** | `intelligence/decay_engine.py` | CLI + Compose `decay-worker` (V1 closeout) | -| Outcome nodes | **Schema only** | `V004__outcome_nodes.cypher` | No linker — #63 / #75 | -| Coverage indices | **Schema only** | `V005__coverage_indices.cypher` | No scorer; UI optional field — #62 / #76 | -| Claim / Evidence | **Missing** | — | V2 P0 #68–#69 | -| Temporal state-at-time API | **Missing** | Partial `valid_at`/`invalid_at` on edges | V2 P0 #70 | -| Reliability Gate | **Missing** | — | V2 P0 #71 | -| Procedure memory | **Missing** | — | V2 P1 #74 | -| Memory Firewall | **Partial** | quarantine + CMVK | Needs PII/injection guards — #73 | -| Abstention / gaps | **Missing** | — | V2 P2 #76 | -| CortexBench | **Missing** | — | V2 P0 #72 | -| MCP query/inject/remember | **Full** | `mcp/server.js` — `cortex_query`, `cortex_remember`, `cortex_inject` | | -| MCP evaluate/explain/outcome | **Missing** | — | #71, #77, #75 | -| Meetings connector | **Missing** | README historically overclaimed | #64 | +| Decay engine | **Full** | `intelligence/decay_engine.py` | CLI + Compose `decay-worker` (V1 closeout / #36) | +| Claim / Evidence | **Scaffold** | `shared/v2_models.py`, `evidence/*`, V010, `api/evidence.py` | Behind `CORTEX_EVIDENCE_GRAPH` (default off); deepen write-path | +| Temporal state-at-time | **Scaffold** | `temporal/*`, `api/memory_state.py` | State-at-time API landed; deepen ranking vs query path | +| Reliability Gate | **Scaffold** | `reliability/*`, `api/reliability.py`, MCP `cortex_evaluate` | ACT\|VERIFY\|ASK\|ESCALATE\|BLOCK; wire into inject | +| MCP explain | **Scaffold** | MCP `cortex_explain_memory` | Explain path present; tie to signature demo | +| CortexBench | **Baseline** | `evals/*`, `docs/EVALS.md` | Baseline scenarios; expand suite + CI report | +| Memory Firewall | **Scaffold** | `firewall/*` | write/retrieval/secret helpers; not fully on hot path | +| Procedure memory | **Models** | `procedures/models.py` | Need extractor / versioning / env compatibility | +| Outcome ledger | **Scaffold** | `outcomes/ledger.py`, `api/outcomes.py` | Usefulness API; linker to metrics/incidents still #63 | +| Coverage / abstention | **Scaffold** | `gaps/*`, `api/gaps.py` | Not yet on `/query` + MCP inject (#62 depth) | +| MCP query/inject/remember | **Full** | `mcp/server.js` | V1 tools | +| Meetings connector | **Missing** | — | #64 | | Dashboard | **Full** | `frontend/` | Ask/Explore/Review/Assist | | Live demo URL | **Partial** | Vercel UI; API needs `CORTEX_API_ORIGIN` | #61 | --- -## 3. README / docs mismatches (post V1 closeout) +## 3. README / docs mismatches -Addressed in V1 closeout (`docs/V1_RELEASE.md`): +Addressed in V1 closeout + this hygiene pass: -- Meetings / coverage / outcome tracking no longer claimed as shipped -- ARCHITECTURE.md status updated from “Design only” +- Meetings / coverage / outcome linking not claimed as fully shipped +- V2 described as **scaffold on main**, with deepening called out +- `ARCHITECTURE.md` body may still describe longer-term designs — read as target where marked future -Still stale / watch: +Still watch: -- `ARCHITECTURE.md` body still describes Phase 8+ designs as future — OK if read as target architecture - `docs/PRODUCTION_READINESS.md` is a historical 2026-06-10 report (test counts outdated) -- Frontend `coverage_score?` type remains for forward UI; API does not return it +- Frontend `coverage_score?` type remains for forward UI; query path may not always populate it --- -## 4. Reuse points for V2 +## 4. Reuse points (unchanged) | V2 need | Reuse | |---|---| -| Claim adapter | `DecisionEvent` + `Provenance` in `shared/models.py`; `GraphWriter.write()` | -| Evidence / authority | Extend `scoring/trust_scorer.py` priors (#42); write path in `write_pipeline.py` | -| Temporal | Existing `valid_at` / `invalid_at` / `SUPERSEDES` in `graph/writer.py`, `graph/query.py` causal chain | -| Reliability Gate | New `reliability/` package; call from `api/` + MCP; consume RBAC from `graph/rbac.py` | -| Firewall | Extend `write_pipeline.py` + `memory/quarantine.py`; retrieval guard beside `MemoryService.query_decisions` | -| Outcomes | V004 Outcome indices; new `intelligence/` or `outcomes/` module | -| Coverage / gaps | V005 indices; new scorer; wire into `/query` response schema | -| Eval | New `evals/`; seed via `scripts/demo_catalog.py` patterns | -| Migrations | Continue `graph/migrations/V010+` via `graph/migrate.py` | - -**Feature flags (recommended):** `CORTEX_EVIDENCE_GRAPH`, `CORTEX_RELIABILITY_GATE` — default off until P0 stable. +| Claim adapter | `DecisionEvent` + `Provenance`; `evidence/adapters.py` | +| Evidence / authority | `evidence/authority.py` + trust priors (#42) | +| Temporal | `temporal/validity.py`, `temporal/state.py` | +| Reliability Gate | `reliability/gate.py` ← API + MCP | +| Firewall | Extend `write_pipeline.py` + query guards with `firewall/*` | +| Outcomes | `outcomes/ledger.py` + V004 indices; linker still open | +| Coverage / gaps | `gaps/detector.py`; wire into `/query` | +| Eval | `evals/`; expand scenarios | +| Migrations | `graph/migrations/V010+` via `graph/migrate.py` | + +**Feature flags:** `CORTEX_EVIDENCE_GRAPH` (and gate flags as added) — default off until hot-path stable. --- -## 5. Duplicate / inconsistent logic +## 5. Merged V2 PRs (reference) -| Area | Note | +| PR | Scope | |---|---| -| Trust vs importance | Separate scores — keep; V2 adds usefulness / action_confidence (do not overload trust) | -| Quarantine vs CMVK reject | Both reject writes; firewall should unify reason codes | -| Decision as only memory unit | V2 Claim sits beside Decision via adapter — avoid forking two write paths long-term | -| Aggregate Kafka topics | GitHub/Jira use `cortex.raw.*.events` (D-013) — keep for V2 connectors | - ---- - -## 6. Recommended module placement - -Prefer **new packages** only where V2 domains are first-class (per CORTEX_V2 §21): - -```text -reliability/ # gate, risk, policies, reason_codes, confidence -evidence/ # models helpers, authority, explain (or under graph/) -temporal/ # validity helpers used by graph/query -procedures/ # P1 -outcomes/ # P1 — or intelligence/outcome_linker.py -firewall/ # P1 — or scoring/firewall_*.py -gaps/ # P2 -evals/ # CortexBench -``` - -Do **not** create empty folders until the implementing PR needs them. +| #82 | Claim/Evidence foundations + Neo4j V010 (#68) | +| #83 | Evidence Graph + explain API (#69) | +| #84 | Temporal Truth + `/memory/state` (#70) | +| #85 | Reliability Gate + MCP `cortex_evaluate` (#71) | +| #86 | MCP `cortex_explain_memory` (#77) | +| #87 | CortexBench baseline (#72) | +| #88 | Memory Firewall (#73) | +| #89 | Procedural memory models (#74) | +| #90 | Outcome ledger + usefulness (#75) | +| #91 | Abstention + KnowledgeGap (#76) | --- -## 7. Suggested V2 implementation order (locked) - -1. **Phase 0 (this doc)** — #67 -2. **P0:** #68 data foundations → #69 Evidence → #70 Temporal → #71 Gate + #77 explain → #72 CortexBench -3. **P1:** #73 Firewall → #74 Procedures → #75 Outcomes -4. **P2:** #76 Abstention / gaps (+ #62 coverage) +## 6. Remaining work (priority) -**Do not start #68 until this Phase 0 audit is merged.** +1. **Deepen V2 hot path** — Evidence/Firewall/Gate/coverage on write + `/query` + MCP inject +2. **Signature demo** — CORTEX_V2.md §19 “restart payments?” → BLOCK + explain +3. **#61** — live demo `CORTEX_API_ORIGIN` +4. **#65** — thread-aware extraction +5. **#63 / #64** — outcome linker; meeting connector +6. Expand CortexBench + keep README claims honest --- -## 8. Definition of ready for V2 Phase 1 +## 7. Definition of ready — Phase 0 (complete) - [x] V1 closed (`docs/V1_RELEASE.md`) - [x] `docs/CORTEX_V2.md` checked in -- [x] This `CURRENT_STATE.md` merged -- [ ] Issue #67 closed by maintainer -- [ ] D-018 / ACTIVE instruction Phase 0 marked DONE +- [x] Phase 0 `CURRENT_STATE` audit merged (#80) +- [x] V2 P0–P2 scaffold PRs #82–#91 merged +- [ ] Hot-path deepening + signature demo (next) +- [ ] Epic #78 closed when control-plane DoD is demonstrable