Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,10 +93,10 @@ LinkedIn: linkedin.com/in/abhinaysai-kamineni

## Current Phase

**Version:** **V1 closed**. **Cortex V2 — Phase 0 complete**; P0 Must Have is next.
**Status:** Spec + gap report on `main`: `docs/CORTEX_V2.md`, `docs/CURRENT_STATE.md`, `docs/V1_RELEASE.md`
**Next:** V2 P0 starting at #68 (Claim/Evidence data foundations). Do not skip to Reliability Gate before Evidence + Temporal.
**Target:** Evidence Graph + Temporal Truth + Reliability Gate + CortexBench baseline
**Version:** **V1 closed**. **Cortex V2 scaffold on `main`** (PRs #82–#91).
**Status:** Spec + state: `docs/CORTEX_V2.md`, `docs/CURRENT_STATE.md`, `docs/V1_RELEASE.md`
**Next:** Deepen V2 hot path (Evidence/Firewall/Gate/coverage on write + `/query` + MCP inject) and prove the signature BLOCK + explain demo (`CORTEX_V2.md` §19). Then #61 demo API origin, #65 thread extraction.
**Target:** Demonstrable memory control plane (not scaffold-only APIs)

---

Expand All @@ -113,7 +113,7 @@ LinkedIn: linkedin.com/in/abhinaysai-kamineni
| 6 | React dashboard + knowledge graph explorer | Week 3-4 | ✅ Done |
| 7 | Live demo URL + README polish | Week 4 | ✅ Done |
| **V1** | Organizational memory MVP (phases 0–7) | — | ✅ **Closed** |
| V2 | Memory control plane (see `docs/CORTEX_V2.md`) | — | ⏳ Phase 0 next |
| V2 | Memory control plane (see `docs/CORTEX_V2.md`) | — | ✅ Scaffold on main; ⏳ hot-path deepening |
| ~~8–10~~ | Coverage / outcomes / elicitation / federation | — | Folded into V2 |

---
Expand Down
12 changes: 10 additions & 2 deletions DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,21 @@ Agent picks up OPEN instructions at session start, executes, marks DONE.

## ACTIVE INSTRUCTIONS

### 2026-09-22 — V2 hygiene closed; deepen hot path next
Priority: HIGH
Status: OPEN
Detail:
- V2 scaffold PRs #82–#91 are on `main`. Docs (`CURRENT_STATE.md`, CLAUDE.md, README) reflect scaffold ≠ finished control plane.
- Close completed GitHub issues when permissions allow: #33, #36, #67–#77 (implementation landed). Keep #78 open until DoD / signature demo.
- **Next build work:** wire Evidence/Firewall/Gate/coverage into write + `/query` + MCP inject; prove CORTEX_V2 §19 BLOCK+explain demo. Then #61, #65, #63, #64.

### 2026-09-18 — Close V1 then V2 Phase 0 only
Priority: HIGH
Status: DONE — V1 closed (#79); Phase 0 merged (#80) (2026-09-18)
Status: DONE — V1 closed (#79); Phase 0 merged (#80); V2 scaffold #82–#91 on main (2026-09-22)
Detail:
- V1 = phases 0–7. Coverage/outcomes/meetings are V2, not V1.
- Phase 0 delivered: `docs/CORTEX_V2.md` + `docs/CURRENT_STATE.md`.
- Next allowed work: V2 P0 starting at #68 (Claim/Evidence). Do not skip to Reliability Gate before Evidence + Temporal.
- P0–P2 scaffold implemented via #82–#91. Remaining: hot-path deepening (see 2026-09-22 instruction).

### 2026-06-10 — LLM-backed CMVK verifiers (production)
Priority: HIGH
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ When any agent touches the payments service, Cortex enriches its context automat
| **RBAC** | Graph-level access control — contractors don't see salary decisions |
| **GDPR erasure** | Cascade delete with audit trail; query cache invalidated per workspace |

**V1 scope note:** Coverage scoring, outcome linking, and meeting connectors are **not shipped in V1** — they are tracked for Cortex V2. See [docs/V1_RELEASE.md](docs/V1_RELEASE.md), [docs/CORTEX_V2.md](docs/CORTEX_V2.md), and [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md).
**V1 / V2 honesty:** V1 = phases 0–7 (see [docs/V1_RELEASE.md](docs/V1_RELEASE.md)). Cortex V2 **modules and APIs are on `main`** (Evidence Graph behind flag, Temporal Truth, Reliability Gate, Firewall helpers, procedure models, outcome ledger, gaps/abstention, CortexBench baseline, MCP `cortex_evaluate` / `cortex_explain_memory`) — treat them as **scaffold**, not a finished control plane. Meeting connectors and full hot-path wiring remain open. Details: [docs/CORTEX_V2.md](docs/CORTEX_V2.md), [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md).

---

Expand Down Expand Up @@ -389,7 +389,7 @@ cortex/
| Phase 6 | React dashboard (Ask, memory map, guide, agent inject) | ✅ Shipped |
| Phase 7 | Live demo URL + README polish | ✅ Done (wire `CORTEX_API_ORIGIN` for API-backed search) |
| **V1** | Phases 0–7 — organizational memory MVP | ✅ **Closed** — [docs/V1_RELEASE.md](docs/V1_RELEASE.md) |
| V2 | Memory control plane | ⏳ Phase 0 audit — [docs/CORTEX_V2.md](docs/CORTEX_V2.md) · [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) · [#78](https://github.com/askmy-stack/cortex/issues/78) |
| V2 | Memory control plane | ✅ Scaffold on `main` (#82–#91); ⏳ deepen hot path + signature demo — [docs/CORTEX_V2.md](docs/CORTEX_V2.md) · [docs/CURRENT_STATE.md](docs/CURRENT_STATE.md) · [#78](https://github.com/askmy-stack/cortex/issues/78) |
| ~~Phase 8–10~~ | Coverage / outcomes / elicitation / federation | Folded into **Cortex V2** roadmap |

**CI:** GitHub Actions runs `pytest` + seed dry-run on push/PR ([`.github/workflows/ci.yml`](.github/workflows/ci.yml)).
Expand Down
26 changes: 26 additions & 0 deletions SESSIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -649,3 +649,29 @@
1. Merge Phase 0 PR; mark #67 done
2. Then start V2 P0 #68 (Claim/Evidence data foundations) only
3. Do not jump to Reliability Gate before Evidence + Temporal

---

## Session — 2026-09-22 — V2 close-the-loop hygiene
**Duration:** ~20m
**Phase:** V2 docs/issue hygiene (no feature code)

### Built
- Refreshed `docs/CURRENT_STATE.md` for post-#82–#91 reality (scaffold vs deepening)
- Updated `CLAUDE.md` Current Phase + build table
- Updated README V1/V2 honesty note + roadmap row
- ACTIVE instruction: deepen hot path next; keep epic #78 until DoD
- Attempted close of completed issues #33, #36, #67–#77

### State at end
- V2 modules remain on `main`; docs no longer claim Phase 0 / #68 as next
- Remaining product work: hot-path wiring, signature demo, #61/#65/#63/#64

### Decisions made
- Document V2 as scaffold-on-main; do not claim control-plane DoD until §19 demo works

### Next session starts with
1. Merge this hygiene PR
2. Wire Evidence/Firewall/Gate/coverage into write + `/query` + MCP inject
3. Prove CORTEX_V2 §19 BLOCK + explain scenario

141 changes: 65 additions & 76 deletions docs/CURRENT_STATE.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,21 @@
# Cortex CURRENT_STATE — V2 Phase 0 Audit
# Cortex CURRENT_STATE — Post V2 Scaffold Land

**Date:** 2026-09-18
**Date:** 2026-09-22
**Baseline:** Cortex **V1 closed** (phases 0–7) — see [V1_RELEASE.md](V1_RELEASE.md)
**Spec:** [CORTEX_V2.md](CORTEX_V2.md)
**Tracking:** GitHub [#67](https://github.com/askmy-stack/cortex/issues/67) · epic [#78](https://github.com/askmy-stack/cortex/issues/78)
**Tracking:** Epic [#78](https://github.com/askmy-stack/cortex/issues/78) (close when hot-path deepening + signature demo land)

> Phase 0 deliverable only. No Claim/Evidence migrations or Reliability Gate code in this document’s accompanying PR.
> Phase 0 audit (2026-09-18) delivered the gap report. PRs **#82–#91** then landed V2 modules on `main`. This document reflects **code on main as of 2026-09-22**, not the pre-implementation audit.

---

## 1. Executive summary

Cortex V1 is a working decision-capture → Kafka → extract → score → Neo4j → query/inject/MCP pipeline with RBAC, GDPR, contradictions, decay, and a React dashboard. It is **not** yet a memory control plane: there is no Evidence Graph, no Reliability Gate (`ACT|VERIFY|ASK|ESCALATE|BLOCK`), no temporal state-at-time API, no procedure memory, and no CortexBench.
Cortex V1 remains a working decision-capture → Kafka → extract → score → Neo4j → query/inject/MCP pipeline with RBAC, GDPR, contradictions, decay, and a React dashboard.

V2 should **extend** existing modules (`scoring/`, `intelligence/`, `graph/`, `memory/`, `api/`, `mcp/`) rather than replace Kafka, Neo4j, or the dashboard.
**V2 scaffold is on `main`:** Evidence Graph (flagged), Temporal Truth, Reliability Gate + MCP `cortex_evaluate` / `cortex_explain_memory`, CortexBench baseline, Memory Firewall helpers, Procedure models, Outcome ledger API, Abstention/gaps API.

**Not done yet:** full hot-path wiring (write pipeline / `/query` / MCP inject), procedure extractor + versioning, outcome→incident linker, coverage on query responses, signature BLOCK demo, and live demo API origin (#61). Treat V2 as **landed scaffold + APIs**, not a complete memory control plane.

---

Expand All @@ -27,110 +29,97 @@ V2 should **extend** existing modules (`scoring/`, `intelligence/`, `graph/`, `m
| Trust scoring | **Full** | `scoring/trust_scorer.py` | Bayesian; hardcoded source priors (#42) |
| CMVK | **Full** | `scoring/cmvk.py`, `cmvk_llm.py` | Heuristic / openai / ollama |
| Write pipeline gate | **Full** | `scoring/write_pipeline.py` | importance → CMVK → trust |
| Quarantine audit | **Partial** | `memory/quarantine.py` | Write rejects only; no retrieval firewall |
| Quarantine audit | **Partial** | `memory/quarantine.py` | Write rejects; firewall helpers exist but not fully unified |
| Neo4j Decision graph | **Full** | `graph/writer.py`, `graph/query.py` | MERGE idempotent |
| RBAC (graph) | **Full** | `graph/rbac.py`, V002/V009 | Read-path filter |
| GDPR erase | **Full** | `graph/gdpr.py`, `api/gdpr.py` | + Qdrant/Timescale purge |
| Redis query cache | **Full** | `api/memory.py`, `memory/cache_epoch.py` | Epoch bump on GDPR + **writes** (V1 closeout) |
| Redis query cache | **Full** | `api/memory.py`, `memory/cache_epoch.py` | Epoch bump on GDPR + writes (V1 closeout / #33) |
| Semantic (Qdrant) | **Partial** | `memory/semantic.py` | Flag `CORTEX_SEMANTIC_ENABLED` |
| Episodic (Timescale) | **Partial** | `memory/episodic.py` | Optional append / GDPR purge |
| Contradiction detector | **Full** | `intelligence/contradiction_detector.py` | + API resolve |
| Decay engine | **Full** | `intelligence/decay_engine.py` | CLI + Compose `decay-worker` (V1 closeout) |
| Outcome nodes | **Schema only** | `V004__outcome_nodes.cypher` | No linker — #63 / #75 |
| Coverage indices | **Schema only** | `V005__coverage_indices.cypher` | No scorer; UI optional field — #62 / #76 |
| Claim / Evidence | **Missing** | — | V2 P0 #68–#69 |
| Temporal state-at-time API | **Missing** | Partial `valid_at`/`invalid_at` on edges | V2 P0 #70 |
| Reliability Gate | **Missing** | — | V2 P0 #71 |
| Procedure memory | **Missing** | — | V2 P1 #74 |
| Memory Firewall | **Partial** | quarantine + CMVK | Needs PII/injection guards — #73 |
| Abstention / gaps | **Missing** | — | V2 P2 #76 |
| CortexBench | **Missing** | — | V2 P0 #72 |
| MCP query/inject/remember | **Full** | `mcp/server.js` — `cortex_query`, `cortex_remember`, `cortex_inject` | |
| MCP evaluate/explain/outcome | **Missing** | — | #71, #77, #75 |
| Meetings connector | **Missing** | README historically overclaimed | #64 |
| Decay engine | **Full** | `intelligence/decay_engine.py` | CLI + Compose `decay-worker` (V1 closeout / #36) |
| Claim / Evidence | **Scaffold** | `shared/v2_models.py`, `evidence/*`, V010, `api/evidence.py` | Behind `CORTEX_EVIDENCE_GRAPH` (default off); deepen write-path |
| Temporal state-at-time | **Scaffold** | `temporal/*`, `api/memory_state.py` | State-at-time API landed; deepen ranking vs query path |
| Reliability Gate | **Scaffold** | `reliability/*`, `api/reliability.py`, MCP `cortex_evaluate` | ACT\|VERIFY\|ASK\|ESCALATE\|BLOCK; wire into inject |
| MCP explain | **Scaffold** | MCP `cortex_explain_memory` | Explain path present; tie to signature demo |
| CortexBench | **Baseline** | `evals/*`, `docs/EVALS.md` | Baseline scenarios; expand suite + CI report |
| Memory Firewall | **Scaffold** | `firewall/*` | write/retrieval/secret helpers; not fully on hot path |
| Procedure memory | **Models** | `procedures/models.py` | Need extractor / versioning / env compatibility |
| Outcome ledger | **Scaffold** | `outcomes/ledger.py`, `api/outcomes.py` | Usefulness API; linker to metrics/incidents still #63 |
| Coverage / abstention | **Scaffold** | `gaps/*`, `api/gaps.py` | Not yet on `/query` + MCP inject (#62 depth) |
| MCP query/inject/remember | **Full** | `mcp/server.js` | V1 tools |
| Meetings connector | **Missing** | — | #64 |
| Dashboard | **Full** | `frontend/` | Ask/Explore/Review/Assist |
| Live demo URL | **Partial** | Vercel UI; API needs `CORTEX_API_ORIGIN` | #61 |

---

## 3. README / docs mismatches (post V1 closeout)
## 3. README / docs mismatches

Addressed in V1 closeout (`docs/V1_RELEASE.md`):
Addressed in V1 closeout + this hygiene pass:

- Meetings / coverage / outcome tracking no longer claimed as shipped
- ARCHITECTURE.md status updated from “Design only”
- Meetings / coverage / outcome linking not claimed as fully shipped
- V2 described as **scaffold on main**, with deepening called out
- `ARCHITECTURE.md` body may still describe longer-term designs — read as target where marked future

Still stale / watch:
Still watch:

- `ARCHITECTURE.md` body still describes Phase 8+ designs as future — OK if read as target architecture
- `docs/PRODUCTION_READINESS.md` is a historical 2026-06-10 report (test counts outdated)
- Frontend `coverage_score?` type remains for forward UI; API does not return it
- Frontend `coverage_score?` type remains for forward UI; query path may not always populate it

---

## 4. Reuse points for V2
## 4. Reuse points (unchanged)

| V2 need | Reuse |
|---|---|
| Claim adapter | `DecisionEvent` + `Provenance` in `shared/models.py`; `GraphWriter.write()` |
| Evidence / authority | Extend `scoring/trust_scorer.py` priors (#42); write path in `write_pipeline.py` |
| Temporal | Existing `valid_at` / `invalid_at` / `SUPERSEDES` in `graph/writer.py`, `graph/query.py` causal chain |
| Reliability Gate | New `reliability/` package; call from `api/` + MCP; consume RBAC from `graph/rbac.py` |
| Firewall | Extend `write_pipeline.py` + `memory/quarantine.py`; retrieval guard beside `MemoryService.query_decisions` |
| Outcomes | V004 Outcome indices; new `intelligence/` or `outcomes/` module |
| Coverage / gaps | V005 indices; new scorer; wire into `/query` response schema |
| Eval | New `evals/`; seed via `scripts/demo_catalog.py` patterns |
| Migrations | Continue `graph/migrations/V010+` via `graph/migrate.py` |

**Feature flags (recommended):** `CORTEX_EVIDENCE_GRAPH`, `CORTEX_RELIABILITY_GATE` — default off until P0 stable.
| Claim adapter | `DecisionEvent` + `Provenance`; `evidence/adapters.py` |
| Evidence / authority | `evidence/authority.py` + trust priors (#42) |
| Temporal | `temporal/validity.py`, `temporal/state.py` |
| Reliability Gate | `reliability/gate.py` ← API + MCP |
| Firewall | Extend `write_pipeline.py` + query guards with `firewall/*` |
| Outcomes | `outcomes/ledger.py` + V004 indices; linker still open |
| Coverage / gaps | `gaps/detector.py`; wire into `/query` |
| Eval | `evals/`; expand scenarios |
| Migrations | `graph/migrations/V010+` via `graph/migrate.py` |

**Feature flags:** `CORTEX_EVIDENCE_GRAPH` (and gate flags as added) — default off until hot-path stable.

---

## 5. Duplicate / inconsistent logic
## 5. Merged V2 PRs (reference)

| Area | Note |
| PR | Scope |
|---|---|
| Trust vs importance | Separate scores — keep; V2 adds usefulness / action_confidence (do not overload trust) |
| Quarantine vs CMVK reject | Both reject writes; firewall should unify reason codes |
| Decision as only memory unit | V2 Claim sits beside Decision via adapter — avoid forking two write paths long-term |
| Aggregate Kafka topics | GitHub/Jira use `cortex.raw.*.events` (D-013) — keep for V2 connectors |

---

## 6. Recommended module placement

Prefer **new packages** only where V2 domains are first-class (per CORTEX_V2 §21):

```text
reliability/ # gate, risk, policies, reason_codes, confidence
evidence/ # models helpers, authority, explain (or under graph/)
temporal/ # validity helpers used by graph/query
procedures/ # P1
outcomes/ # P1 — or intelligence/outcome_linker.py
firewall/ # P1 — or scoring/firewall_*.py
gaps/ # P2
evals/ # CortexBench
```

Do **not** create empty folders until the implementing PR needs them.
| #82 | Claim/Evidence foundations + Neo4j V010 (#68) |
| #83 | Evidence Graph + explain API (#69) |
| #84 | Temporal Truth + `/memory/state` (#70) |
| #85 | Reliability Gate + MCP `cortex_evaluate` (#71) |
| #86 | MCP `cortex_explain_memory` (#77) |
| #87 | CortexBench baseline (#72) |
| #88 | Memory Firewall (#73) |
| #89 | Procedural memory models (#74) |
| #90 | Outcome ledger + usefulness (#75) |
| #91 | Abstention + KnowledgeGap (#76) |

---

## 7. Suggested V2 implementation order (locked)

1. **Phase 0 (this doc)** — #67
2. **P0:** #68 data foundations → #69 Evidence → #70 Temporal → #71 Gate + #77 explain → #72 CortexBench
3. **P1:** #73 Firewall → #74 Procedures → #75 Outcomes
4. **P2:** #76 Abstention / gaps (+ #62 coverage)
## 6. Remaining work (priority)

**Do not start #68 until this Phase 0 audit is merged.**
1. **Deepen V2 hot path** — Evidence/Firewall/Gate/coverage on write + `/query` + MCP inject
2. **Signature demo** — CORTEX_V2.md §19 “restart payments?” → BLOCK + explain
3. **#61** — live demo `CORTEX_API_ORIGIN`
4. **#65** — thread-aware extraction
5. **#63 / #64** — outcome linker; meeting connector
6. Expand CortexBench + keep README claims honest

---

## 8. Definition of ready for V2 Phase 1
## 7. Definition of ready — Phase 0 (complete)

- [x] V1 closed (`docs/V1_RELEASE.md`)
- [x] `docs/CORTEX_V2.md` checked in
- [x] This `CURRENT_STATE.md` merged
- [ ] Issue #67 closed by maintainer
- [ ] D-018 / ACTIVE instruction Phase 0 marked DONE
- [x] Phase 0 `CURRENT_STATE` audit merged (#80)
- [x] V2 P0–P2 scaffold PRs #82–#91 merged
- [ ] Hot-path deepening + signature demo (next)
- [ ] Epic #78 closed when control-plane DoD is demonstrable
Loading