Recall

An interview coach that interviews you.

Most study tools hand you material to read. Reading feels productive and is a poor predictor of whether you can actually say the thing under pressure. Recall inverts the loop: it asks, you answer, it tells you which specific ideas you missed — then schedules the question to come back exactly when you are about to forget it.

Type an answer the way you would say it out loud. Recall grades it against a rubric derived from the reference answer, shows you point by point what landed and what did not, and feeds the score into a spaced-repetition scheduler so weak topics resurface first.

It runs with no API key and no network. A hosted model is optional and makes the grading of open answers better, not possible.

┌─ Recall ────────────────────────────────────────────────────┐
│  How do you decide the column order in a composite index?   │
│                                                             │
│  > Equality columns first, most selective first, and the    │
│  > range column last, because after a range the rest stop   │
│  > being seekable. I check the plan to confirm.             │
│                                                             │
│  SOLID  78%   graded by rubric-local · next review in 6 days│
│    ✓ A composite index is sorted by the first column…       │
│    ✓ Equality predicates first, then range or ordering      │
│    ✗ A covering index answers from the index alone          │
└─────────────────────────────────────────────────────────────┘

What is interesting here

• Rubric grading of free text, offline. A reference answer is decomposed into weighted rubric points at ingest time. An answer is scored per point on keyword coverage plus latent-semantic similarity, so the feedback names the missing idea instead of returning one opaque number.
• The grader is measured, not asserted. 28 human-labelled answers across four quality bands, scored the way any classifier would be: agreement, rank correlation, and pass/fail separation. recall eval --sweep is how the calibration constant was chosen, and CI fails if the grader stops separating good answers from bad ones.
• Anti-gaming guards. Empty answers, "I don't know", and restating the question are rejected before scoring rather than collecting partial credit for vocabulary.
• Hard-negative distractor mining. Multiple-choice options are opening claims from other topics, selected by embedding similarity so they are plausible rather than obviously irrelevant — with a near-duplicate guard so exactly one option can answer the question.
• SM-2 spaced repetition over durable SQLite, so progress survives restarts and a question you keep failing keeps returning sooner.
• A real ingestion pipeline. A TypeScript study corpus is read using the TypeScript toolchain, then transformed into a graded question bank by Python.

Quick start

Requires uv and Node 20+.

make install          # uv sync + npm install
make bank             # build the question bank from the seed corpus
make drill            # a full drill session in your terminal

For the web interface, in two terminals:

make api              # FastAPI on :8000, docs at /docs
make web              # Vite on :5173

Drill in the terminal

recall drill is the whole product with no browser:

cd backend
uv run recall drill --length 10
uv run recall drill --category "AI, ML & LLMs"
uv run recall progress

How grading works

Multiple choice is exact set comparison. There is nothing to infer, so nothing is inferred.

Open answers are scored per rubric point on two signals:

Signal	Weight	What it catches
Keyword coverage	0.65	Did the answer use the identifying terms?
Semantic similarity	0.35	Was the idea expressed in different words?

Keyword coverage leads deliberately. Latent semantic analysis over a corpus this size relates terms that co-occur, but it cannot reliably equate a rare synonym the corpus never uses, so trusting it as the primary signal would let a fluent but wrong answer score well. A point earns full credit at a combined score of 0.65 rather than 1.0, because a learner restating an idea in their own words rarely reproduces every keyword — and that constant was chosen by sweeping it against the labelled set, not guessed.

The grader's honest weakness

The evaluation makes this measurable rather than leaving it as a surprise: the offline grader's worst cases are confidently wrong answers using the right vocabulary, which land in shaky when a human says missed. That is inherent to keyword-led scoring, it is visible in the disagreement list that recall eval prints, and it is exactly what the optional hosted grader fixes.

Evaluation

make eval     # or: uv run recall eval --sweep

Current measurements on the seed bank, 28 labelled answers across 7 questions:

Metric	Value	What it means
Exact band agreement	67.9%	Matches the human's band outright
Within one band	100%	Never off by two bands
Rank correlation	0.901	Orders answers the way a human does
Pass/fail separation	0.533	Clear gap between good and bad answers
Mean absolute error	0.123	Distance from the labelled band's midpoint

Mean score by human label — the ordering invariant CI enforces:

missed    0.26  ########
shaky     0.41  ############
solid     0.77  #######################
mastered  1.00  ##############################

Rank correlation matters more than absolute error: a grader that is systematically low but orders answers correctly can be recalibrated, while one that ranks a bad answer above a good one cannot be rescued by a constant.

Drilling your own material

The seed corpus ships ~38 questions so the repository is demoable standalone. To drill your own notes, point the extractor at a TypeScript study corpus exposing allTopics with interviewQuestions:

node tools/extract-bank.mjs ../my-notes data/raw-topics.json
cd backend && uv run recall build-bank --corpus ../data/raw-topics.json \
  --out data/personal_bank.json
RECALL_BANK_PATH=backend/data/personal_bank.json make drill

personal_bank.json is gitignored: a bank built from private notes stays private.

Enabling a hosted model

cd backend && uv sync --extra llm
export RECALL_OPENAI_API_KEY=sk-...

Two things change. Open answers are graded for meaning rather than keyword coverage — and the model's judgements are re-validated against the rubric, with the score recomputed locally, so a confident model cannot inflate its own grade. Second, POST /api/questions/{id}/variant writes genuinely new questions instead of falling back to reverse-recall. Generated questions are tagged and never enter the evaluation set, because a generated reference answer is not ground truth.

Architecture

flowchart TB
  UI[React + Vite] --> API[FastAPI]
  CLI[recall drill] --> ENGINE

  API --> ENGINE[Drill engine]
  ENGINE --> SELECT[Selection:<br/>due → unseen → weakest]
  ENGINE --> GRADE[Grading service]
  ENGINE --> SM2[SM-2 scheduler]

  GRADE --> MCQ[Exact match]
  GRADE --> RUBRIC[Rubric grader<br/>keywords + LSA]
  GRADE -. optional .-> LLM[Hosted grader<br/>re-validated locally]

  SM2 --> DB[(SQLite)]
  ENGINE --> DB

  TS[TypeScript corpus] --> EXTRACT[esbuild extractor]
  EXTRACT --> BUILD[Bank builder<br/>rubrics + distractors]
  BUILD --> BANK[(question_bank.json)]
  BANK --> ENGINE

  EVAL[Labelled answers] --> RUBRIC

Loading

Details in docs/architecture.md and docs/evaluation.md.

API

Method	Endpoint	Purpose
GET	/health	Bank size, active grader, embedder (public)
GET	/api/categories	Categories with question counts
POST	/api/sessions	Start a drill session
GET	/api/sessions/{id}/next	The next question — without its answer
POST	/api/sessions/{id}/answer	Grade an answer, reveal the reference
GET	/api/progress	Mastery per topic and the review queue
POST	/api/questions/{id}/variant	A fresh take on a question

Interactive docs at /docs. Setting RECALL_API_KEY requires an x-api-key header on everything except /health.

The question payload never contains the reference answer, the rubric, or which option is correct — serialising the domain object directly would ship the answer to the browser with the question. There is a test for that.

Project layout

recall/
├── backend/
│   ├── src/recall/
│   │   ├── domain/       # question, grade, review-state models
│   │   ├── ingest/       # chunk-free corpus → rubrics → question bank
│   │   ├── grading/      # exact, rubric-local, and hosted graders
│   │   ├── scheduling/   # SM-2
│   │   ├── generation/   # reverse-recall and hosted variants
│   │   ├── evaluation/   # labelled set, metrics, calibration sweep
│   │   ├── store/        # SQLite persistence
│   │   └── api/          # FastAPI
│   ├── data/seed_corpus.json
│   └── tests/            # 101 tests
├── frontend/             # React 19 + Vite, 12 tests
├── tools/                # TypeScript corpus extractor (dev only)
└── docs/

Checks

make check    # ruff + mypy --strict + pytest + eslint + tsc + vitest

The backend is mypy --strict clean. All study content is either freshly written for this repository or supplied by the person running it.