task5_speller_api

Backend Python API for the Brain↔ChatGPT BCI project. Takes a letter prefix (typically 1–3 chars) from the Unity speller grid and returns three predicted complete English words via an OpenAI-compatible LLM endpoint (Groq by default), ready to be rendered as SSVEP-selectable targets.

Scope: BR41N.IO Hackathon Prep · Assignment 1 · Task 5. v0.1 ships as a callable Python function, a CLI smoke-test, and a FastAPI endpoint for web integration.

Owners: Ali Nasser (repo, core script, integration/latency docs) · Mohamed Nady (prompt design, test bench, accuracy writeup, key-setup guide).

Layout

speller_api/
├── task5_speller_api/      # Python package
│   ├── __init__.py         # re-exports predict_words
│   ├── __main__.py         # CLI entry (python -m task5_speller_api)
│   ├── _client.py          # OpenAI client + API-key loader
│   ├── server.py           # FastAPI endpoint
│   └── speller.py          # predict_words() — public contract
├── tests/
│   └── test_cases.json     # M2 test bench (Mohamed)
├── data/                   # M6 unigram frequency data (Mohamed)
├── .env.example            # copy to .env and fill in your key
├── .gitignore
├── requirements.txt
├── README.md               # this file
├── INTEGRATION.md          # UI team integration contract (Ali, A5)
├── LATENCY.md              # latency suggestions writeup (Ali, A6)
├── DESIGN_NOTES.md         # forward-looking v0.2+ notes (Ali, A8)
├── KEY_SETUP.md            # OpenAI key walkthrough (Mohamed, M7)
└── ACCURACY.md             # accuracy suggestions writeup (Mohamed, M3)

Architecture

Dual-Model Design

v0.1 uses a split-brain approach:

• Speller Model (cheap, fast): Groq llama-3.3-70b-versatile by default. Predicts the next 3 words from a BCI prefix—optimized for latency and cost.
• Response Model (smart, contextual): Google Gemini by default. Generates rich, conversational replies to fully-typed sentences—optimized for quality and personality.

Both use OpenAI-compatible endpoints, so you can swap providers independently. See DESIGN_NOTES.md §1 for the forward-looking rationale.

git clone https://github.com/Holmesberg/speller_api.git
cd speller_api
python -m venv .venv && source .venv/bin/activate     # Windows: .venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env                                   # then edit .env and paste your key

See KEY_SETUP.md for the end-to-end account + key walkthrough.

Provider setup (Groq default · also OpenAI, Gemini, or Local LLM)

The client uses the OpenAI Python SDK pointed at whichever OpenAI-compatible endpoint OPENAI_API_BASE_URL names. Two providers are documented:

• Groq (default): set OPENAI_API_BASE_URL=https://api.groq.com/openai/v1, put your Groq key into OPENAI_API_KEY, and set OPENAI_MODEL (e.g. llama-3.3-70b-versatile).
• Google AI Studio (Gemini): set OPENAI_API_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/, put your Gemini key into OPENAI_API_KEY, and set OPENAI_MODEL to a compatible Gemini model (e.g. gemini-3-flash-preview).
• Local LLM (llama.cpp): Ensure you have llama.cpp running, then set the .env variables to point to your local server.

Example .env for Local LLMs:

OPENAI_API_BASE_URL=http://localhost:8080/v1
OPENAI_API_KEY=sk-no-key-required
OPENAI_MODEL=gemma4

Example command to run a local server:

.\llama-server.exe -m "F:\LLMs\gemma-4-E2B-it-RotorQuant-IQ4_XS.gguf" --cache-type-k q8_0 --cache-type-v q8_0 -ngl 99 -fa on --jinja --temp 1.0 --top_p 0.95 --top_k 64

Upstream docs (kept current by the providers):

• Groq API keys: https://console.groq.com/keys
• Groq OpenAI compatibility: https://console.groq.com/docs/openai
• Gemini API keys (Google AI Studio): https://aistudio.google.com/app/apikey
• Gemini OpenAI compatibility: https://ai.google.dev/gemini-api/docs/openai

Run

# Start the local FastAPI server for web integration
python -m uvicorn task5_speller_api.server:app --reload
# Or (if uvicorn is installed globally)
uvicorn task5_speller_api.server:app --reload

# Or run the CLI smoke test
python -m task5_speller_api he --context "writing an email to my professor"
# → ["hello", "hope", "help"]

# Or run the real-time quality simulator (Windows terminal)
python -m task5_speller_api --simulate
# Optional custom output path
python -m task5_speller_api --simulate --output logs/my_session.json

Or in Python:

from task5_speller_api import predict_words, respond_to_sentence

predict_words("he", context="writing an email to my professor")
# → ['hello', 'hope', 'help']

# Generate a contextual response (model decides when to search)
respond_to_sentence(
    "I'm interested in learning Python",
    context="education"
)
# → "That's great! Python is an excellent language to start with..."

Real-time simulator mode

The simulator is designed for BCI recommendation quality testing:

• You enter one topic/context for the session.
• As you type each letter, recommendations refresh immediately.
• Select a recommendation by pressing its number: 1, 2, or 3.
• Press Space to commit your typed prefix manually as a word.
• Press Backspace to delete the last typed letter.
• Press Enter to end the session and save results.

The simulator writes a JSON report containing:

• topic
• started_at and ended_at (UTC timestamps)
• event_count
• events[] (one per selected/committed word), including:
  • selection timestamp
  • prefix at selection
  • number of letters typed before selection
  • shown recommendations
  • selected recommendation index (or null for manual)
  • selected word
  • prediction latency in milliseconds
• final_sentence
• response (model's contextual response to final sentence)

Response generation

The respond_to_sentence() function uses the smarter response model (Gemini by default) to generate contextual replies.

from task5_speller_api import respond_to_sentence

response = respond_to_sentence("What time is it?", context="")
# → "The current time depends on your location and timezone..."

response = respond_to_sentence(
    "How is AI changing healthcare?",
    context="medical technology"
)
# → "AI is revolutionizing healthcare in several key areas. Recent developments include..."

The response uses the model's built-in knowledge to generate thoughtful, conversational replies.

API Endpoint

POST /predict

Accepts a JSON payload:

{
  "prefix": "<optional prefix string>",
  "sentence": "<optional sentence string>",
  "context": "<optional context string>"
}

Returns:

{
  "predictions": ["word1", "word2", "word3"]
}

• All fields are optional, but at least one should be provided for meaningful predictions.
• The endpoint enforces the prefix constraint and capitalization rules as described above.

Example request:

POST http://localhost:8000/predict
Content-Type: application/json

{
  "prefix": "ch",
  "sentence": "Barcelona is the",
  "context": "football"
}

Example response:

{
  "predictions": ["Champions", "Chelsea", "Choice"]
}

Docs

Doc	Audience	What's in it
INTEGRATION.md	Unity UI team	Function signature, error model, latency envelope, sequence diagram, mocking guide
KEY_SETUP.md	Anyone on the team	Step-by-step account / key / spending-cap walkthrough (OpenAI-era; see README for Groq setup)
LATENCY.md	Future-us (v0.2+)	Ten concrete ways to reduce per-call latency
ACCURACY.md	Future-us (v0.2+)	Ten concrete ways to improve prediction quality
DESIGN_NOTES.md	Whole project team	Forward-looking design (WBS 4.1 hook, FastAPI sketch)