Nigel Ask-Bird

Nigel is a voice-activated tiki bartender parrot running on a Dell thin client. He answers drink questions, tells jokes, does impressions, and serves as a physical animatronic character with motorized beak, blinking eyes, and body movement — all driven by a local three-tier response system with real-time audio.

New here? Start with HOW-IT-WORKS.md — the full technical deep dive: hardware, wake word, dispatch chain, the beak-sync design, and the war stories behind them. Quick answers in the FAQ; a map of all the docs in docs/README.md.

What this repo is: a published snapshot of the code literally running my bird. It assumes my machines, my paths, and my bar. It is not (yet) a kit — genericizing it into something you can point at your own hardware is in progress. Read it, steal ideas from it, open an issue if you're building something similar.

Quick Start

# Text mode (CLI)
ask-bird "what is in a mai tai"

# Wake-word mode (hands-free — say "hey Nigel" then your question)
# Normally runs as nigel-wake.service; this is the same binary run directly:
nigel_wake

How a question flows

Whichever way a question comes in, it converges on the same dispatcher and playback path:

mic ──sherpa-onnx KWS "hey Nigel"──► nigel_wake ──Deepgram STT──►
  ask-bird ──routine/cache/DB/LLM──► nigel-voice (TTS) ──►
    beakplay ──POST /nigel/audio/play──► nigel_api (envelope + beak timeline)
      ──USB serial──► esp32-bridge.service ──► ESP32 (motor2 beak, motor1 body)

ask-bird "..." in text mode skips straight to the dispatcher. See docs/WAKE_WORD.md for the wake-word half and docs/BEAK_SYNC.md for the playback half.

Architecture

Nigel uses a three-tier dispatch chain running entirely locally on the Dell:

Tier	Source	Latency	What it handles
Routines	Hard-coded handlers	Instant	Impressions, "make me X", parrot taunts, greetings
Tier 1	~/.nigel_cache/cache_index.json	Instant	Exact drink recipes (29 drinks)
Tier 2	~/.nigel_cache/recommend_cache.json	Instant	Fuzzy-matched Q&A, banter, dynamic pools (10 pools)
Tier 3a	Kapu Tracker DB (localhost:8521)	~200ms	Structured drink recipes from the Kapu Tracker database, spoken via bin/format_db_recipe
Tier 3b	Local Hermes LLM	5-45s	Novel questions — body animation + stalling banter while waiting

All tiers run on the Dell. No Mac dependency. No SSH/SCP overhead. Cache hits return instantly with motor-synchronized audio. LLM calls play stalling banter and body animation while the response generates.

Files

Executables (bin/)

Script	Language	Function
nigelctl	Bash	Mode manager. CLI and menu for mode control — mode [name], status, menu
ask-bird	Bash	Main dispatcher. Process question, route to routine/cache/DB/LLM, generate audio, play with motors
nigel-voice	Bash	TTS pipeline: edge-tts (en-GB-RyanNeural) then ffmpeg shrill chain to /tmp/bird_out.wav
speak-pi	Bash	Short-form TTS for stalling/banter phrases (same shrill chain)
nigel_wake / nigel_wake.py	Python	Wake-word listener ("hey Nigel") — sherpa-onnx keyword spotter, then Deepgram STT
beakplay	Bash	The CLI's entry point for playback. Hands the WAV to nigel_api's /nigel/audio/play, which computes the beak envelope and drives the ESP32 timeline. Falls back to plain paplay when the API is unreachable
nigel_body_anim.py	Python	Standalone body animation for the CLI path
refresh_nigel_pool.py	Python	Background pool refresher: pops used entries, generates new ones via LLM
pool.sh	Bash	Reusable pool API: pool_get (read random entry), pool_refresh_bg (fire refresh)
pool_config.json	JSON	Central pool configuration: max sizes, LLM prompts, skills, static/dynamic flags
nigel-api / nigel-api-dev	Python / Bash	Production and dev launchers for the FastAPI daemon
esp32_serial_bridge.py	Python	USB serial ↔ TCP bridge daemon — owns /dev/nigel-esp32, exposes it as 127.0.0.1:5000. Runs as esp32-bridge.service

The idle and annoying daemons no longer live in bin/ — they moved into nigel_api/daemon_manager.py, along with beak sync (nigel_api/audio.py, see docs/BEAK_SYNC.md).

Bench and measurement tools (bin/)

These drive a board directly and default to the bench board, esp-2.local.

Script	Function
bodycheck	Body-cam repeatability harness — homing, park spread, gesture landing
camtrace / campro	Capture and profile raw cam-index switch traces
measure_squint.py	Measurement pass over a recorded squint video (a video-analysis tool that sends nothing to the board)
blink_pwm_sweep.py	Sweep blink PWM, measure lid-cycle time from firmware counters
switchtest / motortest	Manual switch and motor exercisers
transcribe_demos.py	Demo transcription (Deepgram)

Deployment (deploy/)

File	Description
nigel-api.service	Systemd user unit for the API daemon — sets NIGEL_ESP32_HOST=127.0.0.1 (the USB serial bridge)
nigel-wake.service	Systemd user unit for the wake-word listener (sherpa-onnx KWS)
esp32-bridge.service	Systemd user unit for the USB serial ↔ TCP bridge — starts before nigel-api.service
99-nigel-esp32.rules	udev rule for the stable /dev/nigel-esp32 symlink the bridge depends on

All three services are systemd user units — install and control with systemctl --user, not sudo systemctl. See AGENTS.md's Dev-Test-Deploy Loop for the full deploy recipe.

Runtime Data — ~/.nigel_cache/ (not in repo)

Enumerated from the code that reads/writes each path (nigel_api/config.py, nigel_api/modes.py, nigel_api/ask.py, nigel_api/main.py, bin/ask-bird, bin/refresh_nigel_pool.py, bin/nigel_wake.py):

Path	What it is
mode, mode_last_activity	Current mode + last-activity timestamp for timeout tracking
nigel.conf	Beak sync, audio volume, and other hot-reloaded playback settings
timeouts.conf	Per-mode timeout durations
idle.conf, annoying.conf	Per-daemon animation tuning (timing, PWM, weights)
auto_modes.json	Named auto-mode schemas: routines, timers, motor ranges, blink settings
pool_config.json	Fallback copy of pool config, checked second. nigel_api/main.py's _find_pool_config() looks in ~/.local/bin/pool_config.json first (the authoritative deployed copy), then here, then ~/nigel-ask-bird/bin/pool_config.json last
cache_index.json	Tier 1: 29 exact drink recipes (manually curated)
recommend_cache.json	Tier 2: dynamic pools + static QA entries
wav_cache/	Pre-rendered pool WAVs (refresh_nigel_pool.py); scanned by /nigel/beak/test
annoying/	Squawk WAVs for annoying mode and the wake-word chirp ack
demo/	Demo-reel audio for the auto-mode "demo" pool
poof_sounds/	"Make me a ___" routine's magic sound effect
sherpa-kws/	sherpa-onnx keyword-spotter model files — see docs/WAKE_WORD.md
ask_history.log	Q&A log written by nigel_api/ask.py
refresh_nigel_pool.log	Fatal-startup log for the detached pool refresher — it runs nohup'd, so this is the only trace of a startup crash
*.lock	File locks guarding concurrent-safe read-modify-write on the cache files

~/.local/bin/ is a separate directory, not under ~/.nigel_cache/ — deployed script copies (cp'd from bin/), which is what actually runs.

Documentation (docs/)

Reference

File	Description
ARCHITECTURE.md	System design, dispatch chain, audio/motor pipeline, current status
BEAK_SYNC.md	Official beak-sync design — the ESP32 timeline pipeline, calibration, per-bird settings
WAKE_WORD.md	Wake-word listener architecture (sherpa-onnx KWS), model layout, threshold tuning, debugging
EYE_PRIMITIVES.md	Official eye-motion contract — blink/squint/open, all switch-referenced on the ESP32
ROUTINES.md	Routine handler documentation (triggers, behaviors, code locations)
GIT_WORKFLOW.md	Branch-first workflow, commit conventions, deployment steps
HOME_ASSISTANT.md	HA infrastructure on the Dell, integrations, API token, Alexa bridge

Hardware

File	Description
HARDWARE.md	Start here for wiring — parts list, ESP32 pin map, TB6612 hookup, motor/switch topology
SWITCHES_AND_MOTION_PRIMITIVES.md	Canonical switch/motion measurement background behind EYE_PRIMITIVES.md
BODY_CAM_CALIBRATION.md	Body-cam period table and calibration procedure
SQUAWKERS_BABY_INTEL.md / SQUAWKERS_LIMIT_SWITCH_INTEL.md	Reverse-engineering notes on the donor Squawkers Baby toy
GRAB_BAG_IDENTIFICATION_GUIDE.md	General-purpose parts-identification reference
takes/	Working notes and in-progress measurement takes

Archive — superseded docs kept for their reasoning and hard-won detail; each has its own README explaining what changed and why, and none describes the system as it exists today

Path	Description
archive/	Early Nigel API planning docs (handoff summary, phase tracker, an AsyncTCP/WiFi optimization plan overtaken by the current design)
electronics/archive/	Abandoned hardware beak-sync approaches (analog envelope follower, relay wiring) — superseded by the ESP32-timeline design

Character: Nigel

Nigel hails from the fictional south Pacific isle of Kalapu Atoll — a lost volcanic speck where the rum flows freely, the sand is black glass, and the local parrots have strong opinions about your tip selection. He brings that island attitude to every drink question, impression, and squawk.

British tiki bartender parrot. Knows rums, tiki cocktails, and bars. Response style:

• One short conversational sentence by default
• Offers to elaborate ("tell me more" returns the full answer from cache)
• Full words, not abbreviations ("ounces" not "oz")
• Dry, authoritative tone
• Unknown queries get a random opener + helper combo

Motor Control

Nigel drives two motors via an ESP32 microcontroller. Production talks to it over USB serial via esp32-bridge.service (a TCP↔serial bridge on 127.0.0.1:5000, board at /dev/nigel-esp32) — the mDNS hostname is no longer in this data path. Mac dev talks directly over WiFi to a bench board (esp-2.local:5000 by default; the boards swapped roles on 2026-08-01 and kept their hostnames, so the lower number is the live one in production but the dev default).

Motor	Channel	Pins	Controls
Motor 1 (Body)	TB6612 A	D16, D17, PWMA=D4	Body tilt/eye cam movement
Motor 2 (Beak)	TB6612 B	D18, D19, PWMB=D21	Beak open/close + eye blinks

• Beak envelope: Motor 2 follows the speech waveform in real time (50ms RMS windows)
• Blinks: Motor 2, firmware-counted and switch-referenced (eye:blink:<n>:<pwm>). In-speech blinks are on by default (blink_in_speech) — placed in measured silences off the same envelope that builds the beak timeline, with the firmware holding and restoring the beak's position around each one so the two can't drift apart. See docs/EYE_PRIMITIVES.md.
• Body animation: Motor 1 runs random movements in a background thread throughout audio
• Idle: the API's idle daemon runs between questions with independent blink/body/beak-twitch timers

Troubleshooting recipes and detailed motor tuning parameters live in the nigel-ask-bird-workflow Hermes skill; the durable design contracts are docs/BEAK_SYNC.md and docs/EYE_PRIMITIVES.md.

Motor parameters are now configurable via the Daemon Tuning UI card on the dashboard at nigel.local:8500/nigel/ui/. Switch between Idle and Annoying modes, adjust timing intervals and PWM values, save, and restart the daemon — all without code changes.

Development Workflow

Branch-first, always. Never commit to master directly.

cd ~/nigel-ask-bird
git checkout master
git checkout -b feature/name-of-feature
# ... make changes, commit ...
git diff master..feature/name-of-feature   # review before merge
git checkout master && git merge --ff-only feature/name-of-feature
git branch -d feature/name-of-feature

Deploy before testing. The repo (~/nigel-ask-bird/bin/) is the source. ~/.local/bin/ is what runs.

cp ~/nigel-ask-bird/bin/<script> ~/.local/bin/<script>

See docs/GIT_WORKFLOW.md for full conventions.

Debugging

# Test a specific query
ask-bird "what is a daiquiri"

# Verbose debug mode (bash trace)
bash -x ~/.local/bin/ask-bird "what is a mai tai"

# Check cache state
python3 -c "import json; d=json.load(open('$HOME/.nigel_cache/recommend_cache.json')); print(len(d), 'keys')"

# Kill stuck processes (don't kill the hermes gateway)
pkill -9 -f 'hermes chat'
pkill -9 -f 'ask-bird'

# Check motor connectivity — on the Dell, via the local USB serial bridge:
echo -n "motor1:forward:100" | nc -w1 127.0.0.1 5000
# Bench board over WiFi (Mac dev): nc -w1 esp-2.local 5000

Current Status

• All tiers running locally on Dell (no Mac dependency)
• Cache self-healing (novel answers cached automatically)
• Stalling banter with body animation during LLM generation
• Motor control: beak sync, blinks, body animation, idle daemon — all inside nigel_api now; beak sync uses an ESP32-side timeline (see docs/BEAK_SYNC.md)
• Fully independent from Mac — no SSH/SCP overhead

Modes

Nigel has four behavior modes managed by nigelctl:

Mode	Body	Audio	Behavior
awake	Full motors	Greeting + chirp + TTS	Normal conversation
idle	Subtle animation (blinks, twitches, shifts)	None	Silent background life, wakes on "hey Nigel"
annoying	Aggressive animation (high PWM, bursts, jitter)	Random squawks 30-90s	Playful nuisance, 30min timeout → idle
auto	Per-schema	Per-schema	Runs a named schema of weighted routine pools — configured in the dashboard's Auto Modes card, stored in ~/.nigel_cache/auto_modes.json. 30min timeout → idle

Use nigelctl, nigelctl menu, or voice commands (go idle, annoying mode, wake up).