Skip to content

feat(rules): add keyword-rule matcher with config loading - #16

Merged
johnnyhuy merged 1 commit into
mainfrom
agent/issue-15-keyword-rule-matcher
Nov 8, 2026
Merged

johnnyhuy merged 1 commit into
mainfrom
agent/issue-15-keyword-rule-matcher

Conversation

@johnnyhuy

Copy link
Copy Markdown
Contributor

What changed

Adds a deterministic operator-rule matcher so Wingman can return a pinned, approved answer for known questions instead of a generated one. Rules load from config, and the RAG ask path checks them before retrieval and generation.

Part of #6
Closes #15

How it was verified

  • mise run install - passed (backend via uv, frontend via bun, runtime deps)
  • mise run test-backend - 47 passed, including 21 new tests in backend/tests/test_rules.py
  • mise run test-frontend - passed (the repo task currently reports no frontend tests yet)
  • Offline HTTP proof against the local backend (see below). Slack auth was stubbed with the repo test pattern, so nothing connected to a live workspace.

Proof

Backend started locally with OPERATOR_RULES_JSON=[{"id":"vpn-reset","keywords":["reset vpn"],"response":"Use the IT portal to reset VPN access."}], a sqlite database and the Pi runtime pointed at a dead local port, so a matched answer can only come from the rule.

Matched question returns the canned answer with the rule as its source:

$ curl -s -X POST http://127.0.0.1:8000/api/ask -H 'Content-Type: application/json' -d '{"question":"How do I reset VPN access?"}'
{"answer":"Use the IT portal to reset VPN access.","sources":[{"source":"operator_rule","rule_id":"vpn-reset"}],"confidence":"high"}

Unmatched question leaves the rule path and enters normal retrieval (fails here only because Chroma is not running in this offline setup):

$ curl -s -X POST http://127.0.0.1:8000/api/ask -H 'Content-Type: application/json' -d '{"question":"What is RAG?"}'
{"detail":"Could not connect to a Chroma server. Are you sure it is running?"}

New tests, all passing: TestMatching (7), TestOrderingAndEnabled (3), TestConfigLoading (9), TestRagShortCircuit (2). The short-circuit tests assert retrieval and generation are never called for a matched question.

Reviewer notes

  • Rules cover the /api/ask path through RAGEngine.generate_response in this slice. Slack delivery still goes through the Pi Durable job flow, so wiring rules into the Slack handlers is a follow-up slice. That work would need a live Slack workspace to verify end to end, which this run must not use.
  • Bad config fails safe. Invalid rules are logged and skipped, invalid JSON or a missing file loads as no rules, and an empty config is a no-op.
  • No migration in this slice. Config is two new settings, OPERATOR_RULES_JSON and OPERATOR_RULES_FILE, documented in .env.example and docs/operator-rules.md.
  • One environment note: the worktree venv was recreated with the mise-pinned Python 3.13, because the default Python 3.14 cannot install onnxruntime. No lockfile change was needed and the incidental uv.lock refresh was reverted.

Operator rules pin a canned answer to a question pattern. The matcher
supports contains, exact and regex modes with priority ordering, loads
from OPERATOR_RULES_JSON or OPERATOR_RULES_FILE, and skips invalid rules
instead of breaking startup. RAGEngine.generate_response checks rules
before retrieval and generation, so a matched question returns without
a Pi job. Adds unit tests and operator docs.

Part of #6
@johnnyhuy
johnnyhuy merged commit dc7dfed into main Nov 8, 2026
10 checks passed
@johnnyhuy
johnnyhuy deleted the agent/issue-15-keyword-rule-matcher branch November 8, 2026 06:35
@github-actions github-actions Bot mentioned this pull request Oct 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add keyword-rule matcher with config loading and unit tests

1 participant