ttft-optimization

Hi, I'm IncidentIQ!! A Slack-based SRE assistant built with Google ADK, FastAPI, and multi-agent orchestration. Works via Slack, API, or ADK Web UI; supports incident briefs, reliability reviews, AWS cost/core ops, Kubernetes checks, RAG over runbooks/past incidents, alert triage/dedupe, various LLM modes, eval benchmarks, and Docker deployment.

A production-style LLM evaluation pipeline spanning vLLM serving, lm-eval-harness integration, performance metrics (TTFT/TPOT/p95), deterministic guardrails, and statistically significant benchmark improvements.

LLM inference system with adaptive routing, multi-precision quantization, and speculative decoding (2.54× speedup). Achieves <200ms TTFT at 18.9 tok/s on 2.29GB VRAM via vLLM-PagedAttention with semantic caching and reproducible benchmarks.

Algorithmic context manager and execution engine for multi-agent workflows. Reduces TTFT and isolates attention by routing the minimal topological R/W subset to parallel LLMs.