🔱 Trishula-StructuredGen

Sovereign Structured LLM Generation Engine

Trishula-StructuredGen is a structured output generation engine that combines SGLang's RadixAttention KV-cache sharing with Pydantic schema enforcement to guarantee valid, typed JSON outputs from local LLMs — with automatic Ollama + Instructor fallback.

No hallucinated keys. No invalid JSON. Exact regex-constrained outputs.

Features

• ⚡ SGLang RadixAttention — 3–5x faster KV-cache sharing on multi-turn agent loops
• ✅ Pydantic schema enforcement — guaranteed structured outputs, no hallucinated fields
• 🎯 Regex-constrained generation — exact format outputs (e.g. PK, -3.5, +7)
• 🔄 Automatic Ollama fallback — routes to Ollama + Instructor if SGLang server offline
• 📊 SQA latency gate — alerts when TTFT exceeds 5 seconds
• 📋 Audit trail — latency receipt on every generation

Quick Start

pip install trishula-structuredgen
pip install sglang pydantic

from trishula_structured_gen import TrishulaStructuredGen
from pydantic import BaseModel, Field

class PickPrediction(BaseModel):
    home_team: str
    away_team: str
    predicted_winner: str
    spread: str = Field(description="e.g. 'PK', '-3.5', '+7'")
    confidence: float = Field(ge=0.0, le=1.0)
    reasoning: str

gen = TrishulaStructuredGen(model="llama3.2:3b")

# Guaranteed structured output — no hallucinated keys
result = gen.generate_json(
    prompt="Analyze: Chiefs (14-3) vs Bills (13-4). Who wins?",
    schema=PickPrediction,
)
print(result["result"].spread)       # e.g. "-3.5"
print(result["result"].confidence)   # e.g. 0.74
print(result["latency_ms"])          # e.g. 847.3

# Regex-constrained spread output
spread = gen.generate_regex(
    prompt="What is the spread for Chiefs vs Bills?",
    pattern=r"(PK|[+-][0-9]+\.?[0-9]*)"
)
print(spread["result"])  # e.g. "-3.5"

How It Works

Request
  ↓
SGLang Server available?
  ├─ YES → RadixAttention structured generation (3-5x faster)
  └─ NO  → Ollama + Instructor fallback (always works)
       ↓
Pydantic validation → JSONL ledger receipt → Return typed result

Requirements

• Python 3.10+
• pip install sglang pydantic instructor
• Ollama running locally (fallback, always available)
• SGLang server (optional, for RadixAttention acceleration)

License

MIT — see LICENSE