ci-gate

An API contract consumer-impact analyzer that treats every version bump like a seismic event. Go CLI + TypeScript viewer, deterministic reports, CI exit-code gate.

AI run variance and reproducibility analyzer - answer stability, route consistency and timing variance in one deterministic report.

Human-centered frontend quality gate and theory-based UX audit skill with measurable fast-gate checks, weighted scoring, and CI strict mode.

Self-hosted Agent quality engineering platform — evaluation as a release gate. 8 eval types with AI judges, datasets, pre-eval scripts, analytics & A/B testing. Gate loop (API keys, CI CLI, webhooks) lands with M1. FastAPI + Vue 3 + Celery.

Record and replay AI agent runs offline to catch tool regressions and verify changes to files and databases.

Java 原生 LLM 可观测与评测：同源 traceId 贯通、国产模型中文评测集、CI 回归门禁。不是 Langfuse 的替代品。

pip install gauntlet-evals · v0.1.0. Merge-blocking evaluation gates for generative AI features: YAML suites run against any HTTP endpoint or Python callable, fail the build on a miss, and emit both a diffable JSON pack and a reviewer document cross-referenced to California's published GenAI risk framework. Aligned to, never approved by.

Calibration-first evaluation toolkit for Cloudflare's Clef decision models. Judge cheap, audit confidence.

Julia, standard-library-only diagnostics console that scans polyglot repositories for language-policy violations and architectural anti-patterns, with evidence-backed classifications and scan timing.

v0.2.0. Fail-closed evaluation harness for government-facing chat systems: reproducible, provenance-stamped audit verdicts, byte-identical across Python 3.11 to 3.14, with no third-party dependencies. A silent or unreadable target scores zero rather than passing by absence. Two public projects of my own pin it by exact commit.

Complete security framework for AI Agents, MCP Servers, and RAG pipelines. Features automated MCP fuzzing, cross-agent poisoning detection, persistent memory audits, unified risk scoring, and CI/CD security gates.

Agent reliability, measured: an evaluation harness with 15 programmatically-verified tasks, 11 fault classes, a permission model, and a CI gate — plus the LangGraph agent it evaluates.

Self-hosted accuracy evals + correction loop for document extraction. Bring your own model.

Prove whether an Arm binary can actually use the chip's matrix hardware. Static ISA verifier + CI gate for AArch64 AI runtimes (SME/SME2, i8mm, dotprod).

A CI gate to optimize GitHub stacked PRs (gh stack). Stop rerunning CI on every layer of stacked pull requests.

CLI launch-readiness validator for product, marketing, sales and operations specifications. Scores a spec against a token map and can gate CI.

Field-level evaluation and release gate for LLM document extraction: exact golden set, per-field precision/recall, CI gate that blocks regressions

Agent and LLM evaluation harness: golden datasets, multi-scorer execution, regression detection across model versions, cost-quality leaderboards, and CI gates for model promotion.

A Claude Code skill that brands every deliverable by default — and the gate that refuses any file that is off-brand

Deterministic LLM prompt security & quality gate (prompt's ESLint): zero LLM cost, byte-level reproducible verdicts, SARIF/CI-native, OWASP LLM Top 10 2026 mapped. Python CLI · GitHub Action · MCP · VS Code · Docker