truthfulqa

Production-grade LLM Evaluation & Benchmarking Framework - GPT-4, Claude, Gemini, Mistral. Accuracy, latency, cost, hallucination, reasoning metrics.

Code for NAACL paper When Quantization Affects Confidence of Large Language Models?

Enterprise-grade LLM evaluation framework | Multi-model benchmarking, honest dashboards, system profiling | Academic metrics: MMLU, TruthfulQA, HellaSwag | Zero fake data | PyPI: llm-benchmark-toolkit | Blog: https://dev.to/nahuelgiudizi/building-an-honest-llm-evaluation-framework-from-fake-metrics-to-real-benchmarks-2b90

LLM hallucination evaluation pipeline: two Claude models scored on TruthfulQA (817 questions, 38 categories) via AWS Bedrock and DeepEval LLM-as-judge, with per-category failure analysis and model comparison

A hallucination detection pipeline for Large Language Models (LLMs).

A deterministic honesty layer for LLM agents + a reproducible TruthfulQA eval that measures calibration honestly — reporting the wins and the costs.

LOGIC-HALT: multi-signal LLM hallucination detection (DeBERTa NLI, self-consistency, entropy, NCD) tuned with Optuna — evaluated on TruthfulQA

Does inter-agent LLM disagreement signal hallucination? An 817-question TruthfulQA study with judge-based evaluation and a fair single-model baseline.

Does instruction tuning make language models more sycophantic? A paired causal study across Qwen, Llama, and Gemma on TruthfulQA, showing the effect is family-dependent in both magnitude and direction. 7,200 evaluations, 12 ATE estimates with paired t-tests and bootstrap CIs.

Evaluating how semantically equivalent prompt variations affect LLM predictions and uncertainty estimates.

Evaluation of Llama-3.1-8B Base vs Instruct on TruthfulQA using few-shot prompting and automatic judge models

A hybrid 4-stage pipeline for detecting and mitigating hallucinations in Large Language Models using RAG retrieval, self-consistency sampling, uncertainty estimation, and NLI-based verification. Evaluated on TruthfulQA.

Official code for "From Fact to Judgment: Investigating the Impact of Task Framing on LLM Conviction in Dialogue Systems" (IWSDS 2026)

A tool to evaluate and compare local LLMs running on Ollama or LM Studio under identical conditions using deepeval's public benchmarks (MMLU, TruthfulQA, GSM8K).

Prototype that asks several LLMs to review an answer and flags likely hallucinations, with a balanced TruthfulQA evaluation.

Transformer transfer learning for sentiment analysis and commonsense reasoning: RoBERTa, PIQA, TruthfulQA, WinoGrande, and semantic similarity. PyTorch/Hugging Face coursework, ECE NTUA (2023).

Adaptive, probe-controlled activation steering that cuts LLM hallucination rate by ~10pp (64.6%→58.5%) on TruthfulQA — steers only when a real-time risk probe flags danger, unlike fixed-strength ITI/CAA/TSV baselines reproduced here for comparison.

Your LLM judge says a model is wrong. Can it show the false sentence? Evidence-backed grading, checked against human labels.

PT-GAT Transformer Diagnostics: task-relative hallucination diagnosis with adequacy triggers, evidence conditioning, and anti-collapse baselines.

Multilingual hallucination evaluation framework for Large Language Models across Indian languages using TruthfulQA, NLLB-200, and mechanistic interpretability.