All Documents
3,528 documents available
13-02-PLAN
Defines 100 annotated queries for retrieval evaluation and a CSV export service that writes aggregate and detailed metrics to configurable output.
Configuration Reference
Guides an LLM Evaluator agent through designing evaluation frameworks, test plans, and quality gate decisions for AI systems.
Study Guide: RAG Evaluation (RAGAS-Lite)
Explains a reference-free RAG evaluation framework that scores context relevancy, faithfulness, and answer relevancy using embedding similarity and n-gram overlap.
Repository Intelligence: Building the Next Generation of Agent Evaluation Data
Describes a pipeline for generating synthetic evaluation data that tests repository-level agent capabilities across QA, code generation, and debugging tasks.
評估系統
Documents a multi-layered evaluation system for LLM-generated SEO Q&A, covering LLM-as-Judge, reasoning models, retrieval metrics, maturity assessment, and pipeline step-by-step golden sets.
Evaluation and Observability
Defines evaluation methodology, monitoring signals, and feedback loops for LLM applications in production.
Claim Extraction Evaluation Matrix
Defines a matrix evaluation framework for comparing YouTube claim extraction quality across multiple LLM models and video types.
AI System Evaluation & Testing / Đánh Giá và Kiểm Thử Hệ Thống AI
Teaches a four-level evaluation framework, three LLM testing patterns, regression CI pipeline, and production monitoring for AI features.
Voice AI Leaderboards, Benchmarks, and Evaluation Gaps (Jan 2025 -- Feb 2026)
Surveys 20+ voice AI benchmarks from Jan 2025, Feb 2026, identifies evaluation gaps, and provides leaderboard data for STT, TTS, and end-to-end voice agents.
LLM-as-Judge Reliability Patterns
Documents five reliability patterns for LLM-as-Judge systems and maps them against gaps in the LLMTrace security judge implementation.
LLM Evaluation & Metrics — Complete Guide
Covers the full spectrum of LLM evaluation metrics, from perplexity and BLEU to RAGAS and LLM-as-judge, with production pipeline code.
EVALS.md — LLM & RAG Evaluation Playbook
Guides model selection, regression testing, and production-grade evaluation for LLMs and RAG systems with research-backed practices.
Evaluating AI Agent Systems: Metrics, Benchmarks, and Quality Assurance (2024-2026)
Surveys 2024-2026 metrics, benchmarks, and monitoring tools for evaluating AI agent systems, with recommendations for a self-improving coding agent.
Evalyn Roadmap
Tracks planned and completed features for the Evalyn observability and evaluation framework, organized by category.
RAG Evaluation Frameworks: Comprehensive Analysis for VERA
Surveys RAG evaluation frameworks and metrics, focusing on grounding verification for a verification engine called VERA.
Exact match
Covers eval types, metrics, LLM-as-judge patterns, regression testing, CI integration, and tooling for testing non-deterministic AI systems.
The Evaluation & Optimization Loop
Describes a closed-loop system that generates synthetic data, applies human review, optimizes routing with DSPy, evaluates results, and annotates telemetry for continuous improvement.
When "Better" Prompts Hurt: Evaluation-Driven Iteration for LLM Applications
Presents an evaluation-driven workflow for LLM applications with a Minimum Viable Evaluation Suite (MVES) framework and shows that generic prompt improvements can degrade structured task performance.
RAG Evaluation: Complete Guide to Evaluating Retrieval-Augmented Generation Systems
Teaches how to evaluate RAG systems using faithfulness, relevancy, precision, and recall metrics with code examples.
LLM Evaluation — Interview Grill
Presents 115 active-recall questions across 14 sections to test and reinforce knowledge of LLM evaluation concepts.
Knowledge MCP Query Reference for Evaluation Timing
Teaches you to query a Knowledge MCP server for evaluation placement, methods, anti-patterns, and decision support, with query templates and result interpretation.
Evaluation Discipline: The Missing Loss Function of the Humanities
Argues for rigorous evaluation as a core design discipline, outlining a protocol for building test harnesses to bound failure modes in AI systems.
Design an Evaluation Pipeline for an LLM-Based Product
Designs an end-to-end evaluation pipeline for LLM-based products, covering offline benchmarks, online A/B tests, safety testing, and human evaluation.
evaluation/ — Evaluation Skill
Defines an offline evaluation framework that tests a WHO IMCI classification system against 20+ golden scenarios and produces versioned accuracy reports.