All Documents
237 documents available
Marketing Audit & Benchmarking Module - Complete Feature List
Lists 280+ features for a dental CRM marketing audit and benchmarking module, all marked 100% complete and production-ready.
Intelligent Research Assistant - Technical Documentation
Documents the architecture, components, API endpoints, and security features of a multi-agent research assistant platform.
Evaluation of RAG Systems + Presentation Outline
Outlines evaluation strategies for a RAG system and proposes a presentation structure for a project on international student queries.
Research Report: Using LLMs as Oracle for Entity Matching Ground Truth
Summarises research on using LLMs (DeepSeek, GPT-4, Claude) for entity matching ground truth, covering accuracy, prompt engineering, ensembles, cost analysis, validation, and regression test conversion.
Evaluation Framework
Defines a three-layer evaluation system for an AI investment assistant: guardrails, online RAGAS scoring, and offline golden dataset regression tests.
Day 20: Evaluation & Benchmarks 📏
Teaches evaluation of generative AI outputs using n-gram metrics, LLM-as-judge, and RAGAS, with runnable code examples for each technique.
[BEE-30004] Evaluating and Testing LLM Applications
Explains why traditional testing fails for LLMs and gives a practical evaluation strategy combining offline metrics, LLM-as-judge, red-teaming, and production monitoring.
GenAI Benchmarks & Evaluation — Product-Based Companies
Understanding how to **benchmark, evaluate, and compare LLMs** is essential for roles at Google, OpenAI, Anthropic, Cohere, and AI research teams. This file covers the most important benchmarks, evaluation methodologies, and how to build custom evaluation harnesses.
After LangGraph node execution, convert messages
Explains how to integrate RAGAS evaluation metrics into a LangGraph RAG pipeline, covering offline and online evaluation strategies.
Domain 5: Testing, Validation, and Troubleshooting
Maps the AIP-C01 exam Domain 5 topics, evaluation systems, metrics, and troubleshooting, into a structured study guide with code examples and exam tips.
Topic: Evaluation & Benchmarking
Surveys evaluation and benchmarking for LLMs, covering metrics, methods, pipelines, and trade-offs across offline and online settings.
Using Performance Metrics to Evaluate RAG Systems
Explains how to evaluate RAG systems using Top-K parameter optimization and auto prompt optimization with Qdrant and Relari.
RAG System Testing Methodologies: A Comprehensive Guide
Surveys RAG evaluation frameworks, metrics, and testing methodologies with implementation guidance and tool comparisons.
Data-Driven RAG Evaluation: Testing Qdrant Apps with Relari AI
Explains how to evaluate RAG applications using Qdrant and Relari, covering Top-K parameter tuning and Auto Prompt Optimization with code examples.
Instructions for Claude Code: n8n Meal Feedback LLM Evaluation Workflow
Guides building an n8n workflow that uses a thinking model to generate ground truth answers and evaluates cheaper prompts against them.
Using Performance Metrics to Evaluate RAG Systems
Walks through evaluating RAG systems with Qdrant and Relari, covering Top-K parameter tuning and auto prompt optimization.
EGG Rubric: Corporate Sustainability Evaluation Framework
Provides a detailed scoring rubric for evaluating corporate sustainability across five themes using commitment strength, specificity, and temporal evolution.
Agent Evaluation Reference Guide
Documents the agent-eval CLI for evaluating ADK agents with deterministic and LLM-as-judge metrics.
RAG Evaluation Guide
Defines a repeatable workflow to evaluate RAG retrieval and answer quality across multiple retriever configurations using RAGAS and custom metrics.
PRD-010 — Evaluation Framework
Specifies how agent output quality is measured, scored, and gated in CI using LangSmith evaluation primitives.
The Evals Gap
Explains why LLM non-determinism breaks traditional testing and demonstrates the effect with a temperature experiment.
Evaluation Metrics for Language Model Comparative Analysis
Defines three metrics, Accuracy, Bug Frequency, Feature Completeness, for evaluating language model generated Python game scripts.
NEAR Protocol Project Scoring Rubric
Defines an 8-category scoring rubric for evaluating NEAR Protocol projects, each with point ranges and AI evaluation prompts.
Text Retrieval Benchmark
Runs a text retrieval benchmark using BEIR datasets, measuring recall, precision, NDCG, and MRR without LLM generation.