Recently Added
3,528 documents
Week 2 Requirements - Completion Status
Tracks completion status of 13 project requirements for a chatbot with LLM and RAG, marking 8 as done and 5 as partial or missing.
Landing Page Copy: Claru
Provides a complete set of copy and structure for a landing page that positions Claru as an embedded expert intelligence service for AI labs.
Testing
Documents a 3,292-test suite for a markdown vault tool, covering performance, concurrency, fuzzing, security, and retrieval benchmarks against HotpotQA and LoCoMo.
Day 4 - Podcast Transcript
Summarises a white paper on evaluating AI agent quality, covering pillars, observability, and a continuous improvement flywheel.
Analysis of Paper 50
Structured analysis of a research paper's use of BigCloneBench, answering 12 specific questions about dataset usage and validity.
Page Architecture: Claru
Defines a seven-section landing page layout with animated hero, comparison grid, testimonials, and a terminal-style progressive disclosure form.
Advanced RAG Techniques
Surveys advanced RAG architectures, Agentic, Corrective, Self-RAG, Graph RAG, RAPTOR, Multimodal, Temporal, with design trade-offs and decision guidance.
Training Readiness Checklist
Lists five gates to verify before launching a training or tuning run, with checkboxes for each item.
-PLUIE: Personalisable metric with Llm Used for Improved Evaluation
Lists 16 arXiv papers on LLM evaluation, fine-tuning, and interpretability, each with title, authors, link, and abstract.
From Heuristics to Hybrid: A Methodology for Building a Testable, Sequential Log Anomaly Detection Engine
Describes a sequential rule-then-ML log anomaly detector and a manual golden set evaluation to avoid circular validation.
Weeks 3–4: Production RAG & Evaluation
Teaches production RAG design, chunking, embedding, vector search, reranking, and evaluation with RAGAS and LangSmith over two weeks.
Quick start: Evaluation sets
Walks through creating an evaluation set of questions and optional ground-truth answers to measure RAG application quality before stakeholder review.
Evaluation Findings (Sample Run)
Reports results from two evaluation pipelines: agent response scoring and recommendation accuracy, with aggregate metrics and improvement steps.
Configuration Reference
Guides an LLM Evaluator agent through designing evaluation frameworks, test plans, and quality gate decisions for AI systems.
Superpipe Studio
Introduces a free observability and experimentation app for Superpipe pipelines, with logging, dataset management, and experiment tracking.
Repository Intelligence: Building the Next Generation of Agent Evaluation Data
Describes a pipeline for generating synthetic evaluation data that tests repository-level agent capabilities across QA, code generation, and debugging tasks.
── 数据结构定义 ──────────────────────────────────────────────
Explains why agent evaluation is harder than model evaluation and provides a five-dimension framework plus a runnable Python evaluation class.
Evaluation Strategy
Defines a four-tier evaluation taxonomy for a clinical reasoning system, prioritizing deterministic checks over LLM-as-judge for safety and correctness.
Agent Eval - Complete UI Flow Visualization
Maps every screen of an agent evaluation application to its URL, UI layout, and the evaluation features it implements.
LLM Judge — Setup & Operations
Explains how to enable and configure a three-tier LLM judge cascade for prompt-injection detection, with shadow-mode rollout and golden-set calibration.
LLM Evaluation Overview
Surveys evaluation methods for LLMs: automated metrics, human eval, AI-as-judge, and task-specific checks, with guidance on dataset design and cadence.
When "Better" Prompts Hurt: Evaluation-Driven Iteration for LLM Applications
Presents an evaluation-driven workflow (Define-Test-Diagnose-Fix) for LLM applications and demonstrates that generic prompt improvements can degrade task-specific performance.
LLM Evaluation & Metrics — Complete Guide
Covers the full spectrum of LLM evaluation metrics, from perplexity and BLEU to RAGAS and LLM-as-judge, with production pipeline code.
Traditional function - easy to test
Introduces AI model evaluation concepts, metrics, and code examples for measuring accuracy, reliability, robustness, efficiency, and safety.