Eric Fett — NGP Capital - Evals Are the New Moat—And RL Turns Them into Your Product Advantage - October 2025
FreeEvals Are the New Moat—And RL Turns Them into Your Product Advantage
FreeFree tier
About Eric Fett — NGP Capital - Evals Are the New Moat—And RL Turns Them into Your Product Advantage - October 2025
This article by Eric Fett argues that as AI scaling laws face diminishing returns, the competitive advantage shifts from model size to post-training techniques like reinforcement learning (RL) and evaluation frameworks. It explains how RL uses evals as objective functions to align model behavior, turning raw capability into reliable, task-specific performance. The piece highlights the emerging 'EvalOps' frontier, the use of LLM-as-a-judge for autonomous evaluation, and a real-world example of a foundation model designing a novel antibiotic. It positions mercor's rapid growth as market validation.
Key Features
Reinforcement learning as post-training technique
Evaluation frameworks (evals) as objective functions
LLM-as-a-judge for autonomous orchestration of evals
EvalOps and democratized fine-tuning
Agentic tools for reliable workflow execution
Pros & Cons
Pros
- Turns black-box model output into explainable, auditable results
- Improves reliability and reduces hallucination via RL
- Enables task-specific performance without requiring larger base models
- Creates a concrete framework for measuring AI accuracy and cost
Cons
- Compute-hungry approach (reportedly 100x more compute than basic training)
- Requires significant investment in evaluation data pipelines and infrastructure
Best For
Drug discovery: AI-designed antibiotics validated in miceEnterprise AI: fine-tuning models for industry-specific workflowsAgentic evaluation: scoring each step in complex decision chains
FAQ
Why is reinforcement learning important for AI products?
RL allows models to learn from rewards and penalties, turning evaluation scores into objective functions that align behavior with desired outcomes, reducing hallucinations and improving reliability on complex tasks.
What is EvalOps?
EvalOps refers to the operationalization of evaluations, including automated testing, scoring, and feedback loops that enable teams to continuously improve model performance in production.