Improving Deep Agents with Harness Engineering — LangChain
FreeTerminalBench 2.0 case study: 52.8% → 66.5%, same model
About Improving Deep Agents with Harness Engineering — LangChain
This blog post from LangChain details a case study on improving agent performance through harness engineering. The authors describe how they improved their coding agent (deepagents-cli) from 52.8% to 66.5% on the TerminalBench 2.0 benchmark (moving from Top 30 to Top 5) by only modifying the agent's harness—system prompt, tools, and middleware—while keeping the underlying model (GPT-5.2-Codex) fixed. The post introduces a repeatable trace analysis skill that uses LangSmith to automatically analyze failure modes across experiment runs, and highlights the importance of self-verification loops where agents test and correct their own outputs. It also discusses trade-offs such as overfitting and the need for generalization.
Key Features
Pros & Cons
- Self-verification significantly boosts agent performance without model changes
- Trace analysis skill saves hours of manual debugging
- Methodology is repeatable and can be generalized to other agent tasks
- Uses real-world observability data (LangSmith) to drive improvements
- Changes that overfit to specific tasks can cause regressions on other tasks
- Requires careful generalization and human validation in the feedback loop
- Models remain black-box, limiting interpretability of internal reasoning
- Benchmarks like TerminalBench 2.0 may not capture all real-world scenarios