Harness Engineering — Martin Fowler
FreeArchitecture perspective
FreeFree tier
About Harness Engineering — Martin Fowler
Harness engineering is a mental model for building trust in AI-generated code produced by coding agents. It combines feedforward controls (guides that anticipate and steer agent behavior before action) and feedback controls (sensors that observe output and enable self-correction). These controls can be computational (deterministic, fast—like tests, linters, type checkers) or inferential (semantic analysis, LLM-as-judge). The goal is to increase the probability of correct results on the first attempt, provide self-correction loops, and ultimately reduce human review toil while improving system quality.
Key Features
Feedforward controls (guides) to steer agent behavior before action
Feedback controls (sensors) for self-correction after action
Computational controls: deterministic tests, linters, type checkers
Inferential controls: semantic analysis, LLM-as-judge
Custom linter messages with self-correction instructions (positive prompt injection)
Separation of agent into Model + Harness concept
Pros & Cons
Pros
- Provides a structured framework to build trust in non-deterministic LLM outputs
- Combines deterministic and inferential checks for robust quality assurance
- Enables self-correcting agents that reduce repeated mistakes
- Reduces wasted tokens and human review effort
- Practical guidance derived from real-world experience at Thoughtworks
Cons
- Requires upfront effort to design and implement harness components
- Metaphor of 'harness' can be stretched when applied to nested contexts
- Inferential controls (LLM-as-judge) are slower and more expensive than computational ones
- Not a turnkey tool; requires adaptation to specific agent setups
Best For
Increasing confidence in AI-generated code for production systemsReducing human review burden in AI-assisted developmentImplementing quality gates for coding agents in CI/CD pipelinesDesigning outer harness for specific codebases and contexts
FAQ
What is harness engineering?
Harness engineering is a mental model for coding agent users that combines feedforward controls (guides) and feedback controls (sensors) to increase confidence in AI-generated code. It treats everything around the model as the harness: Agent = Model + Harness.
What are feedforward and feedback controls?
Feedforward controls (guides) anticipate and steer agent behavior before it acts, like system prompts or retrieval mechanisms. Feedback controls (sensors) observe outputs after action and help the agent self-correct, such as custom linter errors with fix instructions.
What is the difference between computational and inferential controls?
Computational controls are deterministic and fast (e.g., tests, linters, type checkers, run by CPU). Inferential controls use semantic analysis (e.g., AI code review, LLM-as-judge) and are slower, more expensive, and less deterministic.