AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design

Analysis

Why This Paper Matters

AutoDesign addresses a critical gap in current AI agent systems: the lack of recursive self-improvement. Most existing agentic frameworks are static, relying on fixed prompts or hand-crafted harnesses that do not adapt based on experience. AutoDesign introduces a meta-harness optimization approach that allows the system to learn from its own rollouts, aligning with human design priors and accumulating reusable experience. This is a significant step toward truly autonomous AI systems that can improve themselves over time.

The paper also introduces PosterBench, a new benchmark for evaluating long-horizon agentic design tasks. By focusing on the concrete task of converting academic papers into posters, the authors provide a realistic and challenging testbed that requires multimodal understanding, structured output generation, and iterative refinement. This benchmark will likely become a standard for evaluating future agentic design systems.

Technical Contributions

• Meta-Harness Optimization: The core innovation is a meta-harness optimizer that iteratively improves the harness (the system that controls the code agent) based on rollout feedback. This is a form of meta-learning applied to agentic systems, enabling recursive self-improvement.
• Human Design Priors: The framework explicitly aligns with human design principles, ensuring that the generated posters are aesthetically pleasing and follow standard design conventions.
• PosterBench Benchmark: A new benchmark with 100 papers across five disciplines, providing a diverse and realistic evaluation set. The mini version allows for controlled experiments with different code-agent-model configurations.
• Autonomous Long-Horizon Loop: The system operates fully autonomously, executing 253 tool calls and 11 editing turns within 40 minutes, demonstrating practical efficiency and cost-effectiveness (under $3).

Results

AutoDesign achieves the highest score of 78.32 on the PosterBench Main Track, surpassing the closed-source commercial system Claude Design by 7.45 points. Across seven controlled code-agent-model configurations, integrating the learned DesignHarness consistently improves performance, increasing the average PosterBench Score from 54.99 to 67.39, a +12.4% improvement. In a system-blind human study, AutoDesign receives the highest human preference among evaluated systems, and its outputs reach average conference-poster quality. These results highlight the effectiveness of the meta-harness optimization approach.

Significance

The broader impact of AutoDesign extends beyond paper-to-poster generation. The concept of meta-harness optimization can be applied to any long-horizon agentic task, such as software development, scientific discovery, or content creation. By enabling systems to learn from their own experience, AutoDesign paves the way for AI agents that become more capable over time without human intervention. The introduction of PosterBench also provides a standardized evaluation framework, encouraging further research in this area. As AI systems become more autonomous, frameworks like AutoDesign will be crucial for ensuring they can adapt and improve in dynamic environments.