LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks (April 2026)

LiveClawBench is a benchmark designed to evaluate LLM agents on complex, real-world assistant tasks with dual fidelity: faithfulness both to the distribution of real assistant tasks and to the execution semantics of the environments in which those tasks unfold. It addresses the limitations of existing benchmarks that often lose fidelity by underrepresenting real-world difficulties such as cross-service dependency, contaminated state, implicit intent, and runtime change, or by using endpoints that remove sessions, artifacts, state transitions, and downstream side effects. LiveClawBench combines a Triple-Axis Complexity Framework for difficulty-driven task construction with reproducible full-stack mock applications that preserve stateful execution semantics. It includes 134 executable cases across 10 domains with 22 mocked services, supporting controlled, extensible, and factor-level diagnostic evaluation of realistic agentic tasks. The benchmark resources (benchmark code, leaderboard, and trajectories) are publicly released.