HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark

Analysis

Why This Paper Matters

Humanoid motion tracking is critical for teleoperation and whole-body imitation, yet existing evaluation metrics often fail to capture what humans perceive as errors. Traditional kinematic metrics, such as average per-frame pose differences, overlook physical artifacts like foot skating and unstable support, which are visually salient and important for real-world performance. This paper addresses a fundamental gap: evaluation that aligns with human perception.

The paper also highlights the inadequacy of current test suites, which are small and lack diversity, especially for contact-rich and long-horizon behaviors. By introducing a large-scale benchmark with 153 hours of professional motion data, the authors enable more robust and comprehensive evaluation. This is a significant step toward making humanoid tracking evaluation both perceptually meaningful and scalable.

Technical Contributions

• HumanTracker Benchmark: A large-scale dataset with ~153 hours of optical motion trajectories from multiple professional performers, organized into four motion families. Text labels enable fine-grained diagnosis of tracking failures.
• HumanScore Metric: A preference-aligned metric trained on 12K motion pairs (24K motions) using human preference annotations. This metric learns to predict human judgments, providing a more perceptually relevant evaluation than kinematic errors.
• Evaluation Framework: The benchmark and metric together offer a scalable way to stress-test trackers on contact-rich, long-horizon behaviors, which are often underrepresented in existing suites.
• Diagnostic Capability: The text labels and motion family organization allow for detailed analysis of failure modes, such as foot skating and mistimed touch-downs.

Results

The paper evaluates representative state-of-the-art trackers using HumanScore and compares it with kinematic metrics. The key finding is that HumanScore better predicts human preferences, indicating that it captures perceptual quality more accurately. Moreover, HumanScore reveals contact and stability failures that kinematic metrics often miss, demonstrating its diagnostic power. While specific numerical metrics are not provided in the abstract, the qualitative improvement in alignment with human perception is clear.

Significance

This work has broad implications for the humanoid robotics and AI communities. By providing a perceptually aligned evaluation metric, it encourages the development of trackers that produce physically plausible and human-acceptable motions, which is crucial for teleoperation and imitation learning. The benchmark's scale and diversity also set a new standard for evaluating long-horizon, contact-rich behaviors. This could accelerate progress in whole-body control and human-robot interaction, as researchers can now more reliably measure and improve perceptual quality. The approach of using preference learning for evaluation metrics may also inspire similar efforts in other domains where human perception is key.