PAC-MAN: Perception-Aware CBF-RL for Whole-Body Safety in Humanoid Dodgeball

Analysis

Why This Paper Matters

PAC-MAN addresses a critical challenge in legged robotics: achieving whole-body safety in dynamic environments with only onboard perception. While prior CBF-RL methods assume perfect state information, PAC-MAN explicitly couples barrier safety with perception-aware training, making it deployment-realistic. The paper's focus on humanoid dodgeball provides a demanding benchmark that tests reactive evasion, recovery, and multi-throw scenarios, pushing beyond static obstacle avoidance.

The key insight is that the effectiveness of barrier structure depends on perceptual observability. The authors show that Joint-CBF, which uses full body-state information, performs well with accurate ball states but degrades when only fixed-camera observations are available. This highlights a fundamental issue: safety guarantees derived from privileged information may not transfer to real-world perception. By analyzing this degradation and proposing a Link-CBF policy that is robust to imperfect perception, PAC-MAN offers a practical solution for sim-to-real transfer.

Technical Contributions

• Perception-aware CBF-RL framework: Integrates control-barrier functions as training-time guidance, not runtime constraints, allowing the policy to learn evasive reflexes that respect safety while relying on onboard sensing.
• Segmentation-masked depth: The deployed policy uses only depth data masked by semantic segmentation, reducing the perceptual burden and improving generalization to different ball types.
• Adversarial motion prior: Regularizes the policy to produce natural evasive movements, preventing overfitting to specific throw patterns.
• Any-link contact benchmark: A controlled evaluation with seeded throws in single and deployment-loop regimes, measuring contact to any body link.
• Zero-shot real-world deployment: The Link-CBF policy is transferred to a Unitree G1 without fine-tuning, demonstrating practical viability.

Results

In simulation, the PAC-MAN policy achieves performance within a few points of a privileged state oracle, indicating that a fixed onboard camera is adequate for evasion. The paper reports that Joint-CBF performs best with accurate ball states but degrades under fixed-camera observations when used only as training guidance. However, performance recovers with a ball-tracking gimbal or a privileged runtime filter, showing the importance of observability.

In real-world tests on the Unitree G1, the deployed Link-CBF policy succeeds on 95% of throws, tolerating imperfect perception and using semantic segmentation to dodge different balls. This is a strong result, especially given the zero-shot transfer and the dynamic nature of dodgeball.

Significance

PAC-MAN contributes to the growing field of safe reinforcement learning for robotics by demonstrating that perception-aware training can mitigate the sim-to-real gap. The findings on barrier observability provide guidance for designing CBF-based safety layers in systems with limited sensing. The successful real-world deployment on a humanoid robot opens avenues for applications in human-robot interaction, dynamic obstacle avoidance, and agile locomotion. Future work could extend this framework to other perception modalities and more complex environments, potentially leading to safer autonomous systems in unstructured settings.