$π\mathbf{R}^2$: Reactive Real-time Flow Policies

Analysis

Why This Paper Matters

Generalist manipulation policies have recently adopted action-chunking flow architectures built on large pretrained backbones. While these models achieve impressive multi-modal action prediction, they execute chunks open-loop, meaning the policy cannot react to new sensory input until the entire chunk finishes. This lack of reactivity makes them unsuitable for dynamic, closed-loop control where fast adaptation is critical. Replanning more frequently would restore reactivity, but the perception-to-action pipeline—a large backbone plus multiple denoising steps—introduces prohibitive latency. πR² directly addresses this tension by making such policies both reactive and real-time without discarding the benefits of large backbones or multi-action prediction.

The paper is significant because it offers a practical, minimal-modification solution that can be applied to existing pretrained policies. By finetuning GR00T-N1.7 on a real robot platform, the authors demonstrate that πR² achieves ~25 Hz closed-loop replanning, a 4× improvement over the base policy. This brings large-scale pretrained manipulation policies closer to real-world deployment where latency and reactivity are paramount.

Technical Contributions

• Dual-channel conditioning: πR² separates conditioning into a fast channel (proprioception, updated every control tick) and a slow channel (vision-language features, updated asynchronously). This allows the policy to react to proprioceptive changes within a chunk while tolerating stale visual information.
• Latency-adaptive flow schedule: The method treats in-flight actions as inpainting conditioning and emits actions in a single denoising step per call. This lets a single trained model adapt to varying hardware latency without retraining.
• Minimal architectural changes: πR² can be finetuned from a pretrained policy (e.g., GR00T-N1.7) with only minor modifications, making it easy to adopt in existing pipelines.
• Real-time closed-loop control: On an xArm6+XHand platform with an A5000 GPU, the system replans every 40 ms, achieving ~25 Hz operation.

Results

• Replanning speed: πR² achieves ~25 Hz closed-loop replanning on real hardware, approximately 4× faster than the base GR00T-N1.7 policy.
• Simulation success rate: Up to 23% improvement over the strongest baseline across various manipulation tasks.
• Real-world success rate: Up to 30% improvement over the strongest baseline on the xArm6+XHand platform.
• Observation cycle: The system acts on a fresh observation every 40 ms, enabling tight closed-loop control.

Significance

πR² bridges the gap between large-scale pretrained manipulation policies and real-time reactive control. By enabling frequent replanning without sacrificing model expressiveness, it opens the door for deploying generalist robot policies in dynamic environments where quick adaptation is essential. The approach is architecture-agnostic and can be applied to any action-chunking flow policy, potentially accelerating the adoption of such models in industrial and service robotics. Future work could explore extending the dual-channel idea to other sensory modalities or handling cases where both channels become stale.