SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them

Analysis

Why This Paper Matters

Vision-language models (VLMs) are increasingly deployed in embodied agents, where they must interpret visual inputs, reason about spatial relationships, and make task-level decisions. However, a fundamental capability mismatch persists: general VLMs excel at high-level reasoning but often miss fine-grained visual details, while specialist vision models capture those details but cannot translate them into task-level decisions. SpatialCLI directly addresses this gap by proposing a framework that teaches VLMs to use spatial tools and then internalize the specialist perceptual capabilities those tools provide. This is a significant step toward more capable and autonomous embodied agents.

The paper also introduces SpatialCLI-Bench, a 516-example benchmark for compositional perception across localization, segmentation, depth, and pose. This benchmark fills a gap in evaluating VLMs' spatial reasoning abilities in a structured way, which is crucial for progress in embodied AI. The results on MindCube are striking: SpatialCLI boosts Qwen3-VL-8B-Instruct from 29.3% to 84.6% with tools, surpassing even GPT-5.6 Sol with tools (72.1%). Moreover, after internalization, the model retains 73.8% accuracy without tools, demonstrating that the learned capabilities are not merely dependent on external tools.

Technical Contributions

• Three-stage framework: SpatialCLI proceeds in three stages: Call, Learn, and Internalize. In the Call stage, specialist vision models are exposed as spatial tools to augment the VLM's perception. In the Learn stage, Cold-Start SFT and agentic RL are used to improve tool use. In the Internalize stage, successful tool-use trajectories are verbalized to internalize specialist perceptual capabilities.
• Cold-Start SFT and agentic RL: The Learn stage combines supervised fine-tuning with reinforcement learning to teach the model when and how to use tools effectively.
• Internalization via verbalization: The Internalize stage converts tool-use trajectories into natural language descriptions, allowing the VLM to learn from them without requiring the tools at inference time.
• SpatialCLI-Bench: A new benchmark with 516 examples covering four spatial perception tasks: localization, segmentation, depth, and pose, enabling compositional evaluation.

Results

On MindCube, SpatialCLI raises Qwen3-VL-8B-Instruct from 29.3% to 84.6% with tools, surpassing GPT-5.6 Sol with tools (72.1%). After internalization, the model retains 73.8% without tools, showing that the internalized capabilities are robust. These results highlight the effectiveness of the three-stage approach in bridging the gap between general VLMs and specialist perception models.

Significance

SpatialCLI offers a practical pathway for enhancing VLMs' spatial reasoning without requiring architectural changes or massive retraining. By leveraging external tools during training and then internalizing their capabilities, the framework can be applied to various VLM backbones and tasks. The introduction of SpatialCLI-Bench also provides a standardized evaluation for compositional perception, which could accelerate research in embodied AI. This work has the potential to influence how VLMs are trained for spatial tasks, making them more reliable for real-world applications such as robotics and autonomous navigation.