Improving Vision-language Models with Perception-centric Process Reward Models

Analysis

Why This Paper Matters

Vision-language models (VLMs) have made remarkable progress in tasks like image captioning and visual question answering, but they often struggle with complex reasoning that requires multiple steps. Traditional reward models provide only outcome-level feedback, which is insufficient for guiding intermediate reasoning. This paper addresses a critical gap by introducing perception-centric process reward models (PRMs) that offer step-level supervision grounded in visual information. This is particularly important for applications where reasoning transparency and correctness are paramount, such as medical imaging or autonomous driving.

The work draws inspiration from the success of process reward models in mathematical reasoning, but extends the concept to the multimodal domain. By focusing on perception-centric signals, the authors ensure that each reasoning step is not only logically sound but also visually grounded. This dual emphasis on logical coherence and perceptual accuracy makes the approach uniquely suited for vision-language tasks.

Technical Contributions

• Perception-centric process reward models: Novel reward functions that evaluate intermediate reasoning steps based on their alignment with visual evidence.
• Step-level supervision: Provides fine-grained feedback during training, enabling VLMs to learn more robust reasoning chains.
• Integration with existing VLMs: The proposed PRMs can be plugged into various VLM architectures without major modifications.
• Benchmarking: Extensive experiments on multiple vision-language reasoning datasets demonstrate consistent improvements.

Results

The paper reports significant gains over baseline VLMs and existing reward model approaches. On the VQA-v2 dataset, the method achieves an accuracy improvement of 3.2% over the best baseline. On the NLVR2 dataset, it outperforms prior work by 2.8% in accuracy. The authors also show that the PRM-based training leads to more interpretable reasoning chains, as validated by human evaluation.

Significance

This research opens a new direction for improving VLMs through process-level supervision. By making reasoning steps more transparent and grounded, it enhances trustworthiness and reliability. The approach could be extended to other multimodal domains, such as video understanding or robotics, where step-by-step reasoning is essential. Furthermore, it provides a framework for combining logical reasoning with perceptual grounding, which is a key challenge in AI.