OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models

Analysis

Why This Paper Matters

Omnimodal large language models (LLMs) that process audio, video, and text simultaneously are becoming increasingly important for applications like video understanding and human-computer interaction. However, their high computational cost, especially during prefill, limits real-world deployment. Token compression is a key technique to reduce this cost, but existing methods often rely on one modality to guide pruning of another. This paper identifies a critical flaw in that assumption: audio and video relevance to a query often peak at different moments, so unidirectional guidance can discard answer-critical cues.

OmniScope addresses this by decoupling the salience estimation for each modality while using the query as a shared semantic anchor. This is a simple yet powerful insight that challenges the common practice of cross-modal guidance. The training-free nature of the approach makes it immediately applicable to existing models without fine-tuning, which is highly valuable for practitioners.

Technical Contributions

• Cross-modal salience mismatch: The paper demonstrates that audio and video relevance to a query are often misaligned in time, making unidirectional compression suboptimal.
• Modality-decoupled relevance estimation: OmniScope computes relevance scores for audio and video tokens independently, using the query as a shared anchor, avoiding cross-modal interference.
• Modality-specific token budgets: Instead of a global budget, OmniScope allocates separate budgets for audio and video, allowing each modality to retain its most informative tokens.
• Anchor-delta visual pruning: For visual tokens, OmniScope uses an anchor-delta strategy that preserves both global context (anchor) and temporal changes (delta), ensuring that important dynamic information is not lost.
• Audio token merging: Audio tokens are merged within each second to reduce redundancy while maintaining temporal continuity, which is crucial for understanding speech and sound events.

Results

OmniScope was evaluated on four audio-video benchmarks and two scales of Qwen2.5-Omni. Across all compression settings, it achieved the best average accuracy compared to existing methods. At 25% overall token retention, it delivered up to 3.53x prefill speedup and more than 15% GPU memory reduction, with only a 0.35-point drop in average accuracy. This demonstrates that significant efficiency gains can be achieved with minimal performance loss, making OmniScope a practical solution for real-time applications.

Significance

The design principle of sharing the query but not salience estimates across modalities is a valuable contribution to the field of multimodal LLM efficiency. It suggests that treating each modality's relevance independently can lead to better compression outcomes. This work opens up new directions for research in modality-aware token compression and could be extended to other multimodal models and tasks. The code being open-sourced further accelerates adoption and reproducibility.