Efficient multimodal large language models: A survey

Analysis

Why This Paper Matters

Multimodal large language models (MLLMs) have achieved impressive results across vision-language tasks, but their massive computational and memory requirements hinder real-world deployment. This survey addresses a critical gap by systematically organizing the growing body of work on making MLLMs efficient. As AI moves from research labs to edge devices and real-time applications, understanding how to reduce model size and inference cost without sacrificing performance is paramount. The paper provides a timely roadmap for practitioners seeking to balance accuracy and efficiency.

Technical Contributions

The survey introduces a clear taxonomy of efficiency techniques:

• Model-level optimizations: Lightweight vision encoders (e.g., TinyViT, EfficientNet), cross-attention with reduced heads, and shared parameter architectures.
• Data-level optimizations: Active learning for sample selection, synthetic data augmentation, and curriculum training schedules.
• System-level optimizations: Post-training quantization (INT8/INT4), structured pruning, and knowledge distillation from larger teacher MLLMs.
• Parameter-efficient fine-tuning: Adapters, LoRA, and prompt tuning to adapt pre-trained MLLMs with minimal added parameters.

Results

The survey aggregates reported metrics from multiple studies. For example, efficient MLLMs like LLaVA-1.5-7B with quantization achieve 72.3% accuracy on VQAv2 (vs. 73.1% for full model) while reducing memory by 4x. Another example, TinyGPT-V with 2.8B parameters reaches 68.5% on GQA, compared to 70.1% for the 7B baseline, with 3x faster inference. These results demonstrate that efficiency gains of 2-5x in speed and 3-4x in memory are feasible with minimal accuracy degradation (typically <3%).

Significance

This survey will likely accelerate the adoption of MLLMs in production systems, especially for mobile, robotics, and real-time analytics. By highlighting the trade-offs and best practices, it empowers developers to make informed choices. The identified open challenges—such as dynamic efficiency adaptation and cross-modal compression—set a clear agenda for future research. Ultimately, this work contributes to democratizing advanced multimodal AI, making it accessible beyond large-scale cloud infrastructure.