Parameter-efficient fine-tuning for pre-trained vision models: A survey and benchmark

Analysis

Why This Paper Matters

Parameter-efficient fine-tuning (PEFT) has become crucial for adapting large pre-trained models to downstream tasks without prohibitive computational costs. This paper addresses the growing need for a systematic understanding of PEFT methods in the vision domain, where such surveys have been less common than in NLP. By providing both a taxonomy of existing approaches and a controlled benchmark, the paper helps practitioners navigate the landscape of PEFT techniques and make informed decisions based on empirical evidence.

The timing of this work is significant as vision models grow in size and complexity, making full fine-tuning increasingly impractical. The survey consolidates knowledge from disparate sources, while the benchmark offers reproducible comparisons that can guide both research and deployment.

Technical Contributions

The paper's main technical contributions include:

• A structured taxonomy of PEFT methods for vision, categorizing approaches such as adapter modules, prompt tuning, and low-rank adaptations.
• A unified benchmark framework that evaluates multiple PEFT methods on standard vision tasks (e.g., image classification, object detection) using the same pre-trained backbone.
• Analysis of key factors affecting PEFT performance, including the number of trainable parameters, placement of adapters, and interaction with model architecture.
• Identification of best practices and trade-offs, such as when prompt tuning outperforms adapter-based methods.

Results

The benchmark results show that PEFT methods can achieve performance close to full fine-tuning while updating only 1-10% of model parameters. For instance, adapter-based methods typically retain over 95% of full fine-tuning accuracy on classification tasks, while prompt tuning shows competitive results on certain datasets. The paper also reports that the optimal PEFT method varies by task and model size, with no single approach dominating across all settings. Detailed tables compare accuracy, parameter count, and training time for each method.

Significance

This survey and benchmark provide a foundational reference for the vision community, similar to how PEFT surveys have accelerated progress in NLP. By standardizing evaluation, the paper enables fair comparisons and encourages development of more efficient adaptation techniques. The findings have practical implications for deploying large vision models in resource-constrained environments, such as edge devices or applications requiring rapid task switching. Future work can build on this benchmark to explore new PEFT designs and extend the analysis to video and multimodal models.