Low-bit model quantization for deep neural networks: A survey

Analysis

Why This Paper Matters

Model quantization is a critical technique for deploying deep neural networks on edge devices, mobile phones, and other resource-constrained hardware. As AI models grow larger and more complex, reducing their memory footprint and computational cost without sacrificing accuracy becomes paramount. This survey paper provides a timely and comprehensive overview of low-bit quantization methods, which are essential for achieving efficient inference in production systems.

The paper's curated list of resources and systematic categorization of quantization approaches make it a valuable starting point for both newcomers and experienced practitioners. By summarizing the state of the art, it helps the community understand which techniques work best under different constraints, such as bit-width, hardware target, and accuracy requirements.

Technical Contributions

• Taxonomy of quantization methods: The paper organizes techniques by quantization bit-width (e.g., 8-bit, 4-bit, 2-bit, binary), granularity (per-tensor, per-channel, per-group), and training strategy (post-training quantization vs. quantization-aware training).
• Comprehensive resource list: Includes a curated GitHub repository with links to papers, code, and benchmarks, facilitating reproducibility and further research.
• Discussion of trade-offs: Highlights the accuracy-compression trade-off, noting that 4-bit and 8-bit quantization often retain near-full-precision accuracy, while lower-bit methods require more sophisticated techniques like mixed-precision or learned quantization.
• Identification of open challenges: Points out issues such as handling outliers, hardware-specific constraints, and the need for standardized benchmarks.

Results

The survey does not present new experimental results but aggregates findings from the literature. Key observations include: 8-bit quantization is now standard and often lossless; 4-bit quantization can achieve <1% accuracy drop on ImageNet with quantization-aware training; 2-bit and binary methods typically suffer 3-10% accuracy degradation, though recent advances in ternary and binary networks are narrowing the gap.

Significance

This survey has broad impact by consolidating knowledge in a rapidly evolving field. It enables AI practitioners to make informed decisions about which quantization method to adopt for their specific deployment scenario. By highlighting open problems, it also guides future research toward areas that need further innovation, such as hardware-aware quantization and automatic mixed-precision selection. Ultimately, this work supports the broader goal of making deep learning accessible on low-power devices, enabling applications in IoT, autonomous systems, and mobile AI.