Accelerating Diffusion LLMs

Analysis

Why This Paper Matters

Diffusion-based large language models (dLLMs) offer a promising alternative to autoregressive models by enabling parallel token generation through iterative denoising. However, existing parallel decoding strategies rely on fixed, input-agnostic heuristics like confidence thresholds, which fail to adapt to input-specific characteristics, leading to suboptimal speed-quality trade-offs. This paper tackles this limitation by introducing a learned, adaptive filter that predicts which tokens are correctly predicted, thereby approximating an oracle decoding strategy. This is significant because it moves beyond static heuristics to a data-driven approach that can dynamically adjust decoding behavior, potentially unlocking the full throughput benefits of dLLMs.

The proposed method, Learn2PD, is particularly notable for its efficiency: the filter model is trained in a post-training manner requiring only minute-level GPU time, making it practical for real-world adoption. Additionally, the introduction of End-of-Text Prediction (EoTP) addresses the often-overlooked issue of redundant decoding of padding tokens, further improving efficiency. These contributions collectively address critical bottlenecks in dLLM inference, making them more viable for production use.

Technical Contributions

• Learn2PD Framework: Trains a lightweight filter model to predict, for each token position, whether the current prediction matches the final output. This filter approximates an oracle that unmasks tokens only when correctly predicted, enabling adaptive parallel decoding.
• Post-Training Optimization: The filter is learned in a post-training manner, requiring only a small amount of computation (minute-level GPU time), which is a stark contrast to full model retraining.
• End-of-Text Prediction (EoTP): A mechanism to detect decoding completion at the end of the sequence, avoiding redundant computation on padding tokens and further speeding up inference.
• Compatibility with KV-Cache: The method can be combined with KV-Cache to achieve even greater speedups, demonstrating its flexibility.

Results

The experiments on the LLaDA benchmark show that Learn2PD achieves up to 22.58x speedup without any performance drop, and up to 57.51x when combined with KV-Cache. These results highlight the effectiveness of the adaptive filter compared to fixed heuristics, which likely yield lower speedups or degrade quality. The ability to maintain performance while dramatically accelerating inference is a key achievement.

Significance

This work has significant implications for the deployment of diffusion LLMs in latency-sensitive applications such as real-time chat, code generation, and interactive AI systems. By enabling faster parallel decoding without sacrificing quality, Learn2PD could make dLLMs more competitive with autoregressive models in practical settings. The adaptive approach also opens avenues for further research into learned decoding strategies, potentially leading to more intelligent and efficient inference methods across various generative models.