SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD

Analysis

Why This Paper Matters

This paper addresses a critical gap in the AI infrastructure landscape: the ability to efficiently post-train trillion-parameter Mixture-of-Experts (MoE) models on non-GPU hardware. As the industry diversifies beyond NVIDIA GPUs, the Ascend NPU SuperPOD represents a significant alternative, but its adoption has been hindered by the lack of optimized training recipes. The authors present a comprehensive solution that not only achieves competitive Model FLOPs Utilization (MFU) but also demonstrates a full-stack pathway from infrastructure optimization to domain-specific model specialization.

The focus on Operations Research (OR) tasks is particularly timely. OR problems—such as routing, scheduling, and resource allocation—are ubiquitous in logistics, manufacturing, and finance, yet they remain challenging for general-purpose LLMs. By combining efficient post-training with solver-verified synthetic data, the paper shows how frontier models can be grounded in mathematical reasoning, moving beyond pattern matching to genuine problem-solving.

Technical Contributions

The paper's technical innovations are organized into three layers:

• Model-level parallelism: Optimized tensor and pipeline parallelism strategies tailored for the Ascend NPU architecture, addressing memory pressure and communication bottlenecks inherent in MoE models.
• Computation-communication orchestration: Techniques to overlap communication with computation, reducing non-overlapped overhead and improving overall throughput.
• Low-level kernel execution: Custom kernel optimizations for Ascend NPUs, including efficient attention and MoE routing implementations.
• Domain-specific data pipeline: A CPT and SFT workflow that combines collected OR domain resources with solver-verified synthetic optimization documents, yielding 10K high-quality SFT samples spanning four task categories and three problem representations.

Results

The system achieves a 34.22% MFU, representing a 2.93x improvement over the open-source baseline recipe, while maintaining training stability. On the OR benchmark, the specialized DeepSeek-V4-Flash model achieves the highest average zero-shot Pass@1 score among evaluated models at 71.81%, outperforming GPT-5.4-Mini by 3.98 percentage points and the base DeepSeek-V4-Flash model by 11.27 percentage points. These results demonstrate that domain-specific post-training on Ascend infrastructure can yield state-of-the-art performance on complex reasoning tasks.

Significance

This work has broad implications for the AI field. First, it provides a practical blueprint for organizations invested in Ascend NPU infrastructure to train and deploy trillion-parameter models efficiently. Second, it demonstrates that domain-specialized post-training—particularly with solver-verified synthetic data—can significantly boost performance on mathematical reasoning tasks, suggesting a path toward more reliable AI systems for operations research and other structured problem domains. Finally, the full-stack approach from hardware optimization to application-specific fine-tuning serves as a model for future work aiming to bridge the gap between large-scale training infrastructure and real-world deployment.