ConvNeXt

Analysis

Why This Paper Matters

For years, Vision Transformers (ViTs) were hailed as the successors to Convolutional Neural Networks (ConvNets), claiming superior accuracy and scalability. However, ConvNeXt flips the narrative by showing that a carefully modernized ConvNet can match or even beat Transformers across image classification, object detection, and semantic segmentation. This paper matters because it decouples performance gains from the attention mechanism itself, attributing much of the success to training recipes and architectural refinements that are equally applicable to ConvNets. For AI practitioners, this means that convolutional architectures remain a powerful, efficient choice, especially when computational resources are constrained.

How It Works

The authors start with a standard ResNet-50 and apply a series of modernizations, grouped into macro and micro design changes.

Training Techniques: First, they adopt the training recipe used for Transformers: 300 epochs, AdamW optimizer, data augmentation (Mixup, Cutmix, RandAugment, Random Erasing), and regularization (Stochastic Depth, Label Smoothing). This alone boosts ResNet-50 accuracy from 76.1% to 78.8%, indicating that training methodology accounts for a large part of the performance gap.

Macro Design:

• Stage compute ratio is adjusted to 1:1:3:1 (matching Swin-T), improving accuracy to 79.4%.
• The stem cell is replaced with a 4×4, stride 4 convolution (patchify layer), yielding 79.5%.
• Depthwise convolutions reduce FLOPs, but accuracy drops; widening the network from 64 to 96 channels recovers performance to 80.5%.

• Inverted bottleneck design (4× expansion ratio) reduces FLOPs and improves accuracy to 80.6%.
• Moving the depthwise conv layer up and using a 7×7 kernel maintains 80.6%.

Micro Design:

• ReLU is replaced with GELU (no change in accuracy).
• Reducing activation functions to one per block (like Transformers) boosts accuracy to 81.3%.
• Reducing normalization layers to one BatchNorm before the 1×1 conv yields 81.4%.
• Substituting BatchNorm with Layer Normalization (LN) achieves 81.5%.
• Separating downsampling with a 2×2 conv and adding normalization at resolution changes stabilizes training and pushes accuracy to 82.0%, surpassing Swin-T's 81.3%.

The final ConvNeXt architecture uses a hierarchical design with four stages, each containing blocks with depthwise conv, LN, GELU, and inverted bottlenecks.

Results

• ConvNeXt variants (T/S/B/L/XL) show scaling behavior similar to Transformers, with ConvNeXt-B at 384² outperforming Swin-B by 0.6% and achieving 12.5% higher inference throughput.

• On COCO, ConvNeXt matches or exceeds Swin Transformer across model sizes, especially for larger models (B/L/XL) in both box and mask AP.

• On ADE20K semantic segmentation, ConvNeXt models are competitive across all capacities, confirming the architecture's generality.

Significance

ConvNeXt challenges the prevailing assumption that attention mechanisms are necessary for state-of-the-art vision performance. By systematically modernizing a classic ConvNet, the authors show that convolutional architectures can be just as effective and scalable as Transformers. For practitioners, this means that ConvNets remain a viable option, especially when hardware constraints or latency requirements favor their efficiency. The paper also provides a clear blueprint for future architectural innovations, emphasizing that training recipes and careful design choices often matter more than the underlying paradigm. ConvNeXt has already influenced subsequent work, such as ConvNeXt V2 and other hybrid models, cementing its place as a landmark in computer vision.