ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation

Analysis

Why This Paper Matters

ERFNet addresses a critical bottleneck in deploying deep neural networks for real-world autonomous driving: the tradeoff between segmentation accuracy and inference speed. Prior state-of-the-art models like DeepLab or PSPNet achieved high accuracy but required heavy computational resources, making them unsuitable for real-time embedded systems. ERFNet's innovation lies in showing that a carefully designed lightweight architecture can match the accuracy of much larger models while running at over 80 FPS on a GPU and even 7 FPS on an embedded Jetson TX1. This makes it a landmark paper for practical intelligent vehicle perception, where both accuracy and latency are paramount.

The paper also contributes to the broader trend of efficient neural network design, particularly for dense prediction tasks. By combining residual connections with factorized convolutions, ERFNet provides a template for building efficient architectures that can be adapted to other real-time vision tasks beyond segmentation.

Technical Contributions

• Efficient Residual Factorized (ERF) Layer: The core innovation is a layer that decomposes a standard 3x3 convolution into a 3x1 followed by a 1x3 convolution (factorization), reducing parameters by 33%. Residual connections are added to maintain gradient flow and accuracy.
• Non-bottleneck Design: Unlike ResNet's bottleneck blocks, ERFNet uses a non-bottleneck design with two 3x3 convolutions replaced by factorized versions, preserving spatial resolution and feature richness.
• Downsampling Block: A specialized block for downsampling that uses factorized convolutions with stride 2 and max pooling, avoiding information loss common in strided convolutions.
• Encoder-Decoder Architecture: The network follows an encoder-decoder structure where the encoder uses ERF layers for feature extraction and the decoder upsamples with transposed convolutions, keeping the overall design simple and efficient.

Results

• Cityscapes test set: 68.3% mean IoU (comparable to DeepLab v2 at 70.4% but much faster).
• Speed: 83 FPS on a single Titan X (512x1024 input), 7 FPS on Jetson TX1 (embedded).
• Comparison: Outperforms ENet (51.3% mIoU, 76.9 FPS) and SegNet (57.0% mIoU, 16.7 FPS) in both accuracy and speed tradeoff.
• Ablation studies: Show that the factorized residual design improves accuracy by 2-3% over plain factorized convolutions without residuals.

Significance

ERFNet's impact extends beyond autonomous driving. It demonstrates that efficient architectural design can achieve state-of-the-art accuracy without relying on heavy backbones or complex multi-scale processing. This principle has influenced subsequent efficient segmentation networks like ICNet and BiSeNet. The paper also highlights the importance of targeting embedded platforms (Jetson TX1) for real-world deployment, setting a benchmark for mobile-friendly semantic segmentation. Its open-source release has facilitated widespread adoption and further research in efficient deep learning for perception.