Regularized Evolution for Image Classifier Architecture Search

Analysis

Why This Paper Matters

This paper marks a pivotal moment in neural architecture search (NAS) by demonstrating that evolutionary algorithms can finally outperform human-designed architectures on a major benchmark. Prior to this work, evolutionary NAS had consistently lagged behind both hand-crafted networks and reinforcement learning (RL) approaches. By introducing a simple yet effective regularization technique—age-based selection—the authors show that evolution can not only match but exceed the performance of complex RL methods, often with faster convergence. This is particularly significant for practitioners with limited compute budgets, as the method yields strong results earlier in the search process.

The paper also provides a controlled comparison against RL, offering empirical evidence that evolution is a competitive and simpler alternative. This challenges the prevailing assumption that RL is the default choice for NAS and opens the door for more widespread adoption of evolutionary methods in automated machine learning.

Technical Contributions

• Age-regularized tournament selection: Modifies standard tournament selection by assigning each genotype an age (number of times it survives selection) and favoring younger individuals. This prevents premature convergence and maintains population diversity.
• AmoebaNet-A architecture: The discovered architecture uses a combination of normal and reduction cells, similar to NASNet but evolved rather than RL-optimized.
• Efficient search on CIFAR-10: The search is performed on CIFAR-10 and then transferred to ImageNet, a common paradigm in NAS to reduce computational cost.
• Scalability demonstration: Shows that the evolved architecture scales well to larger models, achieving state-of-the-art results on ImageNet.

Results

• On ImageNet, AmoebaNet-A achieves 83.9% top-1 accuracy and 96.6% top-5 accuracy at larger scale, surpassing previous state-of-the-art models.
• When matched in size, AmoebaNet-A has comparable accuracy to models discovered by more complex NAS methods (e.g., NASNet, PNASNet).
• In a controlled comparison with RL (using identical hardware and search space), evolution reaches a given accuracy threshold faster, especially in the early stages of search.
• The age regularization is shown to improve search efficiency over standard tournament selection without it.

Significance

This work has broad implications for the NAS community and AI practitioners. It demonstrates that evolution, a conceptually simple and well-understood optimization method, can be highly effective for architecture search when properly regularized. This reduces the barrier to entry for automated model design, as evolutionary algorithms are easier to implement and tune than RL-based approaches. The paper also highlights the importance of algorithmic innovations (like age regularization) over sheer computational power, making NAS more accessible. For Neura Market's audience, this paper reinforces the value of exploring classic optimization techniques with modern twists, and provides a practical, proven method for generating high-performance image classifiers.