ImageNet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever et al.
5.2k
Citations
303
Influential Citations
Physical Review Letters
Venue
2001
Year
We introduce the concept of efficiency of a network as a measure of how efficiently it exchanges information. By using this simple measure, small-world networks are seen as systems that are both globally and locally efficient. This gives a clear physical meaning to the concept of "small world," and also a precise quantitative analysis of both weighted and unweighted networks. We study neural networks and man-made communication and transportation systems and we show that the underlying general principle of their construction is in fact a small-world principle of high efficiency.
This paper introduces a simple yet powerful measure—network efficiency—that captures how well a network exchanges information. By applying this measure to small-world networks, the authors reveal that these networks are not only globally efficient (short average path lengths) but also locally efficient (high clustering). This dual efficiency gives a clear physical meaning to the concept of a 'small world' and provides a quantitative tool for analyzing real-world networks.
The significance lies in its generality: the efficiency measure works for both weighted and unweighted networks, making it applicable to diverse domains such as neural networks, communication systems, and transportation networks. This work has become a cornerstone in network science, influencing how researchers design and evaluate network architectures in AI and beyond.
The paper demonstrates that small-world networks achieve high global efficiency (close to that of random networks) while maintaining high local efficiency (close to that of regular lattices). For example, the C. elegans neural network shows both high global and local efficiency, explaining its ability to process information rapidly and robustly. The authors also show that man-made networks like the Internet and power grids follow a similar small-world principle of high efficiency.
This work has had a lasting impact on network science and AI. The efficiency measure is now widely used to analyze and design networks, from social networks to deep learning architectures. It provides a principled way to optimize networks for both information propagation and fault tolerance, influencing fields such as neuromorphic computing, communication network design, and transportation planning.
Alex Krizhevsky, Ilya Sutskever et al.
Ashish Vaswani, Noam Shazeer et al.
Douglas M. Bates, Martin Mächler et al.
Diederik P. Kingma, Jimmy Ba