ImageNet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever et al.
1.9k
Citations
117
Influential Citations
Computer Methods in Applied Mechanics and Engineering
Venue
1995
Year
An approach is developed for deriving variational methods capable of representing multiscale phenomena. The ideas are first illustrated on the exterior problem for the Helmholtz equation. This leads to the well-known Dirichlet-to-Neumann formulation. Next, a class of subgrid scale models is developed and the relationships to ‘bubble function’ methods and stabilized methods are established. It is shown that both the latter methods are approximate subgrid scale models. The identification for stabilized methods leads to an analytical formula for τ, the ‘intrinsic time scale’, whose origins have been a mystery heretofore.
This paper, by Thomas J.R. Hughes, is a seminal contribution that unifies several seemingly disparate numerical methods under a single multiscale variational framework. Before this work, stabilized methods (like SUPG) and bubble function methods were developed heuristically, with the intrinsic time scale τ being a parameter tuned for stability and accuracy. Hughes's analysis demystified τ by showing it emerges naturally from the fine-scale Green's function, thus placing these methods on a solid theoretical footing.
The paper's significance extends beyond computational mechanics. It introduced the concept of variational multiscale (VMS) methods, which have become a powerful paradigm for modeling problems with multiple scales, from turbulence to fluid-structure interaction. The idea of explicitly representing fine scales and then modeling their effect on coarse scales resonates with modern machine learning approaches that use hierarchical or multiscale representations, such as in computer vision and graph neural networks.
The paper is theoretical and does not present numerical experiments. However, its main result is the analytical formula for τ, which is derived for the Helmholtz equation and can be extended to other problems. The formula shows that τ depends on the element size and the operator's Green's function, providing a rational basis for choosing stabilization parameters. The paper also establishes the equivalence between bubble functions and subgrid scale models, which had been observed empirically but not rigorously explained.
The impact of this paper on computational mechanics is profound. It has been cited nearly 2000 times and has inspired a large body of research on variational multiscale methods. The framework has been applied to incompressible flows, turbulence, and even to problems in biomechanics and geophysics. In the broader AI context, the idea of separating scales and modeling fine-scale effects via subgrid models is analogous to techniques in multi-scale deep learning, such as U-Nets and multi-resolution transformers. The paper's emphasis on deriving parameters from first principles rather than tuning them is a lesson that resonates with the current push for interpretable and physics-informed machine learning. Overall, this work is a classic example of how deep theoretical insights can lead to practical and widely adopted numerical methods.
Alex Krizhevsky, Ilya Sutskever et al.
Ashish Vaswani, Noam Shazeer et al.
Douglas M. Bates, Martin Mächler et al.
Diederik P. Kingma, Jimmy Ba