Scientific machine learning through physics–informed neural networks: Where we are and what's next

Analysis

Why This Paper Matters

Physics-informed neural networks (PINNs) have emerged as a powerful paradigm for solving partial differential equations (PDEs) by embedding physical laws into the neural network training process. This review paper is significant because it provides a structured overview of the field, which has expanded rapidly since the introduction of the vanilla PINN. By cataloging the many collocation-based variants, the authors help researchers understand the design space and identify which innovations address specific challenges such as training instability, loss weighting, and boundary condition enforcement.

The paper's timing (2022) captures a critical phase in the maturation of PINNs, moving from proof-of-concept to practical applications. For AI practitioners, this review serves as a roadmap, highlighting the strengths and weaknesses of different approaches and offering a clear entry point for those looking to apply PINNs to their own scientific problems. It also underscores the interdisciplinary nature of the field, bridging deep learning and numerical analysis.

Technical Contributions

The review's primary contribution is its taxonomy of collocation-based PINNs. Key innovations covered include:

• Vanilla PINN: The foundational approach where the residual of the PDE is added as a loss term, and automatic differentiation is used to compute derivatives.
• Loss weighting strategies: Methods to balance the contributions of data, boundary, and PDE residual losses, which are crucial for training convergence.
• Architectural modifications: Variations such as using Fourier features, adaptive activation functions, or domain decomposition to improve expressivity and handle complex geometries.
• Training improvements: Techniques like curriculum learning, transfer learning, and optimization strategies to mitigate the well-known failure modes of PINNs.
• Extended formulations: Including fractional derivatives, stochastic PDEs, and inverse problems, broadening the applicability of PINNs.

The review also discusses the theoretical underpinnings, such as the universal approximation theorem and the impact of collocation point selection.

Results

Since this is a review paper, it does not present new experimental results. Instead, it synthesizes findings from the literature, noting that while PINNs have shown promise on benchmark problems (e.g., Burgers' equation, Navier-Stokes), they still face significant challenges in terms of accuracy and convergence for stiff or multi-scale problems. The review highlights that no single variant universally outperforms others, and the choice of method often depends on the specific PDE and domain. It also points out that training PINNs can be computationally expensive, and that careful hyperparameter tuning is often required.

Significance

The broader impact of this review lies in its potential to accelerate progress in scientific machine learning. By providing a clear map of the field, it enables researchers to build on prior work more effectively and avoid redundant efforts. It also emphasizes the importance of collaboration between the machine learning and scientific computing communities, which is essential for developing robust and scalable PINN methods. As PINNs continue to evolve, this review will serve as a foundational reference, helping to establish best practices and guiding future innovations that could lead to real-world applications in engineering design, climate modeling, and biomedical simulations.