Recently, Prof. Sun Hao's team published a paper entitled Encoding physics to learn reaction-diffusion process on Nature Machine Intelligence. The journal of Nature Machine Intelligence publishes high-quality original research and reviews in a wide range of topics in machine learning, robotics and AI. The journal also explores and discusses the significant impact of the above academic fields on other scientific disciplines, as well as on many aspects of our society and industry. This is the first time that our faculty at Gaoling School of Artificial Intelligence has published a paper on this top-notch journal.
The paper proposes a novel network, namely, the physics-encoded recurrent convolutional neural network (PeRCNN), for modelling and discovery of nonlinear spatiotemporal dynamical systems based on sparse and noisy data. The proposed approach can be applied to a variety of problems such as reaction–diffusion processes and other PDE systems, including forward and inverse analysis, data-driven modelling and discovery of PDEs. The prior physics knowledge is forcibly ‘encoded’, which makes the network possess interpretability. In particular, the paper proposes a deep learning framework which forcibly encodes a given physics structure in a recurrent convolutional neural network to facilitate learning of the spatiotemporal dynamics in sparse data regimes. The study finds that such a computational paradigm shows high accuracy, robustness, interpretability and generalizability. In this paper, Sun’s team demonstrated the capabilities of the proposed network architecture by applying it to various tasks in scientific modelling of spatiotemporal dynamics such as reaction–diffusion processes.
Generally speaking, predicting the evolution of complex spatiotemporal dynamical systems is a challenging task for many cases owing to insufficient prior knowledge and a lack of explicit PDE formulation for describing the nonlinear process of the system variables. Common machine learning methods need to rely on a large amount of training data, possessing problems such as poor interpretability, weak generalization, and uncontrollable modeling errors. Thanks to the recent development of data-driven approaches, it is possible to learn spatiotemporal dynamics from measurement data while adding prior physics knowledge. However, existing physics-informed machine learning paradigms impose physical laws or governing equations through soft penalty constraints, and the solution quality largely depends on a trial-and-error proper setting of hyperparameters. Therefore, it is of great necessity to develop new knowledge-embedded learning model to learn the underlying spatiotemporal dynamics from data.
To this end, the paper proposes a novel network, namely, the PeRCNN model, as shown in Fig. 1 (see the paper). One major advantage of the PeRCNN is that the prior physics knowledge can be encoded into the network, which guarantees that the resulting network strictly obeys given physics (for example, ICs and BCs, general PDE structure, and known terms in PDEs). This brings distinct benefits for improving the convergence and accuracy of the model. By encoding given physical structure into the recurrent convolutional neural network, the performance of modeling spatiotemporal dynamical systems based on sparse and noisy data was improved.
The paper shows with extensive numerical experiments how the proposed approach can be applied to model and discover a variety of reaction–diffusion processes and other PDE systems. By comparing the proposed approach with some existing methods (aka., baseline models), the team found that the approach outperforms (if not performs as good as) the considered baselines consistently under different noise levels and data richness.
In addition, the paper integrates the sparse regression technique with the PeRCNN model for discovering the explicit form of PDEs. The proposed framework for the PDE discovery is presented in Fig. 5 (see the paper) with the example of 2D GS RD equation. The entire procedure consists of three steps, data reconstruction, sparse regression and fine-tuning of coefficients.
Although the paper demonstrates the effectiveness of the PeRCNN on various RD systems, the model is in theory applicable to other types of spatiotemporal PDEs (for example, the 2D Burgers’ equation with the convection term shown in Supplementary Note F.4, and the Kolmogorov turbulent flows at Reynolds number 1,000, discussed in Supplementary Note J).
This research is expected to boost the development of data-driven modeling of complex spatiotemporal dynamical systems, providing scientists and engineers with more powerful tools to understand and predict natural and engineering phenomena. This approach, which combines deep learning and our prior physics knowledge, is expected to be applicable in multiple disciplines and play an important role, including fluid mechanics, biochemistry, environmental science, engineering, materials science, and more. We look forward to seeing the further development and application of this new approach, which may reveal more mysteries about complex spatiotemporal dynamical systems and bring new breakthroughs for future scientific and technological development.
About the author:
Hao Sun is an Associate Professor with Tenure in the Gaoling School of Artificial Intelligence at Renmin University of China (RUC). He received his Ph.D. in Engineering Mechanics from Columbia University in 2014 and did his Postdoc training at MIT during 2014-2017. He was a Tenure-Track Assistant Professor at the University of Pittsburgh (2017-2018) and at Northeastern University (2018-2021) before joining RUC. His research interests lie in AI for Science as well as AI-enabled Scientific Computing. He has published over 60 peer-reviewed articles in top-tiered journals (e.g., Nature Machine Intelligence, Nature Communications) and top computer science conferences (e.g., ICLR, NeurIPS). Dr. Sun has been a PI or a major Co-PI for dozens of research projects (with a total funding level of 30 Million RMB) sponsored by National Natural Science Foundation of China, National Science Foundation (US), Beijing Natural Science Foundation, etc. Thanks to his academic contributions, Dr. Sun was named to the prestigious 2018 Forbes “30 Under 30”: Science, a list of the world’s most inspiring young innovators, bright rising stars and the leaders of tomorrow who are transforming the world. He also received the 2019 Top Ten Outstanding Chinese American Youth Award and the 2022 DeepTech Intelligent Computing Innovator Award of China.
