×

Deep learning of thermodynamics-aware reduced-order models from data. (English) Zbl 1506.80005

Summary: We present an algorithm to learn the relevant latent variables of a large-scale discretized physical system and predict its time evolution using thermodynamically-consistent deep neural networks. Our method relies on sparse autoencoders, which reduce the dimensionality of the full order model to a set of sparse latent variables with no prior knowledge of the coded space dimensionality. Then, a second neural network is trained to learn the metriplectic structure of those reduced physical variables and predict its time evolution with a so-called structure-preserving neural network. This data-based integrator is guaranteed to conserve the total energy of the system and the entropy inequality, and can be applied to both conservative and dissipative systems. The integrated paths can then be decoded to the original full-dimensional manifold and be compared to the ground truth solution. This method is tested with two examples applied to fluid and solid mechanics.

MSC:

80M99 Basic methods in thermodynamics and heat transfer
65Q99 Numerical methods for difference and functional equations, recurrence relations
80A99 Thermodynamics and heat transfer
68T05 Learning and adaptive systems in artificial intelligence

Software:

Adam; PyTorch
PDFBibTeX XMLCite
Full Text: DOI arXiv Link

References:

[1] Fefferman, Charles; Mitter, Sanjoy; Narayanan, Hariharan, Testing the manifold hypothesis, J. Amer. Math. Soc., 29, 4, 983-1049 (2016) · Zbl 1359.62122
[2] Niroomandi, Siamak; Alfaro, Icíar; Cueto, Elías; Chinesta, Francisco, Real-time deformable models of non-linear tissues by model reduction techniques, Comput. Methods Programs Biomed., 91, 3, 223-231 (2008)
[3] Du, Juan; Fang, Fangxin; Pain, Christopher C.; Navon, I. M.; Zhu, Jiang; Ham, David A., Pod reduced-order unstructured mesh modeling applied to 2d and 3d fluid flow, Comput. Math. Appl., 65, 3, 362-379 (2013) · Zbl 1319.76026
[4] Prud’Homme, Christophe; Rovas, Dimitrios V.; Veroy, Karen; Machiels, Luc; Maday, Yvon; Patera, Anthony T.; Turinici, Gabriel, Reliable real-time solution of parametrized partial differential equations: Reduced-basis output bound methods, J. Fluids Eng., 124, 1, 70-80 (2002)
[5] Rowley, Clarence W.; Colonius, Tim; Murray, Richard M., Model reduction for compressible flows using pod and galerkin projection, Physica D, 189, 1-2, 115-129 (2004) · Zbl 1098.76602
[6] Farrell, P. E.; Maddison, J. R., Conservative interpolation between volume meshes by local galerkin projection, Comput. Methods Appl. Mech. Engrg., 200, 1-4, 89-100 (2011) · Zbl 1225.76193
[7] Badías, Alberto; Curtit, Sarah; González, David; Alfaro, Icíar; Chinesta, Francisco; Cueto, Elías, An augmented reality platform for interactive aerodynamic design and analysis, Internat. J. Numer. Methods Engrg., 120, 1, 125-138 (2019)
[8] Moya, Beatriz; González, David.; Alfaro, Icíar; Chinesta, Francisco.; Cueto, E., Learning slosh dynamics by means of data, Comput. Mech., 64, 2, 511-523 (2019) · Zbl 1469.76092
[9] Moya, Beatriz; Alfaro, Iciar; Gonzalez, David; Chinesta, Francisco; Cueto, Elías, Physically sound self-learning digital twins for sloshing fluids, PLOS ONE, 15, 6, Article e0234569 pp. (2020)
[10] Goodfellow, Ian; Bengio, Yoshua; Courville, Aaron, Deep Learning (2016), MIT press · Zbl 1373.68009
[11] Farina, Marco; Nakai, Yuichiro; Shih, David, Searching for new physics with deep autoencoders, Phys. Rev. D, 101, 7, Article 075021 pp. (2020)
[12] Liu, Qi; Allamanis, Miltiadis; Brockschmidt, Marc; Gaunt, Alexander, Constrained graph variational autoencoders for molecule design, (Advances in Neural Information Processing Systems (2018)), 7795-7804
[13] Lee, Kookjin; Carlberg, Kevin T., J. Comput. Phys., 404, Article 108973 pp. (2020)
[14] Marco, Julio; Hernandez, Quercus; Munoz, Adolfo; Dong, Yue; Jarabo, Adrian; Kim, Min H.; Tong, Xin; Gutierrez, Diego, Deeptof: off-the-shelf real-time correction of multipath interference in time-of-flight imaging, ACM Trans. Graph. (ToG), 36, 6, 1-12 (2017)
[15] Raissi, Maziar; Perdikaris, Paris; Karniadakis, George E., Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378, 686-707 (2019) · Zbl 1415.68175
[16] Kelly, Jacob; Bettencourt, Jesse; Johnson, Matthew James; Duvenaud, David, Learning differential equations that are easy to solve, (Neural Information Processing Systems (2020))
[17] Bertalan, Tom; Dietrich, Felix; Mezić, Igor; Kevrekidis, Ioannis G., On learning hamiltonian systems from data, Chaos, 29, 12, Article 121107 pp. (2019)
[18] Greydanus, Samuel; Dzamba, Misko; Yosinski, Jason, Hamiltonian neural networks, (Advances in Neural Information Processing Systems (2019)), 15379-15389
[19] Toth, Peter; Rezende, Danilo Jimenez; Jaegle, Andrew; Racanière, Sébastien; Botev, Aleksandar; Higgins, Irina, Hamiltonian generative networks (2019), arXiv preprint arXiv:1909.13789
[20] Zhong, Yaofeng Desmond; Dey, Biswadip; Chakraborty, Amit, Symplectic ode-net: Learning hamiltonian dynamics with control (2019), arXiv preprint arXiv:1909.12077
[21] Tong, Yunjin; Xiong, Shiying; He, Xingzhe; Pan, Guanghan; Zhu, Bo, Symplectic neural networks in taylor series form for hamiltonian systems (2020), arXiv preprint arXiv:2005.04986
[22] Jin, Pengzhan; Zhang, Zhen; Kevrekidis, Ioannis G.; Karniadakis, George Em, Learning Poisson systems and trajectories of autonomous systems via Poisson neural networks (2020), arXiv preprint arXiv:2012.03133
[23] Öttinger, Hans Christian; Grmela, Miroslav, Dynamics and thermodynamics of complex fluids. ii. illustrations of a general formalism, Phys. Rev. E, 56, 6, 6633 (1997)
[24] Grmela, Miroslav; Öttinger, Hans Christian, Dynamics and thermodynamics of complex fluids. i. development of a general formalism, Phys. Rev. E, 56, 6, 6620 (1997)
[25] Hernandez, Quercus; Badias, Alberto; Gonzalez, David; Chinesta, Francisco; Cueto, Elias, Structure-preserving neural networks (2020), arXiv preprint arXiv:2004.04653
[26] Weinan, E., A proposal on machine learning via dynamical systems, Commun. Math. Stat., 5, 1, 1-11 (2017) · Zbl 1380.37154
[27] Öttinger, Hans Christian, Preservation of thermodynamic structure in model reduction, Phys. Rev. E, 91, 3, Article 032147 pp. (2015)
[28] Ng, Andrew, Sparse autoencoder, CS294A Lect. Notes, 72, 2011, 1-19 (2011)
[29] Paszke, Adam; Gross, Sam; Chintala, Soumith; Chanan, Gregory; Yang, Edward; DeVito, Zachary; Lin, Zeming; Desmaison, Alban; Antiga, Luca; Lerer, Adam, Automatic differentiation in pytorch. Autodiff workshop: The future of gradient-based machine learning software and techniques (2017)
[30] Ruder, Sebastian, An overview of gradient descent optimization algorithms (2016), arXiv preprint arXiv:1609.04747
[31] Battaglia, Peter W.; Hamrick, Jessica B.; Bapst, Victor; Sanchez-Gonzalez, Alvaro; Zambaldi, Vinicius; Malinowski, Mateusz; Tacchetti, Andrea; Raposo, David; Santoro, Adam; Faulkner, Ryan, Relational inductive biases, deep learning, and graph networks (2018), arXiv preprint arXiv:1806.01261
[32] Morrison, Philip J., A paradigm for joined hamiltonian and dissipative systems, Physica D, 18, 1-3, 410-419 (1986) · Zbl 0661.70025
[33] Schmidhuber, Jürgen, Deep learning in neural networks: An overview, Neural networks, 61, 85-117 (2015)
[34] Laso, Manuel; Öttinger, Hans Christian, Calculation of viscoelastic flow using molecular models: the connffessit approach, J. Non-Newton. Fluid Mech., 47, 1-20 (1993) · Zbl 0774.76012
[35] Le Bris, Claude; Lelievre, Tony, Multiscale modelling of complex fluids: a mathematical initiation, (Multiscale Modeling and Simulation in Science (2009), Springer), 49-137 · Zbl 1157.76002
[36] He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing; Sun, Jian, Delving deep into rectifiers: Surpassing human-level performance on imagenet classification, (Proceedings of the IEEE International Conference on Computer Vision (2015), ICCV), 1026-1034
[37] Kingma, Diederik P.; Ba, Jimmy, Adam: A method for stochastic optimization (2014), arXiv preprint arXiv:1412.6980
[38] González, David; Chinesta, Francisco; Cueto, Elías, Thermodynamically consistent data-driven computational mechanics, Contin. Mech. Thermodyn., 31, 1, 239-253 (2019)
[39] González, D.; Chinesta, F.; Cueto, E., Consistent data-driven computational mechanics, AIP Conf. Proc., 1960, 1, Article 090005 pp. (2018)
[40] Kingma, Diederik P.; Welling, Max, Auto-encoding variational bayes (2013), arXiv preprint arXiv:1312.6114 · Zbl 1431.68002
[41] Tompson, Jonathan; Schlachter, Kristofer; Sprechmann, Pablo; Perlin, Ken, Accelerating eulerian fluid simulation with convolutional networks (2016), arXiv preprint arXiv:1607.03597
[42] Zhou, Jie; Cui, Ganqu; Zhang, Zhengyan; Yang, Cheng; Liu, Zhiyuan; Wang, Lifeng; Li, Changcheng; Sun, Maosong, Graph neural networks: A review of methods and applications (2018), arXiv preprint arXiv:1812.08434
[43] Teichert, Gregory H.; Natarajan, A. R.; Ven Van der, A.; Garikipati, Krishna, Machine learning materials physics: Integrable deep neural networks enable scale bridging by learning free energy functions, Comput. Methods Appl. Mech. Engrg., 353, 201-216 (2019) · Zbl 1441.82021
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.