×

Latent-space time evolution of non-intrusive reduced-order models using Gaussian process emulation. (English) Zbl 07493969

Summary: Non-intrusive reduced-order models (ROMs) have recently generated considerable interest for constructing computationally efficient counterparts of nonlinear dynamical systems emerging from various domain sciences. They provide a low-dimensional emulation framework for systems that may be intrinsically high-dimensional. This is accomplished by utilizing a construction algorithm that is purely data-driven. It is no surprise, therefore, that the algorithmic advances of machine learning have led to non-intrusive ROMs with greater accuracy and computational gains. However, in bypassing the utilization of an equation-based evolution, it is often seen that the interpretability of the ROM framework suffers. This becomes more problematic when black-box deep learning methods are used which are notorious for lacking robustness outside the physical regime of the observed data. In this article, we propose the use of a novel latent-space interpolation algorithm based on Gaussian process regression. Notably, this reduced-order evolution of the system is parameterized by control parameters to allow for interpolation in space. The use of this procedure also allows for a continuous interpretation of time which allows for temporal interpolation. The latter aspect provides information, with quantified uncertainty, about full-state evolution at a finer resolution than that utilized for training the ROMs. We assess the viability of this algorithm for an advection-dominated system given by the inviscid shallow water equations.

MSC:

65P99 Numerical problems in dynamical systems
37M99 Approximation methods and numerical treatment of dynamical systems
68T07 Artificial neural networks and deep learning
76B15 Water waves, gravity waves; dispersion and scattering, nonlinear interaction
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Verstappen, R.; Veldman, A., Direct numerical simulation of turbulence at lower costs, J. Eng. Math., 32, 2-3, 143-159 (1997) · Zbl 0911.76072
[2] Long, Z.; Lu, Y.; Dong, B., PDE-Net 2.0: Learning PDEs from data with a numeric-symbolic hybrid deep network, J. Comput. Phys., Article 108925 pp. (2019) · Zbl 1454.65131
[3] Raissi, M., Deep hidden physics models: Deep learning of nonlinear partial differential equations, J. Mach. Learn. Res., 19, 1, 932-955 (2018) · Zbl 1439.68021
[4] Raissi, M.; Perdikaris, P.; Karniadakis, G. E., Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378, 686-707 (2019) · Zbl 1415.68175
[5] Sirignano, J.; Spiliopoulos, K., DGM: A deep learning algorithm for solving partial differential equations, J. Comput. Phys., 375, 1339-1364 (2018) · Zbl 1416.65394
[6] Bui-Thanh, T.; Damodaran, M.; Willcox, K., Aerodynamic data reconstruction and inverse design using proper orthogonal decomposition, AIAA J., 42, 8, 1505-1516 (2004)
[7] Renganathan, S. A.; Liu, Y.; Mavris, D. N., Koopman-based approach to nonintrusive projection-based reduced-order modeling with black-box high-fidelity models, AIAA J., 56, 10, 4087-4111 (2018)
[8] Noack, B. R.; Morzynski, M.; Tadmor, G., Reduced-Order Modelling for Flow Control, Vol. 528 (2011), Springer Science & Business Media
[9] Kutz, J. N.; Brunton, S. L.; Brunton, B. W.; Proctor, J. L., Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems (2016), SIAM · Zbl 1365.65009
[10] Chattopadhyay, A.; Hassanzadeh, P.; Pasha, S., Predicting clustered weather patterns: A test case for applications of convolutional neural networks to spatio-temporal climate data, Sci. Rep., 10, 1, 1-13 (2020)
[11] Chattopadhyay, A.; Nabizadeh, E.; Hassanzadeh, P., Analog forecasting of extreme-causing weather patterns using deep learning, J. Adv. Model, 12, 2 (2020), e2019MS001958
[12] Maulik, R.; Egele, R.; Lusch, B.; Balaprakash, P., Recurrent neural network architecture search for geophysical emulation (2020), arXiv preprint arXiv:2004.10928
[13] Hsieh, W. W.; Tang, B., Applying neural network models to prediction and data analysis in meteorology and oceanography, Bull. Am. Meteorol. Soc., 79, 9, 1855-1870 (1998)
[14] Mannarino, A.; Mantegazza, P., Nonlinear aeroelastic reduced order modeling by recurrent neural networks, J. Fluid Struct., 48, 103-121 (2014)
[15] Zhang, W.; Kou, J.; Wang, Z., Nonlinear aerodynamic reduced-order model for limit-cycle oscillation and flutter, AIAA J., 3304-3311 (2016)
[16] Kou, J.; Zhang, W., Layered reduced-order models for nonlinear aerodynamics and aeroelasticity, J. Fluid Struct., 68, 174-193 (2017)
[17] San, O.; Maulik, R., Neural network closures for nonlinear model order reduction, Adv. Comput. Math, 44, 6, 1717-1750 (2018) · Zbl 1404.37101
[18] Hesthaven, J. S.; Ubbiali, S., Non-intrusive reduced order modeling of nonlinear problems using neural networks, J. Comput. Phys., 363, 55-78 (2018) · Zbl 1398.65330
[19] O. San, J. Borggaard, Basis selection and closure for POD models of convection dominated Boussinesq flows, in: 21st International Symposium on Mathematical Theory of Networks and Systems, Vol. 5, 2014.
[20] Korda, M.; Putinar, M.; Mezić, I., Data-driven spectral analysis of the Koopman operator, Appl. Comput. Harmon. Anal. (2018)
[21] Kalb, V. L.; Deane, A. E., An intrinsic stabilization scheme for proper orthogonal decomposition based low-dimensional models, Phys. Fluids, 19, 5, Article 054106 pp. (2007) · Zbl 1146.76431
[22] Thuerey, N.; Weißenow, K.; Prantl, L.; Hu, X., Deep learning methods for Reynolds-averaged Navier-Stokes simulations of airfoil flows, AIAA J., 58, 1, 25-36 (2020)
[23] Kim, B.; Azevedo, V. C.; Thuerey, N.; Kim, T.; Gross, M.; Solenthaler, B., (Deep Fluids: A Generative Network for Parameterized Fluid Simulations. Deep Fluids: A Generative Network for Parameterized Fluid Simulations, Comput. Graph. Forum, vol. 38 (2019), Wiley Online Library), 59-70
[24] Gonzalez, F. J.; Balajewicz, M., Learning low-dimensional feature dynamics using deep convolutional recurrent autoencoders (2018), arXiv preprint arXiv:1808.01346
[25] Maulik, R.; Mohan, A.; Lusch, B.; Madireddy, S.; Balaprakash, P.; Livescu, D., Time-series learning of latent-space dynamics for reduced-order model closure, Physica D, Article 132368 pp. (2020) · Zbl 07477434
[26] Kalashnikova, I.; Barone, M. F., On the stability and convergence of a Galerkin reduced order model (ROM) of compressible flow with solid wall and far-field boundary treatment, Internat. J. Numer. Methods Engrg., 83, 10, 1345-1375 (2010) · Zbl 1202.74123
[27] Mohebujjaman, M.; Rebholz, L. G.; Iliescu, T., Physically constrained data-driven correction for reduced-order modeling of fluid flows, Internat. J. Numer. Methods Fluids, 89, 3, 103-122 (2019)
[28] Carlberg, K.; Bou-Mosleh, C.; Farhat, C., Efficient non-linear model reduction via a least-squares Petrov-Galerkin projection and compressive tensor approximations, Internat. J. Numer. Methods Engrg., 86, 2, 155-181 (2011) · Zbl 1235.74351
[29] Xiao, D.; Fang, F.; Du, J.; Pain, C. C.; Navon, I. M.; Buchan, A. G.; Elsheikh, A. H.; Hu, G., Non-linear Petrov-Galerkin methods for reduced order modelling of the Navier-Stokes equations using a mixed finite element pair, Comput. Methods Appl. Mech. Engrg., 255, 147-157 (2013) · Zbl 1297.76107
[30] Fang, F.; Pain, C. C.; Navon, I.; Elsheikh, A. H.; Du, J.; Xiao, D., Non-linear Petrov-Galerkin methods for reduced order hyperbolic equations and discontinuous finite element methods, J. Comput. Phys., 234, 540-559 (2013) · Zbl 1284.65132
[31] Lee, K.; Carlberg, K. T., Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders, J. Comput. Phys., 404, Article 108973 pp. (2020) · Zbl 1454.65184
[32] Vlachas, P. R.; Byeon, W.; Wan, Z. Y.; Sapsis, T. P.; Koumoutsakos, P., Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks, Proc. R. Soc. A, 474, 2213, Article 20170844 pp. (2018) · Zbl 1402.92030
[33] Ahmed, S. E.; San, O.; Rasheed, A.; Iliescu, T., A long short-term memory embedding for hybrid uplifted reduced order models, Physica D, Article 132471 pp. (2020) · Zbl 1505.76071
[34] Mohan, A.; Daniel, D.; Chertkov, M.; Livescu, D., Compressed convolutional LSTM: An efficient deep learning framework to model high fidelity 3D turbulence (2019), arXiv preprint arXiv:1903.00033
[35] Mohan, A. T.; Gaitonde, D. V., A deep learning based approach to reduced order modeling for turbulent flow control using LSTM neural networks (2018), arXiv preprint arXiv:1804.09269
[36] Wang, Q.; Ripamonti, N.; Hesthaven, J. S., Recurrent neural network closure of parametric POD-Galerkin reduced-order models based on the Mori-Zwanzig formalism, J. Comput. Phys., Article 109402 pp. (2020) · Zbl 1436.65093
[37] Xu, J.; Duraisamy, K., Multi-level convolutional autoencoder networks for parametric prediction of spatio-temporal dynamics, Comput. Methods Appl. Mech. Engrg., 372, Article 113379 pp. (2020) · Zbl 1506.68111
[38] Maulik, R.; Lusch, B.; Balaprakash, P., Non-autoregressive time-series methods for stable parametric reduced-order models, Phys. Fluids, 32, 8, Article 087115 pp. (2020)
[39] Champion, K.; Lusch, B.; Kutz, J. N.; Brunton, S. L., Data-driven discovery of coordinates and governing equations, Proc. Natl. Acad. Sci., 116, 45, 22445-22451 (2019) · Zbl 1433.68396
[40] Kalnay, E., Atmospheric Modeling, Data Assimilation and Predictability (2003), Cambridge University Press
[41] Wang, Z.; Akhtar, I.; Borggaard, J.; Iliescu, T., Proper orthogonal decomposition closure models for turbulent flows: a numerical comparison, Comput. Methods Appl. Mech. Engrg., 237, 10-26 (2012) · Zbl 1253.76050
[42] Kutz, J.; Fu, X.; L. Brunton, S., Multi-resolution dynamic mode decomposition, SIAM J. Appl. Dyn. Syst., 15, 2, 713-735 (2016) · Zbl 1338.37121
[43] Mendible, A.; Brunton, S. L.; Aravkin, A. Y.; Lowrie, W.; Kutz, J. N., Dimensionality reduction and reduced order modeling for traveling wave physics (2019), arXiv preprint arXiv:1911.00565
[44] Kosambi, D. D., Statistics in function space, J. Indian Math. Soc., 7, 76-88 (1943) · Zbl 0063.03317
[45] Berkooz, G.; Holmes, P.; Lumley, J. L., The proper orthogonal decomposition in the analysis of turbulent flows, Annu. Rev. Fluid Mech., 25, 1, 539-575 (1993)
[46] Taira, K.; Hemati, M. S.; Brunton, S. L.; Sun, Y.; Duraisamy, K.; Bagheri, S.; Dawson, S.; Yeh, C.-A., Modal analysis of fluid flows: Applications and outlook (2019), arXiv preprint arXiv:1903.05750
[47] Kingma, D. P.; Welling, M., Auto-encoding variational bayes (2013), arXiv preprint arXiv:1312.6114
[48] Rasmussen, C. E.; Williams, C. K.I., Gaussian Processes for Machine Learning (2006), MIT Press Cambridge: MIT Press Cambridge MA · Zbl 1177.68165
[49] Williams, C. K.; Rasmussen, C. E., Gaussian Processes for regression, (Adv. Neur. In. (1996)), 514-520
[50] De G. Matthews, A. G.; Van Der Wilk, M.; Nickson, T.; Fujii, K.; Boukouvalas, A.; León-Villagrá, P.; Ghahramani, Z.; Hensman, J., GPflow: A Gaussian process library using TensorFlow, J. Mach. Learn. Res., 18, 1, 1299-1304 (2017) · Zbl 1437.62127
[51] Lloyd, J. R.; Duvenaud, D.; Grosse, R.; Tenenbaum, J. B.; Ghahramani, Z., Automatic construction and natural-language description of nonparametric regression models, Stat, 1050, 24 (2014)
[52] Bishop, C. M., Pattern Recognition and Machine Learning (2006), Springer · Zbl 1107.68072
[53] Duvenaud, D., Automatic Model Construction with Gaussian Processes (2014), University of Cambridge, (Ph.D. thesis)
[54] Rowley, C. W.; Colonius, T.; Murray, R. M., Model reduction for compressible flows using POD and Galerkin projection, Physica D, 189, 1-2, 115-129 (2004) · Zbl 1098.76602
[55] San, O.; Borggaard, J., Principal interval decomposition framework for POD reduced-order modeling of convective Boussinesq flows, Internat. J. Numer. Methods Fluids, 78, 1, 37-62 (2015)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.