×

Deep learning-enhanced ensemble-based data assimilation for high-dimensional nonlinear dynamical systems. (English) Zbl 07652811

Summary: Data assimilation (DA) is a key component of many forecasting models in science and engineering. DA allows one to estimate better initial conditions using an imperfect dynamical model of the system and noisy/sparse observations available from the system. Ensemble Kalman filter (EnKF) is a DA algorithm that is widely used in applications involving high-dimensional nonlinear dynamical systems. However, EnKF requires evolving large ensembles of forecasts using the dynamical model of the system. This often becomes computationally intractable, especially when the number of states of the system is very large, e.g., for weather prediction. With small ensembles, the estimated background error covariance matrix in the EnKF algorithm suffers from sampling error, leading to an erroneous estimate of the analysis state (initial condition for the next forecast cycle). In this work, we propose hybrid ensemble Kalman filter (H-EnKF), which is applied to a two-layer quasi-geostrophic turbulent flow as a test case. This framework utilizes a pre-trained deep learning-based data-driven surrogate that inexpensively generates and evolves a large data-driven ensemble of the states to accurately compute the background error covariance matrix with smaller sampling errors. The H-EnKF framework outperforms EnKF with only dynamical model or only the data-driven surrogate, and estimates a better initial condition without the need for any ad-hoc localization strategies. H-EnKF can be extended to any ensemble-based DA algorithm, e.g., particle filters, which are currently too expensive to use for high-dimensional systems.

MSC:

86Axx Geophysics
93Exx Stochastic systems and control
60-XX Probability theory and stochastic processes

Software:

U-Net
PDFBibTeX XMLCite
Full Text: DOI arXiv

References:

[1] Kalnay, E., Atmospheric Modeling, Data Assimilation and Predictability (2003), Cambridge University Press
[2] Law, K.; Stuart, A.; Zygalakis, K., Data Assimilation (2015), Springer · Zbl 1353.60002
[3] Asch, M.; Bocquet, M.; Nodet, M., Data Assimilation: Methods, Algorithms, and Applications (2016), SIAM · Zbl 1361.93001
[4] Morzfeld, M.; Adams, J.; Lunderman, S.; Orozco, R., Feature-based data assimilation in geophysics, Nonlinear Process. Geophys., 25, 2, 355-374 (2018)
[5] Morzfeld, M.; Hodyss, D., Gaussian approximations in filters and smoothers for data assimilation, Tellus, Ser. A Dyn. Meteorol. Oceanogr., 71, 1, Article 1600344 pp. (2019)
[6] Lorenz, E. N., Predictability: a problem partly solved, (Proc. Seminar on Predictability, vol. 1 (1996)), 1-18
[7] Carrassi, A.; Bocquet, M.; Bertino, L.; Evensen, G., Data assimilation in the geosciences: an overview of methods, issues, and perspectives, Wiley Interdiscip. Rev.: Clim. Change, 9, 5, e535 (2018)
[8] Chen, N.; Fu, S.; Manucharyan, G. E., An efficient and statistically accurate Lagrangian data assimilation algorithm with applications to discrete element sea ice models, J. Comput. Phys., 455, Article 111000 pp. (2022) · Zbl 07518084
[9] Eliashiv, J.; Subramanian, A. C.; Miller, A. J., Tropical climate variability in the community earth system model: data assimilation research testbed, Clim. Dyn., 54, 1, 793-806 (2020)
[10] Gleiter, T.; Janjić, T.; Chen, N., Ensemble Kalman filter based data assimilation for tropical waves in the MJO skeleton model, Q. J. R. Meteorol. Soc., 148, 743, 1035-1056 (2022)
[11] Belyaev, K.; Kuleshov, A.; Tuchkova, N.; Tanajura, C. A., An optimal data assimilation method and its application to the numerical simulation of the ocean dynamics, Math. Comput. Model. Dyn. Syst., 24, 1, 12-25 (2018) · Zbl 1484.86018
[12] D’Amore, L.; Arcucci, R.; Marcellino, L.; Murli, A., HPC computation issues of the incremental 3D variational data assimilation scheme in OceanVar software, J. Numer. Anal. Ind. Appl. Math., 7, 3-4, 91-105 (2012) · Zbl 1432.86008
[13] Arcucci, R.; Mottet, L.; Pain, C.; Guo, Y.-K., Optimal reduced space for variational data assimilation, J. Comput. Phys., 379, 51-69 (2019) · Zbl 07581563
[14] Yi, T.; Gutmark, E. J., Online prediction of the onset of combustion instability based on the computation of damping ratios, J. Sound Vib., 310, 1-2, 442-447 (2008)
[15] Bell, J.; Day, M.; Goodman, J.; Grout, R.; Morzfeld, M., A Bayesian approach to calibrating hydrogen flame kinetics using many experiments and parameters, Combust. Flame, 205, 305-315 (2019)
[16] Croci, M. L.; Sengupta, U.; Juniper, M. P., Data assimilation using heteroscedastic Bayesian neural network ensembles for reduced-order flame models, (International Conference on Computational Science (2021), Springer), 408-419
[17] Brunton, S. L.; Nathan Kutz, J.; Manohar, K.; Aravkin, A. Y.; Morgansen, K.; Klemisch, J.; Goebel, N.; Buttrick, J.; Poskin, J.; Blom-Schieber, A. W.; Hogan, T.; McDonald, D., Data-driven aerospace engineering: reframing the industry with machine learning, AIAA J., 59, 8, 2820-2847 (2021)
[18] Liu, Y.; Gupta, H. V., Uncertainty in hydrologic modeling: Toward an integrated data assimilation framework, Water Resour. Res., 43, 7 (2007)
[19] Maday, Y.; Patera, A. T.; Penn, J. D.; Yano, M., A parameterized-background data-weak approach to variational data assimilation: formulation, analysis, and application to acoustics, Int. J. Numer. Methods Eng., 102, 5, 933-965 (2015) · Zbl 1352.65529
[20] Gu, Y.; Oliver, D. S., An iterative ensemble Kalman filter for multiphase fluid flow data assimilation, SPE J., 12, 04, 438-446 (2007)
[21] Habibi, M.; D’Souza, R. M.; Dawson, S. T.; Arzani, A., Integrating multi-fidelity blood flow data with reduced-order data assimilation, Comput. Biol. Med., Article 104566 pp. (2021)
[22] Evensen, G., Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics, J. Geophys. Res., Oceans, 99, C5, 10143-10162 (1994)
[23] Fletcher, S. J., Data Assimilation for the Geosciences: From Theory to Application (2017), Elsevier
[24] Bannister, R., A review of operational methods of variational and ensemble-variational data assimilation, Q. J. R. Meteorol. Soc., 143, 703, 607-633 (2017)
[25] Kondo, K.; Miyoshi, T., Impact of removing covariance localization in an ensemble Kalman filter: experiments with 10 240 members using an intermediate AGCM, Mon. Weather Rev., 144, 12, 4849-4865 (2016)
[26] Carrassi, A.; Bocquet, M.; Demaeyer, J.; Grudzien, C.; Raanes, P.; Vannitsem, S., Data assimilation for chaotic dynamics, (Data Assimilation for Atmospheric, Oceanic and Hydrologic Applications (vol. IV) (2022)), 1-42
[27] De Cruz, L.; Schubert, S.; Demaeyer, J.; Lucarini, V.; Vannitsem, S., Exploring the Lyapunov instability properties of high-dimensional atmospheric and climate models, Nonlinear Process. Geophys., 25, 2, 387-412 (2018)
[28] Leutbecher, M., Ensemble size: how suboptimal is less than infinity?, Q. J. R. Meteorol. Soc., 145, 107-128 (2019)
[29] Miyoshi, T.; Kondo, K.; Imamura, T., The 10,240-member ensemble Kalman filtering with an intermediate AGCM, Geophys. Res. Lett., 41, 14, 5264-5271 (2014)
[30] Janjic, T.; Lukacova, M.; Ruckstuhl, Y.; Spichtinger, P.; Wiebe, B., A test of an alternative approach for uncertainty representation in weather forecasting, (EGU General Assembly Conference Abstracts (2021)), EGU21-13077
[31] Yang, L. M.; Grooms, I., Machine learning techniques to construct patched analog ensembles for data assimilation, J. Comput. Phys., 443, Article 110532 pp. (2021) · Zbl 07515431
[32] Chattopadhyay, A.; Mustafa, M.; Hassanzadeh, P.; Bach, E.; Kashinath, K., Towards physics-inspired data-driven weather forecasting: integrating data assimilation with a deep spatial-transformer-based U-NET in a case study with ERA5, Geosci. Model Dev., 15, 5, 2221-2237 (2022)
[33] Tsuyuki, T.; Tamura, R., Nonlinear data assimilation by deep learning embedded in an ensemble Kalman filter, J. Meteorol. Soc. Jpn. II
[34] Maulik, R.; Rao, V.; Wang, J.; Mengaldo, G.; Constantinescu, E.; Lusch, B.; Balaprakash, P.; Foster, I.; Kotamarthi, R., Aieada 1.0: efficient high-dimensional variational data assimilation with machine-learned reduced-order models, Geosci. Model Dev. Discuss., 1-20 (2022)
[35] Penny, S. G.; Smith, T. A.; Chen, T.-C.; Platt, J. A.; Lin, H.-Y.; Goodliff, M.; Abarbanel, H. D., Integrating recurrent neural networks with data assimilation for scalable data-driven state estimation, J. Adv. Model. Earth Syst., 14, 3, Article e2021MS002843 pp. (2022)
[36] Chen, N.; Li, Y., BAMCAFE: a Bayesian machine learning advanced forecast ensemble method for complex turbulent systems with partial observations, Chaos, Interdiscip. J. Nonlinear Sci., 31, 11, Article 113114 pp. (2021)
[37] Brajard, J.; Carrassi, A.; Bocquet, M.; Bertino, L., Combining data assimilation and machine learning to infer unresolved scale parametrization, Philos. Trans. R. Soc. A, 379, 2194, Article 20200086 pp. (2021)
[38] Pawar, S.; Ahmed, S. E.; San, O.; Rasheed, A.; Navon, I. M., Long short-term memory embedded nudging schemes for nonlinear data assimilation of geophysical flows, Phys. Fluids, 32, 7, Article 076606 pp. (2020)
[39] Mojgani, R.; Chattopadhyay, A.; Hassanzadeh, P., Discovery of interpretable structural model errors by combining Bayesian sparse regression and data assimilation: a chaotic Kuramoto-Sivashinsky test case, arXiv preprint
[40] Lutsko, N. J.; Held, I. M.; Zurita-Gotor, P., Applying the fluctuation-dissipation theorem to a two-layer model of quasigeostrophic turbulence, J. Atmos. Sci., 72, 8, 3161-3177 (2015)
[41] Nabizadeh, E.; Hassanzadeh, P.; Yang, D.; Barnes, E. A., Size of the atmospheric blocking events: scaling law and response to climate change, Geophys. Res. Lett., 46, 22, 13488-13499 (2019)
[42] Ronneberger, O.; Fischer, P.; Brox, T., U-NET: convolutional networks for biomedical image segmentation, (International Conference on Medical Image Computing and Computer-Assisted Intervention (2015), Springer), 234-241
[43] Wang, R.; Kashinath, K.; Mustafa, M.; Albert, A.; Yu, R., Towards physics-informed deep learning for turbulent flow prediction, (Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2020)), 1457-1466
[44] Chattopadhyay, A.; Mustafa, M.; Hassanzadeh, P.; Kashinath, K., Deep spatial transformers for autoregressive data-driven forecasting of geophysical turbulence, (Proceedings of the 10th International Conference on Climate Informatics (2020)), 106-112
[45] Wang, R.; Walters, R.; Yu, R., Incorporating symmetry into deep dynamics models for improved generalization, arXiv preprint
[46] Kashinath, K.; Mustafa, M.; Albert, A.; Wu, J.; Esmaeilzadeh, S.; Azizzadenesheli, K.; Wang, R.; Chattopadhyay, A.; Singh, A.; Manepalli, A.; Chirila, D.; Yu, R.; Walters, R.; White, B.; Xiao, H.; Tchelepi, A.; Marcus, P.; Anandkumar, A.; Hassanzadeh, P.; Prabhat, Physics-informed machine learning: case studies for weather and climate modelling, Philos. Trans. R. Soc. A, 379, 2194, Article 20200093 pp. (2021)
[47] Pathak, J.; Subramanian, S.; Harrington, P.; Raja, S.; Chattopadhyay, A.; Mardani, M.; Kurth, Thorsten; Hall, D.; Li, Z.; Azizzadenesheli, K.; Hassanzadeh, P.; Kashinath, K.; Anandkumar, A., FourCastNet: a global data-driven high-resolution weather model using adaptive Fourier neural operators, arXiv preprint
[48] von Holstein, C.-A. S. Stael, An experiment in probabilistic weather forecasting, J. Appl. Meteorol. Climatol., 10, 4, 635-645 (1971)
[49] Keisler, R., Forecasting global weather with graph neural networks, arXiv preprint
[50] Chattopadhyay, A.; Pathak, J.; Nabizadeh, E.; Bhimji, W.; Hassanzadeh, P., Long-term stability and generalization of observationally-constrained stochastic data-driven models for geophysical turbulence, arXiv preprint
[51] Fearnhead, P.; Künsch, H. R., Particle filters and data assimilation, Annu. Rev. Stat. Appl., 5, 421-449 (2018)
[52] Van Leeuwen, P. J.; Künsch, H. R.; Nerger, L.; Potthast, R.; Reich, S., Particle filters for high-dimensional geoscience applications: a review, Q. J. R. Meteorol. Soc., 145, 723, 2335-2365 (2019)
[53] Zeng, Y.; Janjić, T., Study of conservation laws with the local ensemble transform Kalman filter, Q. J. R. Meteorol. Soc., 142, 699, 2359-2372 (2016)
[54] Farchi, A.; Bocquet, M., On the efficiency of covariance localisation of the ensemble Kalman filter using augmented ensembles, Front. Appl. Math. Stat., 5, 3 (2019)
[55] Crystalng, G.-H.; Mclaughlin, D.; Entekhabi, D.; Ahanin, A., The role of model dynamics in ensemble Kalman filter performance for chaotic systems, Tellus, Ser. A Dyn. Meteorol. Oceanogr., 63, 5, 958-977 (2011)
[56] Bach, E.; Ghil, M., A multi-model ensemble Kalman filter for data assimilation and forecasting, arXiv preprint
[57] Wan, E. A.; Van Der Merwe, R.; Haykin, S., The unscented Kalman filter, Kalman filtering and neural networks, 5, 2007, 221-280 (2001)
[58] Weyn, J. A.; Durran, D. R.; Caruana, R., Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere, J. Adv. Model. Earth Syst., 12, 9, Article e2020MS002109 pp. (2020)
[59] Guan, Y.; Subel, A.; Chattopadhyay, A.; Hassanzadeh, P., Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES, Phys. D: Nonlinear Phenom., Article 133568 pp. (2022)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.