×

Deep learning of biological models from data: applications to ODE models. (English) Zbl 1460.92014

Summary: Mathematical equations are often used to model biological processes. However, for many systems, determining analytically the underlying equations is highly challenging due to the complexity and unknown factors involved in the biological processes. In this work, we present a numerical procedure to discover dynamical physical laws behind biological data. The method utilizes deep learning methods based on neural networks, particularly residual networks. It is also based on recently developed mathematical tools of flow-map learning for dynamical systems. We demonstrate that with the proposed method, one can accurately construct numerical biological models for unknown governing equations behind measurement data. Moreover, the deep learning model can also incorporate unknown parameters in the biological process. A successfully trained deep neural network model can then be used as a predictive tool to produce system predictions of different settings and allows one to conduct detailed analysis of the underlying biological process. In this paper, we use three biological models – SEIR model, Morris-Lecar model and the Hodgkin-Huxley model – to show the capability of our proposed method.

MSC:

92B20 Neural networks for/in biological studies, artificial life and related topics
68T07 Artificial neural networks and deep learning
92D30 Epidemiology
92C20 Neural biology
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: large-scale machine learning on heterogeneous systems. https://www.tensorflow.org/. Software available from tensorflow.org
[2] Bianchini, M.; Scarselli, F., On the complexity of neural network classifiers: a comparison between shallow and deep architectures, IEEE Trans Neural Netw Learn Syst, 25, 1553-1565 (2014) · doi:10.1109/TNNLS.2013.2293637
[3] Bongard, J.; Lipson, H., Automated reverse engineering of nonlinear dynamical systems, Proc Natl Acad Sci, 104, 9943-9948 (2007) · Zbl 1155.37044 · doi:10.1073/pnas.0609476104
[4] Chan, S.; Elsheikh, A., A machine learning approach for efficient uncertainty quantification using multiscale methods, J Comput Phys, 354, 494-511 (2018) · Zbl 1380.65331 · doi:10.1016/j.jcp.2017.10.034
[5] Chen RTQ, Rubanova Y, Bettencourt J, Duvenaud D (2018) Neural ordinary differential equations. Preprint arXiv:1806.07366
[6] Chollet F et al (2015) Keras. https://github.com/fchollet/keras
[7] Daniels, BC; Nemenman, I., Automated adaptive inference of phenomenological dynamical models, Nat Commun, 6, 8133 (2015) · doi:10.1038/ncomms9133
[8] Daniels, BC; Nemenman, I., Efficient inference of parsimonious phenomenological models of cellular dynamics using S-systems and alternating regression, PLoS ONE, 10, e0119821 (2015) · doi:10.1371/journal.pone.0119821
[9] DeAngelis DL, Yurek S (2015) Equation-free modeling unravels the behavior of complex ecological systems. doi:10.1073/pnas.1503154112
[10] Du, KL; Swamy, M., Neural networks and statistical learning (2014), London: Springer, London · Zbl 1279.62003 · doi:10.1007/978-1-4471-5571-3
[11] Eldan, R.; Shamir, O., The power of depth for feedforward neural networks, Conf Learn Theory, 2016, 907-940 (2016)
[12] Fu, X.; Chang, LB; Xiu, D., Learning reduced systems via deep neural networks with memory, J Mach Learn Model Comput, 2020, 1 (2020)
[13] Giannakis, D.; Majda, AJ, Nonlinear Laplacian spectral analysis for time series with intermittency and low-frequency variability, Proc Natl Acad Sci, 109, 2222-2227 (2012) · Zbl 1256.62053 · doi:10.1073/pnas.1118984109
[14] Gonzalez-Garcia, R.; Rico-Martinez, R.; Kevrekidis, IG, Identification of distributed parameter systems: a neural net based approach, Comput Chem Eng, 22, S965-S968 (1998) · doi:10.1016/S0098-1354(98)00191-4
[15] Goodfellow, I.; Bengio, Y.; Courville, A., Deep learning (2016), London: MIT Press, London · Zbl 1373.68009
[16] Grimm, V.; Railsback, SF, Individual-based modeling and ecology (2005), Princeton: Princeton University Press, Princeton · Zbl 1085.92043 · doi:10.1515/9781400850624
[17] Hau DT, Coiera EW (1995) Learning qualitative models from physiological signals internal accession date only. Technical report
[18] Hesthaven, J.; Ubbiali, S., Non-intrusive reduced order modeling of nonlinear problems using neural networks, J Comput Phys, 363, 55-78 (2018) · Zbl 1398.65330 · doi:10.1016/j.jcp.2018.02.037
[19] Hethcote, HW, The mathematics of infectious diseases, SIAM Rev, 42, 599-653 (2000) · Zbl 0993.92033 · doi:10.1137/S0036144500371907
[20] He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770-778
[21] Hodgkin AL, Huxley AF (Aug 1952) A quantitative description of membrane current and its application to conduction and excitation in nerve. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1392413/
[22] Kevrekidis, IG; Gear, CW; Hyman, JM; Kevrekidid, PG; Runborg, O.; Theodoropoulos, C., Equation-free, coarse-grained multiscale computation: enabling mocroscopic simulators to perform system-level analysis, Commun Math Sci, 1, 715-762 (2003) · Zbl 1086.65066 · doi:10.4310/CMS.2003.v1.n4.a5
[23] Khoo Y, Lu J, Ying L (2018) Solving parametric PDE problems with artificial neural networks. Preprint arXiv:1707.03351
[24] Kingma DP, Ba J (2017) Adam: a method for stochastic optimization. arXiv:1412.6980
[25] Long Z, Lu Y, Ma X, Dong B (2018) PDE-net: learning PDEs from data. In: Proceedings of the 35th international conference on machine learning, 10-15 edn. vol 80 of proceedings of machine learning research, Stockholmsmässan, Stockholm, Sweden, pp 3208-3216
[26] Mangan, NM; Brunton, SL; Proctor, JL; Kutz, JN, Inferring biological networks by sparse identification of nonlinear dynamics, IEEE Trans Mol Biol Multiscale Commun, 2, 52-63 (2016) · doi:10.1109/TMBMC.2016.2633265
[27] Mardt, A.; Pasquali, L.; Wu, H.; Noe, F., VAMPnets for deep learning of molecular kinetics, Nat Commun, 9, 5 (2018) · doi:10.1038/s41467-017-02388-1
[28] Montufar, GF; Pascanu, R.; Pascanu, K.; Bengio, Y., On the number of linear regions of deep neural networks, Adv Neural Inf Process Syst, 2014, 2924-2932 (2014)
[29] Morris C, Lecar H (1981) Voltage oscillations in the barnacle giant muscle fiber. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1327511/
[30] Perretti, CT; Munch, SB; Sugihara, G., Model-free forecasting outperforms the correct mechanistic model for simulated and experimental data, Proc Natl Acad Sci USA, 110, 5253-5257 (2013) · doi:10.1073/pnas.1216076110
[31] Poggio, T.; Mhaskar, H.; Rosasco, L.; Miranda, B.; Liao, Q., Why and when can deep-but not shallow-networks avoid the curse of dimensionality: a review, Int J Autom Comput, 14, 503-519 (2017) · doi:10.1007/s11633-017-1054-2
[32] Qin, T.; Wu, K.; Xiu, D., Data driven governing equations approximation using deep neural networks, J Comput Phys, 395, 620-635 (2019) · Zbl 1455.65125 · doi:10.1016/j.jcp.2019.06.042
[33] Qin, T.; Chen, Z.; Jakeman, J.; Xiu, D., Data-driven learning of non-autonomous systems, SIAM J Sci Comput, 2020, 1 (2020)
[34] Radhakrishnan K, Hindmarsh AC (1993) Description and use of LSODE, the Livemore solver for ordinary differential equations. Technical report, Lawrence Livermore National Laboratory (LLNL), Livermore. doi:10.2172/15013302. http://www.osti.gov/servlets/purl/15013302-7Xcq5X/native/
[35] Ray, D.; Hesthaven, J., An artificial neural network as a troubled-cell indicator, J Comput Phys, 367, 166-191 (2018) · Zbl 1415.65229 · doi:10.1016/j.jcp.2018.04.029
[36] Rinzel, J.; Ermentrout, G.; Koch, C.; Segev, I., Analysis of neural excitability and oscillations, Methods in neuronal modeling, 251-291 (1998), London: MIT Press, London
[37] Schmidhuber, J., Deep learning in neural networks: an overview, Neural Netw, 61, 85-117 (2015) · doi:10.1016/j.neunet.2014.09.003
[38] Schmidt, M.; Lipson, H., Distilling free-form natural laws from experimental data, Science, 324, 81-85 (2009) · doi:10.1126/science.1165893
[39] Schmidt, MD; Vallabhajosyula, RR; Jenkins, JW; Hood, JE; Soni, AS; Wikswo, JP; Lipson, H., Automated refinement and inference of analytical models for metabolic networks, Phys Biol, 8, 055011 (2011) · doi:10.1088/1478-3975/8/5/055011
[40] Sugihara, G.; May, R.; Ye, H.; Hsieh, C.; Deyle, E.; Fogarty, M.; Munch, S., Detecting causality in complex ecosystems, Science, 338, 496-500 (2012) · Zbl 1355.92144 · doi:10.1126/science.1227079
[41] Tripathy, R.; Bilionis, I., Deep UQ: learning deep neural network surrogate model for high dimensional uncertainty quantification, J Comput Phys, 375, 565-588 (2018) · Zbl 1419.68084 · doi:10.1016/j.jcp.2018.08.036
[42] Tsumoto, K.; Kitajima, H.; Yoshinaga, T.; Aihara, K.; Kawakami, H., Bifurcations in Morris-Lecar neuron model, Neurocomputing, 69, 293-316 (2006) · doi:10.1016/j.neucom.2005.03.006
[43] Voss, HU; Kolodner, P.; Abel, M.; Kurths, J., Amplitude equations from spatiotemporal binary-fluid convection data, Phys Rev Lett, 83, 3422 (1999) · doi:10.1103/PhysRevLett.83.3422
[44] Wang Y, Cheung SW, Chung ET, Efendiev Y, Wang M (2018) Deep multiscale model learning. Preprint arXiv:1806.04830
[45] Weinan, E.; Engquist, B.; Huang, Z., Heterogeneous multiscale method: a general methodology for multiscale modeling, Phys. Rev. B, 67, 092101 (2003) · doi:10.1103/PhysRevB.67.092101
[46] Wood, SN; Thomas, MB, Super-sensitivity to structure in biological models, Proc R Soc B Biol Sci, 266, 565-570 (1999) · doi:10.1098/rspb.1999.0673
[47] Wu, K.; Xiu, D., Numerical aspects for approximating governing equations using data, J Comput Phys, 384, 200-221 (2019) · Zbl 1451.65008 · doi:10.1016/j.jcp.2019.01.030
[48] Ye, H.; Beamish, RJ; Glaser, SM; Grant, SCH; Hsieh, C.; Richards, LJ; Schnute, JT; Sugihara, G., Equation-free mechanistic ecosystem forecasting using empirical dynamic modeling, Proc Natl Acad Sci, 112, 1569-E1576 (2015) · doi:10.1073/pnas.1417063112
[49] Yodzis P (1988) The indeterminacy of ecological interactions as perceived through perturbation. Technical report, p 2
[50] Zhu, Y.; Zabaras, N., Bayesian deep convolutional encoder-decoder networks for surrogate modeling and uncertainty quantification, J Comput Phys, 366, 415-447 (2018) · Zbl 1407.62091 · doi:10.1016/j.jcp.2018.04.018
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.