Multilayer perceptrons and radial basis function neural network methods for the solution of differential equations: a survey.

*(English)*Zbl 1236.65107Summary: Since neural networks have universal approximation capabilities, therefore it is possible to postulate them as solutions for given differential equations that define unsupervised errors. In this paper, we present a wide survey and classification of different Multilayer Perceptron (MLP) and Radial Basis Function (RBF) neural network techniques, which are used for solving differential equations of various kinds. Our main purpose is to provide a synthesis of the published research works in this area and stimulate further research interest and effort in the identified topics. Here, we describe the crux of various research articles published by numerous researchers, mostly within the last 10 years to get a better knowledge about the present scenario.

##### MSC:

65L99 | Numerical methods for ordinary differential equations |

68T05 | Learning and adaptive systems in artificial intelligence |

34-04 | Software, source code, etc. for problems pertaining to ordinary differential equations |

35-02 | Research exposition (monographs, survey articles) pertaining to partial differential equations |

##### Keywords:

differential equations; neural network; multilayer perceptron; radial basis functions; backpropagation algorithm
PDF
BibTeX
XML
Cite

\textit{M. Kumar} and \textit{N. Yadav}, Comput. Math. Appl. 62, No. 10, 3796--3811 (2011; Zbl 1236.65107)

Full Text:
DOI

##### References:

[1] | Its, A.R.; Izergin, A.G.; Korepin, V.E.; Slavnov, N.A., Differential equations for quantum correlation functions, International journal of quantum physics B, 1003-1037, (1990) · Zbl 0719.35091 |

[2] | Kotikov, A.V., Differential equations method: the calculation of vertex-type Feynman diagrams, Physics letters B, 259, 314-322, (1991) |

[3] | Peng, Y.Z., Exact solutions for some nonlinear partial differential equations, Physics letters A, 314, 401-408, (2003) · Zbl 1040.35102 |

[4] | Verwer, J.G.; Blom, J.G.; Loon, M.V.; Spee, E.J., A comparison of stiff ODE solvers for atmospheric chemistry problems, Atmospheric environment, 30, 49-58, (1996) |

[5] | Behlke, J.; Ristau, O., A new approximate whole boundary solution of the lamm differential equation for the analysis of sedimentation velocity experiments, Biophysical chemistry, 95, 59-68, (2002) |

[6] | Salzner, U.; Otto, P.; Ladik, J., Numerical solution of partial differential equation system describing chemical kinetics and diffusion in a cell with the aid of compartmentalization, Journal of computational chemistry, 11, 194-204, (1990) |

[7] | Culshaw, R.V.; Ruan, S., A delay differential equation model of HIV infection of CD4+T-cells, Mathematical biosciences, 165, 27-39, (2000) · Zbl 0981.92009 |

[8] | Bocharov, G.A.; Rihan, F.A., Numerical modeling in biosciences using delay differential equations, Journal of computational and applied mathematics, 125, 183-199, (2000) · Zbl 0969.65124 |

[9] | Noberg, R., Differential equations for moments of present values in life insurance, Mathematics and economics, 17, 171-180, (1995) |

[10] | Butcher, J.C., The numerical analysis of ordinary differential equations: Runge Kutta and general linear methods, (1987), Wiley Interscience New York, NY, USA · Zbl 0616.65072 |

[11] | Verwer, J.G., Expicit runge – kutta methods for parabolic partial differential equations, Applied numerical mathematics, 22, 359-379, (1996) · Zbl 0868.65064 |

[12] | Douglas, J.; Jones, B.F., On predictor – corrector methods for nonlinear parabolic differential equations, Journal of the society for industrial and applied mathematics, 11, 195-204, (1963) · Zbl 0116.09104 |

[13] | Hamming, R.W., Stable predictor corrector methods for ordinary differential equations, Journal of the ACM, 6, 37-47, (1959) · Zbl 0086.11201 |

[14] | Petravic, M.; Petravic, G. Kuo; Roberts, K.V., Automatic production of programmes for solving partial differential equations by finite difference methods, Computer physics communications, 4, 82-88, (1972) · Zbl 0252.68016 |

[15] | Tseng, A.A.; Gu, S.X., A finite difference scheme with arbitrary mesh system for solving high order partial differential equations, Computers & structures, 31, 319-328, (1989) · Zbl 0674.73062 |

[16] | Wu, B.; White, R.E., One implementation variant of finite difference method for solving ODEs/daes, Computers and chemical engineering, 28, 303-309, (2004) |

[17] | Korneev, V.G., Iterative methods of solving system of equations of the finite element method, USSR computational mathematics and mathematical physics, 17, 109-129, (1977) · Zbl 0389.65048 |

[18] | Kumar, M.; Kumar, P., Computational method for finding various solutions for a quasilinear elliptic equation of Kirchhoff type, Advances in engineering software, 40, 1104-1111, (2009) · Zbl 1171.74044 |

[19] | Thomee, V., From finite differences to finite elements: a short history of numerical analysis of partial differential equations, Journal of computational and applied mathematics, 128, 1-54, (2001) · Zbl 0977.65001 |

[20] | Kumar, M.; Srivastava, P.K., Computational techniques for solving differential equations by cubic, quintic and sextic spline, International journal for computational methods in engineering science & mechanics, 10, 108-115, (2009) · Zbl 1183.65097 |

[21] | Sallam, S.; Ameen, W., Numerical solution of general \(n\)th order differential equations via splines, Applied numerical mathematics, 6, 225-238, (1990) · Zbl 0692.65035 |

[22] | Ei-Hawary, H.M.; Mahmoud, S.M., Spline collocation methods for solving delay differential equations, Applied mathematics and computation, 146, 359-372, (2003) · Zbl 1033.65066 |

[23] | Kumar, M.; Gupta, Y., Methods for solving singular boundary value problems using splines: a survey, Applied mathematics and computation, 32, 265-278, (2010) · Zbl 1186.65104 |

[24] | Shawagfeh, N.; Kaya, D., Comparing numerical methods for the solutions of systems of ordinary differential equations, Applied mathematics letters, 17, 323-328, (2004) · Zbl 1061.65062 |

[25] | Liu, J.; Hou, G., Numerical solutions of the space- and time-fractional coupled Burgers equations by generalized differential transform method, Applied mathematics and computation, 217, 7001-7008, (2011) · Zbl 1213.65131 |

[26] | Darnia, P.; Ebadian, A., A method for the numerical solution of the integro-differential equations, Applied mathematics and computation, 188, 657-668, (2007) · Zbl 1121.65127 |

[27] | Kumar, M.; Parul, Methods for solving singular perturbation problems arising in science and engineering, Mathematical and computer modelling, 54, 556-575, (2011) · Zbl 1225.65077 |

[28] | Saadatmandi, A.; Dehghan, M., Numerical solution of higher order linear Fredholm integro-differential difference equation with variable coefficients, Computers & mathematics with applications, 59, 2996-3004, (2010) · Zbl 1193.65229 |

[29] | Tocino, A.; Ardanuy, R., Runge Kutta methods for the numerical solution of stochastic differential equations, Journal of computational and applied mathematics, 138, 219-241, (2002) · Zbl 0993.65012 |

[30] | Abbasbandy, S.; Taati, A., Numerical solution of nonlinear Volterra integro-differential equations with nonlinear differential part by the operational tau method and error estimation, Journal of computational and applied mathematics, 231, 106-113, (2009) · Zbl 1170.65101 |

[31] | Kumar, M.; Singh, N., A collection of computational techniques for solving singular boundary value problems, Advances in engineering software, 40, 288-297, (2009) · Zbl 1159.65076 |

[32] | Momani, S.; Al-Khaled, K., Numerical solutions for systems of fractional differential equations by the decomposition method, Applied mathematics and computation, 162, 1351-1365, (2005) · Zbl 1063.65055 |

[33] | Sommeijer, B.P.; Vander Houwen, P.J.; Neta, B., Symmetric multistep methods for second-order differential equations with periodic solutions, Applied numerical mathematics, 2, 69-77, (1986) · Zbl 0596.65046 |

[34] | Ghasemi, M.; Kajani, M.T.; Babolian, E., Numerical solutions of the nonlinear integro differential equations: wavelet Galerkin method and homotopy perturbation method, Applied mathematics and computation, 188, 450-455, (2007) · Zbl 1114.65368 |

[35] | Kumar, M.; Kumar, P., Computational method for finding various solutions for a quasilinear elliptic equation of Kirchhoff type, Advances in engineering software, 40, 1104-1111, (2009) · Zbl 1171.74044 |

[36] | Maleknejad, K.; Arzhang, A., Numerical solution of the Fredholm singular integro-differential equation with Cauchy kernel by using Taylor-series expansion and Galerkin method, Applied mathematics and computation, 182, 888-897, (2006) · Zbl 1107.65118 |

[37] | Kumar, M.; Mishra, H.K.; Singh, P., A boundary value approach for singularly perturbed boundary value problems, Advances in engineering software, 40, 298-304, (2009) · Zbl 1159.65075 |

[38] | Wang, Q.; Cheng, D., Numerical solution of damped nonlinear klein – gordon equations using variational method and finite element approach, Applied mathematics and computation, 162, 381-401, (2005) · Zbl 1063.65107 |

[39] | Kumar, M.; Singh, N., Modified Adomian decomposition method and computer implementation for solving singular boundary value problems arising in various physical problems, Computers and chemical engineering, 34, 1750-1760, (2010) |

[40] | Yang, X.; Liu, Y.; Bai, S., A numerical solution of second order linear partial differential equation by differential transform, Applied mathematics and computation, 173, 792-802, (2006) · Zbl 1090.65134 |

[41] | Erturk, V.S.; Momani, S., Solving system of fractional differential equations using differential transform method, Journal of computational and applied mathematics, 215, 142-151, (2008) · Zbl 1141.65088 |

[42] | Meadre, A.J.; Fernandez, A.A., The numerical solution of linear ordinary differential equations by feed forward neural networks, Mathematical and computer modelling, 19, 1-25, (1994) |

[43] | Lagaris, I.E.; Likas, A., Artificial neural networks for solving ordinary and partial differential equations, IEEE transactions on neural networks, 9, 987-1000, (1998) |

[44] | He, S.; Reif, K.; Unbehauen, R., Multilayer neural networks for solving a class of partial differential equations, Neural networks, 13, 385-396, (2000) |

[45] | Lagaris, I.E.; Likas, A.C., Neural network methods for boundary value problems with irregular boundaries, IEEE transactions on neural networks, 11, 5, 1041-1049, (2000) |

[46] | Aarts, L.P.; Veer, P.V., Neural network method for solving partial differential equations, Neural processing letters, 14, 261-271, (2001) · Zbl 0985.68048 |

[47] | Alli, H.; Ucar, A.; Demir, Y., The solution of vibration control problems using artificial neural networks, Journal of the franklin institute, 340, 307-325, (2003) · Zbl 1032.74564 |

[48] | Smaoui, N.; Al-Enezi, S., Modelling the dynamics of nonlinear partial differential equations using neural networks, Journal of computational and applied mathematics, 170, 27-58, (2004) · Zbl 1049.65108 |

[49] | Malek, A.; Beidokhti, R.S., Numerical solution for high order differential equations using a hybrid neural network-optimization method, Applied mathematics and computation, 183, 260-271, (2006) · Zbl 1105.65340 |

[50] | Saxen, H.; Petterson, F., Method for the selection of inputs and structure of feed forward neural networks, Computers and chemical engineering, 30, 1038-1045, (2006) |

[51] | X. Li-Ying, W. Hui, Z. Zhe-Zhao, The algorithm of neural networks on the initial value problems in ordinary differential equations, in: 2nd IEEE Conference on Industrial Electronics and Applications, 2007, pp. 813-816. |

[52] | Shirvany, Y.; Hayati, M.; Moradian, R., Numerical solution of the nonlinear schrodinger equation by feedforward neural networks, Communications in nonlinear science and numerical simulation, 13, 2132-2145, (2008) · Zbl 1221.65315 |

[53] | Beidokhti, R.S.; Malek, A., Solving initial boundary value problems for system of partial differential equations using neural network and optimization techniques, Journal of the franklin institute, 346, 898-913, (2009) · Zbl 1298.65155 |

[54] | Kolmogrov, A.N., On the representation of continuous functions of several variables by superpositions of continuous functions of one variable and addition, American mathematical society translations, 28, 55-59, (1963) |

[55] | Cybenko, G., Approximation of superpositions of a sigmoidal function, Mathematics of control, signals, and systems, 2, 304-314, (1989) · Zbl 0679.94019 |

[56] | Nelder, J.A.; Mead, R., A simplex method for function minimization, The computer journal, 7, 308-313, (1965) · Zbl 0229.65053 |

[57] | Shirvany, Y.; Hayati, M.; Moradian, R., Multilayer perceptron neural network with novel unsupervised training method for the solution of partial differential equations, Applied soft computing, 9, 20-29, (2009) |

[58] | Tsoulos, I.G.; Gavrilis, D.; Glavas, E., Solving differential equations with constructed neural networks, Neurocomputing, 72, 2385-2391, (2009) |

[59] | Tsoulos, I.; Gavrilis, D.; Glavas, E., Neural network construction and training using grammatical evolution, Neurocomputing, 72, 269-277, (2008) |

[60] | O’Neill, M.; Ryan, C., Grammatical evolution, IEEE transactions on evolutionary computation, 5, 349-358, (2001) |

[61] | Filici, C., Error estimation in the neural network solution of ordinary differential equations, Neural networks, 23, 614-617, (2010) · Zbl 06944397 |

[62] | Zadunaisky, P.E., On the estimation of errors propagated in the numerical integration of ordinary differential equations, Numerische Mathematik, 27, 21-39, (1976) · Zbl 0324.65035 |

[63] | Zadunaisky, P.E., On the accuracy in the numerical solution of the \(N\)-body problem, Celestial mechanics, 20, 209-230, (1979) · Zbl 0424.65035 |

[64] | Effati, S.; Pakdaman, M., Artificial neural network approach for solving fuzzy differential equations, Information sciences, 180, 1434-1457, (2010) · Zbl 1185.65114 |

[65] | Masmoudi, N.K.; Rekik, C.; Djemel, M.; Derbel, N., Two coupled neural network based solution of the hamilton – jacobi – bellman equation, Applied soft computing, 11, 2946-2963, (2011) |

[66] | Dua, V., An artificial neural network approximation based decomposition approach for parameter estimation of system of ordinary differential equations, Computers and chemical engineering, 35, 545-553, (2011) |

[67] | Mai-Duy, N.; Tran-Cong, T., Numerical solution of differential equations using multiquadric radial basis function networks, Neural networks, 14, 185-199, (2001) |

[68] | L. Jianyu, L. Siwei, Q. Yingjian, H. Yaping, Numerical solution of differential equations by radial basis function neural networks, in: Proceedings of International Joint Conference on Neural Networks, 2002, pp. 773-777. |

[69] | Mai-Duy, N.; Tran-Cong, T., Mesh-free radial basis function network methods with domain decomposition for approximation of functions and numerical solution of poisson’s equation, Engineering analysis with boundary elements, 26, 133-156, (2002) · Zbl 0996.65131 |

[70] | Mai-Duy, N.; Tran-Cong, T., Approximation of function and its derivatives using radial basis function networks, Applied mathematical modelling, 27, 197-220, (2003) · Zbl 1024.65012 |

[71] | L. Jianyu, Luo Siwei, Qi Yingjian, Huang Yaping, Numerical solution of elliptic partial differential equation by growing radial basis function neural networks, in: Proceedings of the International Joint Conference on Neural Networks, 2003, pp. 85-90. |

[72] | Kansa, E.J.; Power, H.; Fasshauer, G.E.; Ling, L., A volumetric integral radial basis function method for time dependent partial differential equations I formulation, Engineering analysis with boundary elements, 28, 1191-1206, (2004) · Zbl 1159.76363 |

[73] | H. Zou, J. Lei, C. Pan, Design of a new kind of RBF neural network based on differential reconstruction, in: International Joint Conference on Neural Network and Brain, vol. 1, 2005, pp. 456-460. |

[74] | Mai-Duy, N., Solving high order ordinary differential equations with radial basis function networks, International journal for numerical methods in engineering, 62, 824-852, (2005) · Zbl 1077.74057 |

[75] | Mai-Duy, N.; Tran-Cong, T., Solving biharmonic problems with scattered-point discretization using indirect radial basis function networks, Engineering analysis with boundary elements, 30, 77-87, (2006) · Zbl 1195.65179 |

[76] | Golbabai, A.; Seifollahi, S., Radial basis function networks in the numerical solution of linear integro-differential equations, Applied mathematics and computation, 188, 427-432, (2007) · Zbl 1114.65369 |

[77] | Golbabai, A.; Mammadov, M.; Seifollahi, S., Solving a system of nonlinear integral equations by an RBF network, Computers and mathematics with applications, 57, 1651-1658, (2009) · Zbl 1186.45009 |

[78] | Mai-Duy, N.; Tran-Cong, T., A Cartesian-grid collocation method based on radial-basis-function networks for solving PDEs in irregular domains, Numerical methods for partial differential equations, 23, 1192-1210, (2007) · Zbl 1129.65089 |

[79] | Aminataei, A.; Mazarei, M.M., Numerical solution of poisson’s equation using radial basis function networks on the polar coordinate, Computers and mathematics with applications, 56, 2887-2895, (2008) · Zbl 1165.65401 |

[80] | Chen, H.; Kong, Li; Leng, W., Numerical solution of PDEs via integrated radial basis function networks with adaptive training algorithm, Applied soft computing, 11, 855-860, (2011) |

[81] | Sarra, S., Integrated radial basis functions based differential quadrature method and its performance, Computers & mathematics with applications, 43, 1283-1296, (2002) · Zbl 1146.65327 |

[82] | Choi, B.; Lee, J., Comparison of generalizing ability on solving differential equations using backpropagation and reformulated radial basis function network, Neurocomputing, 73, 115-118, (2009) |

[83] | K. Valasoulis, D.L. Fotadis, I.E. Lagaris, A. Likas, Solving differential equations with neural networks: implementation on a DSP platform, in: 14th International Conference on Digital Signal Processing, vol. 2, 2002, pp. 1265-1268. |

[84] | Vaziri, N.; Hojabri, A.; Erfani, A.; Monsefi, M.; Nilforooshan, B., Critical heat flux prediction by using radial basis function and multilayer perceptron neural networks: a comparison study, Nuclear engineering and design, 237, 377-385, (2007) |

[85] | Chiddarwar, S.S.; Babu, N.R., Comparison of RBF and MLP neural networks to solve inverse kinematic problem for 6R serial robot by a fusion approach, Engineering applications of artificial intelligence, 23, 1083-1092, (2010) |

[86] | Yilmaz, A.S.; Ozer, Z., Pitch angle control in wind turbines above the rated wind speed by multi-layer perceptron and radial basis function neural networks, Expert systems with applications, 36, 9767-9775, (2009) |

[87] | Kashaninejad, M.; Dehghani, A.A.; Kashiri, M., Modeling of wheat soaking using two artificial neural networks (MLP and RBF), Journal of food engineering, 91, 602-607, (2009) |

[88] | Suchacz, B.; Wesołowski, M., The recognition of similarities in trace elements content in medicinal plants using MLP and RBF neural networks, Talanta, 69, 37-42, (2006) |

[89] | Chandnok, J.S.; Kar, I.N.; Tuli, S., Estimation of furnace exit gas temperature (FEGT) using optimized radial basis and back-propagation neural networks, Energy conversion and management, 49, 1989-1998, (2008) |

[90] | Afkhami, A.; Tarighat, M.A.; Bahram, M., Artificial neural networks for determination of enantiomeric composition of \(\alpha\)-phenylglycine using UV spectra of cyclodextrin host – guest complexes: comparison of feed-forward and radial basis function networks, Talanta, 75, 91-98, (2008) |

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.