×

Neural network for constrained nonsmooth optimization using Tikhonov regularization. (English) Zbl 1325.90104

Summary: This paper presents a one-layer neural network to solve nonsmooth convex optimization problems based on the Tikhonov regularization method. Firstly, it is shown that the optimal solution of the original problem can be approximated by the optimal solution of a strongly convex optimization problems. Then, it is proved that for any initial point, the state of the proposed neural network enters the equality feasible region in finite time, and is globally convergent to the unique optimal solution of the related strongly convex optimization problems. Compared with the existing neural networks, the proposed neural network has lower model complexity and does not need penalty parameters. In the end, some numerical examples and application are given to illustrate the effectiveness and improvement of the proposed neural network.

MSC:

90C59 Approximation methods and heuristics in mathematical programming
90C25 Convex programming
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Attouch, H., Viscosity solutions of minimization problems, SIAM Journal on Optimization, 6, 769-806 (1996) · Zbl 0859.65065
[2] Attouch, H.; Cominetti, R., A dynamical approach to convex minimization coupling approximation with the steepest descent method, Journal of Differential Equations, 128, 519-540 (1996) · Zbl 0886.49024
[3] Attouch, H.; Czarnecki, M.-O., Asymptotic behavior of coupled dynamical systems with multiscale aspects, Journal of Differential Equations, 248, 1315-1344 (2010) · Zbl 1190.37090
[4] Aubin, J. P., Viability theory (1991), Birkhäuser: Birkhäuser Cambridge, MA · Zbl 0755.93003
[5] Aubin, J. P.; Cellina, A., Differential inclusions (1984), Springer-Verlag: Springer-Verlag Berlin, Germany
[6] Bazaraa, M. S.; Sherali, H. D.; Shetty, C. M., Nonlinear programming: Theory and algorithms (1993), John Wiley & Sons · Zbl 0774.90075
[7] Bellen, A.; Jackiewicz, Z.; Zennaro, M., Stability analysis of one-step methods for neutral delay-differential equations, Numerische Mathematik, 52, 605-619 (1988) · Zbl 0644.65049
[8] Bian, W.; Xue, X., Subgradient-based neural networks for nonsmooth nonconvex optimization problems, IEEE Transactions on Neural Networks, 20, 1024-1038 (2009)
[9] Bian, W.; Xue, X., Neural network for solving constrained convex optimization problems with global attractivity, IEEE Transactions on Circuits and Systems I: Regular Papers, 60, 710-723 (2013) · Zbl 1468.90081
[10] Charalambous, C.; Conn, A., An efficient method to solve the minimax problem directly, SIAM Journal on Numerical Analysis, 15, 162-187 (1978) · Zbl 0384.65032
[11] Cheng, L.; Hou, Z.-G.; Lin, Y.; Tan, M.; Zhang, W. C.; Wu, F.-X., Recurrent neural network for non-smooth convex optimization problems with application to the identification of genetic regulatory networks, IEEE Transactions on Neural Networks, 22, 714-726 (2011)
[12] Clarke, F., Optimization and nonsmooth analysis (1983), Wiley: Wiley New York · Zbl 0582.49001
[13] Cominetti, R.; Peypouquet, J.; Sorin, S., Strong asymptotic convergence of evolution equations governed by maximal monotone operators with Tikhonov regularization, Journal of Differential Equations, 245, 3753-3763 (2008) · Zbl 1169.34045
[14] Deng, M.; Bu, N., Robust control for nonlinear systems using passivity-based robust right coprime factorization, IEEE Transactions on Automatic Control, 57, 2599-2604 (2012) · Zbl 1369.93243
[15] Di Pillo, G.; Grippo, L.; Lucidi, S., Smooth transformation of the generalized minimax problem, Journal of Optimization Theory and Applications, 95, 1-24 (1997) · Zbl 0890.90165
[16] Filippov, A., Differential equations with discontinuous right-hand side, American Mathematical Society Translations, 42, 199-231 (1964) · Zbl 0148.33002
[17] Forti, M.; Nistri, P.; Quincampoix, M., Generalized neural network for nonsmooth nonlinear programming problems, IEEE Transactions on Circuits and Systems I: Regular Papers, 51, 1741-1754 (2004) · Zbl 1374.90356
[18] Forti, M.; Nistri, P.; Quincampoix, M., Convergence of neural networks for programming problems via a nonsmooth Lojasiewicz inequality, IEEE Transactions on Neural Networks, 17, 1471-1486 (2006)
[19] Gao, X.-B.; Liao, L.-Z., A new one-layer neural network for linear and quadratic programming, IEEE Transactions on Neural Networks, 21, 918-929 (2010)
[20] Guo, Z.; Liu, Q.; Wang, J., A one-layer recurrent neural network for pseudoconvex optimization subject to linear equality constraints, IEEE Transactions on Neural Networks, 22, 1892-1900 (2011)
[21] Hung, P. G.; Muu, L. D., The Tikhonov regularization extended to equilibrium problems involving pseudomonotone bifunctions, Nonlinear Analysis. Theory, Methods & Applications, 74, 6121-6129 (2011) · Zbl 1262.47084
[22] Kennedy, M. P.; Chua, L. O., Neural networks for nonlinear programming, IEEE Transactions on Circuits and Systems, 35, 554-562 (1988)
[23] Liu, Q.; Cao, J.; Chen, G., A novel recurrent neural network with finite-time convergence for linear programming, Neural Computation, 22, 2962-2978 (2010) · Zbl 1208.68175
[24] Liu, Q.; Dang, C.; Cao, J., A novel recurrent neural network with one neuron and finite-time convergence for \(k\)-winners-take-all operation, IEEE Transactions on Neural Networks, 21, 1140-1148 (2010)
[25] Liu, Q.; Guo, Z.; Wang, J., A one-layer recurrent neural network for constrained pseudoconvex optimization and its application for dynamic portfolio optimization, Neural Networks, 26, 99-109 (2012) · Zbl 1273.90216
[26] Liu, Q.; Wang, J., A one-layer recurrent neural network with a discontinuous activation function for linear programming, Neural Computation, 20, 1366-1383 (2008) · Zbl 1135.68535
[27] Liu, Q.; Wang, J., A one-layer recurrent neural network with a discontinuous hard-limiting activation function for quadratic programming, IEEE Transactions on Neural Networks, 19, 558-570 (2008)
[28] Liu, Q.; Wang, J., A one-layer recurrent neural network for constrained nonsmooth optimization, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 40, 1323-1333 (2011)
[29] Mannshardt, R., One-step methods of any order for ordinary differential equations with discontinuous right-hand sides, Numerische Mathematik, 31, 131-152 (1978) · Zbl 0373.65037
[30] Oliveira, P.; Santos, P.; Silva, A., A Tikhonov-type regularization for equilibrium problems in Hilbert spaces, Journal of Mathematical Analysis and Applications (2012) · Zbl 1259.47075
[31] Qin, S.; Xue, X., Dynamical analysis of neural networks of subgradient system, IEEE Transactions on Automatic Control, 55, 2347-2352 (2010) · Zbl 1368.49043
[32] Qin, S.; Xue, X., Dynamical behavior of a class of nonsmooth gradient-like systems, Neurocomputing, 73, 2632-2641 (2010)
[33] Qin, S.; Xue, X.; Wang, P., Global exponential stability of almost periodic solution of delayed neural networks with discontinuous activations, Information Sciences, 220, 367-378 (2013) · Zbl 1291.92015
[34] Stewart, D. E., High accuracy numerical methods for ordinary differential equations with discontinuous right-hand side, Bulletin of the Australian Mathematical Society, 42, 169-170 (1990) · Zbl 0701.34028
[35] Stewart, D. E., Dynamics with inequalities (2011), Society for Industrial and Applied Mathematics
[36] Tank, D.; Hopfield, J., Simple ‘neural’ optimization networks: an A/D converter, signal decision circuit, and a linear programming circuit, IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 33, 533-541 (1986)
[37] Taubert, D. K., Converging multistep methods for initial value problems involving multivalued maps, Computing, 27, 123-136 (1981) · Zbl 0465.65038
[38] Tuy, H., Convex analysis and global optimization (1998), Springer-Verlag: Springer-Verlag New York · Zbl 0904.90156
[39] Wang, J., A deterministic annealing neural network for convex programming, Neural Networks, 7, 629-641 (1994) · Zbl 0818.90090
[40] Xia, Y.; Wang, J., A one-layer recurrent neural network for support vector machine learning, IEEE Transactions on Systems, Man, and Cybernetics, Part B, 34, 1261-1269 (2004)
[41] Xia, Y.; Wang, J., A recurrent neural network for solving nonlinear convex programs subject to linear constraints, IEEE Transactions on Neural Networks, 16, 379-386 (2005)
[42] Xue, X.; Bian, W., Subgradient-based neural networks for nonsmooth convex optimization problems, IEEE Transactions on Circuits and Systems I: Regular Papers, 55, 2378-2391 (2008)
[43] Yang, Y.; Cao, J., The optimization technique for solving a class of non-differentiable programming based on neural network method, Nonlinear Analysis: Real World Applications, 11, 1108-1114 (2010) · Zbl 1181.90251
[44] Zhang, S.; Constantinides, A., Lagrange programming neural networks, IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, 39, 441-452 (1992) · Zbl 0758.90067
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.