×

An inertial Bregman generalized alternating direction method of multipliers for nonconvex optimization. (English) Zbl 1486.90160


MSC:

90C26 Nonconvex programming, global optimization
90C30 Nonlinear programming

Software:

iPiano; GADMM; BADMM
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Adona, VA; Gonçalves, ML; Melo, JG, Iteration-complexity analysis of a generalized alternating direction method of multipliers, J. Global Optim., 73, 2, 331-348 (2019) · Zbl 1482.90143 · doi:10.1007/s10898-018-0697-z
[2] Attouch, H.; Bolte, J., On the convergence of the proximal algorithm for nonsmooth functions involving analytic features, Math. Program., 116, 1-2, 5-16 (2009) · Zbl 1165.90018 · doi:10.1007/s10107-007-0133-5
[3] Attouch, H.; Bolte, J.; Redont, P.; Soubeyran, A., Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the Kurdyka-Łojasiewicz inequality, Math. Oper. Res., 35, 2, 438-457 (2010) · Zbl 1214.65036 · doi:10.1287/moor.1100.0449
[4] Bregman, LM, The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex programming, USSR Comput. Math. Math. Phys., 7, 3, 200-217 (1964) · doi:10.1016/0041-5553(67)90040-7
[5] Bolte, J.; Sabach, S.; Teboulle, M., Proximal alternating linearized minimization for nonconvex and nonsmooth problems, Math. Program., 146, 1-2, 459-494 (2014) · Zbl 1297.90125 · doi:10.1007/s10107-013-0701-9
[6] Boţ, RI; Csetnek, ER; László, SC, An inertial forward-backward algorithm for the minimization of the sum of two nonconvex functions, EURO J. Comput. Optim., 4, 1, 3-25 (2016) · Zbl 1338.90311 · doi:10.1007/s13675-015-0045-8
[7] Boţ, RI; Nguyen, DK, The proximal alternating direction method of multipliers in the nonconvex setting: convergence analysis and rates, Math. Oper. Res. (2020) · Zbl 1480.90198 · doi:10.1287/moor.2019.1008
[8] Boyd, S.; Parikh, N.; Chu, E.; Peleato, B.; Eckstein, J., Distributed optimization and statistical learning via the alternating direction method of multipliers, Found. Trends Mach. Learn., 3, 1, 1-122 (2011) · Zbl 1229.90122 · doi:10.1561/2200000016
[9] Cai, X.; Han, D., O (1/t) complexity analysis of the generalized alternating direction method of multipliers, Sci. China. Math., 62, 4, 795-808 (2019) · Zbl 1431.65083 · doi:10.1007/s11425-016-9184-4
[10] Chao, M.; Zhang, Y.; Jian, J., An inertial proximal alternating direction method of multipliers for nonconvex optimization, Int. J. Comput. Math. (2020) · Zbl 1479.90165 · doi:10.1080/00207160.2020.1812585
[11] Chao, M.; Cai, X.; Han, D., Convergence of the Peaceman-Rachford splitting method for a class of nonconvex programs, Numer. Math., 14, 2, 438-460 (2021) · Zbl 1488.90142
[12] Chartrand, R.; Staneva, V., Restricted isometry properties and nonconvex compressive sensing, Inverse. Probl., 24, 3, 035020 (2008) · Zbl 1143.94004 · doi:10.1088/0266-5611/24/3/035020
[13] Chen, C.; Ma, S.; Yang, J., A general inertial proximal point algorithm for mixed variational inequality problem, SIAM J. Optim., 25, 2120-2142 (2015) · Zbl 1327.65106 · doi:10.1137/140980910
[14] Chen, C.; Chan, RH; Ma, S.; Yang, J., Inertial proximal ADMM for linearly constrained separable convex optimization, SIAM J. Imaging. Sci., 8, 4, 2239-2267 (2015) · Zbl 1328.65134 · doi:10.1137/15100463X
[15] Combettes, PL; Wajs, VR, Signal recovery by proximal forward-backward splitting, Multiscale Model. Simul., 4, 4, 1168-1200 (2005) · Zbl 1179.94031 · doi:10.1137/050626090
[16] Douglas, J.; Rachford, HH, On the numerical solution of heat conduction problems in two or three space variables, Trans. Am. Math. Soc., 82, 421-439 (1956) · Zbl 0070.35401 · doi:10.1090/S0002-9947-1956-0084194-4
[17] Eckstein, J.; Bertsekas, DP, On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators, Math. Program., 55, 1-3, 293-318 (1992) · Zbl 0765.90073 · doi:10.1007/BF01581204
[18] Fan, J.; Li, R., Variable selection via nonconcave penalized likelihood and its oracle properties, J. Am. Stat. Assoc., 96, 456, 1348-1360 (2001) · Zbl 1073.62547 · doi:10.1198/016214501753382273
[19] Fang, EX; He, B.; Liu, H.; Yuan, X., Generalized alternating direction method of multipliers: new theoretical insights and applications, Math. Program. Comput., 7, 2, 149-187 (2015) · Zbl 1353.90110 · doi:10.1007/s12532-015-0078-2
[20] Gabay, D.; Mercier, B., A dual algorithm for the solution of nonlinear variational problems via finite element approximation, Comput. Math. Appl., 2, 1, 17-40 (1976) · Zbl 0352.65034 · doi:10.1016/0898-1221(76)90003-1
[21] Gao, X.; Cai, X.; Han, D., A Gauss-Seidel type inertial proximal alternating linearized minimization for a class of nonconvex optimization problems, J. Global Optim., 76, 4, 863-887 (2020) · Zbl 1476.90256 · doi:10.1007/s10898-019-00819-5
[22] Glowinski, R.; Marrocco, A., Sur l’approximation, par elements finis d’ordre un, et la resolution, par penalisation-dualit’e, d’une classe de problems de Dirichlet non lineares, Ann. Math. Stat., 9, 41-76 (1975) · Zbl 0368.65053
[23] Goncalves, M.L.N., Melo, J.G., Monteiro, R.D.C.: Convergence rate bounds for a proximal ADMM with over-relaxation stepsize parameter for solving nonconvex linearly constrained problems (2017). arXiv:1702.01850 · Zbl 1484.65135
[24] Guo, K.; Han, D.; Wu, T., Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints, Int. J. Comput. Math., 94, 8, 1653-1669 (2017) · Zbl 1375.90239 · doi:10.1080/00207160.2016.1227432
[25] Hien, L. T. K., Gillis, N.,Patrinos, P.: A Framework of inertial alternating direction method of multipliers for non-convex non-smooth optimization (2021). arXiv:2102.05433v1
[26] Hien, L.T.K., Phan, D.N., Gillis, N.: Inertial block proximal method for non-convex non-smooth optimization. In: Presented at the International Conference on Machine Learning (2020)
[27] Hong, M.; Luo, Z.; Razaviyayn, M., Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems, SIAM J. Optim., 26, 1, 337-364 (2016) · Zbl 1356.49061 · doi:10.1137/140990309
[28] Jian, J., Liu, P., Jiang, X.: A partially symmetric regularized alternating direction method of multipliers for nonconvex multi-block optimization. Acta. Math. Sin. Chin. Ser. (2020). https://kns.cnki.net/kcms/detail/11.2038.o1.20201127.1522.014.html
[29] Li, G.; Pong, T., Global convergence of splitting methods for nonconvex composite optimization, SIAM J. Optim., 25, 4, 2434-2460 (2015) · Zbl 1330.90087 · doi:10.1137/140998135
[30] Lions, PL; Mercier, B., Splitting algorithms for the sum of two nonlinear operators, SIAM J. Numer. Anal., 16, 964-979 (1979) · Zbl 0426.65050 · doi:10.1137/0716071
[31] Liu, J., Chen, J., Ye, J.: Large-scale sparse logistic regression. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 547-556 (2009)
[32] Mordukhovich, BS, Variational Analysis and Generalized Differentiation I: Basic Theory (2006), Berlin: Springer, Berlin · doi:10.1007/3-540-31246-3
[33] Nesterov, Y., A method of solving a convex programming problem with convergence rate \(O(1/k^2)\), Soviet Math. Dokl., 27, 2, 372-375 (1983) · Zbl 0535.90071
[34] Nesterov, Y., On an approach to the construction of optimal methods of minimization of smooth convex functions, Ekonom. i. Mat. Metody., 24, 509-514 (1998) · Zbl 0659.90068
[35] Nesterov, Y., Introductory Lectures on Convex Optimization: A Basic Course (2004), Cambridge: Kluwer Academic Publishing, Cambridge · Zbl 1086.90045 · doi:10.1007/978-1-4419-8853-9
[36] Nesterov, Y., Smooth minimization of non-smooth functions, Math. Prog., 103, 1, 127-152 (2005) · Zbl 1079.90102 · doi:10.1007/s10107-004-0552-5
[37] Ochs, P.; Chen, Y.; Brox, T.; Pock, T., iPiano: Inertial proximal algorithm for nonconvex optimization, SIAM J. Imaging Sci, 7, 1388-1419 (2014) · Zbl 1296.90094 · doi:10.1137/130942954
[38] Peaceman, DW; Rachford, HH Jr, The numerical solution of parabolic and elliptic differential equations, J. Soc. Ind. Appl. Math., 3, 28-41 (1955) · Zbl 0067.35801 · doi:10.1137/0103003
[39] Polyak, B., Some methods of speeding up the convergence of iteration methods, USSR Comput. Math. Math. Phys., 4, 1-17 (1964) · Zbl 0147.35301 · doi:10.1016/0041-5553(64)90137-5
[40] Pock, T.; Sabach, S., Inertial proximal alternating linearized minimization (iPALM) for nonconvex and nonsmooth problems, SIAM J. Imaging. Sci., 9, 4, 1756-1787 (2016) · Zbl 1358.90109 · doi:10.1137/16M1064064
[41] Rockafellar, RT; Wets, RJB, Variational Analysis (2009), Berlin: Springer, Berlin · Zbl 0888.49001
[42] Sahu, DR; Singh, AK, Inertial iterative algorithms for common solution of variational inequality and system of variational inequalities problems, J. Appl. Math. Comput., 76, 351-378 (2021) · Zbl 1510.47096 · doi:10.1007/s12190-020-01395-8
[43] Tu, K.; Zhang, H.; Gao, H.; Feng, J., A hybrid Bregman alternating direction method of multipliers for the linearly constrained difference-of-convex problems, J. Global Optim., 76, 4, 665-693 (2020) · Zbl 1448.90088 · doi:10.1007/s10898-019-00828-4
[44] Wang, F., Xu, Z., Xu, H.: Convergence of Bregman alternating direction method with multipliers for nonconvex composite problems (2014). arXiv:1410.8625
[45] Wu, Z.; Li, C.; Li, M.; Lim, A., Inertial proximal gradient methods with Bregman regularization for a class of nonconvex optimization problems, J. Global. Optim. (2020) · Zbl 1466.90081 · doi:10.1007/s10898-020-00943-7
[46] Wu, Z.; Li, M.; Wang, DZ; Han, D., A symmetric alternating direction method of multipliers for separable nonconvex minimization problems, Asia-Pac. J. Oper. Res., 34, 6, 1750030 (2017) · Zbl 1383.90028 · doi:10.1142/S0217595917500300
[47] Xiu, X.; Liu, W.; Li, L.; Kong, L., Alternating direction method of multipliers for nonconvex fused regression problems, Comput. Stat. Data Anal., 136, 59-71 (2019) · Zbl 1507.62191 · doi:10.1016/j.csda.2019.01.002
[48] Xu, Z.; Chang, X.; Xu, F.; Zhang, H., \( l_{1/2}\) regularization: A thresholding representation theory and a fast solver, IEEE. Trans. Neural. Netw. Learn. Syst., 23, 7, 1013-1027 (2012) · doi:10.1109/TNNLS.2012.2197412
[49] Zavriev, S.; Kostyuk, F., Heavy-ball method in nonconvex optimization problems, Comput. Math. Model., 4, 336-341 (1993) · Zbl 1331.90056 · doi:10.1007/BF01128757
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.