Actuarial risk matrices: the nearest positive semidefinite matrix problem. (English) Zbl 1414.91177

Summary: The manner in which a group of insurance risks are interrelated is commonly presented via a correlation matrix. Actuarial risk correlation matrices are often constructed using output from disparate modeling sources and can be subjectively adjusted, for example, increasing the estimated correlation between two risk sources to confer reserving prudence. Hence, while individual elements still obey the assumptions of correlation values, the overall matrix is often not mathematically valid (not positive semidefinite). This can prove problematic in using the matrix in statistical models. The first objective of this article is to review existing techniques that address the nearest positive semidefinite matrix problem in a very general setting. The chief approaches studied are semidefinite programming (SDP) and the alternating projections method (APM). The second objective is to finesse the original problem specification to consider imposition of a block structure on the initial risk correlation matrix. This commonly employed technique identifies off-diagonal subsets of the matrix where values can or should be set equal to some constant. This may be due to similarity of the underlying risks and/or with the goal of increasing computational efficiency for processes involving large matrices. Implementation of further linear constraints of this nature requires adaptation of the standard SDP and APM algorithms. In addition, a new shrinking method is proposed to provide an alternative solution in the context of this increased complexity. “Nearness” is primarily considered in terms of two summary measures for differences between matrices: the Chebychev norm (maximum element distance) and the Frobenius norm (sum of squared element distances). Among the existing methods, adapted to function appropriately for actuarial risk matrices, APM is extremely efficient in producing solutions that are optimal in the Frobenius norm. An efficient algorithm that would return a positive semidefinite matrix that is optimal in Chebychev norm is currently unknown. However, APM is used to highlight the existence of matrices close to such an optimum and exploited, via the shrinking method, to find high-quality solutions. All methods are shown to work well both on artificial and real actuarial risk matrices provided under collaboration with Tokio Marine Kiln (TMK). Convergence speeds are calculated and compared and sample data and MATLAB code is provided. Ultimately the APM is identified as being superior in Frobenius distance and convergence speed. The shrinking method, building on the output of the APM algorithm, is demonstrated to provide excellent results at low computational cost for minimizing Chebychev distance.


91B30 Risk theory, insurance (MSC2010)
15B48 Positive matrices and their generalizations; cones of matrices
65F30 Other matrix algorithms (MSC2010)
90C22 Semidefinite programming


Mosek; Matlab; SeDuMi
Full Text: DOI


[1] Borsdorf, R.; Higham, N. J.; Raydan, M., Computing a Nearest Correlation Matrix with Factor Structure, SIAM J. Matrix Anal. Appl., 31, 5, 2603-2622, (2010) · Zbl 1213.65022
[2] Boyle, J. P.; Dykstra, R. L., A Method for Finding Projections onto the Intersection of Convex Sets in Hilbert Spaces, Lecture Notes in Statistics, 37, 28-47, 4, (1986)
[3] Burtschell, X.; Gregory, J.; Laurent, J. P., A Comparative Analysis of CDO Pricing Models, ISFA Actuarial School and BNP Parisbas, (2005)
[4] Byers, R., A Bisection Method for Measuring the Distance of a Stable Matrix to the Unstable Matrices, SIAM Journal on Scientific and Statistical Computing, 9, 875-881, (1988) · Zbl 0658.65044
[5] Cantrell, C. D., Modern Mathematical Methods for Physicists and Engineers, (2000), London: Cambridge University Press, London · Zbl 0983.00005
[6] Coleman, T. F.; Li, Y., On the Convergence of Interior-Reflective Newton Methods for Nonlinear Minimization Subject to Bounds, Mathematical Programming, 67, 189-224, (1994) · Zbl 0842.90106
[7] Deutsch, F., The Angle between Subspaces of a Hilbert Space, NATO ASI Series C Mathematical and Physical Sciences-Advanced Study Institute, 454, 107-130, (1995) · Zbl 0848.46010
[8] Deutsch, F.; Hundal, H., The Rate of Convergence of Dykstra’s Cyclic Projections Algorithm: The Polyhedral Case, Numerical Functional Analysis and Optimization, 15, 537-565, (1994) · Zbl 0807.41019
[9] Dykstra, R. L., An Algorithm for Restricted Least Squares Regression, Journal of the American Statistical Association, 78, 837-842, (1983) · Zbl 0535.62063
[10] Ecker, J. G., Geometric Programming: Methods, Computations and Applications, SIAM review, 22, 338-362, (1980) · Zbl 0438.90088
[11] Golub, G.; VanLoan, C. F., Matrix Computations, (1996), Baltimore: Johns Hopkins University Press, Baltimore
[12] Higham, N. J., Computing the Nearest Correlation Matrix—A Problem from Finance, IMA Journal of Numerical Analysis, 22, 3, 329-343, (2002) · Zbl 1006.65036
[13] Higham, N. J.; Strabic, N.; Šego, V., Restoring Definiteness via Shrinking, with an Application to Correlation Matrices with a Fixed Block., (2014) · Zbl 1386.15062
[14] Higham, N. J.; Strabic, N.; Šego, V., Restoring Definiteness via Shrinking, with an Application to Correlation Matrices with a Fixed Block., (2014) · Zbl 1386.15062
[15] Horn, R. A.; Johnson, C. R., Matrix Analysis, (1985), London: Cambridge University Press, London · Zbl 0576.15001
[16] Johnson, C. R.; Kroschel, B.; Wolkowicz, H., An Interior-Point Method for Approximate Positive Semidefinite Completions, Computational Optimization and Applications, 9, 2, 175-190, (1998) · Zbl 0907.90207
[17] Luenberger, D. G., Optimization by Vector Space Methods, (1969), New York: John Wiley & Sons, New York · Zbl 0176.12701
[18] Mishra, S. K., Optimal Solution of the Nearest Correlation Matrix Problem by Minimization of the Maximum Norm, (2004)
[19] Mishra, S. K., The Nearest Correlation Matrix Problem: Solution by Differential Evolution Method of Global optimization, Available at SSRN 980403, (2007)
[20] The MOSEK Optimization Toolbox for MATLAB Manual. Version 7.1 (Revision 28)., (2015)
[21] Pearson, K., Notes on Regression and Inheritance in the Case of Two Parents, Proceedings of the Royal Society of London, 58, 242-242, (1895)
[22] Shih-Ping, H., A Successive Projection Method, Mathematical Programming, 40, 1-14, (1988) · Zbl 0685.90074
[23] Spearman, C., The Proof and Measurement of Association between Two Things, The American Journal of Psychology, 15, 72-101, (1904)
[24] Sturm, J. F., Using SeDuMi 1.02, a MATLAB Toolbox for Optimization over Symmetric Cones., Optimization Methods and Software, 11, 625-653, (1999) · Zbl 0973.90526
[25] Todd, M. J., Semidefinite Optimization, Acta Numerica, 10, 515-560, (2001) · Zbl 1105.65334
[26] Von Neumann, J., Functional Operators: Measures and Integrals (Vol. 1), (1950), Princeton, NJ: Princeton University Press, Princeton, NJ · Zbl 0039.28401
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.