Orthogonal dual graph-regularized nonnegative matrix factorization for co-clustering. (English) Zbl 1466.62359

Summary: Nonnegative Matrix Factorization (NMF) has received great attention in the era of big data, owing to its roles in efficiently reducing data dimension and producing feature-based data representation. In this paper, we first propose two new NMF optimization models, called an orthogonal dual graph regularized nonnegative matrix factorization (ODGNMF) method and its modified version: an orthogonal dual graph regularized nonnegative matrix tri-factorization (ODGNMTF) method. Compared with the existing models, our models can preserve the geometrical structures of data manifold and feature manifold by constructing two graphs, and ensure the orthogonality of factor matrices such that they have better NMF performance. Then, two efficient algorithms are developed to solve the models, and the convergence theory of the algorithms is established. Numerical tests by applying our algorithms to mine randomly generated data sets and well-known public databases demonstrate that ODGNMF and ODGNMTF have better numerical performance than the state-of-the-art algorithms in view of computational cost, robustness, sensitivity and sparseness.


62H30 Classification and discrimination; cluster analysis (statistical aspects)
62H22 Probabilistic graphical models
15A23 Factorization of matrices
68T05 Learning and adaptive systems in artificial intelligence
Full Text: DOI


[1] Zhang, L.; Liu, Z.; Pu, J., Adaptive graph regularized nonnegative matrix factorization for data representation, Appl. Intell., 50, 438-447 (2020)
[2] Gao, Z.; Wang, Y.; Wu, Q., Graph regularized L2,1-nonnegative matrix factorization for miRNA-disease association prediction, BMC Bioinform., 21, 61 (2020)
[3] Wan, Z.; Tang, J.; Ren, L., Optimization techniques to deeply mine the transcriptomic profile of the sub-genomes in hybrid fish lineage, Front. Genet., 10, 911 (2019)
[4] Jolliffe, IT; Cadima, J., Principal component analysis: a review and recent developments, Philos. Trans. R. Soc. A Math. Phys. Eng. Sci., 374, 20150202 (2016) · Zbl 1353.62067
[5] Comon, P.; Jutten, C., Handbook of Blind Source Separation: Independent Component Analysis and Applications (2010), Cambridge: Academic Press, Cambridge
[6] Lieven, DL; Bart, DM; Joos, V., A multilinear singular value decomposition, SIAM J. Matrix Anal. Appl., 21, 1253-1278 (2000) · Zbl 0962.15005
[7] Wang, H.; Zheng, C.; Zhao, X., jNMFMA: a joint non-negative matrix factorization meta-analysis of transcriptomics data, Bioinformatics, 31, 572-580 (2015)
[8] Shang, R.; Song, J.; Jiao, L., Double feature selection algorithm based on low-rank sparse non-negative matrix factorization, Int. J. Mach. Learn. Cybernet., 11, 1891-1908 (2020)
[9] Belachew, MT, Efficient algorithm for sparse symmetric nonnegative matrix factorization, Pattern Recogn. Lett., 125, 735-741 (2019)
[10] Tosyali, A.; Kim, J.; Choi, J., Regularized asymmetric nonnegative matrix factorization for clustering in directed networks, Pattern Recogn. Lett., 125, 750-757 (2019)
[11] Peng, S.; Ser, W.; Chen, B., Robust nonnegative matrix factorization with local coordinate constraint for image clustering, Eng. Appl. Artif. Intell., 88, 103354 (2020)
[12] Chen, G.; Xu, C.; Wang, J., Graph regularization weighted nonnegative matrix factorization for link prediction in weighted complex network, Neurocomputing, 369, 50-60 (2020)
[13] Hoyer, PO, Non-negative matrix factorization with sparseness constraints, J. Mach. Learn. Res., 5, 1457-1469 (2004) · Zbl 1222.68218
[14] Cai, D.; He, X.; Han, J., Graph regularized nonnegative matrix factorization for data representation, IEEE Trans. Pattern Anal. Mach. Intell., 33, 1548-1560 (2011)
[15] Li, X.; Chen, M.; Wang, Q., Discrimination-aware projected matrix factorization, IEEE Trans. Knowl. Data Eng., 32, 809-814 (2020)
[16] Wang, Q.; He, X.; Jiang, X., Robust Bi-stochastic graph regularized matrix factorization for data clustering, IEEE Trans. Pattern Anal. Mach. Intell. (2020)
[17] Gu, Q., Zhou, J.: Co-clustering on manifolds. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 359-368 (2009). doi:10.1145/1557019.1557063
[18] Shang, F.; Jiao, LC; Wang, F., Graph dual regularization non-negative matrix factorization for co-clustering, Pattern Recogn., 45, 2237-2250 (2012) · Zbl 1234.68356
[19] Wang, Q.; Chen, M.; Nie, F., Detecting coherent groups in crowd scenes by multi-view clustering, IEEE Trans. Pattern Anal. Mach. Intell., 42, 46-58 (2020)
[20] Sun, J.; Wang, Z.; Sun, F., Sparse dual graph-regularized NMF for image co-clustering, Neurocomputing, 316, 156-165 (2018)
[21] Ding, C., Li, T., Peng, W., et al.: Orthogonal nonnegative matrix t-factorizations for clustering. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 126-135 (2006). doi:10.1145/1150402.1150420
[22] Yoo, J.; Choi, S., Orthogonal nonnegative matrix tri-factorization for co-clustering: Multiplicative updates on Stiefel manifolds, Inf. Process. Manage., 46, 559-570 (2010)
[23] Dhillon, I.S.: Co-clustering documents and words using bipartite spectral graph partitioning. In: Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 269-274 (2001). doi:10.1145/502512.502550
[24] Dhillon, I.S., Mallela, S., Modha, D.S.: Information-theoretic co-clustering. Proceedings of the 9h ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 89-98 (2003). doi:10.1145/956750.956764
[25] Wang, S., Chang, T., Cui, Y., et al.: Clustering by orthogonal non-negative matrix factorization: a sequential non-convex penalty approach. In: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5576-5580 (2019). doi:10.1109/ICASSP.2019.8683466
[26] He, P.; Xu, X.; Ding, J., Low-rank nonnegative matrix factorization on Stiefel manifold, Inf. Sci., 514, 131-148 (2020) · Zbl 1462.62773
[27] Abe, H.; Yadohisa, H., Orthogonal nonnegative matrix tri-factorization based on Tweedie distributions, Adv. Data Anal. Classif., 13, 825-853 (2019) · Zbl 07176223
[28] Edelman, A.; Arias, TA; Smith, ST, The geometry of algorithms with orthogonality constraints, SIAM J. Matrix Anal. Appl., 20, 303-353 (1998) · Zbl 0928.65050
[29] Lee, DD; Seung, HS, Learning the parts of objects by non-negative matrix factorization, Nature, 401, 788-791 (1999) · Zbl 1369.68285
[30] Stiefel, E.: Richtungsfelder und Fernparallelismus in n-dimensionalen Mannigfaltigkeiten. Commentarii Mathematici Helvetici 8, 305-353 (1935-1936) · JFM 62.0662.02
[31] Lv, J.; Deng, S.; Wan, Z., An efficient single-parameter scaling memoryless Broyden-Fletcher-Goldfarb-Shanno algorithm for solving large scale unconstrained optimization problems, IEEE Access, 8, 85664-85674 (2020)
[32] Li, T.; Wan, Z., New adaptive Barzilar-Borwein step size and its application in solving large scale optimization problems, The ANZIAM J., 61, 76-98 (2019) · Zbl 1409.90189
[33] Guo, J.; Wan, Z., A modified spectral PRP conjugate gradient projection method for solving large-scale monotone equations and its application in compressed sensing, Math. Probl. Eng., 2019, 5261830 (2019) · Zbl 1435.65078
[34] Gaussier, E., Goutte, C.: Relation between PLSA and NMF and implications. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 601-602 (2005). doi:10.1145/1076034.1076148
[35] Ding, C.; Li, T.; Peng, W., On the equivalence between non-negative matrix factorization and probabilistic latent semantic indexing, Comput. Stat. Data Anal., 52, 3913-3927 (2008) · Zbl 1452.62053
[36] Cai, D.; He, X.; Han, J., Document clustering using locality preserving indexing, IEEE Trans. Knowl. Data Eng., 17, 1624-1637 (2005)
[37] Huang, S.; Wan, Z.; Zhang, J., An extended nonmonotone line search technique for large-scale unconstrained optimization, J. Comput. Appl. Math., 330, 586-604 (2018) · Zbl 1373.90150
[38] Ding, C., He, X., Simon, H.D.: On the equivalence of nonnegative matrix factorization and spectral clustering. In: Proceedings of the 2005 SIAM International Conference on Data Mining, pp. 606-610 (2005). doi:10.1137/1.9781611972757.70
[39] Sun, J.; Cai, X.; Sun, F., Dual graph-regularized constrained nonnegative matrix factorization for image clustering, KSII Trans. Internet Inf. Syst., 11, 2607-2627 (2017)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.