Orthogonal nonnegative matrix tri-factorization based on Tweedie distributions. (English) Zbl 07176223

Summary: Orthogonal nonnegative matrix tri-factorization (ONMTF) is a biclustering method using a given nonnegative data matrix and has been applied to document-term clustering, collaborative filtering, and so on. In previously proposed ONMTF methods, it is assumed that the error distribution is normal. However, the assumption of normal distribution is not always appropriate for nonnegative data. In this paper, we propose three new ONMTF methods, which respectively employ the following error distributions: normal, Poisson, and compound Poisson. To develop the new methods, we adopt a \(k\)-means based algorithm but not a multiplicative updating algorithm, which was the main method used for obtaining estimators in previous methods. A simulation study and an application involving document-term matrices demonstrate that our method can outperform previous methods, in terms of the goodness of clustering and in the estimation of the factor matrix.


62H30 Classification and discrimination; cluster analysis (statistical aspects)
68T10 Pattern recognition, speech recognition
15A23 Factorization of matrices


openBliSSART; WebACE
Full Text: DOI


[1] Ailem, M.; Role, F.; Nadif, M., Graph modularity maximization as an effective method for co-clustering text data, Knowl Based Syst, 109, 160-173 (2016)
[2] Banerjee A, Dhillon I, Ghosh J, Sra S (2003) Generative model-based clustering of directional data. In: Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, ACM, pp 19-28
[3] Berry, MW; Browne, M.; Langville, AN; Pauca, VP; Plemmons, RJ, Algorithms and applications for approximate nonnegative matrix factorization, Comput Stat Data Anal, 52, 155-173 (2007) · Zbl 1452.90298
[4] Boley D (1998) Hierarchical taxonomies using divisive partitioning. Technical Report TR-98-012, Department of Computer Science, University of Minnesota, Minneapolis
[5] Boley, D.; Gini, M.; Gross, R.; Han, EHS; Hastings, K.; Karypis, G.; Kumar, V.; Mobasher, B.; Moore, J., Document categorization and query generation on the world wide web using webace, Artif Intell Rev, 13, 365-391 (1999)
[6] Carabias-Orti, JJ; Rodríguez-Serrano, FJ; Vera-Candeas, P.; Cañadas-Quesada, FJ; Ruiz-Reyes, N., Constrained non-negative sparse coding using learnt instrument templates for realtime music transcription, Eng Appl Artif Intell, 26, 1671-1680 (2013)
[7] Chen, G.; Wang, F.; Zhang, C., Collaborative filtering using orthogonal nonnegative matrix tri-factorization, Inf Process Manag, 45, 368-379 (2009)
[8] Choi S (2008) Algorithms for orthogonal nonnegative matrix factorization. In: Neural Networks, 2008. IJCNN 2008. IEEE international joint conference on IEEE world congress on computational intelligence, IEEE, pp 1828-1832
[9] Cichocki, A.; Amari, Si, Families of alpha-beta-and gamma-divergences: flexible and robust measures of similarities, Entropy, 12, 1532-1568 (2010) · Zbl 1229.94030
[10] Costa G, Ortale R (2014) XML document co-clustering via non-negative matrix tri-factorization. In: 2014 IEEE 26th international conference on tools with artificial intelligence (ICTAI), IEEE, pp 607-614
[11] Ding C, Li T, Peng W, Park H (2006) Orthogonal nonnegative matrix tri-factorizations for clustering. In: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, ACM, pp 126-135
[12] Dunn PK, Smyth GK (2001) Tweedie family densities: methods of evaluation. In: Proceedings of the 16th international workshop on statistical modelling, Odense, Denmark, pp 2-6
[13] Févotte, C.; Idier, J., Algorithms for nonnegative matrix factorization with the \(\beta \)-divergence, Neural Comput, 23, 2421-2456 (2011) · Zbl 1231.65072
[14] Févotte, C.; Bertin, N.; Durrieu, JL, Nonnegative matrix factorization with the Itakura-Saito divergence: with application to music analysis, Neural Comput, 21, 793-830 (2009) · Zbl 1156.94306
[15] Govaert G, Nadif M (2013) Co-clustering. Wiley, Hoboken
[16] Hubert, L.; Arabie, P., Comparing partitions, J Classif, 2, 193-218 (1985)
[17] Jørgensen B (1997) The theory of dispersion models. CRC Press, Boca Raton · Zbl 0928.62052
[18] Kim, Y.; Kim, TK; Kim, Y.; Yoo, J.; You, S.; Lee, I.; Carlson, G.; Hood, L.; Choi, S.; Hwang, D., Principal network analysis: identification of subnetworks representing major dynamics using gene expression data, Bioinformatics, 27, 391-398 (2011)
[19] Kimura K, Tanaka Y, Kudo M (2014) A fast hierarchical alternating least squares algorithm for orthogonal nonnegative matrix factorization. In: ACML
[20] Lee, DD; Seung, HS, Learning the parts of objects by non-negative matrix factorization, Nature, 401, 788-791 (1999) · Zbl 1369.68285
[21] Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. In: Advances in neural information processing systems, pp 556-562
[22] Li T, Peng W (2005) A clustering model based on matrix approximation with applications to cluster system log files. In: European conference on machine learning, Springer, pp 625-632
[23] Li, Y.; Zhang, X.; Sun, M., Robust non-negative matrix factorization with \(\beta \)-divergence for speech separation, ETRI J, 39, 21-29 (2017)
[24] Li, Z.; Wu, X.; Peng, H., Nonnegative matrix factorization on orthogonal subspace, Pattern Recognit Lett, 31, 905-911 (2010)
[25] Mauthner T, Kluckner S, Roth PM, Bischof H (2010) Efficient object detection using orthogonal NMF descriptor hierarchies. In: Goesele M, Roth S, Kuijper A, Schiele B, Schindler K (eds) Pattern recognition. Springer, pp 212-221
[26] Mirzal, A., A convergent algorithm for orthogonal nonnegative matrix factorization, J Comput Appl Math, 260, 149-166 (2014) · Zbl 1293.65066
[27] Nakano M, Kameoka H, Le Roux J, Kitano Y, Ono N, Sagayama S (2010) Convergence-guaranteed multiplicative algorithms for nonnegative matrix factorization with \(\beta \)-divergence. In: 2010 IEEE international workshop on machine learning for signal processing (MLSP), IEEE, pp 283-288
[28] Ohnishi T, Dunn PK (2007) Analysis of the rainfall data in queensland using the tweedie glm. In: Proceedings of the 2007 Japanese joint statistical meeting, Japanese joint statistical meeting, pp 18-18
[29] Pompili, F.; Gillis, N.; Absil, PA; Glineur, F., Two algorithms for orthogonal nonnegative matrix factorization with application to clustering, Neurocomputing, 141, 15-25 (2014)
[30] Simsekli U, Cemgil A, Yilmaz YK (2013) Learning the beta-divergence in Tweedie compound poisson matrix factorization models. In: Proceedings of the 30th international conference on machine learning (ICM-13), pp 1409-1417
[31] Smyth, GK; Jørgensen, B., Fitting tweedie’s compound poisson model to insurance claims data: dispersion modelling, Astin Bull, 32, 143-157 (2002) · Zbl 1094.91514
[32] Tan, VY; Févotte, C., Automatic relevance determination in nonnegative matrix factorization with the/spl beta/-divergence, IEEE Trans Pattern Anal Mach Intell, 35, 1592-1605 (2013)
[33] Mechelen, I.; Bock, HH; Boeck, P., Two-mode clustering methods: a structured overview, Stat Methods Med Res, 13, 363-394 (2004) · Zbl 1053.62078
[34] Vichi M (2001) Double k-means clustering for simultaneous classification of objects and variables. In: Borra S, Rocci R, Vichi M, Schader M (eds) Advances in classification and data analysis. Springer, pp 43-52
[35] Virtanen, T., Monaural sound source separation by nonnegative matrix factorization with temporal continuity and sparseness criteria, IEEE Trans Audio Speech Lang Process, 15, 1066-1074 (2007)
[36] Virtanen, T.; Gemmeke, JF; Raj, B.; Smaragdis, P., Compositional models for audio processing: uncovering the structure of sound mixtures, IEEE Signal Process Mag, 32, 125-144 (2015)
[37] Wang, F.; Zhu, H.; Tan, S.; Shi, H., Orthogonal nonnegative matrix factorization based local hidden Markov model for multimode process monitoring, Chin J Chem Eng, 24, 856-860 (2016)
[38] Wang H, Nie F, Huang H, Makedon F (2011) Fast nonnegative matrix tri-factorization for large-scale data co-clustering. In: IJCAI proceedings-international joint conference on artificial intelligence, vol 22, p 1553
[39] Wang, YX; Zhang, YJ, Nonnegative matrix factorization: a comprehensive review, IEEE Trans on Knowl Data Eng, 25, 1336-1353 (2013)
[40] Weninger, F.; Schuller, B., Optimization and parallelization of monaural source separation algorithms in the openBliSSART toolkit, J Signal Process Syst, 69, 267-277 (2012)
[41] Xue, Y.; Tong, CS; Chen, Y.; Chen, WS, Clustering-based initialization for non-negative matrix factorization, Appl Math Comput, 205, 525-536 (2008) · Zbl 1152.68506
[42] Yoo J, Choi S (2008) Orthogonal nonnegative matrix factorization: Multiplicative updates on Stiefel manifolds. In: Intelligent data engineering and automated learning-IDEAL 2008, Springer, pp 140-147
[43] Yoo J, Choi S (2009) Probabilistic matrix tri-factorization. In: Proceedings of the IEEE international conference on acoustics, speech, and signal processing (ICASSP), IEEE, pp 1553-1556
[44] Yoo, J.; Choi, S., Nonnegative matrix factorization with orthogonality constraints, J Comput Sci Eng, 4, 97-109 (2010)
[45] Yoo, J.; Choi, S., Orthogonal nonnegative matrix tri-factorization for co-clustering: multiplicative updates on Stiefel manifolds, Inf Process Manag, 46, 559-570 (2010)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.