×

zbMATH — the first resource for mathematics

Kernel multilogit algorithm for multiclass classification. (English) Zbl 06984116
Summary: An algorithm for multi-class classification is proposed. The soft classification problem is considered, where the target variable is a multivariate random variable. The proposed algorithm transforms the original target variable into a new space using the multilogit function. Assuming Gaussian noise on this transformation and using a standard Bayesian approach the model yields a quadratic functional whose global minimum can easily be obtained by solving a set of linear system of equations. In order to obtain the classification, the inverse multilogit-based transformation should be applied and the obtained result can be interpreted as a ‘soft’ or probabilistic classification. Then, the final classification is obtained by using the ‘Winner takes all’ strategy. A Kernel-based formulation is presented in order to consider the non-linearities associated with the feature space of the data. The proposed algorithm is applied on real data, using databases available online. The experimental study shows that the algorithm is competitive with respect to other classical algorithms for multiclass classification.
MSC:
62-XX Statistics
Software:
LIBLINEAR; UCI-ml
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Bache, K., Lichman, M., UCI machine learning repository, 2013. URL: http://archive.ics.uci.edu/ml.
[2] Balasubramanian, K.; Lebanon, G., The landmark selection method for multiple output prediction, (29th International Conference on Machine Learning (ICML), (2012), icml.cc/Omnipress), 983-990
[3] Bishop, C., Pattern recognition and machine learning, vol. 4, (2006), Springer New York
[4] Blondel, M.; Seki, K.; Uehara, K., Block coordinate descent algorithms for large-scale sparse multiclass classification, Mach. Learn., 93, 1, 31-52, (2013) · Zbl 1293.68216
[5] Bohning, D., Multinomial logistic regression algorithm, Ann. Inst. Statist. Math., 44, 1, 197-200, (1992) · Zbl 0763.62038
[6] Breiman, L.; Friedman, J.; Olshen, R.; Stone, C., Classification and regression trees, (1984), Wadsworth and Brooks Monterey, CA · Zbl 0541.62042
[7] Caudill, S., An advantage of the linear probability model over probit or logit, Oxford Bull. Econ. Stat., 50, 425-427, (1988)
[8] Chandra, B.; Gupta, M., Robust approach for estimating probabilities in naive-Bayes classifier for gene expression data, Expert Syst. Appl., 38, 3, 1293-1298, (2011)
[9] Cortez, P.; Cerdeira, A.; Almeida, F.; Matos, T.; Reis, J., Modeling wine preferences by data mining from physicochemical properties, Decis. Support Syst., 47, 4, 547-553, (1998)
[10] Cuingnet, R.; Chupin, M.; Benali, H.; Colliot, O., Spatial and anatomical regularization of SVM for brain image analysis, (Lafferty, J.; Williams, C. K.I.; Shawe-Taylor, J.; Zemel, R.; Culotta, A., Advances in Neural Information Processing Systems, Vol. 23, (2010), Curran Associates, Inc.), 460-468
[11] Cuingnet, R.; Rosso, C.; Chupin, M.; Lehéricy, S.; Dormont, D.; Benali, H.; Samson, Y.; Colliot, O., Spatial regularization of SVM for the detection of diffusion alterations associated with stroke outcome, Med. Image Anal., 15, 5, 729-737, (2011)
[12] Duda, R. O.; Hart, P. E.; Stork, D. G., Pattern classification, (2001), Wiley-Interscience · Zbl 0968.68140
[13] Fan, R. E.; Chang, K. W.; Hsieh, C. J.; Wang, X. R.; Lin, C. J., LIBLINEAR: a library for large linear classification, J. Mach. Learn. Res., 9, 1871-1874, (2008) · Zbl 1225.68175
[14] Gray, J. B.; Fan, G., Classification tree analysis using target, Comput. Statist. Data Anal., 52, 3, 1362-1372, (2008) · Zbl 1452.62445
[15] Hastie, T.; Tibshirani, R.; Friedman, J. H., The elements of statistical learning, (2003), Springer
[16] Kim, H.; yin Loh, W., Classification trees with unbiased multiway splits, J. Amer. Statist. Assoc., 96, 589-604, (2001)
[17] Krishnapuram, B.; Carin, L.; Figueiredo, M. A.T.; Hartemink, A., Sparse multinomial logistic regression: fast algorithms and generalization bounds, IEEE Trans. Pattern Anal. Mach. Intell., 27, 6, 957-968, (2005)
[18] Lopez-Cruz, P. L.; Bielza, C.; Larrañaga, P., Directional naive Bayes classifiers, Pattern Anal. Appl., 1-22, (2013)
[19] Lu, J.; Yang, Y.; Webb, G. I., Incremental discretization for naive-Bayes classifier, (Proceedings of the Second International Conference on Advanced Data Mining and Applications, ADMA’06, (2006), Springer-Verlag Berlin, Heidelberg), 223-238
[20] Ryali, S.; Supekar, K.; Abrams, D. A.; Menon, V., Sparse logistic regression for whole-brain classification of fMRI data, NeuroImage, 51, 2, 752-764, (2010)
[21] Taheri, S.; Mammadov, M.; Bagirov, A. M., Improving naive Bayes classifier using conditional probabilities, (Proceedings of the Ninth Australasian Data Mining Conference—Volume 121, AusDM’11, (2011), Australian Computer Society, Inc. Darlinghurst, Australia, Australia), 63-68
[22] Tibshirani, R., Regression shrinkage and selection via the lasso, J. R. Stat. Soc. Ser. B, 58, 267-288, (1994) · Zbl 0850.62538
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.