×

GFAM: Evolving Fuzzy ARTMAP neural networks. (English) Zbl 1151.68574

Summary: This paper focuses on the evolution of Fuzzy ARTMAP neural network classifiers, using genetic algorithms, with the objective of improving generalization performance (classification accuracy of the ART network on unseen test data) and alleviating the ART category proliferation problem (the problem of creating more than necessary ART network categories to solve a classification problem). We refer to the resulting architecture as GFAM. We demonstrate through extensive experimentation that GFAM exhibits good generalization and is of small size (creates few ART categories), while consuming reasonable computational effort. In a number of classification problems, GFAM produces the optimal classifier. Furthermore, we compare the performance of GFAM with other competitive ARTMAP classifiers that have appeared in the literature and addressed the category proliferation problem in ART. We illustrate that GFAM produces improved results over these architectures, as well as other competitive classifiers.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Anagnostopoulos, G. C. (2001). Novel approaches in adaptive resonance theory for machine learning. Doctoral Thesis; Anagnostopoulos, G. C. (2001). Novel approaches in adaptive resonance theory for machine learning. Doctoral Thesis
[2] Anagnostopoulos, G. C., Bharadwaj, M., Georgiopoulos, M., Verzi, S. J., & Heileman, G. L. (2003). Exemplar-based pattern recognition via semi-supervised learning. In Proc. of the international joint conference on neural networksVol. 4; Anagnostopoulos, G. C., Bharadwaj, M., Georgiopoulos, M., Verzi, S. J., & Heileman, G. L. (2003). Exemplar-based pattern recognition via semi-supervised learning. In Proc. of the international joint conference on neural networksVol. 4
[3] Anagnostopoulos, G. C., & Georgiopoulos, M. (2001). Ellipsoid ART and ARTMAP for incremental clustering and classification. In Proceedings of the IEEE-INNS international joint conference on neural networks; Anagnostopoulos, G. C., & Georgiopoulos, M. (2001). Ellipsoid ART and ARTMAP for incremental clustering and classification. In Proceedings of the IEEE-INNS international joint conference on neural networks
[4] Anagnostopoulos, G. C.; Georgiopoulos, M., Category regions as new geometrical concepts in Fuzzy ART and Fuzzy ARTMAP, Neuarl Networks, 15, 10, 1205-1221 (2002)
[5] Anagnostopoulos, G. C., Georgiopoulos, M., Verzi, S. J., & Heileman, G. L. (2002). Reducing generalization error and category proliferation in ellipsoid ARTMAP via tunable misclassification error tolerance: Boosted Ellipsoid ARTMAP. In Proc. of the international joint conference on neural networks; Anagnostopoulos, G. C., Georgiopoulos, M., Verzi, S. J., & Heileman, G. L. (2002). Reducing generalization error and category proliferation in ellipsoid ARTMAP via tunable misclassification error tolerance: Boosted Ellipsoid ARTMAP. In Proc. of the international joint conference on neural networks
[6] Bala, J.; De Jong, K.; Huang, J.; Vafaie, H.; Wechsler, H., Using learning to facilitate the evolution of features for recognizing visual concepts, Evolutionary Computation, 4, 3, 297-311 (1996)
[7] Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Wadsworth; Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and regression trees. Wadsworth · Zbl 0541.62042
[8] Brotherton, T. W.; Simpson, P. K., Dynamic feature set training of neural nets for classification, (McDonnel, J. R.; Reynolds, R. G.; Fogel, D. B., Evolutionary programming, Vol. IV (1995), MIT Press: MIT Press Cambridge, MA), 83-94
[9] Burton, A. R, & Vladimirova, T. (1997). Utilisation of an adaptive resonance theory neural network as a genetic algorithm fitness evaluator. In Proceedings of the 1997 IEEE international symposium on information theory; Burton, A. R, & Vladimirova, T. (1997). Utilisation of an adaptive resonance theory neural network as a genetic algorithm fitness evaluator. In Proceedings of the 1997 IEEE international symposium on information theory
[10] Cantu-Paz, E., Feature sub-set selection by estimation of distribution algorithms, (Langdon, W. B.; Cantu-Paz, E.; Mathias, K.; Roy, R.; Davis, D.; Poli, R.; Balakrishna, K.; Honavar, V.; Rudolph, G.; Wegener, J.; Bull, L.; Potter, M. A.; Schultz, A. C.; Miller, J. F.; Burke, E.; Jonoska, N., GECCO 2002: Proceedings of the genetic and evolutionary computation conference (2002), Morgan Kaufmann Publishers: Morgan Kaufmann Publishers San Francisco, CA), 302-310
[11] Carpenter, G. A.; Grossberg, S.; Markuzon, N.; Reynolds, J. H., Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multi-dimensional maps, IEEE Transactions on Neural Networks, 3, 5, 698-713 (1992)
[12] Carpenter, G. A.; Grossberg, S.; Rosen, D. B., Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system, Neural Networks, 4, 759-771 (1991)
[13] Carpenter, G. A.; Milenova, B., Distributed ARTMAP: A neural network for fast distributed supervised learning, Neural Networks, 11, 2, 323-336 (1998)
[14] Charalampidis, D.; Kasparis, T.; Georgiopoulos, M., Classification of noisy signals using Fuzzy ARTMAP neural networks, IEEE Transactions on Neural Networks, 12, 5, 1023-1036 (2001)
[15] Georgiopoulos, M.; Huang, J.; Heileman, G. L., Properties of learning in ARTMAP, Neural Networks, 7, 3, 495-506 (1994)
[16] Goldberg, D. E., Genetic algorithms in search, optimization, and machine learning (1989), Addison-Wesley: Addison-Wesley Reading, MA · Zbl 0721.68056
[17] Gomez-Sanchez, E.; Dimitriadis, Y. A.; Cano-Izquierdo, J. M.; Lopez-Coronado, J., Safe-\( \mu\) ARTMAP: A new solution for reducing category proliferation in Fuzzy ARTMAP, (Proc. of the IEEE International joint conference on neural networks, Vol. 2 (2001)), 1197-1202
[18] Gomez-Sanchez, E.; Dimitriadis, Y. A.; Cano-Izquierdo, J. M.; Lopez-Coronado, J., \( \mu\) ARTMAP: Use of mutual information for category reduction in Fuzzy ARTMAP, IEEE Transactions on Neural Networks, 13, 1, 58-69 (2002)
[19] Grossberg, S., Adaptive pattern recognition and universal recoding II: Feedback, expectation, olfaction, and illusions, Biological Cybernetics, 23, 187-202 (1976) · Zbl 0339.92005
[20] Hancock, P. J.B., Pruning neural networks by genetic algorithm, (Aleksander, I.; Taylor, J., Proceedings of the 1992 international conference on neural networks, Vol. 2 (1992), Elsevier Science: Elsevier Science Amsterdam, The Netherlands), 991-994
[21] Hancock, P. J.B.; Smith, L. S.; Phillips, W. A., A biologically supported errr-correcting learning rule, (Kohonen, T.; Makisara, K.; Simula, O.; Kangas, J., Proceedings international conference on artificial neural networks, Vol. 1 (1991), North Holland: North Holland Amsterdam, The Netherlands), 531-536
[22] Harp, S. A.; Samad, T.; Guha, A., Towards the genetic synthesis of neural networks, (Proceedings of the third international conference on genetic algorithms (1989), Morgan Kaufmann)
[23] Inza, I.; Larranaga, P.; Etxeberria, R.; Sierra, B., Feature sub-set selection by Bayesian networks based optimization, Artificial Intelligence, 123, 1-2, 157-184 (1999) · Zbl 0952.68118
[24] Kasuba, T., Simplified Fuzzy ARTMAP, AI Expert, 18-25 (1993)
[25] Kelly, J. D. Jr., & Davis, L. (1991). Hybridizing the genetic algorithm and the \(K ICGA 1991 \); Kelly, J. D. Jr., & Davis, L. (1991). Hybridizing the genetic algorithm and the \(K ICGA 1991 \)
[26] Koufakou, A.; Georgiopoulos, M.; Anagnostopoulos, G. C.; Kasparis, T., Cross-validation in Fuzzy ARTMAP for large databases, Neural Networks, 14, 1279-1291 (2001)
[27] Lim, T.; Loh, W.; Shih, Y., A comparison of prediction accuracy, complexity, and training time of thirty-three old and new classification algorithms, Machine Learning, 40, 203-229 (2000) · Zbl 0969.68669
[28] Liu, H., Liu, Y., Liu, J., Zhang, B., & Wu, G. (2003). Impulse force based ART network with GA optimization. In Neural networks and signal processing. (Proceedings of the 2003 international conference on neural networks and signal processingVol. 1; Liu, H., Liu, Y., Liu, J., Zhang, B., & Wu, G. (2003). Impulse force based ART network with GA optimization. In Neural networks and signal processing. (Proceedings of the 2003 international conference on neural networks and signal processingVol. 1
[29] Marin, F. J.; Sandoval, F., Genetic synthesis of discrete-time recurrent neural networks, (Proceedings international workshop artificial neural networks (IWANN 1993). Proceedings international workshop artificial neural networks (IWANN 1993), Lecture notes in computer science, Vol. 686 (1993), Springer-Verlag: Springer-Verlag Berlin, Germany)
[30] Marriott, S.; Harrison, R. F., A modified Fuzzy ARTMAP architecture for the approximation of noisy mappings, Neural Networks, 8, 4, 619-641 (1995)
[31] Miller, G. F., Todd, P. M., & Hedge, S. U. (1989). Designing neural networks using genetic algorithms. In: J. D. Scaffer (Ed.) Proceedings of the third international conference on genetic algorithms.; Miller, G. F., Todd, P. M., & Hedge, S. U. (1989). Designing neural networks using genetic algorithms. In: J. D. Scaffer (Ed.) Proceedings of the third international conference on genetic algorithms.
[32] Newman, D. J., Hettich, S., Blake, C. L., & Merz, C. J. (1998). UCI repository of machine learning databases http://www.ics.uci.edu/ mlearn/MLRepository.html; Newman, D. J., Hettich, S., Blake, C. L., & Merz, C. J. (1998). UCI repository of machine learning databases http://www.ics.uci.edu/ mlearn/MLRepository.html
[33] Palmes, P. P.; Hayasaka, T.; Usui, S., Mutation-based genetic neural network, IEEE Transactions on Neural Networks, 16, 3 (2005)
[34] Parrado-Hernandez, E.; Gomez-Sanchez, E.; Dimitriadis, Y. A., Study of distributed learning as a solution to category proliferation in Fuzzy ARTMAP-based neural systems, Neural Networks, 16, 1039-1057 (2003)
[35] Punch, W. F., Goodman, E. D., ChiaShun, M. P. L., Hovland, P., & Enbody, R. (1993). Further research on feature selection and classification using genetic algorithms. In International conference on genetic algorithms 93; Punch, W. F., Goodman, E. D., ChiaShun, M. P. L., Hovland, P., & Enbody, R. (1993). Further research on feature selection and classification using genetic algorithms. In International conference on genetic algorithms 93
[36] Sexton, R. S.; Dorsey, R. E.; Jonson, J. D., Toward global optimization of neural networks: A comparison of the genetic approach and back-propagation, Decision Support Systems, 22, 2, 171-185 (1998)
[37] Siedlecki, W.; Sklansky, J., A note on genetic algorithms for large-scale feature selection, Pattern Recognition Letters, 10, 335-347 (1989) · Zbl 0942.68690
[38] Taghi, M.; Bagmisheh, V.; Pavesic, N., A fast simplified fuzzy artmap network, Neural Processing Letters, 17, 3, 273-316 (2003)
[39] Verzi, S. J., Heileman, G. L., Georgiopoulos, M., & Healy, M. (2001). Rademacher penalization applied to Fuzzy ARTMAP and boosted ARTMAP. In Proc. of the IEEE-INNS international joint conference on neural network; Verzi, S. J., Heileman, G. L., Georgiopoulos, M., & Healy, M. (2001). Rademacher penalization applied to Fuzzy ARTMAP and boosted ARTMAP. In Proc. of the IEEE-INNS international joint conference on neural network
[40] Whitley, D.; Starkweather, T.; Bogart, C., Genetic algorithms and neural networks: Optimizing connections and connectivity, Parallel Computing, 14, 347-361 (1990)
[41] Williamson, J. R., Gaussian ARTMAP: A neural network for fast incremental learning of noisy multi-dimensional maps, Neural Networks, 9, 5, 881-897 (1996)
[42] Williamson, J. R., A constructive, incremental-learning network for mixture modeling and classification, Neural Computation, 9, 1517-1543 (1997)
[43] Yang, J.; Honavar, V., Feature subset selection using genetic algorithms, IEEE Intelligent Systems, 13, 44-49 (1998)
[44] Yao, X., Evolving artificial neural networks, Proceedings of the IEEE, 87, 9, 1423-1447 (1999)
[45] Yao, X.; Liu, Y., Making use of a population information in evolutionary artificial neural networks, IEEE Transactions on Systems, Man and Cybernetics, B, 28, 417-425 (1998)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.