×

zbMATH — the first resource for mathematics

Multi-network evolutionary systems and automatic decomposition of complex problems. (English) Zbl 1140.93333
Summary: Multi-network systems, i.e. multiple neural network systems, can often solve complex problems more effectively than their monolithic counterparts. Modular neural networks (MNNs) tackle a complex problem by decomposing it into simpler subproblems and then solving them. Unlike the decomposition in MNNs, a neural network ensemble usually includes redundant component nets and is often inspired by statistical theories. This paper presents different types of problem decompositions and discusses the suitability of various multi-network systems for different decompositions. A classification of various multi-network systems, in the context of problem decomposition, is obtained by exploiting these differences. Then a specific type of problem decomposition, which gives no information about the subproblems and is often ignored in literature, is discussed in detail and a novel MNN architecture for problem decomposition is presented. Finally, a co-evolutionary model is presented, which is used to design and optimize such MNNs with subtask specific modules. The model consists of two populations. The first population consists of a pool of modules and the second population synthesizes complete systems by drawing elements from the pool of modules. Modules represent a part of the solution, which co-operate with each other to form a complete solution. Using two artificial supervised learning tasks, constructed from smaller subtasks, it can be shown that if a particular task decomposition is better than others, in terms of performance on the overall task, it can be evolved using the co-evolutionary model.
MSC:
93B11 System structure simplification
93A15 Large-scale systems
PDF BibTeX XML Cite
Full Text: DOI
References:
[1] Abu-Mostafa Y.S., Advances in Neural Information Processing Systems 5 pp 73– (1993)
[2] Bishop C.M., Neural Networks for Pattern Recognition (1996) · Zbl 0868.68096
[3] Brown G., PhD thesis (2004)
[4] Bullinaria J.A., Birmingham, in: Proceedings of the 2002 U.K. Workshop on Computational Intelligence (UKCI-02) pp 3– (2002)
[5] Calabretta R., Proceedings of the Sixth International Conference on Artificial Life (ALIFE VI) pp 275– (1998)
[6] Ferdinando A.D., Proceedings of the Sixth Neural Computation and Psychology Workshop pp 253– (2001)
[7] DOI: 10.1016/S0893-6080(02)00095-3 · Zbl 02022277
[8] DOI: 10.1109/TEVC.2005.844158 · Zbl 05451934
[9] Gruau F., Genetic Programming 1996: Proceedings of the First Annual Conference pp 81– (1996)
[10] Hansen J.V., PhD thesis (2000)
[11] Hrycej T., Modular Learning in Neural Networks. A Modularized Appproach to Neural Network Classification (1992) · Zbl 0785.68074
[12] DOI: 10.1109/ECNN.2000.886225
[13] DOI: 10.1080/09540090208559328 · Zbl 02021130
[14] DOI: 10.1016/S0925-2312(01)00700-7 · Zbl 1006.68811
[15] DOI: 10.1207/s15516709cog1502_2
[16] DOI: 10.1162/neco.1991.3.1.79
[17] DOI: 10.1162/neco.1992.4.5.647
[18] Jordan M.I., The Handbook of Brain Theory and Neural Networks pp 579– (1995)
[19] DOI: 10.1007/978-3-540-30217-9_89
[20] Khare V.R., Edinburgh, UK, IEEE Press, in: 2005 IEEE Congress on Evolutionary Computation, CEC 2005 (2005)
[21] DOI: 10.1016/S0004-3702(97)00043-X · Zbl 0904.68143
[22] Lendaris G.G., San Diego, California, Earlbaum/INNS, in: Proceedings of World Conference on Neural Networks (WCNN’94)
[23] Liao Y., Adv. Neural Inf. Process. Syst. 12 pp 921– (1999)
[24] Liu Y., PhD thesis (1998)
[25] DOI: 10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2 · Zbl 1417.37129
[26] DOI: 10.1109/72.788664
[27] DOI: 10.1126/science.267326 · Zbl 1383.92036
[28] Maimon O., Proceedings of the Second International Symposium on Foundations of Information and Knowledge Systems, Lecture Notes in Computer Science pp 178– (2002)
[29] DOI: 10.1162/evco.1997.5.4.373 · Zbl 05412783
[30] Opitz D., Proceedings of the Sixteenth National Conference on Artificial Intelligence, California, AAAI pp 379– (1999)
[31] DOI: 10.1162/106365600568086 · Zbl 05412828
[32] Ronco E., Technical Report CSC-96012 (1997)
[33] DOI: 10.1162/jocn.1989.1.2.171
[34] Schlosser G., Modularity in Development and Evolution (2005)
[35] Sendhoff B., Congress on Evolutionary Computation CEC pp 259– (1999)
[36] Sharkey A.J.C., Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems (1999) · Zbl 0910.00025
[37] DOI: 10.1016/S0921-8890(97)00047-X
[38] DOI: 10.1109/TNN.2004.828772
[39] Yang H., Advances in Intelligent Data Analysis (AIDA), Computational Intelligence Methods and Applications (CIMA), International Computer Science Conventions pp 22– (1999)
[40] Yong C.H., Technical Report AI01-287 (2001)
[41] Zaffalon M., Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence pp 577– (2002)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.