A binary water wave optimization for feature selection.

*(English)*Zbl 1433.68410Summary: A search method that finds a minimal subset of features (over a feature space) that yields maximum classification accuracy is proposed. This method employs rough set theory (RST) along with a newly introduced binary version of the water wave optimization approach (WWO) which is denoted by BWWO. WWO simulates the phenomena of water waves, such as propagation, refraction, and breaking and is one of the newest nature inspired methods for global optimization problems. In our approach, BWWO utilizes the phenomena of water waves propagation, refraction, and breaking in a binary version. Two main experiments based on the rough set approach and wrapper method as a part of the objective function are carried out to verify the performance of the proposed algorithm. In the first experiment, the effectiveness of the proposed approach based on RST is demonstrated on 16 different datasets. The proposed approach is compared with various typical attribute reduction methods and popular optimizers in the literature, such as ant colony, nonlinear great deluge algorithm, scatter search and others. For the second experiment, a feature subset that maximizes the classification accuracy (using cross-validated \(k\)NN classifier) with minimizing the number of selected features is obtained over 17 different datasets. In wrapper experiment BWWO is compared with the binary gray wolf optimization, binary particle swarm optimizer, binary cat swarm optimization, binary dragonfly algorithm and the binary bat algorithm. The computational results demonstrate the efficiency and effectiveness of the proposed approach in finding a minimal features subset that maximize the classification accuracy. Furthermore, Friedman test and Wilcoxon’s rank-sum test are carried out at 5% significance level in this study.

##### MSC:

68T20 | Problem solving in the context of artificial intelligence (heuristics, search strategies, etc.) |

68T05 | Learning and adaptive systems in artificial intelligence |

##### Keywords:

classification; feature selection; metaheuristics; rough set theory; water wave optimization; wrapper approaches
PDF
BibTeX
XML
Cite

\textit{A. M. Ibrahim} et al., Int. J. Approx. Reasoning 120, 74--91 (2020; Zbl 1433.68410)

Full Text:
DOI

##### References:

[1] | E. Gasca, J. Sánchez, Eliminating redundancy and irrelevance using a new mlp-based feature selection method. · Zbl 1080.68640 |

[2] | Hedar, A.; Ibrahim, A. M.; Abdel-Hakim, A. E.; Sewisy, A. A., K-means cloning: adaptive spherical k-means clustering, Algorithms, 11, 10 (2018) · Zbl 07150345 |

[3] | Papakostas, G. A.; Koulouriotis, D. E.; Polydoros, A. S.; Tourassis, V. D., Evolutionary Feature Subset Selection for Pattern Recognition Applications, 443-458 (April 2011), INTECH Open Access Publisher |

[4] | Yang, Z.-M.; He, J.-Y.; Shao, Y.-H., Feature selection based on linear twin support vector machines, International Conference on Information Technology and Quantitative Management. International Conference on Information Technology and Quantitative Management, Proc. Comput. Sci., 17, 1039-1046 (2013) |

[5] | Dash, M.; Liu, H., Feature selection for classification, Intell. Data Anal., 1, 1, 131-156 (1997) |

[6] | Pawlak, Z., Rough Sets: Theoretical Aspects of Reasoning about Data (1991), Kluwer Academic Publishers: Kluwer Academic Publishers Dordrecht, the Netherlands · Zbl 0758.68054 |

[7] | Pawlak, Z., Rough sets: present state and the future, Found. Comput. Decision Sci., 18, 3-4, 157-166 (1993) · Zbl 0803.04007 |

[8] | Pawlak, Z., Rough sets and intelligent data analysis, Inf. Sci., 147, 1, 1-12 (2002) · Zbl 1018.68082 |

[9] | Chouchoulas, A.; Shen, Q., Rough set-aided keyword reduction for text categorization, Appl. Artif. Intell., 15, 9, 843-873 (2001) |

[10] | Jensen, R.; Shen, Q., A Rough Set-Aided System for Sorting WWW Bookmarks, 95-105 (2001), Springer Berlin Heidelberg: Springer Berlin Heidelberg Berlin, Heidelberg · Zbl 1031.68760 |

[11] | Jia, X.; Liao, W.; Tang, Z.; Shang, L., Minimum cost attribute reduction in decision-theoretic rough set models, Inf. Sci., 219, Supplement C, 151-167 (2013) · Zbl 1293.91049 |

[12] | Jensen, R.; Shen, Q., Semantics-preserving dimensionality reduction: rough and fuzzy-rough-based approaches, IEEE Trans. Knowl. Data Eng., 16, 12, 1457-1471 (December 2004) |

[13] | Su, Y.; Guo, J., A novel strategy for minimum attribute reduction based on rough set theory and fish swarm algorithm, Comput. Intell. Neurosci., 2017, 1-7 (August 2017) |

[14] | Emary, E.; Zawbaa, Hossam M.; Hassanien, Aboul Ella, Binary grey wolf optimization approaches for feature selection, Neurocomputing, 172, 371-381 (2016) |

[15] | Khanesar, M. A.; Teshnehlab, M.; Shoorehdeli, M. A., A novel binary particle swarm optimization, (2007 Mediterranean Conference on Control Automation (June 2007)), 1-6 |

[16] | Unler, A.; Murat, A., A discrete particle swarm optimization method for feature selection in binary classification problems, Eur. J. Oper. Res., 206, 3, 528-539 (2010) · Zbl 1188.90280 |

[17] | Chuang, L.-Y.; Chang, H.-W.; Tu, C.-J.; Yang, C.-H., Improved binary pso for feature selection using gene expression data, Comput. Biol. Chem., 32, 1, 29-38 (2008) · Zbl 1142.92319 |

[18] | Lee, S.; Soak, S.; Oh, S.; Pedrycz, W.; Jeon, M., Modified binary particle swarm optimization, Prog. Nat. Sci., 18, 9, 1161-1166 (2008) |

[19] | Liu, Y.; Wang, G.; Chen, H.; Dong, H.; Zhu, X.; Wang, S., An improved particle swarm optimization for feature selection, J. Bionics Eng., 8, 2, 191-200 (2011) |

[20] | Eiben, A. E.; Raue, P.-E.; Ruttkay, Zs., Genetic algorithms with multi-parent recombination, (Parallel Problem Solving from Nature - PPSN III. Parallel Problem Solving from Nature - PPSN III, Lecture Notes in Computer Science, vol. 866 (June 2005), Springer: Springer Berlin, Heidelberg), 78-87, chapter |

[21] | Suresh, K.; Kumarappan, N., Hybrid improved binary particle swarm optimization approach for generation maintenance scheduling problem, Swarm Evol. Comput., 9, 69-89 (2013) |

[22] | Rashedi, E.; Nezamabadi-pour, H.; Saryazdi, S., Bgsa: binary gravitational search algorithm, Nat. Comput., 9, 3, 727-745 (Sep 2010) |

[23] | Yumin, C.; Duoqian, Miao; Ruizhi, Wang, A rough set approach to feature selection based on ant colony optimization, Pattern Recognit. Lett., 31, 3, 226-233 (February 2010) |

[24] | Azad, M. A.K.; Rocha, A. M.A. C.; Fernandes, E. M.G. P., Improved binary artificial fish swarm algorithm for the 0-1 multidimensional knapsack problems, Swarm Evol. Comput., 14, 66-75 (2014) |

[25] | Chandrashekar, G.; Sahin, F., A survey on feature selection methods, 40th-Year Commemorative Issue. 40th-Year Commemorative Issue, Comput. Electr. Eng., 40, 1, 16-28 (2014) |

[26] | Erguzel, T. T.; Tas, C.; Cebi, M., A wrapper-based approach for feature selection and classification of major depressive disorder-bipolar disorders, Comput. Biol. Med., 64, Supplement C, 127-137 (2015) |

[27] | Zarshenas, A.; Suzuki, K., Binary coordinate ascent: an efficient optimization technique for feature subset selection for machine learning, Knowl.-Based Syst., 110, Supplement C, 191-201 (2016) |

[28] | Zheng, Y.-J., Water wave optimization: a new nature-inspired metaheuristic, Comput. Oper. Res., 55, 1-11 (March 2015) |

[29] | Yun, X.; Feng, X.; Lyu, X.; Wang, S.; Liu, B., A novel water wave optimization based memetic algorithm for flow-shop scheduling, (2016 IEEE Congress on Evolutionary Computation (CEC) (2016), IEEE), 1971-1976 |

[30] | Manshahia, M. S., Water wave optimization algorithm based congestion control and quality of service improvement in wireless sensor networks, Trans. Netw. Commun., 5, 4, 31 (2017) |

[31] | M. Siva, R. Balamurugan, L. Lakshminarasimman, Water wave optimization algorithm for solving economic dispatch problems with generator constraints. |

[32] | n Ke, L.; Feng, Z.; Ren, Z., An efficient ant colony optimization approach to attribute reduction in rough set theory, Pattern Recognit. Lett., 29, 9, 1351-1357 (2008) |

[33] | Hedar, A.; Wang, J.; Fukushima, M., Tabu search for attribute reduction in rough set theory, Soft Comput., 12, 9, 909-918 (Jul 2008) |

[34] | Mafarja, M.; Abdullah, S., Modified great deluge for attribute reduction in rough set theory, (2011 Eighth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), vol. 3 (July 2011)), 1464-1469 |

[35] | Jaddi, N. S.; Abdullah, S., Nonlinear great deluge algorithm for rough set attribute reduction, J. Inf. Sci. Eng., 29, 1, 49-62 (January 2013) |

[36] | Jue, W.; Qi, Z.; Hedar, A.; Ibrahim, A. M., A rough set approach to feature selection based on scatter search metaheuristic, J. Syst. Sci. Complex., 27, 1, 157-168 (February 2014) |

[37] | Keller, J. M.; Gray, M. R.; Givens, J. A., A fuzzy k-nearest neighbor algorithm, IEEE Trans. Syst. Man Cybern., SMC-15, 4, 580-585 (July 1985) |

[38] | Mirjalili, S.; Lewis, A., S-shaped versus v-shaped transfer functions for binary particle swarm optimization, Swarm Evol. Comput., 9, Supplement C, 1-14 (2013) |

[39] | Mirjalili, S. M., Dragonfly algorithm: a new meta-heuristic optimization technique for solving single-objective, discrete, and multi-objective problems, Neural Comput. Appl., 27, 4, 1053-1073 (May 2016) |

[40] | Mirjalili, S. M.; Yang, X.-S., Binary bat algorithm, Neural Comput. Appl., 25, 3, 663-681 (Sep 2014) |

[41] | Sharafi, Y.; Khanesar, M. A.; Teshnehlab, M., Discrete binary cat swarm optimization algorithm, (2013 3rd IEEE International Conference on Computer, Control and Communication (IC4) (Sept 2013)), 1-6 |

[42] | Pawlak, Z., Rough sets, Int. J. Comput. Inf. Sci., 11, 5, 341-356 (October 1982) |

[43] | Pawlak, Z., Rough set approach to knowledge-based decision support, Eur. J. Oper. Res., 99, 1, 48-57 (1997) · Zbl 0923.90004 |

[44] | Li, H.; Li, D.; Zhai, Y.; Wang, S.; Zhang, J., A novel attribute reduction approach for multi-label data based on rough set theory, Inf. Sci., 367, Supplement C, 827-847 (2016) · Zbl 1428.68243 |

[45] | Benítez-Caballero, M. J.; Medina, J.; Ramírez-Poussa, E., Attribute Reduction in Rough Set Theory and Formal Concept Analysis, 513-525 (2017), Springer International Publishing: Springer International Publishing Cham |

[46] | Dai, J.; Hu, Q.; Hu, H.; Huang, D., Neighbor inconsistent pair selection for attribute reduction by rough set approach, IEEE Trans. Fuzzy Syst., 26, 2, 937-950 (2017) |

[47] | Hedar, A.; Ibrahim, A. M.; Abdel-Hakim, A. E.; Sewisy, A. A., Modulated clustering using integrated rough sets and scatter search attribute reduction, (Proceedings of the Genetic and Evolutionary Computation Conference Companion. Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO’18 (2018), ACM: ACM New York, NY, USA), 1394-1401 |

[48] | Manish, S., Rough-fuzzy functions in classification, Fuzzy Sets Syst., 132, 353-369 (2002) · Zbl 1008.68582 |

[49] | Das, S., Filters, wrappers and a boosting-based hybrid for feature selection, (Proceedings of the Eighteenth International Conference on Machine Learning. Proceedings of the Eighteenth International Conference on Machine Learning, ICML ’01 (2001), Morgan Kaufmann Publishers Inc.: Morgan Kaufmann Publishers Inc. San Francisco, CA, USA), 74-81 |

[50] | John, G. H.; Kohavi, R.; Pfleger, K., Irrelevant features and the subset selection problem, (Machine Learning: Proceedings of the Eleventh International (1994), Morgan Kaufmann), 121-129 |

[51] | Tawhid, M. A.; Ibrahim, A. M., Hybrid Binary Particle Swarm Optimization and Flower Pollination Algorithm Based on Rough Set Approach for Feature Selection Problem, 249-273 (2020), Springer International Publishing: Springer International Publishing Cham |

[52] | Tawhid, M. A.; Ibrahim, A. M., Feature selection based on rough set approach, wrapper approach, and binary whale optimization algorithm, Int. J. Mach. Learn. Cybern. (Aug 2019) |

[53] | M. Lichman, UCI machine learning repository, 2013. |

[54] | Hochberg, Y.; Benjamini, Y., More powerful procedures for multiple significance testing, Stat. Med., 9, 7, 811-818 (1990) |

[55] | Derrac, J.; García, S.; Molina, D.; Herrera, F., A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms, Swarm Evol. Comput., 1, 1, 3-18 (2011) |

[56] | Kasuya, E., Wilcoxon signed-ranks test: symmetry should be confirmed before the test, Anim. Behav., 79, 3, 765-767 (2010) |

[57] | Taheri, S. M.; Hesamian, G., A generalization of the Wilcoxon signed-rank test and its applications, Stat. Pap., 54, 2, 457-470 (May 2013) |

[58] | Kellerer, H.; Pferschy, U.; Pisinger, D., Introduction to np-completeness of knapsack problems, (Knapsack Problems (2004), Springer), 483-493 |

[59] | Helal, N.; Pichon, F.; Porumbel, D.; Mercier, D.; Lefèvre, É., The capacitated vehicle routing problem with evidential demands, Int. J. Approx. Reason., 95, 124-151 (2018) · Zbl 06890220 |

[60] | Vogiatzis, D.; Tsapatsoulis, N., Active learning for microarray data, Int. J. Approx. Reason., 47, 1, 85-96 (2008) · Zbl 1191.68539 |

[61] | Maji, P.; Paul, S., Rough set based maximum relevance-maximum significance criterion and gene selection from microarray data, Int. J. Approx. Reason., 52, 3, 408-426 (2011) |

[62] | Baioletti, M.; Milani, A.; Santucci, V., Learning Bayesian networks with algebraic differential evolution, (International Conference on Parallel Problem Solving from Nature (2018), Springer), 436-448 |

[63] | Yang, C.; Ji, J.; Liu, J.; Liu, J.; Yin, B., Structural learning of Bayesian networks by bacterial foraging optimization, Int. J. Approx. Reason., 69, 147-167 (2016) · Zbl 1344.68194 |

[64] | Tawhid, M. A.; Dsouza, K. B., Hybrid binary bat enhanced particle swarm optimization algorithm for solving feature selection problems, Appl. Comput. Inform. (2018) |

[65] | Tawhid, M. A.; Dsouza, K. B., Solving feature selection problem by hybrid binary genetic enhanced particle swarm optimization algorithm, Int. J. Hybrid Intell. Syst., 15, 207-219 (2019) |

[66] | Tawhid, M. A.; Dsouza, K. B., Hybrid binary dragonfly enhanced particle swarm optimization algorithm for solving feature selection problems, Math. Found. Comput., 1, 2, 181-200 (2018) |

[67] | Kulkarni, R. V.; Venayagamoorthy, G. K., Particle swarm optimization in wireless-sensor networks: a brief survey, IEEE Trans. Syst. Man Cybern., Part C, Appl. Rev., 41, 2, 262-267 (2011) |

[68] | Kulkarni, R. V.; Forster, A.; Venayagamoorthy, G. K., Computational intelligence in wireless sensor networks: a survey, IEEE Commun. Surv. Tutor., 13, 1, 68-96 (2011) |

[69] | Liu, G.; Zhou, Y.; Zhao, J.; Dai, G.; Li, X.-Y.; Gu, M.; Ma, H.; Mo, L.; He, Y.; Wang, J., Long-term large-scale sensing in the forest: recent advances and future directions of greenorbs, Front. Comput. Sci. China, 4, 3, 334-338 (2010) |

[70] | Li, X.; Yao, X., Cooperatively coevolving particle swarms for large scale optimization, IEEE Trans. Evol. Comput., 16, 2, 210-224 (2012) |

[71] | Teodorovic, D., Transport modeling by multi-agent systems: a swarm intelligence approach, Transp. Plann. Technol., 26, 4, 289-312 (2003) |

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.