×

An improvement on floating search algorithms for feature subset selection. (English) Zbl 1178.68503

Summary: A new improved forward floating selection algorithm for selecting a subset of features is presented. Our proposed algorithm improves the state-of-the-art sequential forward floating selection algorithm. The improvement is to add an additional search step called “replacing the weak feature” to check whether removing any feature in the currently selected feature subset and adding a new one at each sequential step can improve the current feature subset. Our method provides the optimal or quasi-optimal (close to optimal) solutions for many selected subsets and requires significantly less computational load than optimal feature selection algorithms. Our experimental results for four different databases demonstrate that our algorithm consistently selects better subsets than other suboptimal feature selection algorithms do, especially when the original number of features of the database is large.

MSC:

68T10 Pattern recognition, speech recognition
68P10 Searching and sorting
68P15 Database theory

Software:

UCI-ml
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Ferri, F. J.; Pudil, P.; Hatef, M.; Kittler, J., Comparative study of techniques for large-scale feature selection, (Gelsema, E. S.; Kanal, L. N., Pattern Recognition in Practice IV (1994), Elsevier: Elsevier Amsterdam), 403-413
[2] Jain, A.; Zongker, D., Feature selection: evaluation, application, and small sample performance, IEEE Trans. Pattern Anal. Mach. Intell., 19, 2, 153-158 (1997)
[3] Kudo, M.; Sklansky, J., Comparison of algorithms that select features for pattern classifiers, Pattern Recognition, 33, 25-41 (2000)
[4] Fukunaga, K., Introduction to Statistical Pattern Recognition (1992), Academic Press Inc.: Academic Press Inc. New York
[5] Yu, B.; Yuan, B., A more efficient branch and bound algorithm for feature selection, Pattern Recognition, 26, 883-889 (1993)
[6] Somol, P.; Pudil, P.; Kittler, J., Fast branch & bound algorithms for optimal feature selection, IEEE Trans. Pattern Anal. Mach. Intell., 26, 900-912 (2004)
[7] Nakariyakul, S.; Casasent, D., Adaptive branch and bound algorithm for selecting optimal features, Pattern Recognition Lett., 28, 1415-1427 (2007)
[8] Pudil, P.; Novovicova, J.; Kittler, J., Floating search methods in feature selection, Pattern Recognition Lett., 15, 1119-1125 (1994)
[9] Whitney, A. W., A direct method of nonparametric measurement selection, IEEE Trans. Comput., 20, 1100-1103 (1971) · Zbl 0227.68047
[10] Marill, T.; Green, D. M., On the effectiveness of receptors in recognition system, IEEE Trans. Inf. Theory, 9, 917-922 (1963)
[11] S.D. Stearns, On selecting features for pattern classifiers, in: Proceedings of the 3rd International Conference on Pattern Recognition, 1976, pp. 71-75.; S.D. Stearns, On selecting features for pattern classifiers, in: Proceedings of the 3rd International Conference on Pattern Recognition, 1976, pp. 71-75.
[12] Somol, P.; Pudil, P.; Novocicova, J.; Paclik, P., Adaptive floating search methods in feature selection, Pattern Recognition Lett., 20, 1157-1163 (1999)
[13] Siedlecki, W.; Sklansky, J., A note on genetic algorithms for large-scale feature selection, Pattern Recognition Lett., 10, 335-347 (1989) · Zbl 0942.68690
[14] Serpico, S. B.; Bruzzone, L., A new search algorithm for feature selection in hyperspectral remote sensing images, IEEE Trans. Geosci. Remote Sensing, 39, 7, 1360-1367 (2001)
[15] Verikas, A.; Bacauskiene, M., Feature selection with neural networks, Pattern Recognition Lett., 23, 1323-1335 (2002) · Zbl 1010.68915
[16] Liang, J.; Yang, S.; Winstanley, A., Invariant optimal feature selection: a distance discriminant and feature ranking based solution, Pattern Recognition, 41, 1429-1439 (2008) · Zbl 1140.68470
[17] A. Asuncion, D.J. Newman, UCI Machine Learning Repository \(\langle;\) http://www.ics.uci.edu/\( \sim;\) mlearn/MLRepository.html \(\rangle;\). School of Information and Computer Science, University of California, Irvine, CA, 2007.; A. Asuncion, D.J. Newman, UCI Machine Learning Repository \(\langle;\) http://www.ics.uci.edu/\( \sim;\) mlearn/MLRepository.html \(\rangle;\). School of Information and Computer Science, University of California, Irvine, CA, 2007.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.