×

Improved discriminate analysis for high-dimensional data and its application to face recognition. (English) Zbl 1113.68086

Summary: Many pattern recognition applications involve the treatment of high-dimensional data and the small sample size problem. Principal Component Analysis (PCA) is a common used dimension reduction technique. Linear Discriminate Analysis (LDA) is often employed for classification. PCA plus LDA is a famous framework for discriminant analysis in high-dimensional space and singular cases. In this paper, we examine the theory of this framework and find out that even if there is no small sample size problem the PCA dimension reduction cannot guarantee the subsequent successful application of LDA. We thus develop an improved discriminate analysis method by introducing an inverse Fisher criterion and adding a constrain in PCA procedure so that the singularity phenomenon will not occur. Experiment results on face recognition suggest that this new approach works well and can be applied even when the number of training samples is one per class.

MSC:

68T10 Pattern recognition, speech recognition

Software:

KPCA plus LDA
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Raudys, S. J.; Jain, A. K., Small sample size effects in statistical pattern recognition: recommendations for practitioners, IEEE Trans. Pattern Anal. Mach. Intell., 13, 252-264 (1991)
[2] Jain, A. K.; Ross, A.; Prabhakar, S., An introduction to biometric recognition, IEEE Trans. Circuits Syst. Video Technol., 14, 1, 4-20 (2004)
[3] Zhao, W.; Chellappa, R.; Phillips, P. J., Face recognition: a literature survey, ACM Comput. Surv., 35, 4, 399-459 (2003)
[4] Belhumeur, P. N.; Hespanha, J. P.; Kriegman, D. J., Eigenfaces vs. Fisherfaces: recognition using class specific linear projection, IEEE Trans. Pattern Anal. Mach. Intell., 19, 711-720 (1997)
[5] Martínez, A. M.; Kak, A. C., PCA versus LDA, IEEE Trans. Pattern Anal. Mach. Intell., 23, 233-288 (2001)
[6] Turk, M., A random walk through eigenspace, IEICE Trans. Inform. Syst., E84-D, 12, 1586-1695 (2001)
[7] Turk, M. A.; Pentland, A. P., Eigenfaces for recognition, J. Cognitive. Neurosci., 3, 1, 71-86 (1991)
[8] S. Mika, G. Rätsch, J. Weston, B. Schölkopf, K.-R. Müller, Fisher discriminant analysis with kernels, Proceedings of the IEEE International Workshop on Neural Networks for Signal Processing IX, August 1999, pp. 41-48.; S. Mika, G. Rätsch, J. Weston, B. Schölkopf, K.-R. Müller, Fisher discriminant analysis with kernels, Proceedings of the IEEE International Workshop on Neural Networks for Signal Processing IX, August 1999, pp. 41-48.
[9] Mika, S.; Rätsch, G.; Weston, J.; Schölkopf, B.; Smola, A.; Müller, K.-R., Constructing descriptive and discriminative nonlinear features: Rayleigh coefficients in kernel feature spaces, IEEE Trans Pattern Anal. Mach. Intell., 25, 5, 623-628 (2003)
[10] Baudat, G.; Anouar, F., Generalized discriminant analysis using a kernel approach, Neural Comput., 12, 10, 2385-2404 (2000)
[11] Dai, D. Q.; Yuen, P. C., Regularized discriminant analysis and its applications to face recognition, Pattern Recognition, 36, 3, 845-847 (2003) · Zbl 1032.68129
[12] D.Q. Dai, P.C. Yuen, A wavelet-based two-parameter regularization discriminant analysis for face recognition, Proceeding of the Fourth International Conference on Audio and Video Based Personal Authentication, Lecture Notes in Computer Science, vol. 2688 (2003) 137-144.; D.Q. Dai, P.C. Yuen, A wavelet-based two-parameter regularization discriminant analysis for face recognition, Proceeding of the Fourth International Conference on Audio and Video Based Personal Authentication, Lecture Notes in Computer Science, vol. 2688 (2003) 137-144.
[13] Pima, I.; Aladjem, M., Regularized discriminant analysis for face recognition, Pattern Recognition, 37, 9, 1945-1948 (2004)
[14] Liu, C. J.; Wechsler, H., A shape- and texture-based enhanced Fisher classifier for face recognition, IEEE Trans. Image Process., 10, 4, 598-608 (2001) · Zbl 1036.68558
[15] Swets, D. L.; Weng, J., Using discriminant eigenfeatures for image retrieval, IEEE Trans. Pattern Anal. Mach. Intell., 18, 8, 831-836 (1996)
[16] Yu, H.; Yang, J., A direct LDA algorithm for high-dimensional data with application to face recognition, Pattern Recognition, 34, 10, 2067-2070 (2001) · Zbl 0993.68091
[17] Yang, J.; Yang, J. Y., Why can LDA be performed in PCA transformed space?, Pattern Recognition, 36, 2, 563-566 (2003)
[18] Liu, K.; Cheng, Y.-Q.; Yang, J.-Y.; Liu, X., An efficient algorithm for Foley-Sammon optimal set of discriminant vectors by algebraic method, Int. J. Pattern Recognition Artif. Intell., 6, 5, 817-829 (1992)
[19] Chen, L. F.; Liao, H. Y.M.; Lin, J. C.; Kao, M. D.; Yu, G. J., A new LDA-based face recognition system which can solve the small sample size problem, Pattern Recognition, 33, 10, 1713-1726 (2000)
[20] Perlibakas, V., Face recognition using principal component analysis and wavelet packet decomposition, Informatica, 15, 2, 243-250 (2004) · Zbl 1098.68689
[21] J. Yang, J.Y. Yang, Optimal FLD algorithm for facial feature extraction, Proceedings of SPIE on Intelligent Robots and Computer Vision XX: Algorithms, Techniques, and Active Vision, October 2001, pp. 438-444.; J. Yang, J.Y. Yang, Optimal FLD algorithm for facial feature extraction, Proceedings of SPIE on Intelligent Robots and Computer Vision XX: Algorithms, Techniques, and Active Vision, October 2001, pp. 438-444.
[22] Ye, J. P.; Janardan, R.; Park, C. H.; Park, H., An optimization criterion for generalized discriminant analysis on undersampled problems, IEEE Trans. Pattern Anal. Mach. Intell., 26, 8, 982-994 (2004)
[23] Zhang, B.; Zhang, H.; Sam Ge, S., Face recognition by applying wavelet subband representation and kernel associative memory, IEEE Trans. Neural Networks, 15, 1, 166-177 (2004)
[24] Yang, J.; Frangi, A. F.; Yang, J. Y.; Zhang, D.; Jin, Z., KPCA plus LDA: a complete kernel fisher discriminant framework for feature extraction and recognition, IEEE Trans. Pattern Anal. Mach. Intell., 27, 2, 230-244 (2005)
[25] Zhuang, X. S.; Dai, D. Q., Inverse Fisher discriminate criteria for small sample size problem and its application to face recognition, Pattern Recognition, 38, 11, 2192-2194 (2005)
[26] Fukunaga, K., Introduction to Statistical Pattern Recognition (1990), Academic Press: Academic Press New York · Zbl 0711.62052
[27] Dudoit, S.; Fridlyand, J.; Speed, T., Comparison of discrimination methods for the classification of tumors using gene expression data, J. Am. Stat. Assoc., 97, 77-87 (2002) · Zbl 1073.62576
[28] Lee, J. W.; Lee, J. B.; Park, M.; Song, S. H., An extensive comparison of recent classification tools applied to microarray data, Comput. Stat. Data Anal., 48, 869-885 (2005) · Zbl 1429.62252
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.