Fusing magnitude and phase features with multiple face models for robust face recognition

Abstract

High accuracy face recognition is of great importance for a wide variety of real-world applications. Although significant progress has been made in the last decades, fully automatic face recognition systems have not yet approached the goal of surpassing the human vision system, even in controlled conditions. In this paper, we propose an approach for robust face recognition by fusing two complementary features: one is Gabor magnitude of multiple scales and orientations and the other is Fourier phase encoded by spatial pyramid based local phase quantization (SPLPQ). To reduce the high dimensionality of both features, block-wise fisher discriminant analysis (BFDA) is applied and further combined by score-level fusion. Moreover, inspired by the biological cognitive mechanism, multiple face models are exploited to further boost the robustness of the proposed approach. We evaluate the proposed approach on three challenging databases, i.e., FRGC ver2.0, LFW, and CFW-p, that address two face classification scenarios, i.e., verification and identification. Experimental results consistently exhibit the complementarity of the two features and the performance boost gained by the multiple face models. The proposed approach achieved approximately 96% verification rate when FAR was 0.1% on FRGC ver2.0 Exp.4, impressively surpassing all the best known results.

Keywords

face recognition / fisher discriminant analysis / fusion / Gabor magnitude feature / multiple face models / spatial pyramid based local phase quantization

References

[1] Zhao W Y, Chellappa R, Phillips P J, Rosenfeld A. Face recognition: a literature survey. ACM Computing Surveys, 2003, 35(4): 399–458

[2] Liu C J, Wechsler H. Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition. IEEE Transactions on Image Processing, 2002, 11(4): 467–476

[3] Zhang W C, Shan S G, Gao W, Chen X L, Zhang H M. Local gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition. In: Proceedings of the 10th IEEE International Conference on Computer Vision. 2005, 786–791

[4] Su Y, Shan S G, Chen X L, Gao W. Hierarchical ensemble of global and local classifiers for face recognition. IEEE Transactions on Image Processing, 2009, 18(8): 1885–1896

[5] Xie S F, Shan S G, Chen X L, Chen J. Fusing local patterns of gabor magnitude and phase for face recognition. IEEE Transactions on Image Processing, 2010, 19(5): 1349–1361

[6] Li Y, Shan S G, Zhang H H, Lao S H, Chen X L. Fusing magnitude and phase features for robust face recognition. In: Proceedings of Asian Conference on Computer Vision. 2013, 601–612

[7] Tan X Y, Triggs B. Fusing gabor and lbp feature sets for kernel-based face recognition. In: Proceedings of International Conference on Automatic Face and Gesture Recognition. 2007, 235–249

[8] Chan C H, Kittler J, Tahir M A. Kernel fusion of multiple histogram descriptors for robust face recognition. In: Proceedings of Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition and Structural, Syntactic, and Statistical Pattern Recognition. 2010, 718–727

[9] Cai D, He X F, Han J W. Efficient kernel discriminant analysis via spectral regression. In: Proceedings of the 7th IEEE International Conference on Data Mining. 2007, 427–432

[10] Deng W H, Hu J N, Guo J, Cai W, Feng D G. Emulating biological strategies for uncontrolled face recognition. Pattern Recognition, 2010, 43(6): 2210–2223

[11] Deng W H, Hu J N, Lu J W, Guo J. Transform-invariant PCA: a unified approach to fully automatic facealignment, representation, and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(6): 1275–1284

[12] Gabor D. Theory of communication. Part 1: the analysis of information. Journal of the Institution of Electrical Engineers-Part III: Radio and Communication Engineering, 1946, 93(26): 429–441

[13] Ojansivu V, Heikkilä J. Blur insensitive texture classification using local phase quantization. In: Proceedings of International Conference on Image and Signal Processing. 2008, 236–243

[14] Ojala T, Pietikäinen M, Harwood D. A comparative study of texture measures with classification based on featured distributions. Pattern Recognition, 1996, 29(1): 51–59

[15] Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91–110

[16] Bicego M, Lagorio A, Grosso E, Tistarelli M. On the use of sift features for face authentication. In: Proceedings of the 2006 Conference on Computer Vision and Pattern Recognition Workshop. 2006

[17] Luo J, Ma Y, Takikawa E, Lao S, Kawade M, Lu B L. Person-specific sift features for face recognition. In: Proceedings of International Conference on Acoustics, Speech and Signal Processing. 2007

[18] Mian A S, Bennamoun M, Owens R. An efficient multimodal 2D-3D hybrid approach to automatic face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(11): 1927–1943

[19] Dalal N, Triggs B. Histograms of oriented gradients for human detection. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2005, 886–893

[20] Cao Z M, Yin Q, Tang X O, Sun J. Face recognition with learningbased descriptor. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2010, 2707–2714

[21] Albiol A, Monzo D, Martin A, Sastre J, Albiol A. Face recognition using HOG-EBGM. Pattern Recognition Letters, 2008, 29(10): 1537–1543

[22] Liu Z M, Liu C J. Robust face recognition using color information. In: Proceedings of International Conference on Biometrics. 2009, 122–131

[23] Shan S G, Zhang W C, Su Y, Chen X L, Gao W. Ensemble of piecewise FDA based on spatial histograms of local (Gabor) binary patterns for face recognition. In: Proceedings of International Conference on Pattern Recognition. 2006, 606–609

[24] Sinha P, Poggio T. I think I know that face. Nature, 1996, 384(6608): 404

[25] Davies G. Perceiving and Remembering Faces. London: Academic Press, 1981

[26] Ellis H D. Aspects of Face Processing. Boston: Martinus Nijhoff Publishers, 1986

[27] Phillips P J, Flynn P J, Scruggs T, Bowyer K W, Chang J, Hoffman K, Marques J, Min J, Worek W. Overview of the face recognition grand challenge. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2005, 947–954

[28] Huang G B, Mattar M, Berg T, Learned-Miller E. Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical Report 07-49, 2007

[29] Zhang X, Zhang L, Wang X J, Shum H Y. Finding celebrities in billions of Web images. IEEE Transactions on Multimedia, 2012, 14(4): 995–1007

[30] Lades M, Vorbruggen J C, Buhmann J, Lange J, Malsburg V D C, Wurtz R P, Konen W. Distortion invariant object recognition in the dynamic link architecture. IEEE Transactions on Computers, 1993, 42(3): 300–311

[31] Zhang B C, Shan S G, Chen X L, Gao W. Histogram of Gabor phase patterns (HGPP): a novel object representation approach for face recognition. IEEE Transactions on Image Processing, 2007, 16(1): 57–68

[32] Lazebnik S, Schmid C, Ponce J. Beyond bags of features: spatial pyramidmatching for recognizing natural scene categories. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2006, 2169–2178

[33] Grauman K, Darrell T. The pyramid match kernel: discriminative classification with sets of image features. In: Proceedings of International Conference on Computer Vision. 2005, 1458–1465

[34] Hadjidemetriou E, Grossberg M D, Nayar S K. Multiresolution histograms and their use for recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(7): 831–847

[35] Fisher R A. The use of multiple measurements in taxonomic problems. Annals of Human Genetics, 1936, 7(2): 179–188

[36] Kumar N, Berg A C, Belhumeur P N, Nayar S K. Attribute and simile classifiers for face verification. In: Proceedings of International Conference on Computer Vision. 2009, 365–372

[37] Tan X Y, Triggs B. Enhanced local texture feature sets for face recognition under difficult lighting conditions. In: Proceedings of International Conference on Automatic Face and Gesture Recognition. 2007, 168–182

[38] Yang J, Liu C J. Color image discriminant models and algorithms for face recognition. IEEE Transactions on Neural Networks, 2008, 19(12): 2088–2098

[39] Yang J, Liu C J. Horizontal and vertical 2DPCA-based discriminant analysis for face verification on a large-scale database. IEEE Transactions on Information Forensics and Security, 2007, 2(4): 781–792

[40] Shih P, Liu C J. Improving the face recognition grand challenge baseline performance using color configurations across color spaces. In: Proceedings of International Conference on Image Processing. 2006, 1001–1004

[41] Taigman Y, Wolf L, Hassner T. Multiple one-shots for utilizing class label information. In: Proceedings of British Machine Vision Conference. 2009, 1–12

[42] Yang Y, Song J K, Huang Z, Ma Z G, Sebe N, Hauptmann A G. Multi-feature fusion via hierarchical regression for multimedia analysis. IEEE Transaction on Multimedia, 2013, 15(3): 572–581

[43] Ma Z G, Yang Y, Sebe N, Hauptmann A G. Multiple features but few labels? a symbiotic solution exemplified for video analysis. In: Proceedings of the 22nd ACM International Conference on Multimedia. 2014, 77–86

[44] Kumar B, Savvides M, Xie C Y. Correlation pattern recognition for face recognition. Proceedings of the IEEE, 2006, 94(11): 1963–1976

[45] Liu C J. The bayes decision rule induced similarity measures. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(6): 1086–1090

[46] Hwang W, Park G, Lee J, Kee S C. Multiple face model of hybrid fourier feature for large face image set. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2006, 1574–1581

[47] Liu C J. Capitalize on dimensionality increasing techniques for improving face recognition grand challenge performance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(5): 725–737

[48] Parkhi O M, Vedaldi A, Zisserman A. Deep face recognition. In: Proceedings of British Machine Vision Conference. 2015, 1–12

[49] Li P, Fu Y, Mohammed U, Elder J H, Prince S J. Probabilistic models for inference about identity. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(1): 144–157

[50] Berg T, Belhumeur P N. Tom-vs-Pete classifiers and identitypreserving alignment for face verification. In: Proceedings of British Machine Vision Conference. 2012

[51] Chen D, Cao X D, Wen F, Sun J. Blessing of dimensionality: highdimensional feature and its efficient compression for face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 3025–3032

[52] Taigman Y, Yang M, Ranzato M, Wolf L. Deepface: closing the gap to human-level performance in face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 1701–1708

[53] Sun Y, Wang X G, Tang X O. Deep learning face representation from predicting 10,000 classes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 1891–1898

[54] Schroff F, Kalenichenko D, Philbin J. Facenet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 815–823

[55] Yi D, Lei Z, Liao S C, Li S Z. Learning face representation from scratch. 2014, arXiv preprint arXiv:1411.7923v1

RIGHTS & PERMISSIONS

Higher Education Press and Springer-Verlag GmbH Germany, part of Springer Nature