Face recognition using SIFT features under 3D meshes

Abstract

Expression, occlusion, and pose variations are three main challenges for 3D face recognition. A novel method is presented to address 3D face recognition using scale-invariant feature transform (SIFT) features on 3D meshes. After preprocessing, shape index extrema on the 3D facial surface are selected as keypoints in the difference scale space and the unstable keypoints are removed after two screening steps. Then, a local coordinate system for each keypoint is established by principal component analysis (PCA). Next, two local geometric features are extracted around each keypoint through the local coordinate system. Additionally, the features are augmented by the symmetrization according to the approximate left-right symmetry in human face. The proposed method is evaluated on the Bosphorus, BU-3DFE, and Gavab databases, respectively. Good results are achieved on these three datasets. As a result, the proposed method proves robust to facial expression variations, partial external occlusions and large pose changes.

Keywords

3D face recognition / scale-invariant feature transform (SIFT) / expression / occlusion / large pose changes / 3D meshes

References

[1] JainA K, RossA, PrabhakarS. An introduction to biometric recognition [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2004, 14(1): 4-20

[2] JainA K, RossA, PrabhakarS. Biometric identification [J]. Communications of the ACM, 2000, 43(2): 91-98

[3] MedioniG, WaupotitschR. Face modeling and recognition in 3-D [C]. Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures, 2003, Nice, France, IEEE: 232-233

[4] ChangK, BowyerK, FlynnP. A survey of approaches and challenges in 3D and multi-modal 2D+3D face recognition [J]. Computer Vision and Image Understanding, 2006, 101(1): 1-15

[5] BeslP J, MckayN D. A method for registration of 3-D shapes [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992, 14(2): 239-256

[6] LuX-g, JainA K, ColbryD. Matching 2.5D face scans to 3D models [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(1): 31-43

[7] MianA S, BennamounM, OwensR. An efficient multimodal 2D-3D hybrid approach to automatic face recognition [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(11): 1927-1943

[8] MahooraM, Abdel-mottalebM. Face recognition based on 3D ridge images obtained from range data [J]. Pattern Recognition, 2009, 42(3): 445-451

[9] ChangK I, BowyerK W, FlynnP J. Multiple nose region matching for 3D face recognition under varying facial expression [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(10): 1695-1700

[10] WangY-m, PanG, WuZ-h, WangY-gang. Exploring facial expression effects in 3D face recognition using partial ICP [C]. Proceedings of the 7th Asian Conference on Computer Vision, 2006, Berlin, Heidelberg, Springer: 581-590

[11] SalahA A, AlyuzN, AkarunL. Registration of three-dimensional face scans with average face models [J]. Journal of Electronic Imaging, 2008, 17(1): 1-14

[12] HuttenlocherD P, KlandermanG A, RucklidgeW J. Comparing images using the Hausdorff distance [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993, 15(9): 850-863

[13] AchermannB, BunkeH. Classifying range images of human faces with Hausdorff distance [C]. Proceedings of the 15th International Conference on Pattern Recognition, 2000, Barcelona, Spain, IEEE: 809-813

[14] LeeY, ShimJ. Curvature based human face recognition using depth weighted Hausdorff distance [C]. Proceedings of the 2004 International Conference on Image Processing, 2004, Singapore, IEEE: 1429-1432

[15] RussT D, KochM W, LittleC Q. A 2D range Hausdorff approach for 3D face recognition [C]. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005, San Diego, USA, IEEE: 1-8

[16] CondeC, Rodriguez-AragonL J, CabelloE. Automatic 3D face feature points extraction with spin images [C]. Proceedings of the Third International Conference on Image Analysis and Recognition. Povoa de Varzim, Portugal, 2006317-328

[17] RomeroM, PearsN. Point-pair descriptors for 3d facial landmark localization [C]. Proceedings of the IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems, 2009, Washington, DC, USA, IEEE: 1-6

[18] PerakisP, PassalisG, TheoharisT, KakadiarisI A. 3D facial landmark detection under large yaw and expression variations [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(7): 1552-1564

[19] ChuaC, JarvisR. Point signatures: A new representation for 3D object recognition [J]. International Journal of Computer Vision, 1997, 25(1): 63-85

[20] WangY-j, ChuaC-s, HoY-khing. Facial feature detection and face recognition from 2D and 3D images [J]. Pattern Recognition Letters, 2002, 23(10): 1191-1202

[21] IrfanogluM O, GokberkB, AkarunL. 3D shape-based face recognition using automatically registered facial surfaces [C]. Proceedings of the 17th International Conference on Pattern Recognition, 2004, Cambridge, IEEE: 183-186

[22] WangY-j, ChuaC-seng. Robust face recognition from 2D and 3D images using structural Hausdorff distance [J]. Image and Vision Computing, 2006, 24(2): 176-185

[23] NagamineT, UemuraT, MasudaI. 3D facial image analysis for human identification [C]. Proceedings of the 11th IAPR International Conference on Pattern Recognition, 1992, The Hague, Netherlands, IEEE: 324-327

[24] BeumierC, AcheroyM. Automatic 3D face authentication [J]. Image and Vision Computing, 2000, 18(4): 315-321

[25] SamirC, SrivastavaA, DaoudiM. Three-dimensional face recognition using shapes of facial curves [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(11): 1858-1863

[26] ColomboA, CusanoC, SchettiniR. 3D face detection using curvature analysis [J]. Pattern Recognition, 2006, 39(3): 444-455

[27] GokberkB, IrfanogluM O, AkarunL. 3D shape-based face representation and feature extraction for face recognition [J]. Image and Vision Computing, 2006, 24(8): 857-869

[28] GokberkB, DutagaciH, UlasA, AkarunL, SankurB. Representation plurality and fusion for 3-D face recognition [J]. IEEE Transactions on Systems, Man, and Cybernetics, 2008, 38(1): 155-173

[29] TanakaH T, IkedaM, ChiakiH. Curvature-based face surface recognition using spherical correlation — Principal directions for curved object recognition [C]. Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, 1998, Nara, Japan, IEEE: 372-377

[30] BronsteinM A, BronsteinM M, KimmelR. Three dimensional face recognition [J]. International Journal of Computer Vision, 2005, 64(1): 5-30

[31] LoweD G. Object recognition from local scale-invariant features [C]. Proceedings of the Seventh IEEE International Conference on Computer Vision. Kerkyra, 19991150-1157

[32] LoweD G. Distinctive image features from scale-invariant keypoints [J]. International Journal of Computer Vision, 2004, 60(2): 91-110

[33] SeS, LoweD G, LittleJ J. Vision-based mobile robot localization and mapping using scale-invariant features [C]. Proceedings of International Conference on Robotics and Automation, 20012051-2058

[34] BrownM, LoweD G. Automatic panoramic image stitching using invariant features [J]. International Journal of Computer Vision, 2007, 74(1): 59-73

[35] BasharatA, ZhaiY, ShahM. Content based video matching using spatiotemporal volumes [J]. Computer Vision and Image Understanding, 2008, 110(3): 360-377

[36] CheungW, HamarnehG. n-SIFT: n-dimensional scale invariant feature transform [J]. IEEE Transactions on Image Processing, 2009, 18(9): 2012-2021

[37] OsadaK, FuruyaT, OhbuchiR. Shrec’08 entry: Local volumetric features for 3D model retrieval [C]. Proceedings of IEEE International Conference on Shape Modeling and Applications, 2008, New York, USA, IEEE: 245-246

[38] ZouG-y, HuaJ, DongM, QinHong. Surface matching with salient keypoints in geodesic scale space [J]. Computer Animation and Virtual Worlds, 2008, 19(3/4): 399-410

[39] ZaharescuA, BoyerE, VaranasiK, HoraudR. Surface feature detection and description with applications to mesh matching [C]. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2009, Miami, Florida, IEEE: 373-380

[40] SmeetsD, KeustermansJ, VandermeulenD, SuetensP. meshSIFT: Local surface features for 3D face recognition under expression variations and partial data [J]. Computer Vision and Image Understanding, 2013, 117(2): 158-169

[41] DoraiC, JainA K. COSMOS-A representation scheme for 3D free-form objects [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 19(10): 1115-1130

[42] SavranA, AlyuzN, DibekliogluH, CeliktutanO, GokberkB, SankurB, AkarunL. Bosphorus database for 3D face analysis [J]. Biometrics and Identity Management, 2008, 5372: 47-56

[43] YinL-j, WeiX-z, SunY, WangJ, RosatoM J. A 3D facial expression database for facial behavior research [C]. Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition. Southampton, 2006211-216

[44] MorenoA B, SanchezA. GavabDB: A 3D face database [C]. Proceedings of COST Workshop on Biometrics on the Internet: Fundamentals, Advances and Applications. Vigo, Spain, 200477-82

[45] KmanP, FriesenW VThe facial action coding system: A technique for the measurement of facial movement [M], 1978, San Francisco, Consulting Psychologists Press

[46] AlyuzN, GokberkB, AkarunL. Regional registration for expression resistant 3-D face recognition [J]. IEEE Transactions on Information Forensics and Security, 2010, 5(3): 425-440

[47] HuangD, ArdabilianM, WangY-h, ChenL-ming. 3-D face recognition using eLBP-based facial description and local feature hybrid matching [J]. IEEE Transactions on Information Forensics and Security, 2012, 7(5): 1551-1565

[48] LiuP-j, WangY-h, HuangD, ZhangZ-x, ChenL-ming. Learning the spherical harmonic features for 3-D face recognition [J]. IEEE Transactions on Image Processing, 2013, 22(3): 914-925

[49] KaushikV D, BudhwarA, DubeyA, AgrawalR, GuptaS, PathakV K, GuptaP. An efficient 3D face recognition algorithm [C]. Proceedings of the 3rd International Conference on New Technologies, Mobility and Security. Cairo, 20091-5

[50] DaniyalF, NairP, CavallaroA. Compact signatures for 3D face recognition under varying expressions [C]. Proceedings of the Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance. Genova, 2009302-307

[51] WangY-m, LiuJ-z, TangX-ou. Robust 3D face recognition by local shape difference boosting [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(10): 1858-1870

[52] SmeetsD, FabryT, HermansJ, VandermeulenD, SuetensP. Fusion of an isometric deformation modeling approach using spectral decomposition and a region-based approach using ICP for expression-invariant 3D face recognition [C]. Proceedings of the 20th International Conference on Pattern Recognition. Istanbul, 20101172-1175

[53] LeiY-j, BennamounM, El-sallamA A. An efficient 3D face recognition approach based on the fusion of novel local low-level features [J]. Patter Recognition, 2013, 46(1): 24-37

[54] DriraH, AmorB B, DaoudiM, SrivastavaA. Pose and expression-invariant 3D face recognition using elastic radial curves [C]. Proceedings of British Machine Vision Conference. Aberystwyth, UK, 20101-11

[55] LiX-x, JiaT, ZhangHao. Expression-insensitive 3D face recognition using sparse representation [C]. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Miami, Florida, 20092575-2582