E-MobileNeXt: face expression recognition model based on improved MobileNeXt

Abstract

In response to the high complexity and low accuracy of current facial expression recognition networks, this paper proposes an E-MobileNeXt network for facial expression recognition. E-MobileNeXt is built based on our proposed E-SandGlass block. In addition, we also improve the overall performance of the network through RepConv and SGE attention mechanisms. The experimental results show that the network model improves the expression recognition accuracy by 6.5% and 7.15% in RAF-DB and CK+ datasets, respectively, while the parameter and floating-point operations decreased by 0.79 M and 4.2 M compared with MobileNeXt.

References

[1] RevinaI M, EmmanuelW R S. A survey on human face expression recognition techniques[J]. Journal of King Saud University-computer and information sciences, 2021, 33(6):619-628

[2] LiS, DengW. Deep facial expression recognition: a survey[J]. IEEE transactions on affective computing, 2020, 13(3):1195-1215

[3] YuZ, ZhangC. Image based static facial expression recognition with multiple deep network learning[C], 2015, New York, Association for Computing Machinery: 435-442

[4] JiangS, XuX, LiuF, et al.. CS-GResNet: a simple and highly efficient network for facial expression recognition[C], 2022, New York, IEEE: 2599-2603

[5] BarrosP, ChuramaniN, SciuttiA. The facechannel: a light-weight deep neural network for facial expression recognition[C], 2020, New York, IEEE: 652-656

[6] RODOLFO F P, MITRE H H. ResMoNet: a residual mobile-based network for facial emotion recognition in resource-limited systems[EB/OL]. (2020-05-15) [2023-04-10]. https://arxiv.org/abs/2005.07649.

[7] ZhouD, HouQ, ChenY, et al.. Rethinking bottleneck structure for efficient mobile network design[C], 2020, Berlin, Heidelberg, Springer-Verlag: 680-697

[8] DingX, ZhangX, MaN, et al.. Repvgg: making vgg-style convnets great again[C], 2021, New York, IEEE: 13733-13742

[9] HanK, WangY, TianQ, et al.. Ghostnet: more features from cheap operations[C], 2020, New York, IEEE: 1580-1589

[10] SinhaD, El-SharkawyM. Thin mobilenet: an enhanced mobilenet architecture[C], 2019, New York, IEEE: 0280-0285

[11] LI X, HU X, YANG J. Spatial group-wise enhance: improving semantic feature learning in convolutional networks[EB/OL]. (2019-05-23) [2023-04-10]. https://arxiv.org/abs/1905.09646v1.

[12] LiS, DengW, DuJ P. Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild[C], 2017, New York, IEEE: 2852-2861

[13] LuceyP, CohnJ F, KanadeT, et al.. The extended cohn-kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression[C], 2010, New York, IEEE: 94-101

[14] NigamS, SinghR, MisraA K. Efficient facial expression recognition using histogram of oriented gradients in wavelet domain[J]. Multimedia tools and applications, 2018, 77: 28725-28747

[15] LiuP, HanS, MengZ, et al.. Facial expression recognition via a boosted deep belief network[C], 2014, New York, IEEE: 1805-1812

[16] WangZ, ZengF, LiuS, et al.. OAENet: oriented attention ensemble for accurate facial expression recognition[J]. Pattern recognition, 2021, 112: 107694

[17] HuaC H, HuynhT T, SeoH, et al.. Convolutional network with densely backward attention for facial expression recognition[C], 2020, New York, IEEE: 1-6

[18] GanC, XiaoJ, WangZ, et al.. Facial expression recognition using densely connected convolutional neural network and hierarchical spatial attention[J]. Image and vision computing, 2022, 117: 104342

[19] GhoshS, DhallA, SebeN. Automatic group affect analysis in images via visual attribute and feature networks[C], 2018, New York, IEEE: 1967-1971

[20] LiY, ZengJ, ShanS, et al.. Occlusion aware facial expression recognition using CNN with attention mechanism[J]. IEEE transactions on image processing, 2018, 28(5):2439-2450

[21] ZengJ, ShanS, ChenX. Facial expression recognition with inconsistently annotated datasets[C], 2018, Berlin, Heidelberg, Springer-Verlag: 227-243