E-MobileNeXt: face expression recognition model based on improved MobileNeXt

Abstract

In response to the high complexity and low accuracy of current facial expression recognition networks, this paper proposes an E-MobileNeXt network for facial expression recognition. E-MobileNeXt is built based on our proposed E-SandGlass block. In addition, we also improve the overall performance of the network through RepConv and SGE attention mechanisms. The experimental results show that the network model improves the expression recognition accuracy by 6.5% and 7.15% in RAF-DB and CK+ datasets, respectively, while the parameter and floating-point operations decreased by 0.79 M and 4.2 M compared with MobileNeXt.

References

[[1]] Revina I M, Emmanuel W R S. A survey on human face expression recognition techniques[J]. Journal of King Saud University-computer and information sciences, 2021, 33(6): 619-628,

[[2]] Li S, Deng W. Deep facial expression recognition: a survey[J]. IEEE transactions on affective computing, 2020, 13(3): 1195-1215,

[[3]] Yu Z, Zhang C. . Image based static facial expression recognition with multiple deep network learning[C], 2015 New York Association for Computing Machinery 435-442

[[4]] Jiang S, Xu X, Liu F, et al.. . CS-GResNet: a simple and highly efficient network for facial expression recognition[C], 2022 New York IEEE 2599-2603

[[5]] Barros P, Churamani N, Sciutti A. . The facechannel: a light-weight deep neural network for facial expression recognition[C], 2020 New York IEEE 652-656

[[6]] RODOLFO F P, MITRE H H. ResMoNet: a residual mobile-based network for facial emotion recognition in resource-limited systems[EB/OL]. (2020-05-15) [2023-04-10]. https://arxiv.org/abs/2005.07649.

[[7]] Zhou D, Hou Q, Chen Y, et al.. . Rethinking bottleneck structure for efficient mobile network design[C], 2020 Berlin, Heidelberg Springer-Verlag 680-697

[[8]] Ding X, Zhang X, Ma N, et al.. . Repvgg: making vgg-style convnets great again[C], 2021 New York IEEE 13733-13742

[[9]] Han K, Wang Y, Tian Q, et al.. . Ghostnet: more features from cheap operations[C], 2020 New York IEEE 1580-1589

[[10]] Sinha D, El-Sharkawy M. . Thin mobilenet: an enhanced mobilenet architecture[C], 2019 New York IEEE 0280-0285

[[11]] LI X, HU X, YANG J. Spatial group-wise enhance: improving semantic feature learning in convolutional networks[EB/OL]. (2019-05-23) [2023-04-10]. https://arxiv.org/abs/1905.09646v1.

[[12]] Li S, Deng W, Du J P. . Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild[C], 2017 New York IEEE 2852-2861

[[13]] Lucey P, Cohn J F, Kanade T, et al.. . The extended cohn-kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression[C], 2010 New York IEEE 94-101

[[14]] Nigam S, Singh R, Misra A K. Efficient facial expression recognition using histogram of oriented gradients in wavelet domain[J]. Multimedia tools and applications, 2018, 77: 28725-28747,

[[15]] Liu P, Han S, Meng Z, et al.. . Facial expression recognition via a boosted deep belief network[C], 2014 New York IEEE 1805-1812

[[16]] Wang Z, Zeng F, Liu S, et al.. OAENet: oriented attention ensemble for accurate facial expression recognition[J]. Pattern recognition, 2021, 112: 107694,

[[17]] Hua C H, Huynh T T, Seo H, et al.. . Convolutional network with densely backward attention for facial expression recognition[C], 2020 New York IEEE 1-6

[[18]] Gan C, Xiao J, Wang Z, et al.. Facial expression recognition using densely connected convolutional neural network and hierarchical spatial attention[J]. Image and vision computing, 2022, 117: 104342,

[[19]] Ghosh S, Dhall A, Sebe N. . Automatic group affect analysis in images via visual attribute and feature networks[C], 2018 New York IEEE 1967-1971

[[20]] Li Y, Zeng J, Shan S, et al.. Occlusion aware facial expression recognition using CNN with attention mechanism[J]. IEEE transactions on image processing, 2018, 28(5): 2439-2450,

[[21]] Zeng J, Shan S, Chen X. . Facial expression recognition with inconsistently annotated datasets[C], 2018 Berlin, Heidelberg Springer-Verlag 227-243