Robust facial expression recognition via lightweight reinforcement learning for rehabilitation robotics

Abstract

This paper proposes a lightweight reinforcement network (LRN) and auxiliary label distribution learning (ALDL) based robust facial expression recognition (FER) method. Our designed representation reinforcement (RR) network mainly comprises two modules, i.e., the RR module and the auxiliary label space construction (ALSC) module. The RR module highlights key feature messaging nodes in feature maps, and ALSC allows multiple labels with different intensities to be linked to one expression. Therefore, LRN has a more robust feature extraction capability when model parameters are greatly reduced, and ALDL is proposed to contribute to the training effect of LRN in the condition of ambiguous training data. We tested our method on FER-Plus and RAF-DB datasets, and the experiment demonstrates the feasibility of our method in practice during rehabilitation robots.

Keywords

A

References

[1] Yu J, Gao H, Ju Z J, et al. . Deep object detector with attentional spatiotemporal LSTM for space human-robot interaction. IEEE transactions on human-machine systems. 2022, 52: 784-793 J]

[2] Yu J, Gao H, Ju Z J, et al. . Deep temporal model-based identity aware hand detection for space human-robot interaction. IEEE transactions on cybernetics. 2021, 52: 13738-13751 J]

[3] Xu L, Zhao S, Wang T. Aero-optic imaging deviation prediction based on ISSA-ELM. Optoelectronics letters. 2023, 19(7): 425-431 J]

[4] Fang Y, Zhou D, Li K, et al. . Attribute-driven granular model for EMG-based pinch and fingertip force grand recognition. IEEE transactions on cybernetics. 2019, 51(2): 789-800 J]

[5] Li K, Boyd P, Ju Z J, et al. . Electrotactile feedback in a virtual hand rehabilitation platform: evaluation and implementation. IEEE transactions on automation science and engineering. 2018, 16(4): 1556-1565 J]

[6] Li S, Deng W. Reliable crowd sourcing and deep locality-preserving learning for unconstrained facial expression recognition. IEEE transactions on image processing. 2018, 28: 356-370 J]

[7] GOODFELLOW I J, ERHAN D, CARRIER P L, et al. Challenges in representation learning: a report on three machine learning contests[J]. Proceedings of the international conference on neural information processing, 2013: 117–124.

[8] Zeng D, Lin Z, Yan X. Face2exp: combating data biases for facial expression recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 18–24, 2022, New Orleans, LA, USA. 2022, New York, IEEE: 2029120300[C]

[9] Wang K, Peng X, Yang J, et al. . Region attention networks for pose and occlusion robust facial expression recognition. IEEE transactions on image processing. 2020, 29: 4057-4069 J]

[10] Liu J, Ji X, Wang M. SFR-Net: sample-aware and feature refinement network for cross-domain micro-expression recognition. Optoelectronics letters. 2023, 19(7): 437-442 J]

[11] Chen L, Wang K, Li M, et al. . K-means clustering-based kernel canonical correlation analysis for multimodal emotion recognition in human-robot interaction. IEEE transactions on industrial electronics. 2022, 70(1): 1016-1024 J]

[12] Zeng J, Shan S, Chen X. Facial expression recognition with inconsistently annotated data-sets. Proceedings of the European Conference on Computer Vision (ECCV), September 8–14, 2018, Munich, Germany. 2018, Heidelberg, Springer: 222237[C]

[13] FAN Y, LAM J C, LI V O. Multi-region ensemble convolutional neural network for facial expression recognition[J]. Proceedings of the international conference on artificial neural networks, 2018: 84–94.

[14] Li Y, Zeng J, Shan S, et al. . Occlusion aware facial expression recognition using CNN with attention mechanism. IEEE transactions on image processing. 2018, 28: 2439-2450 J]

[15] SHI J, ZHU S. Learning to amend facial expression representation via de-albino and affinity[EB/OL]. (2021-03-18) [2023-10-23]. https://arxiv.org/abs/2103.10189.

[16] Li S, Deng W, Du J. Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, July 21–26, 2017, Honolulu, HI, USA. 2017, New York, IEEE: 28522861[C]

RIGHTS & PERMISSIONS

Tianjin University of Technology