Deep Learning-Enabled Multifunctional Electronic Skin System for Human Activity Recognition and Sign Language Translation

Abstract

The integration of wearable technologies and intelligent algorithms is anticipated to enable synergistic platforms for human motion monitoring and enhanced human-computer interaction. However, the realization of personalized and multi-functional integration of electronic devices through simple fabrication processes remains challenging. Additionally, the current artificial intelligence algorithms lack the universality across different application scenarios. In this work, we present a multifunctional electronic skin (e-skin) system with deep learning-assisted multi-sensing capabilities. The e-skin system integrates a dual-layer 3D-printed e-skin, a flexible multi-channel wireless acquisition circuit, a robust data processing framework, and a multi-action classification neural network (MAC-Net) designed for human activity and sign language recognition. The e-skin can monitor finger bending, joint strain, temperature, humidity, and glucose. Its integrated data processing framework enables an automated workflow from raw signal acquisition to model training and evaluation, enhancing processing efficiency and signal reliability. Experimental results demonstrate that the integration of e-skin with MAC-Net enables accurate daily motion monitoring, sign language recognition, and human-computer interaction, offering a practical and customizable solution for next-generation wearable systems.

Keywords

deep learning / electronic skin / human activity recognition / sign language recognition / wireless sensing

References

[1] M. Guo, Y. Xia, J. Liu, Y. Zhang, M. Li, and X. Wang, “Wearable Pressure Sensor Based on Triboelectric Nanogenerator for Information Encoding, Gesture Recognition, and Wireless Real-Time Robot Control,” Advanced Functional Materials 35, no. 22 (2025): 2419209.

[2] S. Zhang, M. Guo, Y. Xia, S. Li, X. Zhi, and X. Wang, “Noncontact Monolayered Triboelectric Nanogenerator Based on Stretchable MWCNTs/MXene/Ecoflex Film for Human–Machine Interface and High-Accuracy Handwritten Recognition,” Chemical Engineering Journal 505 (2025): 159562.

[3] S. Han, X. Zhi, Y. Xia, et al., “All Resistive Pressure–Temperature Bimodal Sensing E-Skin for Object Classification,” Small 19 (2023): 2301593.

[4] A. S. Pillai, A. Chandran, and S. K. Peethambharan, “Silver Nanoparticle-Decorated Multiwalled Carbon Nanotube Ink for Advanced Wearable Devices,” ACS Applied Materials & Interfaces 14, no. 41 (2022): 46775–46788.

[5] Z. Wang, Z. Zhou, C.-L. Li, et al., “A Single Electronic Tattoo for Multisensory Integration,” Small Methods 7, no. 4 (2023): 2201566.

[6] A. Ahmed, Y.-S. Guan, I. Hassan, et al., “Multifunctional Smart Electronic Skin Fabricated From Two-Dimensional Like Polymer Film,” Nano Energy 75 (2020): 105044.

[7] J. H. Lee, J. S. Heo, Y.-J. Kim, et al., “A Behavior-Learned Cross-Reactive Sensor Matrix for Intelligent Skin Perception,” Advanced Materials 32, no. 22 (2020): 2000969.

[8] Z. Wang, M. Bu, K. Xiu, et al., “A Flexible, Stretchable and Triboelectric Smart Sensor Based on Graphene Oxide and Polyacrylamide Hydrogel for High Precision Gait Recognition in Parkinsonian and Hemiplegic Patients,” Nano Energy 104 (2022): 107978.

[9] J. Sun, K. Xiu, Z. Wang, et al., “Multifunctional Wearable Humidity and Pressure Sensors Based on Biocompatible Graphene/Bacterial Cellulose Bioaerogel for Wireless Monitoring and Early Warning of Sleep Apnea Syndrome,” Nano Energy 108 (2023): 108215.

[10] S. Chen, Y. Wang, B. Fei, et al., “Development of a Flexible and Highly Sensitive Pressure Sensor Based on an Aramid Nanofiber-Reinforced Bacterial Cellulose Nanocomposite Membrane,” Chemical Engineering Journal 430 (2022): 131980.

[11] L. Zhang, Z. Wang, Z. Zhang, et al., “A Multiplexed and In Situ Self-Calibration Biosensor Integrated System for Metabolic Health Analysis in Human Urine,” Advanced Functional Materials (2025). https://doi.org/10.1002/adfm.20252027.

[12] H. Yuan, Z. Wang, K. Yuan, et al., “Ferroelectric-Coupled MXene/BaTiO₃ Nanocomposite Empowering Flexible Triboelectric Nanogenerators for Deep Learning-Enhanced Human-Machine Health Monitoring,” Chemical Engineering Journal 515 (2025): 163758.

[13] Z. Hao, X. Fang, Z. Wang, et al., “Intelligent Wearable Graphene Nano-Electronics With Switchable Surface Wettability Capabilities for Autonomous Sweat Enrichment-Purification-Analysis,” Advanced Functional Materials 34 (2024): 2400947.

[14] K. Chen, K. Liang, H. Liu, et al., “Skin-Inspired Ultra-Tough Supramolecular Multifunctional Hydrogel Electronic Skin for Human–Machine Interaction,” Nano-Micro Letters 15, no. 1 (2023): 102.

[15] W. Niu, Q. Tian, Z. Liu, and X. Liu, “Solvent-Free and Skin-Like Supramolecular Ion-Conductive Elastomers With Versatile Processability for Multifunctional Ionic Tattoos and On-Skin Bioelectronics,” Advanced Materials 35, no. 39 (2023): 2304157.

[16] C. Shang, Q. Xu, N. Liang, J. Zhang, L. Li, and Z. Peng, “Multi-Parameter E-Skin Based on Biomimetic Mechanoreceptors and Stress Field Sensing,” npj Flexible Electronics 7, no. 1 (2023): 19.

[17] Y. Yu, J. Li, S. A. Solomon, et al., “All-Printed Soft Human-Machine Interface for Robotic Physicochemical Sensing,” Science Robotics 7, no. 67 (2022): eabn0495.

[18] C. H. Kim, D. H. Lee, J. Youn, H. Lee, and J. Jeong, “Simple and Cost-Effective Microfabrication of Flexible and Stretchable Electronics for Wearable Multi-Functional Electrophysiological Monitoring,” Scientific Reports 11, no. 1 (2021): 14823.

[19] Y. Chen, S. Lu, S. Zhang, et al., “Skin-Like Biosensor System via Electrochemical Channels for Noninvasive Blood Glucose Monitoring,” Science Advances 3, no. 12 (2017): e1701629.

[20] M. Sang, M. Cho, S. Lim, et al., “Fluorescent-Based Biodegradable Microneedle Sensor Array for Tether-Free Continuous Glucose Monitoring With Smartphone Application,” Science Advances 9, no. 22 (2023): eadh1765.

[21] Y. Wang, H. Wang, H. Wang, et al., “Calcium Gluconate Derived Carbon Nanosheet Intrinsically Decorated With Nanopapillae for Multifunctional Printed Flexible Electronics,” ACS Applied Materials & Interfaces 11, no. 22 (2019): 20272–20280.

[22] X. Chen, H. Wan, R. Guo, et al., “A Double-Layered Liquid Metal-Based Electrochemical Sensing System on Fabric as a Wearable Detector for Glucose in Sweat,” Microsystems & Nanoengineering 8, no. 1 (2022): 48.

[23] S. Jiang, J. Yu, Y. Xiao, Y. Zhu, and W. Zhang, “Ultrawide Sensing Range and Highly Sensitive Flexible Pressure Sensor Based on a Percolative Thin Film With a Knoll-Like Microstructured Surface,” ACS Applied Materials & Interfaces 11, no. 22 (2019): 20500–20508.

[24] H. Zhang, L. Lin, N. Hu, et al., “Pillared Carbon@Tungsten Decorated Reduced Graphene Oxide Film for Pressure Sensors With Ultra-Wide Operation Range in Motion Monitoring,” Carbon 189 (2022): 430–442.

[25] Q. Wei, G. Chen, H. Pan, et al., “MXene-Sponge Based High-Performance Piezoresistive Sensor for Wearable Biomonitoring and Real-Time Tactile Sensing,” Small Methods 6, no. 2 (2022): 2101051.

[26] D. Song, X. Li, X. P. Li, X. Jia, P. Min, and Z.-Z. Yu, “Hollow-Structured MXene-PDMS Composites as Flexible, Wearable and Highly Bendable Sensors With Wide Working Range,” Journal of Colloid and Interface Science 555 (2019): 751–758.

[27] X. Xiao, J. Yin, J. Xu, T. Tat, and J. Chen, “Advances in Machine Learning for Wearable Sensors,” ACS Nano 18, no. 34 (2024): 22734–22751.

[28] T. Someya, Z. Bao, and G. G. Malliaras, “The Rise of Plastic Bioelectronics,” Nature 540, no. 7633 (2016): 379–385.

[29] N. Dai, I. M. Lei, Z. Li, Y. Li, P. Fang, and J. Zhong, “Recent Advances in Wearable Electromechanical Sensors—Moving Towards Machine Learning-Assisted Wearable Sensing Systems,” Nano Energy 105 (2023): 108041.

[30] G. Li, S. Liu, Q. Mao, and R. Zhu, “Multifunctional Electronic Skins Enable Robots to Safely and Dexterously Interact With Human,” Advanced Science 9, no. 11 (2022): 2104969.

[31] G. Ye, T. Jin, X. Wang, et al., “Multimodal Integrated Flexible Electronic Skin for Physiological Perception and Contactless Kinematics Pattern Recognition,” Nano Energy 113 (2023): 108580.

[32] S. Gong, X. Zhang, X. A. Nguyen, et al., “Hierarchically Resistive Skins as Specific and Multimetric On-Throat Wearable Biosensors,” Nature Nanotechnology 18, no. 8 (2023): 889–897.

[33] Y. Qiu, Z. Zou, Z. Zou, et al., “Deep-Learning-Assisted Printed Liquid Metal Sensory System for Wearable Applications and Boxing Training,” npj Flexible Electronics 7, no. 1 (2023): 37.

[34] P. Zhao, R. Zhang, Y. Tong, et al., “Strain-Discriminable Pressure/Proximity Sensing of Transparent Stretchable Electronic Skin Based on PEDOT: PSS/SWCNT Electrodes,” ACS Applied Materials & Interfaces 12, no. 49 (2020): 55083–55093.

[35] S. Li, J. Huang, M. Wang, et al., “Structural Electronic Skin for Conformal Tactile Sensing,” Advanced Science 10, no. 33 (2023): 2304106.

[36] D. Liu, P. Zhu, F. Zhang, et al., “Intrinsically Stretchable Polymer Semiconductor Based Electronic Skin for Multiple Perceptions of Force, Temperature, and Visible Light,” Nano Research 16, no. 1 (2023): 1196–1204.

[37] Y. Lee, X. Tian, J. Park, et al., “Rapidly Self-Healing Electronic Skin for Machine Learning–Assisted Physiological and Movement Evaluation,” Science Advances 11, no. 7 (2025): eads1301.

[38] H. Niu, H. Li, S. Gao, et al., “Perception-to-Cognition Tactile Sensing Based on Artificial-Intelligence-Motivated Human Full-Skin Bionic Electronic Skin,” Advanced Materials 34, no. 31 (2022): 2202622.

[39] J. Li, X. Yang, C. Xu, et al., “Bioinspired Electronic-Skin for Proximity and Pressure Detection in Robot Active Sensing,” Flexible and Printed Electronics 9, no. 2 (2024): 025011.

[40] J. Zhao, K. Dai, C. Liu, et al., “A Comparison Between Strain Sensing Behaviors of Carbon Black/Polypropylene and Carbon Nanotubes/Polypropylene Electrically Conductive Composites,” Composites, Part A: Applied Science and Manufacturing 48 (2013): 129–136.

[41] T. S. Natarajan, S. B. Eshwaran, K. W. Stöckelhuber, et al., “Strong Strain Sensing Performance of Natural Rubber Nanocomposites,” ACS Applied Materials & Interfaces 9, no. 5 (2017): 4860–4872.

[42] M. Li, L. Zong, W. Yang, et al., “Biological Nanofibrous Generator for Electricity Harvest From Moist Air Flow,” Advanced Functional Materials 29, no. 32 (2019): 1901798.

[43] L. Li, Z. Chen, M. Hao, et al., “Moisture-Driven Power Generation for Multifunctional Flexible Sensing Systems,” Nano Letters 19, no. 8 (2019): 5544–5552.

RIGHTS & PERMISSIONS

2026 The Author(s). SmartMat published by Tianjin University and John Wiley & Sons Australia, Ltd.