AIoT-enhanced health management system using soft and stretchable triboelectric sensors for human behavior monitoring

Abstract

Sedentary, inadequate sleep and exercise can affect human health. Artificial intelligence (AI) and Internet of Things (IoT) create the Artificial Intelligence of Things (AIoT), providing the possibility to solve these problems. This paper presents a novel approach to monitor various human behaviors for AIoT-based health management using triboelectric nanogenerator (TENG) sensors. The insole with solely one TENG sensor, creating a most simplified system that utilizes machine learning (ML) for personalized motion monitoring, encompassing identity recognition and gait classification. A cushion with 12 TENG sensors achieves real-time identity and sitting posture recognition with accuracy rates of 98.86% and 98.40%, respectively, effectively correcting sedentary behavior. Similarly, a smart pillow, equipped with 15 sensory channels, detects head movements during sleep, identifying 8 sleep patterns with 96.25% accuracy. Ultimately, constructing an AIoT-based health management system to analyze these data, displaying health status through human-machine interfaces, offers the potential to help individuals maintain good health.

Keywords

AIoT / flexible sensor / health management / machine learning / triboelectric nanogenerator

References

[1] Yoshizaki T, Ishihara J, Kotemori A, et al. Association between irregular daily routine and risk of incident stroke and coronary heart disease in a large Japanese population. Sci Rep. 2022;12(1):15750.

[2] Owen N. Sedentary behavior: understanding and influencing adults' prolonged sitting time. Prev Med. 2012;55(6):535-539.

[3] Xue Y, Huang Z, Liu G, et al. Associations of environment and lifestyle factors with suboptimal health status: a population-based cross-sectional study in urban China. Global Health. 2021;17(1):86.

[4] Kataoka R, Vasenina E, Hammert WB, Ibrahim AH, Dankel SJ, Buckner SL. Is there evidence for the suggestion that fatigue accumulates following resistance exercise? Sports Med. 2022;52(1):25-36.

[5] Arippa F, Nguyen A, Pau M, Harris-Adamson C. Postural strategies among office workers during a prolonged sitting bout. Appl Ergon. 2022;102:103723.

[6] Whitehurst LN, Subramoniam A, Krystal A, Prather AA. Links between the brain and body during sleep: implications for memory processing. Trends Neurosci. 2022;45(3):212-223.

[7] Yum HY, Han SA, Konstantinov K, Kim SW, Kim JH. Smart triboelectric nanogenerators toward human-oriented technologies: health monitoring, wound healing. Drug Deliv Adv Mater Technol. 2023;8(10):2201500.

[8] Ran X, Luo F, Lin Z, Zhu Z, Liu C, Chen B. Blood pressure monitoring via double sandwich-structured triboelectric sensors and deep learning models. Nano Res. 2022;15(6):5500-5509.

[9] Lin Q, Li T, Shakeel PM, Samuel RDJ. Advanced artificial intelligence in heart rate and blood pressure monitoring for stress management. J Ambient Intell Human Comput. 2020;12(3):3329-3340.

[10] Karasneh RA, Al-Azzam SI, Alzoubi KH, Hawamdeh S, Jarab AS, Nusair MB. Smartphone applications for sleep tracking: rating and perceptions about behavioral change among users. Sleep Sci. 2022;15(Spec 1):65-73.

[11] Ran X, Wang C, Xiao Y, Gao X, Zhu Z, Chen B. A portable sitting posture monitoring system based on a pressure sensor array and machine learning. Sens Actuat A: Phys. 2021;331:112900.

[12] Feng Zhang CW, Wang B, Min W, Bugos D, Zhang H, Liu KJR. SMARS: sleep monitoring via ambient radio signals. IEEE Trans Mobile Comput. 2021;20(1):217-231.

[13] Zhang Q, Jin T, Cai J, et al. Wearable triboelectric sensors enabled gait analysis and waist motion capture for IoT-based smart healthcare applications. Adv Sci (Weinh). 2022;9(4):e2103694.

[14] Hu C, Wang F, Cui X, Zhu Y. Recent progress in textile-based triboelectric force sensors for wearable electronics. Adv Compos Hybrid Mater. 2023;6(2):70.

[15] Le X, Shi Q, Sun Z, Xie J, Lee C. Noncontact human-machine Interface using complementary information fusion based on MEMS and triboelectric sensors. Adv Sci (Weinh). 2022;9(21):e2201056.

[16] Tao K, Chen Z, Yu J, et al. Ultra-sensitive, deformable, and transparent triboelectric tactile sensor based on micro-pyramid patterned ionic hydrogel for interactive human-machine interfaces. Adv Sci (Weinh). 2022;9(10):e2104168.

[17] Namiki A, Yokosawa S. Origami folding by multifingered hands with motion primitives. Cyborg Bionic Syst. 2021;2021:9851834.

[18] Tan P, Han X, Zou Y, et al. Self-powered gesture recognition wristband enabled by machine learning for full keyboard and multicommand input. Adv Mater. 2022;34(21):e2200793.

[19] Di Liu YG, Zhou L, Wang J, Wang ZL. Recent advances in high-performance triboelectric nanogenerators. Nano Res. 2023;16(9):1-20.

[20] Walden R, Aazem I, Babu A, Pillai SC. Textile-triboelectric nanogenerators (T-TENGs) for wearable energy harvesting devices. Chem Eng J. 2023;451(Part 3):138741.

[21] Safaei M, Sodano HA, Anton SR. A review of energy harvesting using piezoelectric materials: state-of-the-art a decade later (2008–2018). Smart Mater Struct. 2019;28(11):113001.

[22] Xi Y, Cheng S, Chao S, et al. Piezoelectric wearable atrial fibrillation prediction wristband enabled by machine learning and hydrogel affinity. Nano Res. 2023;16(9):11674-11681.

[23] Costa P, Nunes-Pereira J, Pereira N, Castro N, Gonçalves S, Lanceros-Mendez S. Recent progress on piezoelectric, pyroelectric, and magnetoelectric polymer-based energy-harvesting devices. Energy Technology. 2019;7(7):1800852.

[24] Zou Y, Raveendran V, Chen J. Wearable triboelectric nanogenerators for biomechanical energy harvesting. Nano Energy. 2020;77:105303.

[25] Vidal JV, Slabov V, Kholkin AL, Dos Santos MPS. Hybrid triboelectric-electromagnetic nanogenerators for mechanical energy harvesting: a review. Nanomicro Lett. 2021;13(1):199.

[26] Liu L, Shi Q, Guo X, Zhang Z, Lee C. A facile frequency tuning strategy to realize vibration-based hybridized piezoelectric-triboelectric nanogenerators. EcoMat. 2022;5(1):e12279.

[27] Li Q, Li Z, Meng J. Pyro-phototronic effect enhanced self-powered photodetector. Int J Optomechatron. 2022;16(1):1-17.

[28] Wen H, Yang X, Huang R, et al. Universal energy solution for triboelectric sensors toward the 5G era and internet of things. Adv Sci (Weinh). 2023;10(22):e2302009.

[29] Fan F-R, Tian Z-Q, Lin WZ. Flexible triboelectric generator. Nano Energy. 2012;1(2):328-334.

[30] Zhang Q, Xin C, Shen F, et al. Human body IoT systems based on triboelectrification effect: energy harvesting, sensing, interfacing and communication. Energy Environ Sci. 2022;15(9):3688-3721.

[31] Yi Q, Pei X, Das P, Qin H, Lee SW, Esfandyarpour R. A self-powered triboelectric MXene-based 3D-printed wearable physiological biosignal sensing system for on-demand, wireless, and real-time health monitoring. Nano Energy. 2022;101:107511.

[32] Shi X, Luo J, Luo J, et al. Flexible wood-based triboelectric self-powered smart home system. ACS Nano. 2022;16(2):3341-3350.

[33] Li R, Peng B. Implementing monocular visual-tactile sensors for robust manipulation. Cyborg Bionic Syst. 2022;2022:9797562.

[34] Jin T, Sun Z, Li L, et al. Triboelectric nanogenerator sensors for soft robotics aiming at digital twin applications. Nat Commun. 2020;11(1):5381.

[35] Wen F, Zhang Z, He T, Lee C. AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove. Nat Commun. 2021;12(1):5378.

[36] Wu Y, Li Y, Zou Y, et al. A multi-mode triboelectric nanogenerator for energy harvesting and biomedical monitoring. Nano Energy. 2022;92:106715.

[37] Ning C, Cheng R, Jiang Y, et al. Helical fiber strain sensors based on triboelectric nanogenerators for self-powered human respiratory monitoring. ACS Nano. 2022;16(2):2811-2821.

[38] Li Y, Zhang Y, Yi J, et al. Large-scale fabrication of core-shell triboelectric braided fibers and power textiles for energy harvesting and plantar pressure monitoring. EcoMat. 2022;4(4):e12191.

[39] Chen T, Dai Z, Liu M, et al. 3D multimodal sensing and feedback finger case for immersive dual-way interaction. Adv Mater Technol. 2024;9(5):2301681.

[40] Liang Q, Zhang D, He T, et al. Fiber-based noncontact sensor with Stretchability for underwater wearable sensing and VR applications. ACS Nano. 2024;18(1):600-611.

[41] Shi Q, Zhang Z, Yang Y, Shan X, Salam B, Lee C. Artificial intelligence of things (AIoT) enabled floor monitoring system for smart home applications. ACS Nano. 2021;15(11):18312-18316.

[42] Zhang Z, He T, Zhu M, et al. Deep learning-enabled triboelectric smart socks for IoT-based gait analysis and VR applications. Npj Flex Electron. 2020;4(1):29.

[43] Zou Y, Sun M, Yan F, et al. A high-performance flag-type triboelectric nanogenerator for scavenging wind energy toward self-powered IoTs. Materials (Basel). 2022;15(10):3696.

[44] Qu X, Liu Z, Tan P, et al. Artificial tactile perception smart finger for material identification based on triboelectric sensing. Sci Adv. 2022;8(31):eabq2521.

[45] Yang Y, He T, Ravindran P, et al. All-organic transparent plant e- skin for noninvasive phenotyping. Sci Adv. 2024;10(7):eadk7488.

[46] Zhao T, Fu Y, Sun C, et al. Wearable biosensors for real-time sweat analysis and body motion capture based on stretchable fiber-based triboelectric nanogenerators. Biosens Bioelectron. 2022;205:114115.

[47] Wei C, Cheng R, Ning C, et al. A self-powered body motion sensing network integrated with multiple triboelectric fabrics for biometric gait recognition and auxiliary rehabilitation training. Adv Funct Mater. 2023;33(35):2303562.

[48] Yuan J, Zhang Y, Wei C, Zhu R. A fully self-powered wearable leg movement sensing system for human health monitoring. Adv Sci (Weinh). 2023;10(29):e2303114.

[49] Guo ZH, Zhang Z, An K, et al. A wearable multidimensional motion sensor for AI-enhanced VR sports. Research (Wash DC). 2023;6:154.

[50] Zhu J, Ji S, Yu J, et al. Machine learning-augmented wearable triboelectric human-machine interface in motion identification and virtual reality. Nano Energy. 2022;103(Part A):107766.

[51] Sun Z, Zhu M, Shan X, Lee C. Augmented tactile-perception and haptic-feedback rings as human-machine interfaces aiming for immersive interactions. Nat Commun. 2022;13(1):5224.

[52] Liu Y, Yiu C, Jia H, et al. Thin, soft, garment-integrated triboelectric nanogenerators for energy harvesting and human machine interfaces. EcoMat. 2021;3(4):e12123.

[53] Tao LQ, Tian H, Liu Y, et al. An intelligent artificial throat with sound-sensing ability based on laser induced graphene. Nat Commun. 2017;8:14579.

[54] An S, Pu X, Zhou S, et al. Deep learning enabled neck motion detection using a triboelectric nanogenerator. ACS Nano. 2022;16(6):9359-9367.

[55] Sun Z, Zhu M, Zhang Z, et al. Artificial intelligence of things (AIoT) enabled virtual shop applications using self-powered sensor enhanced soft robotic manipulator. Adv Sci (Weinh). 2021;8(14):e2100230.

[56] Wang T, Jin T, Lin W, et al. Multimodal sensors enabled autonomous soft robotic system with self-adaptive manipulation. ACS Nano. 2024;18(14):9980-9996.

[57] Zhang Z, Du K, Chen X, Xue C, Wang K. An air-cushion triboelectric nanogenerator integrated with stretchable electrode for human-motion energy harvesting and monitoring. Nano Energy. 2018;53:108-115.

[58] Han Y, Yi F, Jiang C, et al. Self-powered gait pattern-based identity recognition by a soft and stretchable triboelectric band. Nano Energy. 2019;56:516-523.

[59] Zhu Y, Sun F, Jia C, et al. A 3D printing triboelectric sensor for gait analysis and virtual control based on human–computer interaction and the internet of things. Sustainability. 2022;14(17):10875.

[60] Sun P, Cai N, Zhong X, Zhao X, Zhang L, Jiang S. Facile monitoring for human motion on fireground by using MiEs-TENG sensor. Nano Energy. 2021;89(Part B):106492.

[61] Zhang W, Zhang Y, Yang G, et al. Wearable and self-powered sensors made by triboelectric nanogenerators assembled from antibacterial bromobutyl rubber. Nano Energy. 2021;82:105769.

[62] Wang Z, Bu M, Xiu K, et al. A flexible, stretchable and triboelectric smart sensor based on graphene oxide and polyacrylamide hydrogel for high precision gait recognition in parkinsonian and hemiplegic patients. Nano Energy. 2022;104(Part B):107978.

[63] Sarkar PK, Kamilya T, Acharya S. Introduction of triboelectric positive bioplastic for powering portable electronics and self-powered gait sensor. ACS Appl Energy Mater. 2019;2(8):5507-5514.

[64] Lin Z, Wu Z, Zhang B, et al. A triboelectric nanogenerator-based smart insole for multifunctional gait monitoring. Adv Mater Technol. 2019;4(2):1800360.

[65] Yang Y, Hou X, Geng W, et al. Human movement monitoring and behavior recognition for intelligent sports using customizable and flexible triboelectric nanogenerator. Sci China Technol Sci. 2022;65(4):826-836.

[66] Zhu M, Shi Q, He T, et al. Self-powered and self-functional cotton sock using piezoelectric and triboelectric hybrid mechanism for healthcare and sports monitoring. ACS Nano. 2019;13(2):1940-1952.

[67] Yeh C, Kao F-C, Wei P-H, et al. Bioinspired shark skin-based liquid metal triboelectric nanogenerator for self-powered gait analysis and long-term rehabilitation monitoring. Nano Energy. 2022;104(Part A):107852.

[68] Jao Y-T, Yang P-K, Chiu C-M, et al. A textile-based triboelectric nanogenerator with humidity-resistant output characteristic and its applications in self-powered healthcare sensors. Nano Energy. 2018;50:513-520.

[69] Li J, Xie Z, Wang Z, et al. A triboelectric gait sensor system for human activity recognition and user identification. Nano Energy. 2023;112:108473.

[70] Mao Y, Zhu Y, Zhao T, Jia C, Wang X, Wang Q. Portable Mobile gait monitor system based on triboelectric nanogenerator for monitoring gait and powering electronics. Energies. 2021;14(16):4996.

[71] Huang H, Dong Y, Wan S, et al. A transient dual-type sensor based on MXene/cellulose nanofibers composite for intelligent sedentary and sitting postures monitoring. Carbon. 2022;200:327-336.

[72] Lei H, Xiao J, Chen Y, et al. Bamboo-inspired self-powered triboelectric sensor for touch sensing and sitting posture monitoring. Nano Energy. 2022;91:106670.

[73] Ma X, Qi Y, Niu Y, et al. Highly sensitive, ultra-reliable flexible piezoelectret sensor for non-contact sitting motion tracking and physiological signal monitoring. Nano Energy. 2023;111:108424.

[74] Hwang HJ, Lee Y, Lee C, et al. Mesoporous highly-deformable composite polymer for a gapless triboelectric nanogenerator via a one-step metal oxidation process. Micromachines (Basel). 2018;9(12):656.

[75] Jiang Y, An J, Liang F, et al. Knitted self-powered sensing textiles for machine learning-assisted sitting posture monitoring and correction. Nano Res. 2022;15(9):8389-8397.

[76] Wang X, Tao Y, Pan S, et al. Biocompatible and breathable healthcare electronics with sensing performances and photothermal antibacterial effect for motion-detecting. Npj Flex Electron. 2022;6(1):95.

[77] Gunasekhar R, Anand PA. Polyvinylidene fluoride/aromatic hyperbranched polyester 2nd generation based triboelectric sensor for polysomnographic and health monitoring applications. Sens Actuat A: Phys. 2023;355:114311.

[78] Gajula P, Muhammad FM, Reza MS, Jaisankar SN, Kim KJ, Kim H. Fabrication of a silicon elastomer-based self-powered flexible triboelectric sensor for wearable energy harvesting and biomedical applications. ACS Appl Electron Mater. 2023;5(3):1750-1760.

[79] Kou H, Wang H, Cheng R, et al. Smart pillow based on flexible and breathable triboelectric nanogenerator arrays for head movement monitoring during sleep. ACS Appl Mater Interfaces. 2022;14(20):23998-24007.

[80] Lin Z, Yang J, Li X, et al. Large-scale and washable smart textiles based on triboelectric nanogenerator arrays for self-powered sleeping monitoring. Adv Funct Mater. 2018;28(1):1704112.

[81] Fahad MM, Reza MS, Prasad G, Jaisankar SN, Kim KJ, Kim H. Polysomnographic observation using triboelectric pressure sensor composed of polymer-pairs having coarse surface. Fibers Polym. 2022;23(6):1490-1499.

[82] Zhou Z, Padgett S, Cai Z, et al. Single-layered ultra-soft washable smart textiles for all-around ballistocardiograph, respiration, and posture monitoring during sleep. Biosens Bioelectron. 2020;155:112064.

[83] Song W, Gan B, Jiang T, et al. Nanopillar arrayed triboelectric nanogenerator as a self-powered sensitive sensor for a sleep monitoring system. ACS Nano. 2016;10(8):8097-8103.

[84] Cao R, Zhao S, Li C. Free deformable nanofibers enhanced tribo-sensors for sleep and tremor monitoring. ACS Appl Electron Mater. 2019;1(11):2301-2307.

[85] Ding X, Cao H, Zhang X, Li M, Liu Y. Large scale triboelectric nanogenerator and self-powered flexible sensor for human sleep monitoring. Sensors (Basel). 2018;18(6):1713.

[86] Yun J, Park J, Jeong S, Hong D, Kim D. A mask-shaped respiration sensor using triboelectricity and a machine learning approach toward smart sleep monitoring systems. Polymers (Basel). 2022;14(17):3549.

[87] Yang Y, Sun N, Wen Z, et al. Liquid-metal-based super-stretchable and structure-designable triboelectric nanogenerator for wearable electronics. ACS Nano. 2018;12(2):2027-2034.

[88] Jiang D, Shi B, Ouyang H, Fan Y, Wang ZL, Li Z. Emerging implantable energy harvesters and self-powered implantable medical electronics. ACS Nano. 2020;14(6):6436-6448.

[89] Shi B, Li Z, Fan Y. Implantable energy-harvesting devices. Adv Mater. 2018;30(44):e1801511.

[90] Wang X, Feng Z, Zhang G, et al. Flexible sensors Array based on frosted microstructured Ecoflex film and TPU nanofibers for epidermal pulse wave monitoring. Sensors (Basel). 2023;23(7):3717.

[91] Feng L, Li Z, Liu C. Are you sitting right?-Sitting posture recognition using RF signals. IEEE Pacific Rim Conference on Communications, Computers and Signal Processing (PACRIM), 1–6. 2019.

[92] Tlili F, Haddad R, Bouallegue R, Shubair R. Design and architecture of smart belt for real time posture monitoring. Internet of Things. 2022;17:100472.

[93] Sun Z, Zhang Z, Lee C. A skin-like multimodal haptic interface. Nature Electron. 2023;6(12):941-942.

[94] Lee J, Hong M, Ryu S. Sleep monitoring system using Kinect sensor. Int J Distrib Sensor Netw. 2015;11(10):875371.

RIGHTS & PERMISSIONS

2024 The Authors. EcoMat published by The Hong Kong Polytechnic University and John Wiley & Sons Australia, Ltd.