Sensory Fiber-Based Electronic Device as Intelligent and Natural User Interface

Abstract

A natural user interface (NUI) with ample information perception capability is a crucial element for the next-generation human–machine interaction and the development of the intelligent era. However, significant challenges remain to be solved in developing intelligent and natural interfaces with satisfactory smart sensing performance. Here, we report an NUI based on an intelligent fabric bracelet empowered with wide-range pressure detectability, enabling invisible and efficient human–machine interaction. The wide-range pressure-sensing ability of the fiber-based pressure sensor can be attributed to the coupling mechanism of contact resistance change and quantum tunneling effect. The fiber-based sensor array is then integrated with a miniaturized wireless flexible printed circuit board, forming an intelligent and compact fabric bracelet system for natural interactive applications in wireless smart home control and virtual reality. It is envisioned that the NUI based on the pressure-sensitive and intelligent fabric bracelet will significantly contribute to the development of next-generation NUIs for more diversified control and interactive applications.

Graphical Abstract

Keywords

Natural user interface / Human–machine interaction / Pressure sensor / Fiber sensor / Wearable electronics / Information and Computing Sciences / Artificial Intelligence and Image Processing / Information Systems / Engineering / Electrical and Electronic Engineering

References

[1] ChenM, LiuJ, LiP, GharaviH, HaoYX, OuyangJY, HuJY, HuL, HouC, HumarI, WeiL, YangGZ, TaoGM. Fabric computing: concepts, opportunities, and challenges. Innovation, 2022, 3100340

[2] ChenM, OuyangJY, JianAJ, LiuJ, LiP, HaoYX, GongYC, HuJY, ZhouJ, WangR, WangJX, HuL, WangYW, OuyangJ, ZhangJ, HouC, WeiL, ZhouHM, ZhangDY, TaoGM. Imperceptible, designable, and scalable braided electronic cord. Nat Commun, 2022, 13: 7097

[3] GouGY, LiXS, JianJM, TianH, WuF, RenJ, GengXS, XuJD, QiaoYC, YanZY, DunG, AhnCW, YangY, RenTL. Two-stage amplification of an ultrasensitive MXene-based intelligent artificial eardrum. Sci Adv, 2022, 8: eabn2156

[4] LuYJ, TianH, ChengJ, ZhuF, LiuB, WeiSS, JiLH, WangZL. Decoding lip language using triboelectric sensors with deep learning. Nat Commun, 2022, 13: 1401

[5] MahmoodM, MzurikwaoD, KimYS, LeeY, MishraS, HerbertR, DuarteA, AngCS, YeoWH. Fully portable and wireless universal brain-machine interfaces enabled by flexible scalp electronics and deep learning algorithm. Nat Mach Intell, 2019, 1: 412-422

[6] NiuHS, LiH, GaoS, LiY, WeiX, ChenYK, YueWJ, ZhouWJ, ShenGZ. Perception-to-cognition tactile sensing based on artificial-intelligence-motivated human full-skin bionic electronic skin. Adv Mater, 2022, 34: 2202622

[7] ShiY, YangP, LeiR, LiuZ, DongX, TaoX, ChuX, WangZL, ChenX. Eye tracking and eye expression decoding based on transparent, flexible and ultra-persistent electrostatic interface. Nat Commun, 2023, 14: 3315

[8] SunZD, ZhuML, ShanXC, LeeCK. Augmented tactile-perception and haptic-feedback rings as human–machine interfaces aiming for immersive interactions. Nat Commun, 2022, 13: 5224

[9] TangX, ShenH, ZhaoSY, LiN, LiuJ. Flexible brain-computer interfaces. Nat Electron, 2023, 6: 109-118

[10] XuJD, LiXS, ChangH, ZhaoBC, TanXC, YangY, TianH, ZhangS, RenTL. Electrooculography and tactile-perception collaborative interface for 3D human–machine interaction. ACS Nano, 2022, 16: 6687-6699

[11] TangZ, SunW, TaoC, PengT, LiH, ChenK, LiJ, ZhaoZ, LiZ, HongX. Rapid response, superior stable, and durable pressure sensor with rGO/CNC interdigital electrode. Nano Energy, 2024, 129110041

[12] WenF, ZhangZX, HeTY, LeeCK. AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove. Nat Commun, 2021, 12: 5378

[13] YanW, NoelG, LokeG, MeiklejohnE, KhudiyevT, MarionJ, RuiGC, LinJN, CherstonJ, SahasrabudheA, WilbertJ, WicaksonoI, HoytRW, MissakianA, ZhuL, MaC, JoannopoulosJ, FinkY. Single fibre enables acoustic fabrics via nanometre-scale vibrations. Nature, 2022, 603: 616-623

[14] ZhouZH, ChenK, LiXS, ZhangSL, WuYF, ZhouYH, MengKY, SunCC, HeQ, FanWJ, FanED, LinZW, TanXL, DengWL, YangJ, ChenJ. Sign-to-speech translation using machine-learning-assisted stretchable sensor arrays. Nat Electron, 2020, 3: 571-578

[15] ChenCY, ChenLJ, WuZY, GuoHY, YuWD, DuZQ, WangZL. 3D double-faced interlock fabric triboelectric nanogenerator for bio-motion energy harvesting and as self-powered stretching and 3D tactile sensors. Mater Today, 2020, 32: 84-93

[16] XuDW, OuyangZF, DongYJ, YuHY, ZhengS, LiSH, TamKC. Robust, breathable and flexible smart textiles as multifunctional sensor and heater for personal health management. Adv Fiber Mater, 2023, 5: 282-295

[17] LiJ, CaiJ, YuJ, LiZ, DingB. The rising of fiber constructed piezo/triboelectric nanogenerators: from material selections, fabrication techniques to emerging applications. Adv Funct Mater, 2023, 33: 2303249

[18] ZhuM, LiJ, YuJ, LiZ, DingB. Superstable and intrinsically self-healing fibrous membrane with bionic confined protective structure for breathable electronic skin. Angew Chem Int Ed, 2022, 61e202200226

[19] DongK, PengX, ChengRW, NingCA, JiangY, ZhangYH, WangZL. Advances in high-performance autonomous energy and self-powered sensing textiles with novel 3D fabric structures. Adv Mater, 2022, 34: 2109355

[20] WangS, WangX, WangQ, MaS, XiaoJ, LiuH, PanJ, ZhangZ, ZhangL. Flexible optoelectronic multimodal proximity/pressure/temperature sensors with low signal interference. Adv Mater, 2023, 35: 2304701

[21] LibanoriA, ChenGR, ZhaoX, ZhouYH, ChenJ. Smart textiles for personalized healthcare. Nat Electron, 2022, 5: 142-156

[22] GalliV, SailapuSK, CuthbertTJ, AhmadizadehC, HanniganBC, MenonC. Passive and wireless all-textile wearable sensor system. Adv Sci, 2023, 10: 2206665

[23] ZengKW, ShiX, TangCQ, LiuT, PengHS. Design, fabrication and assembly considerations for electronic systems made of fibre devices. Nat Rev Mater, 2023, 8: 552-561

[24] LvX, LiuY, YuJ, LiZ, DingB. Smart fibers for self-powered electronic skins. Adv Fiber Mater, 2023, 5: 401-428

[25] LiG, SunF, ZhaoS, XuR, WangH, QuL, TianM. Autonomous electroluminescent textile for visual interaction and environmental warning. Nano Lett, 2023, 23: 8436-8444

[26] KhanAQ, YuKQ, LiJT, LengXQ, WangML, ZhangXS, AnBG, FeiB, WeiW, ZhuangHC, ShafiqM, BaoLL, LiuZF, ZhouX. Spider silk supercontraction-inspired cotton-hydrogel self-adapting textiles. Adv Fiber Mater, 2022, 4: 1572-1583

[27] YangYQ, SunN, WenZ, ChengP, ZhengHC, ShaoHY, XiaYJ, ChenC, LanHW, XieXK, ZhouCJ, ZhongJ, SunXH, LeeST. Liquid-metal-based super-stretchable and structure-designable triboelectric nanogenerator for wearable electronics. ACS Nano, 2018, 12: 2027-2034

[28] YangYJ, XuBG, GaoYY, LiMQ. Conductive composite fiber with customizable functionalities for energy harvesting and electronic textiles. ACS Appl Mater Interfaces, 2021, 13: 49927-49935

[29] MaLY, ZhouMJ, WuRH, PatilA, GongH, ZhuSH, WangTT, ZhangYF, ShenS, DongK, YangLK, WangJ, GuoWX, WangZL. Continuous and scalable manufacture of hybridized nano-micro triboelectric yarns for energy harvesting and signal sensing. ACS Nano, 2020, 14: 4716-4726

[30] MarionJS, GuptaN, CheungH, MonirK, AnikeevaP, FinkY. Thermally drawn highly conductive fibers with controlled elasticity. Adv Mater, 2022, 34: 2201081

[31] WuYP, DaiXY, SunZH, ZhuSX, XiongL, LiangQH, WongMC, HuangLBA, QinQ, HaoJH. Highly integrated, scalable manufacturing and stretchable conductive core/shell fibers for strain sensing and self-powered smart textiles. Nano Energy, 2022, 98107240

[32] ZhuM, YuJ, LiZ, DingB. Self-healing fibrous membranes. Angew Chem Int Ed, 2022, 61e202208949

[33] LiM, LiZ, QuL, ChenF, TianM. Recent progress of the active materials with various micro-structures for flexible textile-based supercapacitors. Adv Fiber Mater, 2022, 4: 1005-1026

[34] MengKY, ZhaoSL, ZhouYH, WuYF, ZhangSL, HeQ, WangX, ZhouZH, FanWJ, TanXL, YangJ, ChenJ. A wireless textile-based sensor system for self-powered personalized health care. Matter, 2020, 2: 896-907

[35] ChenLM, LuMY, YangHS, AvilaJRS, ShiBW, RenL, WeiGW, LiuXQ, YinWL. Textile-based capacitive sensor for physical rehabilitation via surface topological modification. ACS Nano, 2020, 14: 8191-8201

[36] YangQS, LiuN, YinJJ, TianH, YangY, RenTL. Understanding the origin of tensile response in a graphene textile strain sensor with negative differential resistance. ACS Nano, 2022, 16: 14230-14238

[37] ChorsiMT, LeTT, LinF, VinikoorT, DasR, StevensJF, MundraneC, ParkJ, TranKTM, LiuY, PfundJ, ThompsonR, HeW, JainM, Morales-AcostaMD, BilalOR, KazerounianK, IliesH, NguyenTD. Highly piezoelectric, biodegradable, and flexible amino acid nanofibers for medical applications. Sci Adv, 2023, 9: eadg6075

[38] ChenJM, ZhangJ, HuJL, LuoNQ, SunFX, VenkatesanH, ZhaoN, ZhangYT. Ultrafast-response/recovery flexible piezoresistive sensors with DNA-Like double helix yarns for epidermal pulse monitoring. Adv Mater, 2022, 34: 2104313

[39] XuR, XuT, SheM, JiX, LiG, ZhangS, ZhangX, LiuH, SunB, ShenG. Skin-friendly large matrix iontronic sensing meta-fabric for spasticity visualization and rehabilitation training via piezo-ionic dynamics. Nano-Micro Lett, 2025, 17: 90

[40] CaiJ, DuM, LiZ. Flexible temperature sensors constructed with fiber materials. Adv Mater Technol, 2022, 7: 2101182

[41] LuoYY, LiYZ, SharmaP, ShouW, WuK, FosheyM, LiBC, PalaciosT, TorralbaA, MatusikW. Learning human-environment interactions using conformal tactile textiles. Nat Electron, 2021, 4: 193-201

[42] LiLL, ZhaoSF, RanWH, LiZX, YanYX, ZhongBW, LouZ, WangLL, ShenGZ. Dual sensing signal decoupling based on tellurium anisotropy for VR interaction and neuro-reflex system application. Nat Commun, 2022, 13: 5975

[43] DongK, PengX, AnJ, WangAC, LuoJJ, SunBZ, WangJ, WangZL. Shape adaptable and highly resilient 3D braided triboelectric nanogenerators as e-textiles for power and sensing. Nat Commun, 2020, 11: 2868

[44] GuoYJ, YinFF, LiY, ShenGZ, LeeJC. Incorporating wireless strategies to wearable devices enabled by a photocurable hydrogel for monitoring pressure information. Adv Mater, 2023, 35: 2300855

[45] ChenR, HeZX, LuoX, WangM, LiuZD, WuYY, DongXM, ZhangDF, LiuJQ. Bio-inspired ion-conducting foam elastomer for human motion monitoring. ACS Appl Polym Mater, 2023, 5: 391-399

[46] ShiL, LiZ, ChenM, QinYJ, JiangYZ, WuLM. Quantum effect-based flexible and transparent pressure sensors with ultrahigh sensitivity and sensing density. Nat Commun, 2020, 11: 3529

[47] WangXM, TaoLQ, YuanM, WangZP, YuJB, XieDL, LuoF, ChenXP, WongCP. Sea urchin-like microstructure pressure sensors with an ultra-broad range and high sensitivity. Nat Commun, 2021, 12: 1776

[48] LiuR, LiJ, LiM, ZhangQ, ShiG, LiY, HouC, WangH. MXene-coated air-permeable pressure-sensing fabric for smart wear. ACS Appl Mater Interfaces, 2020, 12: 46446-46454

[49] LanL, ZhaoF, YaoY, PingJ, YingY. One-step and spontaneous in situ growth of popcorn-like nanostructures on stretchable double-twisted fiber for ultrasensitive textile pressure sensor. ACS Appl Mater Interfaces, 2020, 12: 10689-10696

[50] WuR, MaL, HouC, MengZ, GuoW, YuW, YuR, HuF, LiuXY. Silk composite electronic textile sensor for high space precision 2D combo temperature-pressure sensing. Small, 2019, 15: 1901558

[51] DingH, WuZ, WangH, ZhouZ, WeiY, TaoK, XieX, WuJ. An ultrastretchable, high-performance, and crosstalk-free proximity and pressure bimodal sensor based on ionic hydrogel fibers for human–machine interfaces. Mater Horiz, 2022, 9: 1935-1946

Funding

the National Key R&D Program of China(2022YFB3603403)

National Natural Science Foundation of China(62301150)

the Nanjing Science and Technology Innovation Project for Returned Overseas Talent(4206002302)

the Postgraduate Research&Practice Innovation Program of Jiangsu Province(SJCX23_0042)

RIGHTS & PERMISSIONS

Donghua University, Shanghai, China