Stretchable spring-sheathed yarn sensor for 3D dynamic body reconstruction assisted by transfer learning

Abstract

A wearable sensing system that can reconstruct dynamic 3D human body models for virtual cloth fitting is highly important in the era of information and metaverse. However, few research has been conducted regarding conformal sensors for accurately measuring the human body circumferences for dynamic 3D human body reshaping. Here, we develop a stretchable spring-sheathed yarn sensor (SSYS) as a smart ruler, for precisely measuring the circumference of human bodies and long-term tracking the movement for the dynamic 3D body reconstruction. The SSYS has a robust property, high resilience, high stability (>18 000), and ultrafast response (12 ms) to external deformation. It is also washable, wearable, tailorable, and durable for long-time wearing. Moreover, geometric, and mechanical behaviors of the SSYS are systematically investigated both theoretically and experimentally. In addition, a transfer learning algorithm that bridges the discrepancy of real and virtual sensing performance is developed, enabling a small body circumference measurement error of 1.79%, noticeably lower than that of traditional learning algorithm. Furtherly, 3D human bodies that are numerically consistent with the actual bodies are reconstructed. The 3D dynamic human body reconstruction based on the wearing sensing system and transfer learning algorithm enables excellent virtual fitting and shirt customization in a smart and highly efficient manner. This wearable sensing technology shows great potential in human-computer interaction, intelligent fitting, specialized protection, sports activities, and human physiological health tracking.

Keywords

3D body reconstruction / remote personalized clothes customization / transfer learning / wearable sensing system / yarn sensor array

References

[1] Tao X, Bruniaux P. Toward advanced three-dimensional modeling of garment prototype from draping technique. Int J Clothing Sci Technol. 2013;25(4):266-283.

[2] Liu K, Zhu C, Tao X, Bruniaux P, Zeng X. Parametric design of garment pattern based on body dimensions. Int J Ind Ergon. 2019;72:212-221.

[3] Yunchu Y, Weiyuan Z. Prototype garment pattern flattening based on individual 3D virtual dummy. Int J Clothing Sci Technol. 2007;19(5):334-348.

[4] Lorussi F, Rocchia W, Scilingo EP, Tognetti A, De Rossi D. Wearable, redundant fabric-based sensor arrays for reconstruction of body segment posture. IEEE Sens J. 2004;4(6):807-818.

[5] Liu Z, He Q, Zou F, Ding Y, Xu B. Apparel ease distribution analysis using three-dimensional motion capture. Text Res J. 2019;89(19-20):4323-4335.

[6] (a) Zeng Y, Fu J, Chao H. 3D human body reshaping with anthropometric modeling. International Conference on Internet Multimedia Computing and Service. Springer Singapore, Singapore. 2017;819:96-107. (b) Thomassey S, Bruniaux P. A template of ease allowance for garments based on a 3D reverse methodology. Int J Ind Ergon. 2013;43(5):406-416.

[7] (a) Wang C, Li X, Gao E, et al. Carbonized silk fabric for ultrastretchable, highly sensitive, and wearable strain sensors. Adv Mater. 2016;28(31):6640. (b) Liu ZF, Fang S, Moura FA, et al. Hierarchically buckled sheath-core fibers for superelastic electronics, sensors, and muscles. Science. 2015;349(6246):400. (c) Hou C, Xu Z, Qiu W, et al. A biodegradable and stretchable protein-based sensor as artificial electronic skin for human motion detection. Small. 2019;15(11):1805084. (d) Wang Q, Jian MQ, Wang CY, Zhang YY. Carbonized silk nanofiber membrane for transparent and sensitive electronic skin. Adv Funct Mater. 2017;27(9):1605657. (e) Lu Y, Yue Y, Ding Q, et al. Environment-tolerant ionic hydrogel-elastomer hybrids with robust interfaces, high transparence, and biocompatibility for a mechanical-thermal multimode sensor. InfoMat. 2023;5(4):e12409. (f) Xiong P, Mengmeng L, Linxuan L, et al. Wearable textile-based in-plane microsupercapacitors. Adv Energy Mater. 2016;6(24):1601254. (g) Wang K, Xu W, Zhang W, et al. Bioinspired water-driven electricity generators: from fundamental mechanisms to practical applications. Nano Res Energy. 2023;2(1):e9120042.

[8] (a) Hammock ML, Chortos A, Tee BC-K, et al. The evolution of electronic skin (e-skin): a brief history, design considerations, and recent progress. Adv Mater. 2013;25(42):5997. (b) Wang M, Yan Z, Wang T, et al. Gesture recognition using a bioinspired learning architecture that integrates visual data with somatosensory data from stretchable sensors. Nature Electron. 2020;3(9):563. (c) Zhou ZH, Chen K, Li XS, et al. Sign-to-speech translation using machine-learning-assisted stretchable sensor arrays. Nature Electron. 2020;3(9):571. (d) Wu R, Seo S, Ma L, Bae J, Kim T. Full-fiber auxetic-interlaced yarn sensor for sign-language translation glove assisted by artificial neural network. Nano-Micro Lett. 2022;14(1):139. (e) Hou C, Tai G, Liu Y, Wu Z, Liang X, Liu X. Borophene-based materials for energy, sensors and information storage applications. Nano Res Energy. 2023;2(2):e9120051. (f) Xu F, Jin X, Lan C, et al. 3D arch-structured and machine-knitted triboelectric fabrics as self-powered strain sensors of smart textiles. Nano Energy. 2023;109:108312. (g) Sheng F, Zhang B, Cheng R, et al. Wearable energy harvesting-storage hybrid textiles as on-body self-charging power systems. Nano Res Energy. 2023;2(4):e9120079. (h) Zhang R, Lin J, He T, et al. High-performance piezoresistive sensors based on transfer-free large-area PdSe₂ films for human motion and health care monitoring. InfoMat. 2023;6:e12484.

[9] (a) Amjadi M, Kyung K-U, Park I, Sitti M. Stretchable, skin-mountable, and wearable strain sensors and their potential applications: a review. Adv Funct Mater. 2016;26(11):1678. (b) Cheng Y, Wang RR, Sun J, Gao L. A stretchable and highly sensitive graphene-based fiber for sensing tensile strain, bending, and torsion. Adv Mater. 2015;27(45):7365. (c) Wu R, Ma L, Patil A, et al. All-textile electronic skin enabled by highly elastic spacer fabric and conductive fibers. ACS Appl Mater Inter-faces. 2019;11(36):33336. (d) Zhou Q, Pan J, Deng S, Xia F, Kim T. Triboelectric nanogenerator-based sensor systems for chemical or biological detection. Adv Mater. 2021;33(35):2008276. (e) Feng ZP, Hao YN, Qin J, et al. Ultrasmall barium titanate nanoparticles modulated stretchable dielectric elastomer sensors with large deformability and high sensitivity. InfoMat. 2023;5:e12413.

[10] (a) Cao X, Ye C, Cao L, Shan Y, Ren J, Ling S. Biomimetic spun silk ionotronic fibers for intelligent discrimination of motions and tactile stimuli. Adv Mater. 2023;35(36):2300447. (b) Ye C, Dong S, Ren J, Ling S. Ultrastable and high-performance silk energy harvesting textiles. Nano-Micro Lett. 2020;12:1.

[11] Wu R, Liu S, Lin Z, Zhu S, Ma L, Wang ZL. Industrial fabrication of 3D braided stretchable hierarchical interlocked fancy-yarn triboelectric nanogenerator for self-powered smart fitness system. Adv Energy Mater. 2022;12(31):2201288.

[12] (a) Pan J, Hao B, Song W, et al. Highly sensitive and durable wearable strain sensors from a core-sheath nanocomposite yarn. Compos B: Eng. 2020;183:107683. (b) Alam MM, Crispin X. The past, present, and future of piezoelectric fluoropolymers: towards efficient and robust wearable nanogenerators. Nano Res Energy. 2023;2(4):107683.

[13] (a) Zhang MC, Wang CY, Wang Q, Jian MQ, Zhang YY. Sheath-core graphite/silk fiber made by dry-meyer-rod-coating for wearable strain sensors. ACS Appl Mater Inter. 2016;8(32):20894. (b) Ma L, Wu R, Liu S, et al. A machine-fabricated 3D honeycomb-structured flame-retardant triboelectric fabric for fire escape and rescue. Adv Mater. 2020;32(28):2003897.

[14] Wu R, Ma L, Liu S, et al. Fibrous inductance strain sensors for passive inductance textile sensing. Mater Today Phys. 2020;15:100243.

[15] (a) Zhang Z, He T, Zhu M, et al. Deep learning-enabled triboelectric smart socks for IoT-based gait analysis and VR applications. npj Flex Electron. 2020;4(1):1. (b) Zhu M, Sun Z, Zhang Z, et al. Haptic-feedback smart glove as a creative human-machine interface (HMI) for virtual/augmented reality applications. Sci Adv. 2020;6(19):eaaz8693.

[16] (a) Zhang HJ, Han WQ, Xu K, et al. Stretchable and ultrasensitive intelligent sensors for wireless human-machine manipulation. Adv Funct Mater. 2021;31(15):2009466.

[17] (a) Chen S, Liu H, Liu S, et al. Transparent and waterproof ionic liquid-based fibers for highly durable multifunctional sensors and strain-insensitive stretchable conductors. ACS Appl Mater Interfaces. 2018;10(4):4305. (b) Wang Y, Hao J, Huang Z, et al. Flexible electrically resistive-type strain sensors based on reduced graphene oxide-decorated electrospun polymer fibrous mats for human motion monitoring. Carbon. 2018:126:360. (c) Wu J, Wang Z, Liu W, Wang L, Xu F. Bioinspired superelastic electroconductive fiber for wearable electronics. ACS Appl Mater Interfaces. 2019;11(47):44735. (d) Duan Z, Jiang Y, Wang S, et al. Inspiration from daily goods: a low-cost, facilely fabricated, and environment-friendly strain sensor based on common carbon ink and elastic core-spun yarn. ACS Sustain Chem Eng. 2019;7(20):17474. (e) Wang L, Tian M, Zhang Y, et al. Helical core-sheath elastic yarn-based dual strain/humidity sensors with MXene sensing layer. J Mater Sci. 2020;55:6187. (f) Li W, Xu F, Liu W, et al. Flexible strain sensor based on aerogel-spun carbon nanotube yarn with a coresheath structure. Compos Part A Appl Sci Manuf. 2018;108:107. (g) Wang Z, Huang Y, Sun J, et al. Flexible strain sensor based on aerogel-spun carbon nanotube yarn with a core-sheath structure. ACS Appl Mater Interfaces. 2016;8(37):24837. (h) Li X, Hua T, Xu B. Electromechanical properties of a yarn strain sensor with graphene-sheath/polyurethane-core. Carbon. 2017;118:686. (i) Li L, Shi P, Hua L, et al. Design of a wearable and shape-memory fibriform sensor for the detection of multimodal deformation. Nanoscale. 2018;10(1):118. (j) Wu J, Ma Z, Hao Z, et al. Sheath-core fiber strain sensors driven by in-situ crack and elastic effects in graphite nanoplate composites. ACS Appl Nano Mater. 2019;2:(2)750. (k) Tang Z, Jia S, Wang F, et al. Highly stretchable core-sheath fibers via wet-spinning for wearable strain sensors. ACS Appl Mater Interfaces. 2018;10(7):6624. (l) Park JJ, Hyun WJ, Mun SC, Park YT, Park OO. Highly stretchable and wearable graphene strain sensors with controllable sensitivity for human motion monitoring. ACS Appl Mater Interfaces. 2015;7(11):6317. (m) Wang X, Qiu Y, Cao W, Hu P. Highly stretchable and conductive core-sheath chemical vapor deposition graphene fibers and their applications in safe strain sensors. Chem Mater. 2015;27(20):6969. (n) Zeng Z, Hao B, Li D, Cheng D, Cai G, Wang X. Large-scale production of weavable, dyeable and durable spandex/CNT/cotton core-sheath yarn for wearable strain sensors. Compos Part A Appl Sci Manuf. 2021;149:106520.

[18] Loper M, Mahmood N, Romero J, Pons-Moll G, Black MJ. SMPL: a skinned multi-person linear model. ACM Trans Graph. 2015;34(6):1-16.

[19] ISO 20685-1:2018. 3-D scanning methodologies for internationally compatible anthropometric databases. Accessed February 22, 2024. https://www.iso.org/standard/63260.html.

[20] Long M, Cao Y, Wang J, Jordan M. Learning transferable features with deep adaptation networks. Presented at Michael Jordan Proceedings of the 32nd International Conference on Machine Learning, Lille, France. 2015;37:97-105.

[21] Cao Z, Hidalgo Martinez G, Simon T, Wei S-E, Sheikh YA. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans Pattern Anal Mach Intell. 2019;43(1):172-186.

RIGHTS & PERMISSIONS

2024 2024 The Authors. InfoMat published by UESTC and John Wiley & Sons Australia, Ltd.