Coupled mechanics in skin-interfaced electronics via computer vision methods

Abstract

Recent advancements in materials and mechanics have paved the way for transforming rigid circuits into flexible electronics. Their ability to laminate onto the skin has led to the development of skin-interfaced electronics, including mechano-acoustic sensors and haptic systems. However, the challenges of the coupled mechanics between the skin and skin-interfaced electronics call for further understanding of biomechanics, bioelectronics, and their interactions. This perspective article highlights the emerging trend of employing computer vision methods to optimize the next generation of skin-interfaced electronics by characterizing associated biomechanics and vice versa. The cyclic research process involves the development of soft electronics, the identification of coupled mechanics, and their quantification using computer vision methods. The article describes state-of-the-art computer vision techniques in the context of skin-interfaced electronics and their potential applications in other forms of soft electronics.

Keywords

Soft electronics / skin-interfaced electronics / mechano-acoustic sensors / haptic interfaces / computer vision

References

[1] Rogers JA,Huang Y.Materials and mechanics for stretchable electronics.Science2010;327:1603-7

[2] Fu H,Bai W.Morphable 3D mesostructures and microelectronic devices by multistable buckling mechanics.Nat Mater2018;17:268-76 PMCID:PMC5877475

[3] Zhu Y,Kim J.Skin-interfaced electronics: a promising and intelligent paradigm for personalized healthcare.Biomaterials2023;296:122075

[4] Li J,Rogers JA.Materials and designs for power supply systems in skin-interfaced electronics.Acc Chem Res2019;52:53-62

[5] Xu C,Gao W.Skin-interfaced sensors in digital medicine: from materials to applications.Matter2020;2:1414-45 PMCID:PMC7274218

[6] Kim JT,Kang S,Lee C.Effect of the aspect ratio on the dynamics of air bubbles within Rayleigh-Bénard convection.Phys Fluids2021;33:095104

[7] Anderson PSL,Kim JT.Taking a stab at quantifying the energetics of biological puncture.Integr Comp Biol2019;59:1586-96

[8] Jung YH,Vázquez-Guardado A.A wireless haptic interface for programmable patterns of touch across large areas of the skin.Nat Electron2022;5:374-85

[9] Adrian RJ.Particle-imaging techniques for experimental fluid mechanics.Annu Rev Fluid Mech1991;23:261-304

[10] Kim JT,Hwang H.Dynamics of plosive consonants via imaging, computations, and soft electronics.Proc Natl Acad Sci U S A2022;119:e2214164119 PMCID:PMC9674252

[11] Pan B.Recent progress in digital image correlation.Exp Mech2011;51:1223-35

[12] Solav D,Jaeger AM,Herr HM.MultiDIC: an open-source toolbox for multi-view 3D digital image correlation.IEEE Access2018;6:30520-35

[13] Yang Q,Seo MH.High-speed, scanned laser structuring of multi-layered eco/bioresorbable materials for advanced electronic systems.Nat Commun2022;13:6518 PMCID:PMC9622701

[14] Kim JT,Liberzon A.Three-dimensional particle tracking velocimetry for turbulence applications: case of a jet flow.J Vis Exp2016;53745 PMCID:PMC4828189

[15] Wu Y,Vázquez-Guardado A.Wireless multi-lateral optofluidic microsystems for real-time programmable optogenetics and photopharmacology.Nat Commun2022;13:5571 PMCID:PMC9500026

[16] Jeong H,Sohn S.Miniaturized wireless, skin-integrated sensor networks for quantifying full-body movement behaviors and vital signs in infants.Proc Natl Acad Sci U S A2021;118:e2104925118 PMCID:PMC8639372

[17] Mathis A,Cury KM.DeepLabCut: markerless pose estimation of user-defined body parts with deep learning.Nat Neurosci2018;21:1281-9

[18] Wu HY,Shih E,Durand F.Eulerian video magnification for revealing subtle changes in the world.ACM Trans Graph2012;31:1-8

[19] Schӧnberger LL, Frahm JM. Structure-from-motion revisited. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016. pp. 4104-13. Available from: https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Schonberger_Structure-From-Motion_Revisited_CVPR_2016_paper.pdf. [Last accessed on 15 Jan 2024]

[20] Seo Y,Enomoto Y,Aonuma K.Endocardial surface area tracking for assessment of regional LV wall deformation with 3D speckle tracking imaging.JACC Cardiovasc Imaging2011;4:358-65

[21] Lee K,Lee JY.Mechano-acoustic sensing of physiological processes and body motions via a soft wireless device placed at the suprasternal notch.Nat Biomed Eng2020;4:148-58 PMCID:PMC7035153

[22] Jeong H,Xu S.Continuous on-body sensing for the COVID-19 pandemic: gaps and opportunities.Sci Adv2020;6:eabd4794 PMCID:PMC7467694

[23] Ni X,Jeong H.Automated, multiparametric monitoring of respiratory biomarkers and vital signs in clinical and home settings for COVID-19 patients.Proc Natl Acad Sci U S A2021;118:e2026610118 PMCID:PMC8126790

[24] Jeong H,Lee K.Differential cardiopulmonary monitoring system for artifact-canceled physiological tracking of athletes, workers, and COVID-19 patients.Sci Adv2021;7:eabg3092 PMCID:PMC8115927

[25] Kang YJ,Yoo JY.Soft skin-interfaced mechano-acoustic sensors for real-time monitoring and patient feedback on respiratory and swallowing biomechanics.NPJ Digit Med2022;5:147 PMCID:PMC9485153

[26] Hiss SG,Stuart A.Effects of age, gender, bolus volume, and trial on swallowing apnea duration and swallow/respiratory phase relationships of normal adults.Dysphagia2001;16:128-35

[27] Yu X,Yu Y.Skin-integrated wireless haptic interfaces for virtual and augmented reality.Nature2019;575:473-9

[28] Kim JT,Yoo JY.Mechanics of vibrotactile sensors for applications in skin-interfaced haptic systems.Extreme Mech Lett2023;58:101940

[29] Kim BH,Kim JT.Three-dimensional electronic microfliers inspired by wind-dispersed seeds.Nature2021;597:503-10

[30] Yoon HJ,Kim JT.Biodegradable, three-dimensional colorimetric fliers for environmental monitoring.Sci Adv2022;8:eade3201 PMCID:PMC9788784

[31] Ni X,Kim JT.Soft shape-programmable surfaces by fast electromagnetic actuation of liquid metal networks.Nat Commun2022;13:5576 PMCID:PMC9508113

[32] Schanz D,Schröder A.Shake-the-box: lagrangian particle tracking at high particle image densities.Exp Fluids2016;57:70

[33] Wang H,Wang S.Dense velocity reconstruction from particle image velocimetry/particle tracking velocimetry using a physics-informed neural network.Phys Fluids2022;34:017116

[34] Boukhtache S,Berry F,Grédiac M.When deep learning meets digital image correlation.Opt Lasers Eng2021;136:106308

[35] Liu C,Yang DS.Multifunctional materials strategies for enhanced safety of wireless, skin-interfaced bioelectronic devices (Adv. Funct. Mater. 34/2023).Adv Funct Mater2023;33:2370203