Teaching oral care via vision-based deformation perception

Abstract

This paper presents a novel, cost-effective sensor platform based on Vision-based Deformation Perception (VBDeformP) for community oral health education. The system integrates a 3D-printed thermoplastic polyurethane soft structure with a rigid resin frame and an ArUco marker to encode interaction information, including the contact region and six-dimensional force and torque. By transforming force estimation into a marker-based pose tracking problem, the VBDeformP sensor achieves accurate and robust force/torque inference under both quasi-static and dynamic conditions, utilizing machine learning models. An adaptive image binarization algorithm extends reliable marker detection across a wide illumination range (10-5,000 lx), ensuring consistent performance of the vision system in realistic community teaching scenarios. Experimental validation involving 10 healthy participants per-forming standardized brushing tasks demonstrated that the sensor attains measurement accuracies comparable to a commercial ATI Axia80-M20 sensor, with mean absolute errors of 0.55 N (2.19% relative error) and 0.067 N·m (2.68% relative error) for quasi-static forces and torques, and 0.16 N (4.10% relative error) and 0.023 N·m (5.75% relative error) under dynamic conditions. Moreover, the system’s real-time brushing region classification algorithm achieved an overall accuracy of 98.12%, further underscoring its potential to provide immediate and personalized guidance on oral hygiene. Its low cost, rapid initialization, portability, and scalable fabrication render it a promising solution for enhancing oral health education in community settings.

Keywords

Vision-based deformation perception / soft force/torque sensor / oral health education

References

[1] Bernabe E,Hernandez CR.GBD 2017 Oral Disorders CollaboratorsGlobal, regional, and national levels and trends in burden of oral conditions from 1990 to 2017: a systematic analysis for the global burden of disease 2017 study.J Dent Res2020;99:362-73 PMCID:PMC7088322

[2] Watt RG,Daly B.Understanding and tackling oral health inequalities in vulnerable adult populations: from the margins to the mainstream.Br Dent J2019;227:49-54

[3] Weik U,Sämann T,Margraf-Stiksrud J.“You should brush your teeth better”: a randomized controlled trial comparing best-possible versus as-usual toothbrushing.BMC Oral Health2023;23:456 PMCID:PMC10327354

[4] Van der Weijden, G. A. F.; van Loveren, C. Mechanical plaque removal in step-1 of care.Periodontol 20002023;

[5] Palanisamy S.Innovations in oral hygiene tools: a mini review on recent developments.Front Dent Med2024;5:1442887 PMCID:PMC11797890

[6] Pindobilowo; Tjiptoningsih, U. G.; Ariani, D. Effective tooth brushing techniques based on periodontal tissue conditions: a narrative review.FJAS2023;2:1649-62

[7] Lee YJ,Kim KS.Toothbrushing region detection using three-axis accelerometer and magnetic sensor.IEEE Trans Biomed Eng2012;59:872-81

[8] Marcon M,Tubaro S.Toothbrush motion analysis to help children learn proper tooth brushing.Comput Vis Image Underst2016;148:34-45

[9] Akther S,Saha M,Kumar S.mTeeth: identifying brushing teeth surfaces using wrist-worn inertial sensors.Proc ACM Interact Mob Wearable Ubiquitous Technol2021;5:1-25 PMCID:PMC8932958

[10] Wang Y,Jiang Y,Liu C.ToothFairy: real-time tooth-by-tooth brushing monitor using earphone reversed signals.Proc ACM Interact Mob Wearable Ubiquitous Technol2023;7:1-19

[11] Herath B,Sukumaran S.Design and development of a novel oral care simulator for the training of nurses.IEEE Trans Biomed Eng2020;67:1314-20

[12] Daigo T,Mitani A.Development of the second prototype of an oral care simulator.JRM2021;33:172-9

[13] Matsuno T,Mitani A.Measurement algorithm for oral care simulator using a single force sensor.Adv Robot2021;35:723-32

[14] Mouri N,Yagimaki T,Igari K.Development of a training simulator for caregivers’ toothbrushing skill using virtual reality.ABE2023;12:91-100

[15] Anwar AI.The influence of demonstration method education in the knowledge of tooth brushing in children age 10-12 years.Enferm Clín2020;30:429-32

[16] Shida H,Yoshioka M.Effectiveness of a digital device providing real-time visualized tooth brushing instructions: a randomized controlled trial.PLoS One2020;15:e0235194 PMCID:PMC7316295

[17] Acherkouk A,Kiesow A.Robot and mechanical testing of a specialist manual toothbrush for cleaning efficacy and improved force control.BMC Oral Health2022;22:225 PMCID:PMC9175444

[18] Rajab LD,El-Smadi LA.Comparison of effectiveness of oral hygiene instruction methods in improving plaque scores among 8-9-year children: a randomized controlled trial.Eur Arch Paediatr Dent2022;23:289-300

[19] Haresaku S,Kubota K.Current status and future prospects for oral care education in Bachelor of Nursing curriculums: a Japanese cross-sectional study.Jpn J Nurs Sci2023;20:e12521

[20] Li Y,Wu S.Mixed reality and haptic-based dental simulator for tooth preparation: research, development, and preliminary evaluation.JMIR Serious Games2022;10:e30653 PMCID:PMC8943556

[21] Ponce-Gonzalez I,Aisenberg G.Improving oral health in migrant and underserved populations: evaluation of an interactive, community-based oral health education program in Washington state.BMC Oral Health2019;19:30 PMCID:PMC6375135

[22] Chandio N,Tadakmadla SK.Barriers and enablers in the implementation and sustainability of toothbrushing programs in early childhood settings and primary schools: a systematic review.BMC Oral Health2022;22:242 PMCID:PMC9206278

[23] Nakre PD.Effectiveness of oral health education programs: a systematic review.J Int Soc Prev Community Dent2013;3:103-15 PMCID:PMC4000911

[24] Qiu Y,Nguyen CC,Do TN.Integrated sensors for soft medical robotics.Small2024;20:e2308805

[25] Zhu J,Zhang M.Recent progress in flexible tactile sensor systems: from design to application.Soft Sci2021;1:3

[26] Guess M,Rigo B.Wireless batteryless soft sensors for ambulatory cardiovascular health monitoring.Soft Sci2023;3:24

[27] Jiang Y,Liu H,Zhou S.A vitrimer-like elastomer with quadruple hydrogen bonding as a fully recyclable substrate for sustainable flexible wearables.Adv Funct Mater2025;2503128

[28] Liu P,Xu Z.Wafer-scale fabrication of wearable all-carbon nanotube photodetector arrays.ACS Nano2024;18:18900-9 PMCID:PMC11271656

[29] Tian Y,Wang M.Ultra-stretchable, tough, and self-healing polyurethane with tunable microphase separation for flexible wearable electronics.Nano Energy2025;139:110908

[30] Aubeeluck DA,Taromsari SM,Diller E.Screen-printed resistive tactile sensor for monitoring tissue interaction forces on a surgical magnetic microgripper.ACS Appl Mater Interfaces2023;15:34008-22

[31] Arshad A,Tiwana MI,Iqbal S.A high sensitivity and multi-axis fringing electric field based capacitive tactile force sensor for robot assisted surgery.Sens Actuators A Phys2023;354:114272

[32] Vijayakanth T,Finkelstein-Zuta G,Gilead S.Perspectives on recent advancements in energy harvesting, sensing and bio-medical applications of piezoelectric gels.Chem Soc Rev2023;52:6191-220 PMCID:PMC10464879

[33] Chen S,Chan H.Liquid metal functionalization innovations in wearables and soft robotics for smart healthcare applications.Adv Funct Mater2024;34:2309989

[34] Li T,Zheng H.An artificial intelligence-motivated skin-like optical fiber tactile sensor.Adv Intell Syst2023;5:2200460

[35] Mun H,Youn JH.Multi-degree-of-freedom force sensor incorporated into soft robotic gripper for improved grasping stability.Soft Robot2024;11:628-38

[36] Dong K,Zhou Q,Gao B.Bionic diffractive meta-silk patch for visually flexible wearables.Laser Photonics Rev2024;18:2300972

[37] Gerald A.Soft sensing and haptics for medical procedures.Nat Rev Mater2024;9:86-8

[38] Wu T,Liu X.Vision-based tactile intelligence with soft robotic metamaterial.Mater Design2024;238:112629

[39] Wong DCY,Yu H.The design of a vision-based bending sensor for PneuNet actuators leveraging ArUco marker detection.IEEE Sens J2023;23:27137-45

[40] Deng Y,Dai S.A miniature triaxial fiber optic force sensor for flexible ureteroscopy.IEEE Trans Biomed Eng2021;68:2339-47

[41] Zhang T,Zuo S.A novel 3-DOF force sensing microneedle with integrated fiber bragg grating for microsurgery.IEEE Trans Ind Electron2022;69:940-9

[42] Di J,Fu W.Using fiber optic bundles to miniaturize vision-based tactile sensors.IEEE Trans Robot2025;41:62-81

[43] Garrido-Jurado S,Madrid-Cuevas F.Automatic generation and detection of highly reliable fiducial markers under occlusion.Pattern Recognit2014;47:2280-92

[44] Yang Z,Wan F,Song C.Scalable tactile sensing for an omni-adaptive soft robot finger. In 2020 3rd IEEE International Conference on Soft Robotics (RoboSoft), New Haven, USA. 15 May - 15 Jul, 2020. IEEE; 2020. pp. 572-7.

[45] Liu X,Hong W,Song C.Proprioceptive learning with soft polyhedral networks.Int J Robot Res2024;43:1916-35

[46] Edwards PJ,Sridhar A,Stoyanov D.Visual kinematic force estimation in robot-assisted surgery - application to knot tying.Comput Methods Biomech Biomed Eng Imaging Vis2021;9:414-20

[47] Fu J,Guo Q,Gan D.A variable stiffness robotic gripper based on parallel beam with vision-based force sensing for flexible grasping.Robotica2024;42:4036-54

[48] Visentin F,Mazzolai B.A new exploration strategy for soft robots based on proprioception. In 2020 3rd IEEE International Conference on Soft Robotics (RoboSoft), New Haven, USA. 15 May - 15 Jul, 2020. IEEE; 2020. pp. 816-21.

[49] Berral-Soler R,Medina-Carnicer R.DeepArUco++: improved detection of square fiducial markers in challenging lighting conditions.Image Vis Comput2024;152:105313

[50] Amiriebrahimabadi M,Mansouri N.A comprehensive survey of multi-level thresholding segmentation methods for image processing.Arch Computat Methods Eng2024;31:3647-97

[51] Ye C,Zhu J,Li X.Soft Thermoplastic polyurethane/silver nanowire membranes with low hysteresis for large strain sensing and joule heating.ACS Appl Polym Mater2024;6:11149-59

[52] Bek M,von Bernstorff B.Viscoelasticity of new generation thermoplastic polyurethane vibration isolators.Phys Fluids2017;29:121614

[53] Zhang Y,Lu Z,Chen L.Analytical and experimental study of thermoplastic polyurethane inclined beam isolator with quasi-zero stiffness and fractional derivative damping.Mech Syst Signal Process2025;224:111962

[54] Zheng H,Wang H.DotView: a low-cost compact tactile sensor for pressure, shear, and torsion estimation.IEEE Robot Autom Lett2023;8:880-7

[55] Funk N,Chalvatzaki G,Peters J.Evetac: an event-based optical tactile sensor for robotic manipulation.IEEE Trans Robot2024;40:3812-32

[56] Fang B,Liu N.Force measurement technology of vision-based tactile sensor.Adv Intell Syst2025;7:2400290

[57] Lin X.Sensing the frictional state of a robotic skin via subtractive color mixing.IEEE Robot Autom Lett2019;4:2386-92

[58] Zhang G,Yu H.DelTact: a vision-based tactile sensor using a dense color pattern.IEEE Robot Autom Lett2022;7:10778-85

[59] Yuan W,Adelson EH.GelSight: high-resolution robot tactile sensors for estimating geometry and force.Sensors2017;17:2762 PMCID:PMC5751610

[60] Sun H,Martius G.A soft thumb-sized vision-based sensor with accurate all-round force perception.Nat Mach Intell2022;4:135-45

[61] Lo Preti, M.; Bernabei, F.; Nardin, A. B.; Beccai, L. Triaxial 3-D-channeled soft optical sensor for tactile robots.IEEE Sens J2024;24:27956-65