Restoring finger-specific tactile sensations with a sensory soft neuroprosthetic hand through electrotactile stimulation

Abstract

Tactile feedback is of great significance for amputees to improve the controllability of prosthetic hands and obtain tactile information regarding the interacting objects, which remains a significant challenge for neuroprosthetic hands. In this study, we present a method to restore finger-specific tactile sensations on the projected finger map of a unilateral forearm amputee with a sensory soft neuroprosthetic hand through electrotactile stimulation. On this basis, five soft touch sensors embedded in the fingertips are first adopted to measure the pressure changes of the soft neuroprosthetic hand with the touched objects. The measured pressure information is then accordingly encoded into electrotactile stimulation patterns to trigger an electrical stimulator that outputs programmable electrical pulses on the projected finger map of the amputee. In this manner, the finger tactile sensation can be elicited, which can help the amputee to distinguish the finger press state and discriminate the curvature and hardness of the touched objects. Experimental results show that, based on the different stimulation regions, the amputee subject can instantaneously distinguish the tactile sensation of a single finger or multiple fingers with an accuracy of 98.57% and 91.71%, respectively. By programming the frequencies of the electrical pulses, the amputee subject can successfully discriminate the touching objects with different curvatures and hardnesses with an accuracy of 97.26% and 97.93%, respectively. Finally, we demonstrate that the amputee subject can achieve closed-loop control of the sensory soft neuroprosthetic hand by integrating a myoelectric control interface and electrotactile feedback to achieve multilevel perception.

Keywords

Tactile sensation / soft neuroprosthetic hand / soft touch sensor / phantom finger map / electrotactile stimulation / sensory feedback

References

[1] Walker JM,Shewokis PA.Tactile feedback of object slip facilitates virtual object manipulation.IEEE Trans Haptics2015;8:454-66

[2] Gibo TL,Okamura AM.Grip force control during virtual object interaction: effect of force feedback,accuracy demands, and training.IEEE Trans Haptics2014;7:37-47

[3] Ajoudani A,Bianchi M.Exploring teleimpedance and tactile feedback for intuitive control of the Pisa/IIT SoftHand.IEEE Trans Haptics2014;7:203-15

[4] Schofield JS,Carey JP.Applications of sensory feedback in motorized upper extremity prosthesis: a review.Expert Rev Med Devices2014;11:499-511

[5] Biddiss E,Chau T.Consumer design priorities for upper limb prosthetics.Disabil Rehabil Assist Technol2007;2:346-57

[6] Cordella F,Sacchetti R.Literature review on needs of upper limb prosthesis users.Front Neurosci2016;10:209 PMCID:PMC4864250

[7] Engdahl SM,Kelly B,Chestek CA.Surveying the interest of individuals with upper limb loss in novel prosthetic control techniques.J Neuroeng Rehabil2015;12:53 PMCID:PMC4465617

[8] Wijk U.Forearm amputees’ views of prosthesis use and sensory feedback.J Hand Ther2015;28:269-77; quiz 278

[9] Belter JT,Dollar AM.Mechanical design and performance specifications of anthropomorphic prosthetic hands: a review.J Rehabil Res Dev2013;50:599-618

[10] Tan DW,Keith MW,Tyler J.A neural interface provides long-term stable natural touch perception.Sci Transl Med2014;6:257ra138 PMCID:PMC5517305

[11] Osborn LE,Betthauser JL.Prosthesis with neuromorphic multilayered e-dermis perceives touch and pain.Sci Robot2018;3 PMCID:PMC7051004

[12] Gu G,Xu H.A soft neuroprosthetic hand providing simultaneous myoelectric control and tactile feedback.Nat Biomed Eng2021;

[13] D’Anna E,Mazzoni A.A closed-loop hand prosthesis with simultaneous intraneural tactile and position feedback.Sci Robot2019;4:eaau8892

[14] Zollo L,Ciancio AL.Restoring Tactile sensations via neural interfaces for real-time force-and-slippage closed-loop control of bionic hands.Sci Robot2019;4:eaau9924 PMCID:PMC6795534

[15] Zhao H,Li S.Optoelectronically innervated soft prosthetic hand via stretchable optical waveguides.Sci Robot2016;1:eaai7529

[16] Boutry CM,Jorda M.A hierarchically patterned, bioinspired e-skin able to detect the direction of applied pressure for robotics.Sci Robot2018;3:eaau6914

[17] Lee S,Hassani FA.Nanomesh pressure sensor for monitoring finger manipulation without sensory interference.Science2020;370:966-70

[18] Ma D,Kaiser G.Simple, Low-Cost Fabrication of Soft Sensors for Shape Reconstruction.IEEE Robot Autom Lett2020;5:4049-54

[19] Chen J,Luo Z.Superelastic, sensitive, and low hysteresis flexible strain sensor based on wave-patterned liquid metal for human activity monitoring.ACS Appl Mater Interfaces2020;12:22200-11

[20] Kim T,Hong T,Kim T.Heterogeneous sensing in a multifunctional soft sensor for human-robot interfaces.Sci Robot2020;5:eabc6878

[21] Bai H,Barreiros J,Pollock CR.Stretchable distributed fiber-optic sensors.Science2020;370:848-52

[22] Yan Y,Yang Z.Soft magnetic skin for super-resolution tactile sensing with force self-decoupling.Sci Robot2021;6:eabc8801

[23] Ge J,Drack M.A bimodal soft electronic skin for tactile and touchless interaction in real time.Nat Commun2019;10:4405 PMCID:PMC6764954

[24] Pu X,Chen X.Ultrastretchable, transparent triboelectric nanogenerator as electronic skin for biomechanical energy harvesting and tactile sensing.Sci Adv2017;3:e1700015 PMCID:PMC5451198

[25] Jin T,Li L.Triboelectric nanogenerator sensors for soft robotics aiming at digital twin applications.Nat Commun2020;11:5381 PMCID:PMC7585441

[26] Li G,Wang L.Skin-inspired quadruple tactile sensors integrated on a robot hand enable object recognition.Sci Robot2020;5:eabc8134

[27] Chai G,Li G,Zhu X.Electrotactile feedback improves grip force control and enables object stiffness recognition while using a myoelectric hand.IEEE Trans Neural Syst Rehabil Eng2022;30:1310-20

[28] Motamedi MR,Duchaine V.The use of vibrotactile feedback to restore texture recognition capabilities, and the effect of subject training.IEEE Trans Neural Syst Rehabil Eng2017;25:1230-9

[29] Franceschi M,Dosen S,Valle M.A system for electrotactile feedback using electronic skin and flexible matrix electrodes: experimental evaluation.IEEE Trans Haptics2017;10:162-72

[30] Wall J,Wang X.Human brain plasticity: an emerging view of the multiple substrates and mechanisms that cause cortical changes and related sensory dysfunctions after injuries of sensory inputs from the body.Brain Res Rev2002;39:181-215

[31] Ramachandran VS.The perception of phantom limbs. The D. O. Hebb lecture.Brain1998;121:1603-30

[32] Chai G,Li S,Lan N.Characterization of evoked tactile sensation in forearm amputees with transcutaneous electrical nerve stimulation.J Neural Eng2015;12:066002

[33] Zhang S,Sun L.Selective release of different neurotransmitters emulated by a p-i-n junction synaptic transistor for environment-responsive action control.Adv Mater2021;33:e2007350

[34] Gong J,Liu J.An artificial visual nerve for mimicking pupil reflex.Matter2022;5:1578-89

[35] Wei H,Sun L.Mimicking efferent nerves using a graphdiyne-based artificial synapse with multiple ion diffusion dynamics.Nat Commun2021;12:1068 PMCID:PMC7886898

[36] Sun L,Yu H,Xu W.An artificial reflex Arc that perceives afferent visual and tactile information and controls efferent muscular actions.Research2022;2022:9851843 PMCID:PMC8858381

[37] Chai G,Zhu X.Developing Non-somatotopic phantom finger sensation to comparable levels of somatotopic sensation through user training with electrotactile stimulation.IEEE Trans Neural Syst Rehabil Eng2017;25:469-80

[38] Zhang J,Yang F.Evaluation of multiple perceptual qualities of transcutaneous electrical nerve stimulation for evoked tactile sensation in forearm amputees.J Neural Eng2022;19:026041

[39] Zhang D,Shull PB,Zhu X.Somatotopical feedback versus non-somatotopical feedback for phantom digit sensation on amputees using electrotactile stimulation.J Neuroeng Rehabil2015;12:44 PMCID:PMC4416276