A sensory–neuromorphic interface capable of environmental perception, sensory coding, and biological stimuli

Abstract

The sensory–neuromorphic interface is key to the application of neuromorphic electronics. Artificial spiking neurons and artificial sensory nerves have been created, and a few studies showed a complete neuromorphic system through cointegration with synaptic electronics. However, artificial synaptic devices and systems often do not work in real environments, which limits their ability to provide realistic neural simulations and interface with biological nerves. We report a sensory–neuromorphic interface that uses a fiber synapse to emulate a biological afferent nerve. For the first time, a sensing–neuromorphic interface is connected to a living organism for peripheral nerve stimulation, allowing the organism to establish a connection with its surrounding environment. The interface converts perceived environmental information into analog electrical signals and then into frequency-dependent pulse signals, which simplify the information interface between the sensor and the pulse-data processing center. The frequency of the interface shows a sublinear dependence on strain amplitude at different stimulus intensities, and can deliver increased frequency spikes at potentially damaging stimulus intensities, similar to the response of biological afferent nerves. To verify the application of this interface, a system that monitors strain and provides an overstrain alarm was constructed based on this afferent neural circuit. The system has a response time of <2 ms, which is compatible with the response time in biological systems. The interface can be potentially extended to process signals from almost any type of sensors for other afferent senses, and these results demonstrate the potential for neuromorphic interfaces to be applied to bionic sensory interfaces.

Keywords

artificial synapse / biological stimuli / neuromorphic electronics / sensory coding / sensory–neuromorphic interface

References

[1] Zhang M, Tang Z, Liu X, Van der Spiegel J. Electronic neural interfaces. Nat Electron. 2020; 3(4): 191-200.

[2] Woods GA, Rommelfanger NJ, Hong G. Bioinspired materials for in vivo bioelectronic neural interfaces. Matter. 2020; 3(4): 1087-1113.

[3] Vázquez-Guardado A, Yang Y, Bandodkar AJ, Rogers JA. Recent advances in neurotechnologies with broad potential for neuroscience research. Nat Neurosci. 2020; 23(12): 1522-1536.

[4] Kim Y, Chortos A, Xu W, et al. A bioinspired flexible organic artificial afferent nerve. Science. 2018; 360(6392): 998-1003.

[5] Chun S, Kim JS, Yoo Y, et al. An artificial neural tactile sensing system. Nat Electron. 2021; 4(6): 429-438.

[6] Wan C, Cai P, Guo X, et al. An artificial sensory neuron with visual-haptic fusion. Nat Commun. 2020; 11(1): 4602.

[7] Keene ST, Lubrano C, Kazemzadeh S, et al. A biohybrid synapse with neurotransmitter-mediated plasticity. Nat Mater. 2020; 19(9): 969-973.

[8] Kim J, Lee M, Shim HJ, et al. Stretchable silicon nanoribbon electronics for skin prosthesis. Nat Commun. 2014; 5(1): 5747.

[9] Prominski A, Shi J, Li P, et al. Porosity-based heterojunctions enable leadless optoelectronic modulation of tissues. Nat Mater. 2022; 21(6): 647-655.

[10] Kim KN, Sung MJ, Park HL, Lee TW. Organic synaptic transistors for bio-hybrid neuromorphic electronics. Adv Electron Mater. 2022; 8(1): 2100935.

[11] Lee Y, Lee TW. Organic synapses for neuromorphic electronics: from brain-inspired computing to sensorimotor nervetronics. Acc Chem Res. 2019; 52(4): 964-974.

[12] Zhu J, Zhang T, Yang Y, Huang R. A comprehensive review on emerging artificial neuromorphic devices. Appl Phys Rev. 2020; 7(1): 011312.

[13] Liu Z, Tang J, Gao B, et al. Neural signal analysis with memristor arrays towards high-efficiency brain–machine interfaces. Nat Commun. 2020; 11(1): 4234.

[14] Wang C, Liang SJ, Wang CY, et al. Scalable massively parallel computing using continuous-time data representation in nanoscale crossbar array. Nat Nanotechnol. 2021; 16(10): 1079-1085.

[15] Li X, Tang J, Zhang Q, et al. Power-efficient neural network with artificial dendrites. Nat Nanotechnol. 2020; 15(9): 776-782.

[16] Choi Y, Oh S, Qian C, Park JH, Cho JH. Vertical organic synapse expandable to 3D crossbar array. Nat Commun. 2020; 11(1): 4595.

[17] Kim SH, Baek GW, Yoon J, et al. A bioinspired stretchable sensory-neuromorphic system. Adv Mater. 2021; 33(44): 2104690.

[18] Van De Burgt Y, Lubberman E, Fuller EJ, et al. A non-volatile organic electrochemical device as a low-voltage artificial synapse for neuromorphic computing. Nat Mater. 2017; 16(4): 414-418.

[19] Tang J, Bishop D, Kim S, et al. ECRAM as scalable synaptic cell for high-speed, low-power neuromorphi. computing. IEEE Int Electron Devices Meet. 2018; 2018: 13.1.1-13.1.4.

[20] Fuller EJ, Gabaly FE, Léonard F, et al. Li-ion synaptic transistor for low power analog computing. Adv Mater. 2017; 29(4): 1604310.

[21] Zhu J, Yang Y, Jia R, et al. Ion gated synaptic transistors based on 2D van der Waals crystals with tunable diffusive dynamics. Adv Mater. 2018; 30(21): 1800195.

[22] Xu W, Min S-Y, Hwang H, Lee T-W. Organic core-sheath nanowire artificial synapses with femtojoule energy consumption. Sci Adv. 2016; 2(6): e1501326.

[23] Chun K-Y, Son YJ, Jeon E-S, Lee S, Han C-S. A self-powered sensor mimicking slow-and fast-adapting cutaneous mechanoreceptors. Adv Mater. 2018; 30(12): 1706299.

[24] Tee BCK, Chortos A, Berndt A, et al. A skin-inspired organic digital mechanoreceptor. Science. 2015; 350(6258): 313-316.

[25] Sun L, Du Y, Yu H, Wei H, Xu W, Xu W. An artificial reflex arc that perceives afferent visual and tactile information and controls efferent muscular actions. Research. 2022; 2022: 9851843-9851854.

[26] Chortos A, Liu J, Bao Z. Pursuing prosthetic electronic skin. Nat Mater. 2016; 15(9): 937-950.

[27] Someya T, Sekitani T, Iba S, Kato Y, Kawaguchi H, Sakurai T. A large-area, flexible pressure sensor matrix with organic field-effect transistors for artificial skin applications. Proc Natl Acad Sci. 2004; 101(27): 9966-9970.

[28] Xu S, Zhang Y, Jia L, et al. Soft microfluidic assemblies of sensors, circuits, and radios for the skin. Science. 2014; 344(6179): 70-74.

[29] Choi S, Lee H, Ghaffari R, Hyeon T, Kim DH. Recent advances in flexible and stretchable bio-electronic devices integrated with nanomaterials. Adv Mater. 2016; 28(22): 4203-4218.

[30] Tang J, Yuan F, Shen X, et al. Bridging biological and artificial neural networks with emerging neuromorphic devices: fundamentals, progress, and challenges. Adv Mater. 2019; 31(49): 1902761.

[31] Mead C. Neuromorphic electronic systems. Proc IEEE. 1990; 78(10): 1629-1636.

[32] Yuan H, Shimotani H, Tsukazaki A, Ohtomo A, Kawasaki M, Iwasa Y. Hydrogenation-induced surface polarity recognition and proton memory behavior at protic-ionic-liquid/oxide electric-double-layer interfaces. J Am Chem Soc. 2010; 132(19): 6672-6678.

[33] Regehr WG. Short-term presynaptic plasticity. Cold Spring Harbor Perspect Biol. 2012; 4(7): a005702.

[34] Desbief S, di Lauro M, Casalini S, et al. Electrolyte-gated organic synapse transistor interfaced with neurons. Org Electron. 2016; 38: 21-28.

[35] John RA, Tiwari N, Patdillah MIB, et al. Self healable neuromorphic memtransistor elements for decentralized sensory signal processing in robotics. Nat Commun. 2020; 11(1): 4030.

[36] Li W, Matsuhisa N, Liu Z, et al. An on-demand plant-based actuator created using conformable electrodes. Nat Electron. 2021; 4(2): 134-142.

[37] Cui Z, Wang W, Guo L, et al. Haptically quantifying Young’s modulus of soft materials using a self-locked stretchable strain sensor. Adv Mater. 2022; 34(25): 2104078.

[38] Cao L, Liu Q, Ren J, et al. Electro-blown spun silk/graphene nanoionotronic skin for multifunctional fire protection and alarm. Adv Mater. 2021; 33(38): 2102500.

[39] Ni Y, Feng J, Liu J, et al. An artificial nerve capable of UV-perception, NIR-vis switchabl. plasticity modulation, and motion state monitoring. Adv Sci. 2022; 9(1): 2102036.

[40] Lee B, Cho H, Park KT, et al. High-performance compliant thermoelectric generators with magnetically self-assembled soft heat conductors for self-powered wearable electronics. Nat Commun. 2020; 11(1): 5948.

[41] Matsuhisa N, Niu S, O’Neill SJK, et al. High-frequency and intrinsically stretchable polymer diodes. Nature. 2021; 600(7888): 246-252.

[42] Sun L, Qu S, Du Y, et al. Bio-inspired vision and neuromorphic image processing using printable metal oxide photonic synapses. ACS Photonics. 2023; 10(1): 242-252.

[43] Fu T, Liu X, Gao H, et al. Bioinspired bio-voltage memristors. Nat Commun. 2020; 11(1): 1861.

[44] Fu T, Liu X, Fu S, et al. Self-sustained green neuromorphic interfaces. Nat Commun. 2021; 12(1): 3351.

[45] Zhou F, Chai Y. Near-sensor and in-sensor computing. Nat Electron. 2020; 3(11): 664-671.

[46] Navarro X, Krueger TB, Lago N, Micera S, Stieglitz T, Dario P. A critical review of interfaces with the peripheral nervous system for the control of neuroprostheses and hybrid bionic systems. J Peripher Nerv Syst. 2005; 10(3): 229-258.

[47] Minev IR, Musienko P, Hirsch A, et al. Electronic dura mater for long-term multimodal neural interfaces. Science. 2015; 347(6218): 159-163.

[48] Dobashi Y, Yao D, Petel Y, et al. Piezoionic mechanoreceptors: force-induced current generation in hydrogels. Science. 2022; 376(6592): 502-507.

[49] Song KI, Seo H, Seong D, et al. Adaptive self-healing electronic epineurium for chronic bidirectional neural interfaces. Nat Commun. 2020; 11(1): 4195.

[50] Shahriari D, Rosenfeld D, Anikeeva P. Emerging frontier of peripheral nerve and organ interfaces. Neuron. 2020; 108(2): 270-285.

[51] Molina-Lopez F, Gao TZ, Kraft U, et al. Inkjet-printed stretchable and low voltage synaptic transistor array. Nat Commun. 2019; 10(1): 2676.

[52] Shim H, Ershad F, Patel S, et al. An elastic and reconfigurable synaptic transistor based on a stretchable bilayer semiconductor. Nat Electron. 2022; 5(10): 660-671.

[53] Lee Y, Oh JY, Xu W, et al. Stretchable organic optoelectronic sensorimotor synapse. Sci Adv. 2018; 4(11): eaat7387.

[54] Nguyen TD, Trung TQ, Lee Y, Lee NE. Stretchable and stable electrolyte-gated organic electrochemical transistor synapse with a Nafion membrane for enhanced synaptic properties. Adv Eng Mater. 2022; 24(4): 2100918.

[55] Xu F, Zhang C, Zhao X, et al. Intrinsically stretchable photonic synaptic transistors for retina-like visual image systems. J Mater Chem C. 2022; 10(29): 10586-10594.

[56] Shim H, Jang S, Thukral A, et al. Artificial neuromorphic cognitive skins based on distributed biaxially stretchable elastomeric synaptic transistors. Proc Natl Acad Sci USA. 2022; 119(23): 2204852119.

[57] Wang Y, Liu D, Zhang Y, et al. Stretchable temperature-responsive multimodal neuromorphic electronic skin with spontaneous synaptic plasticity recovery. ACS Nano. 2022; 16(5): 8283-8293.

[58] Shim H, Sim K, Ershad F, et al. Stretchable elastic synaptic transistors for neurologically integrated soft engineering systems. Sci Adv. 2019; 5(10): eaax4961.

RIGHTS & PERMISSIONS

2024 The Authors. SmartMat published by Tianjin University and John Wiley & Sons Australia, Ltd.