Bio-inspired artificial retina with receptive fields for in-sensor multiply-and-accumulate operations

Abstract

Bio-inspired vision systems based on curved image sensors offer a compelling imaging hardware for mobile robots by geometrically matching the image plane to the Petzval surface of single-lens optics. However, to fully exploit the advantages of biological vision systems, it is essential not only to emulate optical structures but also to integrate sensory-level processing functions into vision hardware. Here, we propose a robotic vision system that leverages the structural advantages of the human eye (compact single-lens imaging architecture) and the functional advantages of the biological receptive fields (sensory-level information pre-processing for efficient signal transmission and downstream computation). It is enabled by a bio-inspired artificial retina composed of curved perovskite photoconductors that form artificial receptive fields (ARFs). Each ARF performs multiply-and-accumulate (MAC) operations through in-sensor computing, executing sensory-level image processing functions (e.g., edge detection). As a result, the artificial retina captures compact yet information-rich edge images without external post-processing, thereby improving the speed and energy efficiency of semantic segmentation.

Keywords

bio-inspired electronics / flexible electronics / in-sensor computing / perovskite photodetector / robotic vision

References

[1] Sandamirskaya Y, Kaboli M, Conradt J, Celikel T. Neuromorphic computing hardware and neural architectures for robotics. Sci Robot. 2022; 7(67):eabl8419.

[2] Billard A, Albu-Schaeffer A, Beetz M, et al. A roadmap for AI in robotics. Nat Mach Intell. 2025; 7(6): 818-824.

[3] Kim S, Choi YY, Kim T, et al. A biomimetic ocular prosthesis system: emulating autonomic pupil and corneal reflections. Nat Commun. 2022; 13(1): 6760.

[4] Chowdhury SS, Sharma D, Kosta A, Roy K. Neuromorphic computing for robotic vision: algorithms to hardware advances. Commun Eng. 2025; 4(1):152.

[5] Rao Z, Lu Y, Li Z, et al. Curvy, shape-adaptive imagers based on printed optoelectronic pixels with a kirigami design. Nat Electron. 2021; 4(7): 513-521.

[6] Chung T, Lee Y, Yang SP, Kim K, Kang BH, Jeong KH. Mining the smartness of insect ultrastructures for advanced imaging and illumination. Adv Funct Mater. 2018; 28(24):1705912.

[7] Gu L, Poddar S, Lin Y, et al. A biomimetic eye with a hemispherical perovskite nanowire array retina. Nature. 2020; 581(7808): 278-282.

[8] Lin D, Hayward TM, Jia W, Majumder A, Sensale-Rodriguez B, Menon R. Inverse-designed multi-level diffractive doublet for wide field-of-view imaging. ACS Photonics. 2023; 10(8): 2661-2669.

[9] Song YM, Xie Y, Malyarchuk V, et al. Digital cameras with designs inspired by the arthropod eye. Nature. 2013; 497(7447): 95-99.

[10] Mikš A, Novák J, Novák P. Method of zoom lens design. Appl Optics. 2008; 47(32): 6088-6098.

[11] Park J, Kim MS, Kim J, et al. Avian eye-inspired perovskite artificial vision system for foveated and multispectral imaging. Sci Robot. 2024; 9(90):eadk6903.

[12] Ude A, Gaskett C, Cheng G. Foveated Vision Systems With Two Cameras Per Eye. IEEE; 2006: 3457-3462.

[13] Ko HC, Stoykovich MP, Song J, et al. A hemispherical electronic eye camera based on compressible silicon optoelectronics. Nature. 2008; 454(7205): 748-753.

[14] Long Z, Qiu X, Chan CLJ, et al. A neuromorphic bionic eye with filter-free color vision using hemispherical perovskite nanowire array retina. Nat Commun. 2023; 14(1): 1972.

[15] Jung I, Xiao J, Malyarchuk V, et al. Dynamically tunable hemispherical electronic eye camera system with adjustable zoom capability. Proc Natl Acad Sci U S A. 2011; 108(5): 1788-1793.

[16] Guenter B, Joshi N, Stoakley R, et al. Highly curved image sensors: a practical approach for improved optical performance. Opt Express. 2017; 25(12): 13010-13023.

[17] Lee M, Lee GJ, Jang HJ, et al. An amphibious artificial vision system with a panoramic visual field. Nat Electron. 2022; 5(7): 452-459.

[18] Garcia M, Edmiston C, York T, et al. Bio-inspired imager improves sensitivity in near-infrared fluorescence image-guided surgery. Optica. 2018; 5(4): 413-422.

[19] Kim M, Chang S, Kim M, et al. Cuttlefish eye-inspired artificial vision for high-quality imaging under uneven illumination conditions. Sci Robot. 2023; 8(75):eade4698.

[20] Li T, Miao J, Fu X, et al. Reconfigurable, non-volatile neuromorphic photovoltaics. Nat Nanotechnol. 2023; 18(11): 1303-1310.

[21] Zhou F, Chai Y. Near-sensor and in-sensor computing. Nat Electron. 2020; 3(11): 664-671.

[22] Tang J, Zhu Y, Jiang G, et al. Human-centred design and fabrication of a wearable multimodal visual assistance system. Nat Mach Intell. 2025; 7(4): 627-638.

[23] Masland RH. The fundamental plan of the retina. Nat Neurosci. 2001; 4(9): 877-886.

[24] Kolb H. How the retina works: much of the construction of an image takes place in the retina itself through the use of specialized neural circuits. Am Sci. 2003; 91(1): 28-35.

[25] Euler T, Haverkamp S, Schubert T, Baden T. Retinal bipolar cells: elementary building blocks of vision. Nat Rev Neurosci. 2014; 15(8): 507-519.

[26] Zhu S, Xie T, Lv Z, et al. Hierarchies in visual pathway: functions and inspired artificial vision. Adv Mater. 2024; 36(6):2301986.

[27] Gollisch T, Meister M. Eye smarter than scientists believed: neural computations in circuits of the retina. Neuron. 2010; 65(2): 150-164.

[28] Thoreson WB, Mangel SC. Lateral interactions in the outer retina. Prog Retin Eye Res. 2012; 31(5): 407-441.

[29] Chai Y. In-sensor computing for machine vision. Nature. 2020; 579(7797): 32-33.

[30] Zhang Z, Wang S, Liu C, Xie R, Hu W, Zhou P. All-in-one two-dimensional retinomorphic hardware device for motion detection and recognition. Nat Nanotechnol. 2022; 17(1): 27-32.

[31] Liao F, Zhou Z, Kim BJ, et al. Bioinspired in-sensor visual adaptation for accurate perception. Nat Electron. 2022; 5(2): 84-91.

[32] Zhou G, Li J, Song Q, et al. Full hardware implementation of neuromorphic visual system based on multimodal optoelectronic resistive memory arrays for versatile image processing. Nat Commun. 2023; 14(1): 8489.

[33] Bae B, Lee D, Park M, et al. Stereoscopic artificial compound eyes for spatiotemporal perception in three-dimensional space. Sci Robot. 2024; 9(90):eadl3606.

[34] Yang Y, Pan C, Li Y, et al. In-sensor dynamic computing for intelligent machine vision. Nat Electron. 2024; 7(3): 225-233.

[35] Dang B, Zhang T, Wu X, Liu K, Huang R, Yang Y. Reconfigurable in-sensor processing based on a multi-phototransistor–one-memristor array. Nat Electron. 2024; 7(11): 991-1003.

[36] Mennel L, Symonowicz J, Wachter S, Polyushkin DK, Molina-Mendoza AJ, Mueller T. Ultrafast machine vision with 2D material neural network image sensors. Nature. 2020; 579(7797): 62-66.

[37] Jang H, Hinton H, Jung W-B, et al. In-sensor optoelectronic computing using electrostatically doped silicon. Nat Electron. 2022; 5(8): 519-525.

[38] Wang C-Y, Liang S-J, Wang S, et al. Gate-tunable van der Waals heterostructure for reconfigurable neural network vision sensor. Sci Adv. 2020; 6(26):eaba6173.

[39] Pang J, Wu H, Li H, Jin T, Tang J, Niu G. Reconfigurable perovskite x-ray detector for intelligent imaging. Nat Commun. 2024; 15(1): 1769.

[40] Widlund T, Yang S, Hsu Y-Y, Lu N. Stretchability and compliance of freestanding serpentine-shaped ribbons. Int J Solids Struct. 2014; 51(23–24): 4026-4037.

[41] Land MF, Nilsson D-E. Animal Eyes. Oxford University Press; 2012.

[42] Schnapf JL, Baylor DA. How photoreceptor cells respond to light. Sci Am. 1987; 256(4): 40-47.

[43] Mead CA, Mahowald MA. A silicon model of early visual processing. Neural Netw. 1988; 1(1): 91-97.

[44] Purves D, Augustine GJ, Groh JM, Huettel SA, LaMantia A-S, White L. Neurosciences. De Boeck Supérieur; 2025.

[45] Lee W, Yoo YJ, Park J, et al. Perovskite microcells fabricated using swelling-induced crack propagation for colored solar windows. Nat Commun. 2022; 13(1): 1946.

[46] Gao W, Zhang X, Yang L, Liu H. An Improved Sobel Edge Detection. IEEE; 2010: 67-71.

[47] Xiong Z, Liang W, Zhang M, Mao D, Xia Q, Xu G. Parallelizing analog in-sensor visual processing with arrays of gate-tunable silicon photodetectors. Nat Commun. 2025; 16(1): 4728.

[48] Wang X. Laplacian operator-based edge detectors. IEEE Trans Pattern Anal Mach Intell. 2007; 29(5): 886-890.

[49] Ding Y, Yang Y, Hao H, et al. Bioinspired target detection pipeline based on two-dimensional optoelectronic van der Waals heterostructures. ACS Nano. 2025; 19(24): 22376-22386.

Rights & permissions

2026 The Author(s). InfoMat published by UESTC and John Wiley & Sons Australia, Ltd.