Diffractive Deep Neural Networks at Visible Wavelengths

Abstract

Optical deep learning based on diffractive optical elements offers unique advantages for parallel processing, computational speed, and power efficiency. One landmark method is the diffractive deep neural network (D2NN) based on three-dimensional printing technology operated in the terahertz spectral range. Since the terahertz bandwidth involves limited interparticle coupling and material losses, this paper
extends D2NN to visible wavelengths. A general theory including a revised formula is proposed to solve any contradictions between wavelength, neuron size, and fabrication limitations. A novel visible light D2NN classifier is used to recognize unchanged targets (handwritten digits ranging from 0 to 9) and targets that have been changed (i.e., targets that have been covered or altered) at a visible wavelength of 632.8 nm. The obtained experimental classification accuracy (84%) and numerical classification accuracy (91.57%) quantify the match between the theoretical design and fabricated system performance. The presented framework can be used to apply a D2NN to various practical applications and design other new applications.

Keywords

Optical computation / Optical neural networks / Deep learning / Optical machine learning / Diffractive deep neural networks

References

[[1]] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521(7553): 436–44.

[[2]] Bohannon J. Helping robots see the big picture. Science 2014;346 (6206):186–7.

[[3]] Hinton G, Deng L, Yu D, Dahl G, Mohamed AR, Jaitly N, et al. Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 2012;29(6):82–97.

[[4]] Schofield D, Nagrani A, Zisserman A, Hayashi M, Matsuzawa T, Biro D, et al. Chimpanzee face recognition from videos in the wild using deep learning. Sci Adv 2019;5(9):eaaw0736.

[[5]] Barbastathis G, Ozcan A, Situ G. On the use of deep learning for computational imaging. Optica 2019;6(8):921–43.

[[6]] Giusti A, Guzzi J, Ciresan DC, He FL, Rodriguez JP, Fontana F, et al. A machine learning approach to visual perception of forest trails for mobile robots. IEEE Robot Autom Lett 2016;1(2):661–7.

[[7]] Silver D, Huang A, Maddison CJ, Guez A, Sifre L, van den Driessche G, et al. Mastering the game of Go with deep neural networks and tree search. Nature 2016;529(7587):484–9.

[[8]] Topol EJ. A decade of digital medicine innovation. Sci Transl Med 2019;11 (498):eaaw7610.

[[9]] Akhtar N, Mian A. Threat of adversarial attacks on deep learning in computer vision: a survey. IEEE Access 2018;6:14410–30.

[[10]] Markram H. The blue brain project. Nat Rev Neurosci 2006;7(2):153–60.

[[11]] Hines ML, Carnevale NT. The NEURON simulation environment. Neural Comput 1997;9(6):1179–209.

[[12]] Deng R, Liu X. Optical multiplexing: tunable lifetime nanocrystals. Nat Photonics 2014;8(1):10–2.

[[13]] Zijlstra P, Chon JWM, Gu M. Five-dimensional optical recording mediated by surface plasmons in gold nanorods. Nature 2009;459(7245):410–3.

[[14]] Li X, Ren H, Chen X, Liu J, Li Q, Li C, et al. Athermally photoreduced graphene oxides for three-dimensional holographic images. Nat Commun 2015;6(1):6984.

[[15]] Ren H, Li X, Zhang Q, Gu M. On-chip noninterference angular momentum multiplexing of broadband light. Science 2016;352(6287):805–9.

[[16]] Deng R, Qin F, Chen R, Huang W, Hong M, Liu X. Temporal full-colour tuning through non-steady-state upconversion. Nat Nanotechnol 2015;10(3): 237–42.

[[17]] Yan T, Wu J, Zhou T, Xie H, Xu F, Fan J, et al. Fourier-space diffractive deep neural network. Phys Rev Lett 2019;123(2):023901.

[[18]] Lin X, Rivenson Y, Yardimci NT, Veli M, Luo Yi, Jarrahi M, et al. All-optical machine learning using diffractive deep neural networks. Science 2018;361 (6406):1004–8.

[[19]] Mengu D, Luo Y, Rivenson Y, Ozcan A. Analysis of diffractive optical neural networks and their integration with electronic neural networks. IEEE J Sel Top Quantum Electron 2019;26(1):3700114.

[[20]] Li J, Mengu D, Luo Y, Rivenson Y, Ozcan A. Class-specific differential detection in diffractive optical neural networks improves inference accuracy. Adv Photonics 2019;1(4):46001.

[[21]] Khoram E, Chen A, Liu D, Ying L, Wang Q, Yuan M, et al. Nanophotonic media for artificial neural inference. Photonics Res 2019;7(8):823–7.

[[22]] Yao K, Unni R, Zheng Y. Intelligent nanophotonics: merging photonics and artificial intelligence at the nanoscale. Nanophotonics 2019;8(3): 339–66.

[[23]] Goi E, Gu M. Laser printing of a nano-imager to perform full optical machine learning [presentation]. In: Conference on Lasers and Electro-Optics/Europe; 2019 Jun 23–27; Munich, Germany; 2019.

[[24]] Gu M, Fang X, Ren H, Goi E. Optically digitalized holography: a perspective for all-optical machine learning. Engineering 2019;5(3):363–5.

[[25]] Yamazato T, Takai I, Okada H, Fujii T, Yendo T, Arai S, et al. Image-sensor-based visible light communication for automotive applications. IEEE Commun Mag 2014;52(7):88–97.

[[26]] Khan LU. Visible light communication: applications, architecture, standardization and research challenges. Digit Commun Netw 2017;3(2):78–88.

[[27]] Uysal M, Ghassemlooy Z, Bekkali A, Kadri A, Menouar H. Visible light communication for vehicular networking: performance study of a V2V system using a measured headlamp beam pattern model. IEEE Veh Technol Mag 2015;10(4):45–53.

[[28]] Arsalan M, Hong HG, Naqvi RA, Lee MB, Kim MC, Kim DS, et al. Deep learningbased iris segmentation for iris recognition in visible light environment. Symmetry 2017;9(11):263.

[[29]] Komine T, Nakagawa M. Fundamental analysis for visible-light communication system using LED lights. IEEE Trans Consum Electron 2004;50(1):100–7.

[[30]] Benitez JM, Castro JL, Requena I. Are artificial neural networks black boxes? IEEE Trans Neural Netw 1997;8(5):1156–64.

[[31]] Zhang Q, Yu H, Barbiero M, Wang B, Gu M. Artificial neural networks enabled by nanophotonics. Light Sci Appl 2019;8:42.

[[32]] Rivenson Y, Göröcs Z, Günaydin H, Zhang Y, Wang H, Ozcan A. Deep learning microscopy. Optica 2017;4(11):1437–43.

[[33]] Li Y, Xue Y, Tian L. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media. Optica 2018;5 (10):1181–90.

[[34]] Lyu M, Wang W, Wang H, Wang H, Li G, Chen Ni, et al. Deep-learning-based ghost imaging. Sci Rep 2017;7:17865.

[[35]] Nehme E, Weiss LE, Michaeli T, Shechtman Y. Deep-STORM: superresolution single-molecule microscopy by deep learning. Optica 2018;5 (4):458–64.

[[36]] Goodman JW. Introduction to Fourier optics. Colorado: Roberts & Company Publishers; 2005.

[[37]] Barlow HB. Unsupervised learning. Neural Comput 1989;1(3):295–311.

[[38]] Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. Generative adversarial nets. In: Proceedings of the 27th International Conference on Neural Information Processing Systems; December 8–13 2014; Montreal, QC, Canada; 2014. p. 2672–80.

[[39]] Bueno J, Maktoobi S, Froehly L, Fischer I, Jacquot M, Larger L, et al. Reinforcement learning in a large-scale photonic recurrent neural network. Optica 2018;5(6):756–60.

[[40]] Naruse M, Terashima Y, Uchida A, Kim SJ. Ultrafast photonic reinforcement learning based on laser chaos. Sci Rep 2017;7:8772.