A small microring array that performs large complex-valued matrix-vector multiplication

Abstract

As an important computing operation, photonic matrix–vector multiplication is widely used in photonic neutral networks and signal processing. However, conventional incoherent matrix–vector multiplication focuses on real-valued operations, which cannot work well in complex-valued neural networks and discrete Fourier transform. In this paper, we propose a systematic solution to extend the matrix computation of microring arrays from the real-valued field to the complex-valued field, and from small-scale (i.e., 4 × 4) to large-scale matrix computation (i.e., 16 × 16). Combining matrix decomposition and matrix partition, our photonic complex matrix–vector multiplier chip can support arbitrary large-scale and complex-valued matrix computation. We further demonstrate Walsh-Hardmard transform, discrete cosine transform, discrete Fourier transform, and image convolutional processing. Our scheme provides a path towards breaking the limits of complex-valued computing accelerator in conventional incoherent optical architecture. More importantly, our results reveal that an integrated photonic platform is of huge potential for large-scale, complex-valued, artificial intelligence computing and signal processing.

Graphical abstract

Keywords

Photonic matrix–vector multiplication / Complex-valued computing / Microring array / Signal/image processing

References

[1] Shi, W. , Caballero, J. , Huszar, F. , Totz, J. , Aitken, A.P. , Bishop, R. , Rueckert, D. , Wang, Z. : Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. Proc. CVPR 1874- 1883 (2016)

[2] Li, X. , Zhang, G. , Huang, H.H. , Wang, Z. , Zheng, W. : Performance analysis of GPU-based convolutional neural networks. Proc. ICPP 67- 76 (2016)

[3] Li, H. , Lin, Z. , Shen, X. , Brandt, J. , Hua, G. : A convolutional neural network cascade for face detection. Proc. CVPR 5325- 5334 (2015)

[4] Kitayama, K.I. , Notomi, M. , Naruse, M. , Inoue, K. , Kawakami, S. , Uchida, A. : Novel frontier of photonics for data processing— photonic accelerator. APL Photonics 4 (9), 090901 (2019)

[5] Krizhevsky, A. , Sutskever, I. , Hinton, G.E. : ImageNet classification with deep convolutional neural networks. Commun. ACM 60 (6), 84- 90 (2017)

[6] LeCun, Y. , Bengio, Y. , Hinton, G. : Deep learning. Nature 521 (7553), 436- 444 (2015)

[7] Xu, X. , Tan, M. , Corcoran, B. , Wu, J. , Boes, A. , Nguyen, T.G. , Chu, S.T. , Little, B.E. , Hicks, D.G. , Morandotti, R. , Mitchell, A. , Moss, D.J. : 11 TOPS photonic convolutional accelerator for optical neural networks. Nature 589 (7840), 44- 51 (2021)

[8] Wu, C. , Yu, H. , Lee, S. , Peng, R. , Takeuchi, I. , Li, M. : Programmable phase-change metasurfaces on waveguides for multimode photonic convolutional neural network. Nat. Commun. 12 (1), 96 (2021)

[9] Feldmann, J. , Youngblood, N. , Karpov, M. , Gehring, H. , Li, X. , Stappers, M. , Le Gallo, M. , Fu, X. , Lukashchuk, A. , Raja, A.S. , Liu, J. , Wright, C.D. , Sebastian, A. , Kippenberg, T.J. , Pernice, W.H.P. , Bhaskaran, H. : Parallel convolutional processing using an integrated photonic tensor core. Nature 589 (7840), 52- 58 (2021)

[10] Ríos, C. , Youngblood, N. , Cheng, Z. , Le Gallo, M. , Pernice, W.H.P. , Wright, C.D. , Sebastian, A. , Bhaskaran, H. : In-memory computing on a photonic platform. Sci. Adv. 5 (2), 5759 (2019)

[11] Feldmann, J. , Youngblood, N. , Wright, C.D. , Bhaskaran, H. , Pernice, W.H.P. : All-optical spiking neurosynaptic networks with self-learning capabilities. Nature 569 (7755), 208- 214 (2019)

[12] Lin, X. , Rivenson, Y. , Yardimci, N.T. , Veli, M. , Luo, Y. , Jarrahi, M. , Ozcan, A. : All-optical machine learning using diffractive deep neural networks. Science 361 (6406), 1004- 1008 (2018)

[13] Zhou, T. , Lin, X. , Wu, J. , Chen, Y. , Xie, H. , Li, Y. , Fan, J. , Wu, H. , Fang, L. , Dai, Q. : Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat. Photonics 15 (5), 367- 373 (2021)

[14] Zhu, W. , Zhang, L. , Lu, Y. , Zhou, P. , Yang, L. : Design and experimental verification for optical module of optical vector-matrix multiplier. Appl. Opt. 52 (18), 4412- 4418 (2013)

[15] Habiby, S.F. , Collins Jr, S.A. : Implementation of a fast digital optical matrix-vector multiplier using a holographic look-up table and residue arithmetic. Appl. Opt. 26 (21), 4639- 4652 (1987)

[16] Bocker, R.P. , Clayton, S.R. , Bromley, K. : Electrooptical matrix multiplication using the twos complement arithmetic for improved accuracy. Appl. Opt. 22 (13), 2019 (1983)

[17] Goodman, J.W. , Dias, A.R. , Woody, L.M. : Fully parallel, highspeed incoherent optical method for performing discrete Fourier transforms. Opt. Lett. 2 (1), 1- 3 (1978)

[18] Hong, J. , Yeh, P. : Photorefractive parallel matrix-matrix multiplier. Opt. Lett. 16 (17), 1343- 1345 (1991)

[19] Cartwright, S. : New optical matrix-vector multiplier. Appl. Opt. 23 (11), 1683- 1684 (1984)

[20] Athale, R.A. , Collins, W.C. : Optical matrix-matrix multiplier based on outer product decomposition. Appl. Opt. 21 (12), 2089- 2090 (1982)

[21] Mukhopadhyay, S. , Das, D.N. , Das, P.P. , Ghosh, P. : Implementation of all-optical digital matrix multiplication scheme with nonlinear material. Opt. Eng. (Redondo Beach, Calif.) 40 (9), 1998- 2002 (2001)

[22] Liu, B. , Liu, L.R. , Shao, L. , Chen, H.Q. : Matrix-vector multiplication in a photorefractive crystal. Opt. Commun. 146 (1-6), 34- 38 (1998)

[23] Gu, C. , Campbell, S. , Yeh, P. : Matrix-matrix multiplication by using grating degeneracy in photorefractive media. Opt. Lett. 18 (2), 146- 148 (1993)

[24] Nitta, T. : Orthogonality of decision boundaries in complexvalued neural networks. Neural Comput. 16 (1), 73- 97 (2004)

[25] Zhou, H. , Zhao, Y. , Xu, G. , Wang, X. , Tan, Z. , Dong, J. , Zhang, X. : Chip-scale optical matrix computation for pagerank algorithm. IEEE J. Sel. Top. Quantum Electron. 26 (2), 1- 10 (2020)

[26] Bogaerts, W. , Pérez, D. , Capmany, J. , Miller, D.A.B. , Poon, J. , Englund, D. , Morichetti, F. , Melloni, A. : Programmable photonic circuits. Nature 586 (7828), 207- 216 (2020)

[27] Clements, W.R. , Humphreys, P.C. , Metcalf, B.J. , Kolthammer, W.S. , Walsmley, I.A. : Optimal design for universal multiport interferometers. Optica 3 (12), 1460- 1465 (2016)

[28] Miller, D.A.B. : Self-configuring universal linear optical component. Photonics Res. 1 (1), 1- 15 (2013)

[29] Mennea, P.L. , Clements, W.R. , Smith, D.H. , Gates, J.C. , Metcalf, B.J. , Bannerman, R.H.S. , Burgwal, R. , Renema, J.J. , Kolthammer, W.S. , Walmsley, I.A. , Smith, P.G.R. : Modular linear optical circuits. Optica 5 (9), 1087- 1090 (2018)

[30] Carolan, J. , Harrold, C. , Sparrow, C. , Martín-López, E. , Russell, N.J. , Silverstone, J.W. , Shadbolt, P.J. , Matsuda, N. , Oguma, M. , Itoh, M. , Marshall, G.D. , Thompson, M.G. , Matthews, J.C.F. , Hashimoto, T. , O’Brien, J.L. , Laing, A. : Universal linear optics. Science 349 (6249), 711- 716 (2015)

[31] Zhou, H. , Zhao, Y. , Wang, X. , Gao, D. , Dong, J. , Zhang, X. : Self-configuring and reconfigurable silicon photonic signal processor. ACS Photonics 7 (3), 792- 799 (2020)

[32] Annoni, A. , Guglielmi, E. , Carminati, M. , Ferrari, G. , Sampietro, M. , Miller, D.A.B. , Melloni, A. , Morichetti, F. : Unscrambling light-automatically undoing strong mixing between modes. Light Sci Appl. 6 (12), e17110 (2017)

[33] Zhou, H. , Zhao, Y. , Wei, Y. , Li, F. , Dong, J. , Zhang, X. : Allin-one silicon photonic polarization processor. Nanophotonics 8 (12), 2257- 2267 (2019)

[34] Shen, Y. , Harris, N.C. , Skirlo, S. , Prabhu, M. , Baehr-Jones, T. , Hochberg, M. , Sun, X. , Zhao, S. , Larochelle, H. , Englund, D. , Soljačić, M. : Deep learning with coherent nanophotonic circuits. Nat. Photonics 11 (7), 441- 446 (2017)

[35] Tait, A.N. , de Lima, T.F. , Zhou, E. , Wu, A.X. , Nahmias, M.A. , Shastri, B.J. , Prucnal, P.R. : Neuromorphic photonic networks using silicon photonic weight banks. Sci. Rep. 7 (1), 7430 (2017)

[36] Yang, L. , Zhang, L. , Ji, R. : On-chip optical matrix-vector multiplier. Optics and Photonics for Information Processing Vii (2013)

[37] Miscuglio, M. , Sorger, V.J. : Photonic tensor cores for machine learning. Appl. Phys. Rev. 7 (3), 031404 (2020)

[38] Zhao, Y. , Wang, X. , Gao, D. , Dong, J. , Zhang, X. : On-chip programmable pulse processor employing cascaded MZI-MRR structure. Front. Optoelectron. 12 (2), 148- 156 (2019)

[39] Roy, A.B. , Dey, D. , Mohanty, B. , Banerjee, D. : Comparison of FFT, DCT, DWT, WHT compression techniques on electrocardiogram and photoplethysmography signals. IJCA Special Issue on International Conference on Computing, Communication and Sensor Network CCSN, 2012. 6- 11

[40] Rahardja, S. , Ser, W. , Lin, Z.N. : UCHT-based complex sequences for asynchronous CDMA system. IEEE Trans. Commun. 51 (4), 618- 626 (2003)

[41] Andrushia, A.D. , Thangarjan, R. : Saliency-based image compression using walsh-hadamard transform (WHT), pp. 21-42. Springer, Biologically rationalized computing techniques for image processing applications (2018)

[42] Strang, G. : The discrete cosine transform. SIAM Rev. 41 (1), 135- 147 (1999)

[43] Oppenheim A.V. , Schafer, R. W. , Buck, J. R. : Discrete-TimeSignal Processing. Norwood: Pearson Education India (1999)

RIGHTS & PERMISSIONS

The Author(s)