Novel neuromorphic architectures based on crossbar arrays of (Co-Fe-B)_x(LiNbO₃)_100−x nanocomposite memristors

Abstract

Memristor-based neuromorphic computing systems (NСSs) provide a fast, high computational and energy efficient approach to neural network (NN) training and solving cognitive problems (pattern recognition, big data processing, prediction, etc.) [1]. Memristors could be organized in large crossbar arrays to perform vector-matrix multiplication (VMM) in a natural one-step way by the weighted electrical current summation (according to the Ohm’s and Kirchhoff’s laws) [1]. In contrast, being the most massively parallel operation in NN learning and inference, VMM is extremely time- and energy-expensive in traditional von Neumann architectures. Owing to this difference, memristor-based NCSs are of high interest. Memristors have already been successfully implemented for diverse NCS realizations, and such schemes as multi-layer perceptron (MLP) [2], long short-term memory and others have been demonstrated. Most of these NCSs are usually trained by various types of gradient descent learning algorithm, the hardware realization of which is challenging due to unreliable cycle-to-cycle (c2c) and device-to-device (d2d) variations of memristive devices. Several approaches have been proposed to partially mitigate these problems, including reservoir computing [3] and fine feature engineering [4]. The general idea of such approaches is to reduce the number of required weights (i.e. memristors) compared with fully connected NNs. In this respect, such novel architectures as convolutional NN (CNN) and MLP-mixer are of high interest as they provide significant weight reduction without classification efficiency drop. Although CNN based on memristors was already demonstrated, different aspects of its realization (such as hybrid hardware-software co-design) have yet to be studied. MLP-mixer was realized only in software. Therefore, in this work we have studied the possibility of hardware realization of CNN and MLP-mixer networks based on crossbar arrays of memristors. For this purpose, we studied (Co-Fe-B)_x(LiNbO₃)_100−x nanocomposite (CFB-LNO NC) memristors, which operate through a multifilamentary resistive switching (RS) mechanism, demonstrate high endurance, long retention and possess multilevel RS [5].

Crossbar array of memristors was fabricated using laser photolithography for patterning electrode buses and ion-beam sputtering on the original facility for active layer deposition (~10 nm thick LiNbO₃ and ~290 nm thick CFB-LNO NC with x ≈10–25 at.%). Details of the fabrication process could be found elsewhere [5].

I-V curves of the fabricated memristors showed small c2c and d2d variations, plasticity with 16 different resistive states and endurance of more than 10⁵ cycles. Using the nanocomposite based crossbar arrays, we implemented a hybrid CNN, consisting of a hardware feature extractor with one/two kernels and a software classifier. Additionally, we have demonstrated in simulation that the usage of the memristors under study in the accurately adapted MLP-Mixer architecture results in high classification accuracy that is resilient to memristive variations and stuck devices.

Keywords

memristor / crossbar array / artificial synapse / neuromorphic computing / convolutional neural networks / MLP-mixer

References

[1] Xia Q, Yang JJ. Memristive crossbar arrays for brain-inspired computing. Nat Mater. 2019;18(4):309–323. Corrected and republished from: Nat Mater. 2019;18(5):518. doi: 10.1038/s41563-019-0291-x

[2] Xia Q., Yang J.J. Memristive crossbar arrays for brain-inspired computing // Nat Mater. 2019. Vol. 18, N 4. P. 309–323. Corrected and republished from: Nat Mater. 2019. Vol. 18, 5. P. 518. doi: 10.1038/s41563-019-0291-x

[3] Shchanikov S, Zuev A, Bordanov I, et al. Designing a bidirectional, adaptive neural interface incorporating machine learning capabilities and memristor-enhanced hardware. Chaos, Solitons and Fractals. 2021;142:110504. doi: 10.1016/j.chaos.2020.110504

[4] Shchanikov S., Zuev A., Bordanov I., et al. Designing a bidirectional, adaptive neural interface incorporating machine learning capabilities and memristor-enhanced hardware // Chaos, Solitons and Fractals. 2021. Vol. 142. P. 110504. doi: 10.1016/j.chaos.2020.110504

[5] Matsukatova AN, Prudnikov NV, Kulagin VA, et al. Combination of organic-based reservoir computing and spiking neuromorphic systems for a robust and efficient pattern classification. Adv Intell Syst. 2023. Vol. 5, N 6. P. 2200407. doi: 10.1002/aisy.202200407

[6] Matsukatova A.N., Prudnikov N.V., Kulagin V.A., et al. Combination of organic-based reservoir computing and spiking neuromorphic systems for a robust and efficient pattern classification // Adv Intell Syst. 2023. Vol. 5, N 6. P. 2200407. doi: 10.1002/aisy.202200407

[7] Matsukatova AN, Vdovichenko AYu, Patsaev TD, et al. Scalable nanocomposite parylene-based memristors: multifilamentary resistive switching and neuromorphic applications. Nano Res. 2023;16:3207–3214. doi: 10.1007/s12274-022-5027-6

[8] Matsukatova A.N., Vdovichenko A.Yu., Patsaev T.D., et al. Scalable nanocomposite parylene-based memristors: multifilamentary resistive switching and neuromorphic applications // Nano Res. 2023. Vol. 16. P. 3207–3214. doi: 10.1007/s12274-022-5027-6

[9] Ilyasov AI, Nikiruy KE, Emelyanov AV, et al. Arrays of nanocomposite crossbar memristors for the implementation of formal and spiking neuromorphic systems. Nanotechnol Russia. 2022;17:118–125. doi: 10.1134/S2635167622010050

[10] Ilyasov A.I., Nikiruy K.E., Emelyanov A.V., et al. Arrays of nanocomposite crossbar memristors for the implementation of formal and spiking neuromorphic systems // Nanotechnol Russia. 2022. Vol. 17. P. 118–125. doi: 10.1134/S2635167622010050

RIGHTS & PERMISSIONS

Eco-Vector