4DoF Rat-SLAM with Memristive Spiking Neural Networks for UAVs Navigation System

Abstract

Unmanned Aerial Vehicles (UAVs) are versatile platforms with potential applications in precision agriculture, disaster management, and more. A core need across these applications is a navigation system that accurately estimates location based on environmental perception. Commercial UAVs use multiple onboard sensors whose fused data improves localization accuracy. The bioinspired Rat-Simultaneous Localization and Mapping (Rat-SLAM) system, is a promising alternative to be explored to tackle the localization and mapping problem of UAVs. Its cognitive capabilities, semi-metric map construction, and loop closure make it attractive for localization in complex environments. This work presents an improved Rat-SLAM algorithm for UAVs, focusing on three innovations. First, Spiking Neural Networks (SNNs) are incorporated into Rat-SLAM’s core modules to emulate biological processing with greater efficiency. Second, Neuromorphic Computing models the neurons of the SNNs, assessing the feasibility of implementing SNNs on specialized hardware to reduce software processing, a key advantage for UAVs with limited onboard resources. Third, SNNs are developed based on the Memristive Leaky Integrate-and-Fire model, integrating memristors into artificial neurons to leverage their low power and memory properties. Our approach was evaluated through trajectory simulations using the Hector Quadrotor UAV in the Gazebo environment within the Robot Operating System, yielding valuable insights and guiding future research directions.

Keywords

Rat-SLAM / Memristors / Neuromorphic Computing / Neuroscience / Spiking Neural Networks / Unmanned Aerial Vehicles

Acknowledgments

Author Bernardo Manuel Pirozzo is working towards his PhD with a grant from the National Science and Technological Research Council—CONICET, Argentina. Part of the experimental equipment was acquired through the project PICT 2016 3814‐RIOMAR, from the National Agency of Science and Techological Research, Argentina.

Author Contributions

B.M.P.: conceptualization, data curation, formal analysis, investigation, methodology, software, validation, visualization, writing—original draft. G.Y.R.: formal analysis, visualization, writing—review and editing. C.R.R.: investigation, writing—review and editing. S.A.V.: conceptualization, formal analysis, validation, visualization, writing—review and editing. M.D.P.: conceptualization, formal analysis, funding acquisition, investigation, methodology, resources, software, supervision, validation, visualization, writing—review and editing. G.G.A.: conceptualization, formal analysis, funding acquisition, investigation, methodology, project administration, resources, supervision, validation, visualization, writing—review and editing.

Ethics Statement

Not applicable. This studie not involving humans or animals.

Informed Consent Statement

Not applicable. This study does not involve humans in experimentations.

Data Availability Statement

Not informed.

Funding

This research was funded by the National Agency for Scientific and Technological Research (grant PICT 2016 3814—RIOMAR), Argentina, the National University of Buenos Aires Province Center (UNCPBA), Argentina and the National Scientific and Technological Research Council (CONICET), Argentina.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

[1] Zalavadiya UB. Drones in agriculture: Mapping, monitoring, and decision-making. In Smart Agritech: Robotics, AI, and Internet of Things (IoT) in Agriculture; Wiley: Hoboken, NJ, USA, 2024; Chapter 14, p. 373.

[2] Singh N, Gupta D, Joshi M, Yadav K, Nayak S, Kumar M, et al. Application of drones technology in agriculture: A modern approach. J. Sci. Res. Rep. 2024, 30, 142-152.

[3] Restas A. Drone applications for supporting disaster management. World J. Eng. Technol. 2015, 3, 316-321.

[4] Daud SMSM, Yusof MYPM, Heo CC, Khoo LS, Singh MKC, Mahmood MS, et al. Applications of drone in disaster management: A scoping review. Sci. Justice 2022, 62, 30-42.

[5] Tuna F. Increasing drones’ combat effectiveness: An alternative analysis for integration into comprehensive military and technological systems. Uludağ Üniversitesi Fen-Edebiyat Fakültesi Sosyal Bilimler Dergisi 2024, 25, 187-204.

[6] Kim GS, Lee S, Woo T, Park S. Cooperative reinforcement learning for military drones over large-scale battlefields. IEEE Trans. Intell. Veh. 2024, 1-11. doi:10.1109/TIV.2024.3472213.

[7] Wang X, Yang Y, Chan APC, Chi H-L, Yung EHK. A regulatory framework for the use of small unmanned aircrafts (SUAs) in the construction industry. Eng. Constr. Archit. Manag. 2024, 31, 3024-3049.

[8] Yin Y, Qing L, Wang D, Cheng TCE, Ignatius J. Exact solution method for vehicle-and-drone cooperative delivery routing of blood products. Comput. Oper. Res. 2024, 164, 106559.

[9] Nguyen NL, Bingi K, Ibrahim R, Korah R, Kumar G, Prusty BR. Autonomous inspection of solar panels and wind turbines using YOLOv8 with quadrotor drones. In Proceedings of the 2024 9th International Conference on Mechatronics Engineering (ICOM), Kuala Lumpur, Malaysia, 13-14 August 2024; pp. 322-326.

[10] Zhang H. Offshore oilfield inspection planning with drone routing optimization. IEEE Access 2024, 12, 20885-20893.

[11] Macoir N, Bauwens J, Jooris B, Herbruggen BV, Rossey J, Hoebeke J, et al. UWB localization with battery-powered wireless backbone for drone-based inventory management. Sensors 2019, 19, 467.

[12] Ribeiro MI. Kalman and extended Kalman filters: Concept, derivation and properties. Inst. Syst. Robot. 2004, 43, 3736-3741.

[13] Kwok C, Fox D, Meila M. Real-time particle filters. Adv. Neural Inf. Process. Syst. 2002, 15, Available online: https://proceedings.neurips.cc/paper_files/paper/2002/file/2d2ca7eedf739ef4c3800713ec482e1a-Paper.pdf (accessed on 10 March 2024).

[14] Wang W, Li D, Yu W. Simultaneous localization and mapping embedded with particle filter algorithm. In Proceedings of the 2016 10th European Conference on Antennas and Propagation (EuCAP), Davos, Switzerland, 10-15 April 2016; pp. 1-4.

[15] Ebrahimi M, Mohammadi RK, Sharafi F. The particle filter and extended Kalman filter methods for the structural system identification considering various uncertainties. Numer. Methods Civ. Eng. 2020, 4, 42-58.

[16] Gageik N, Strohmeier M, Montenegro S. An autonomous UAV with an optical flow sensor for positioning and navigation. Int. J. Adv. Robot. Syst. 2013, 10, 341.

[17] Yang S, Scherer SA, Yi X, Zell A. Multi-camera visual SLAM for autonomous navigation of micro aerial vehicles. Robot. Auton. Syst. 2017, 93, 116-134.

[18] Bazargani H, Laganière R. Camera calibration and pose estimation from planes. IEEE Instrum. Meas. Mag. 2015, 18, 20-27.

[19] Sim R, Dudek G. Learning environmental features for pose estimation. Image Vis. Comput. 2001, 19, 733-739.

[20] Shan D, Su J, Wang X, Liu Y, Zhou T, Wu Z. VID-SLAM: Robust pose estimation with RGBD-inertial input for indoor robotic localization. Electronics 2024, 13, 318.

[21] Arafat MY, Moh S. Bio-inspired approaches for energy-efficient localization and clustering in UAV networks for monitoring wildfires in remote areas. IEEE Access 2021, 9, 18649-18669.

[22] Czy S, Szuniewicz K, Kowalczyk K, Dumalski A, Ogrodniczak M, Zieleniewicz Ł. Assessment of accuracy in unmanned aerial vehicle (UAV) pose estimation with the real-time kinematic (RTK) method on the example of DJI Matrice 300 RTK. Sensors 2023, 23, 2092.

[23] Ekaso D, Nex F, Kerle N. Accuracy assessment of real-time kinematics (RTK) measurements on unmanned aerial vehicles (UAV) for direct geo-referencing. Geo-Spat. Inf. Sci. 2020, 23, 165-181.

[24] Huang G, Du S, Wang D. GNSS techniques for real-time monitoring of landslides: A review. Satell. Navig. 2023, 4, 5.

[25] Fyhn M, Molden S, Witter MP, Moser EI, Moser M-B. Spatial representation in the entorhinal cortex. Science 2004, 305, 1258-1264.

[26] O’Keefe J, Conway DH. Hippocampal place units in the freely moving rat: Why they fire where they fire. Exp. Brain Res. 1978, 31, 573-590.

[27] Rank JB. Head-direction cells in the deep layers of dorsal presubiculum of freely moving rats. Soc. Neurosci. Abstr. 1984, 10, 599.

[28] Milford MJ, Wyeth GF. Mapping a suburb with a single camera using a biologically inspired SLAM system. IEEE Trans. Robot. 2008, 24, 1038-1053.

[29] Yu F, Shang J, Hu Y, Milford M. NeuroSLAM: A brain-inspired SLAM system for 3D environments. Biol. Cybern. 2019, 113, 515-545.

[30] Paredes-Vallés F, Scheper KYW, De Croon GCHE. Unsupervised learning of a hierarchical spiking neural network for optical flow estimation: From events to global motion perception. IEEE Trans. Pattern Anal. Mach. Intell. 2019, 42, 2051-2064.

[31] Vreeken J. Spiking Neural Networks, an Introduction; Utrecht University: Information and Computing Sciences: Utrecht, The Netherlands, 2003.

[32] Marković D, Mizrahi A, Querlioz D, Grollier J. Physics for neuromorphic computing. Nat. Rev. Phys. 2020, 2, 499-510.

[33] Gerstner W, Kistler WM. Spiking Neuron Models: Single Neurons, Populations, Plasticity; Cambridge University Press: Cambridge, UK, 2002.

[34] Chua L. Memristor—The missing circuit element. IEEE Trans. Circuit Theory 1971, 18, 507-519.

[35] Fang X, Liu D, Duan S, Wang L. Memristive LIF spiking neuron model and its application in Morse code. Front. Neurosci. 2022, 16, 853010.

[36] Cheng C, Li X, Xie L, Li L. A unmanned aerial vehicle (UAV)/unmanned ground vehicle (UGV) dynamic autonomous docking scheme in GPS-denied environments. Drones 2023, 7, 613.

[37] Yang B, Yang E, Yu L, Niu C. Adaptive extended Kalman filter-based fusion approach for high-precision UAV positioning in extremely confined environments. IEEE/ASME Trans. Mechatron. 2022, 28, 543-554.

[38] Li Y, Gao Z, Xu Q, Yang C. Comprehensive evaluations of NLOS and linearization errors on UWB positioning. Appl. Sci. 2023, 13, 6187.

[39] Mao G, Drake S, Anderson BDO. Design of an extended Kalman filter for UAV localization. In Proceedings of the 2007 Information, Decision and Control, Adelaide, SA, Australia, 12-14 February 2007; pp. 224-229.

[40] Szczepaniak J, Szlachetko B, Lower M. The influence of temporal disturbances in EKF calculations on the achieved parameters of flight control and stabilization of UAVs. Sensors 2024, 24, 3826.

[41] PX4-ECL GitHub Library. Available online: https://github.com/PX4/PX4-ECL (accessed on 10 March 2024).

[42] Mráz E, Trizuljak A, Rajchl M, Sedláček M, Štec F, Stanko J, et al. Multi-sensor fusion for robust indoor localization of industrial UAVs using particle filter. J. Electr. Eng. 2024, 75, 304-316.

[43] Chowdhury TJS, Elkin C, Devabhaktuni V, Rawat DB, Oluoch J. Advances on localization techniques for wireless sensor networks: A survey. Comput. Netw. 2016, 110, 284-305.

[44] Khalaf-Allah M. Particle filtering for three-dimensional TDOA-based positioning using four anchor nodes. Sensors 2020, 20, 4516.

[45] Rigatos GG. Nonlinear Kalman filters and particle filters for integrated navigation of unmanned aerial vehicles. Robot. Auton. Syst. 2012, 60, 978-995.

[46] Zhong JP, Fung YF. A biological inspired improvement strategy for particle filters. In Proceedings of the 2009 IEEE International Conference on Industrial Technology, Churchill, VIC, Australia, 10-13 February 2009; pp. 1-6.

[47] Li M, Li J, Cao Y, Chen G. A dynamic visual SLAM system incorporating object tracking for UAVs. Drones 2024, 8, 222.

[48] Fischler MA, Bolles RC. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 1981, 24, 381-395.

[49] Klein G, Murray D. Parallel tracking and mapping for small AR workspaces. In Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, Japan, 13-16 November 2007; pp. 225-234.

[50] Miller A, Miller B, Popov A, Stepanyan K. UAV landing based on the optical flow videonavigation. Sensors 2019, 19, 1351.

[51] Chirimuuta M. Your brain is like a computer:Function, analogy, simplification. In Neural Mechanisms: New Challenges in the Philosophy of Neuroscience; Springer: Berlin/Heidelberg, Germany; 2021; pp. 235-261.

[52] Mitchell M. Abstraction and analogy-making in artificial intelligence. Ann. N. Y. Acad. Sci. 2021, 1505, 79-101.

[53] Konar A. Artificial Intelligence and Soft Computing: Behavioral and Cognitive Modeling of the Human Brain; CRC Press: Boca Raton, FL, USA, 2018.

[54] Burger JR. Human Memory Modeled with Standard Analog and Digital Circuits: Inspiration for Man-Made Computers; John Wiley & Sons: Hoboken, NJ, USA, 2009.

[55] Danchin A, Fenton AA. From analog to digital computing: Is Homo sapiens’ brain on its way to become a Turing machine? Front. Ecol. Evol. 2022, 10, 796413.

[56] McCulloch WS, Pitts W. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 1943, 5, 115-133.

[57] Mfetoum IM, Ngoh SK, Molu RJJ, Kenfack BFN, Onguene R, Naoussi SRD, et al. A multilayer perceptron neural network approach for optimizing solar irradiance forecasting in Central Africa with meteorological insights. Sci. Rep. 2024, 14, 3572.

[58] Mottaghi H, Masoodi AR, Gandomi AH. Multiscale analysis of carbon nanotube-reinforced curved beams: A finite element approach coupled with multilayer perceptron neural network. Results Eng. 2024, 23, 102585.

[59] Alghamdi FA, Almanaseer H, Jaradat G, Jaradat A, Alsmadi MK, Jawarneh S, et al. Multilayer perceptron neural network with arithmetic optimization algorithm-based feature selection for cardiovascular disease prediction. Mach. Learn. Knowl. Extr. 2024, 6, 987-1008.

[60] Palabıyık S, Akkan T. Evaluation of water quality based on artificial intelligence: Performance of multilayer perceptron neural networks and multiple linear regression versus water quality indexes. Environ. Dev. Sustain. 2024, 1-24. doi: 10.1007/s10668-024-05075-6.

[61] Kenas F, Saadia N, Ababou A, Ababou N. Model-free based adaptive finite time control with multilayer perceptron neural network estimation for a 10 DOF lower limb exoskeleton. Int. J. Adapt. Control Signal Process. 2024, 38, 696-730.

[62] Specht DF. A general regression neural network. IEEE Trans. Neural Netw. 1991, 2, 568-576.

[63] Zhao X, Wang L, Zhang Y, Han X, Deveci M, Parmar M. A review of convolutional neural networks in computer vision. Artif. Intell. Rev. 2024, 57, 99.

[64] Chen F, Li S, Han J, Ren F, Yang Z. Review of lightweight deep convolutional neural networks. Arch. Comput. Methods Eng. 2024, 31, 1915-1937.

[65] Hochreiter S. Long short-term memory. In Neural Computation; MIT Press: Cambridge, MA, USA, 1997.

[66] Schuster M, Paliwal KK. Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 1997, 45, 2673-2681.

[67] Zhang S, Tong H, Xu J, Maciejewski R. Graph convolutional networks: A comprehensive review. Comput. Soc. Netw. 2019, 6, 1-23.

[68] Wang J, Lu S, Wang S-H, Zhang Y-D. A review on extreme learning machine. Multimed. Tools Appl. 2022, 81, 41611-41660.

[69] Hodgkin AL, Huxley AF. A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol. 1952, 117, 500-544.

[70] Izhikevich EM. Simple model of spiking neurons. IEEE Trans. Neural Netw. 2003, 14, 1569-1572.

[71] Abbott LF. Lapicque’s introduction of the integrate-and-fire model neuron (1907). Brain Res. Bull. 1999, 50, 303-304.

[72] Skocik MJ, Long LN. On the capabilities and computational costs of neuron models. IEEE Trans. Neural Netw. Learn. Syst. 2013, 25, 1474-1483.

[73] Eshraghian JK, Ward M, Neftci EO, Wang X, Lenz G, Dwivedi G, et al. Training spiking neural networks using lessons from deep learning. Proc. IEEE 2023, 111, 1016-1054.

[74] Krauhausen I, Coen C-T, Spolaor S, Gkoupidenis P, van de Burgt Y. Brain-inspired organic electronics: Merging neuromorphic computing and bioelectronics using conductive polymers. Adv. Funct. Mater. 2024, 34, 2307729.

[75] Aguirre F, Sebastian A, Le Gallo M, Song W, Wang T, Yang JJ, et al. Hardware implementation of memristor-based artificial neural networks. Nat. Commun. 2024, 15, 1974.

[76] Isah A, Bilbault J-M. Review on the basic circuit elements and memristor interpretation: Analysis, technology, and applications. J. Low Power Electron. Appl. 2022, 12, 44.

[77] Strukov DB, Snider GS, Stewart DR, Williams RS. The missing memristor found. Nature 2008, 453, 80-83.

[78] Samsonovich A, McNaughton BL. Path integration and cognitive mapping in a continuous attractor neural network model. J. Neurosci. 1997, 17, 5900-5920.

[79] Meyer J, Sendobry A, Kohlbrecher S, Klingauf U, von Stryk O. Comprehensive simulation of quadrotor UAVs using ROS and Gazebo. In Proceedings of the Simulation, Modeling, and Programming for Autonomous Robots: Third International Conference, SIMPAR 2012, Tsukuba, Japan, 5-8 November 2012; Springer: Berlin/Heidelberg, Germany, 2012.

[80] Stanford Artificial Intelligence Laboratory. Robotic Operating System. Available online: https://www.ros.org (accessed on 23 May 2018).