UAV low-altitude obstacle detection based on the fusion of LiDAR and camera

Abstract

In this paper, aiming at the flying scene of the small unmanned aerial vehicle (UAV) in the low-altitude suburban environment, we choose the sensor configuration scheme of LiDAR and visible light camera, and design the static and dynamic obstacle detection algorithms based on sensor fusion. For static obstacles such as power lines and buildings in the low-altitude environment, the way that image-assisted verification of point clouds is used to fuse the contour information of the images and the depth information of the point clouds to obtain the location and size of static obstacles. For unknown dynamic obstacles such as rotary-wing UAVs, the IMM-UKF algorithm is designed to fuse the distance measurement information of point clouds and the high precision angle measurement information of image to achieve accurate estimation of the location and velocity of the dynamic obstacles. We build an experimental platform to verify the effectiveness of the obstacle detection algorithm in actual scenes and evaluate the relevant performance indexes.

Keywords

UAV / Obstacle detection / Sensor fusion

References

[1] Dalamagkidis K, Valavanis K P, Piegl L A., UAS safety assessment and functional requirements, On Integrating Unmanned Aircraft Systems into the National Airspace System, Springer, Dordrecht, 91-123(2012)

[2] W.L. Leong, P. Wang, S. Huang, et al., Vision-based sense and avoid with monocular vision and real-time object detection for UAVs, 2021 international conference on unmanned aircraft systems (ICUAS),1345–1354,(2021)

[3] ClarkM, KernZ, PrazenicaRJ. A vision-based proportional navigation guidance law for UAS sense and avoid. AIAA guidance, navigation, and control conference, 2015 0074

[4] ZhangZ, ZhangY, CaoY, et al.. Infrared and visible airborne targets image fusion with applications to sense and avoid. IFAC-PapersOnLine, 2020, 53(2):14742-14747

[5] WangD, LiW, LiuX, et al.. UAV environmental perception and autonomous obstacle avoidance: a deep learning and depth camera combined solution. Comput. Electron. Agric., 2020, 175: 105523

[6] MaQ, GoshiDS, ShihYC, et al.. An algorithm for power line detection and warning based on a millimeter-wave radar video. IEEE Trans. Image Process., 2011, 20(12):3534-3543

[7] LinY, HyyppäJ, JaakkolaA. Mini-UAV-borne LIDAR for fine-scale mapping. IEEE Geosci. Remote Sens. Lett., 2010, 8(3):426-430

[8] Y. Liu, Y. Zhuang, L. Wan, et al., Binocular vision-based autonomous path planning for UAVs in unknown outdoor scenes, 2018 eighth international conference on information science and technology (ICIST), 492–498(2018)

[9] CarrioA, LinY, SaripalliS, et al.. Obstacle detection system for small UAVs using ADS-B and thermal imaging. J. Intell. Robot. Syst., 2017, 88(2–4):583-595

[10] FasanoG, AccardoD, TirriAE, et al.. Radar/electro-optical data fusion for non-cooperative UAS sense and avoid. Aerosp. Sci. Technol., 2015, 46: 436-450

[11] C. Premebida, G. Monteiro, U. Nunes, et al., A lidar and vision-based approach for pedestrian and vehicle detection and tracking, 2007 IEEE intelligent transportation systems conference, 1044–1049(2007)

[12] K. Kidono, T. Naito, J. Miura, Reliable pedestrian recognition combining high-definition lidar and vision data, 2012 15th international IEEE conference on intelligent transportation systems, 1783–1788(2012)

[13] KaiC, DaZ, ZhangY. Point cloud data processing method of cavity 3D laser scanner. Acta Opt. Sin., 2013, 33(8):0812003

[14] R.G. von Gioi, J. Jakubowicz, J.-M. Morel, G. Randall, LSD: a line segment detector. Image Process, 2, 35–55 (2012)

[15] DollárP, ZitnickCL. Fast edge detection using structured forests. IEEE Trans. Pattern Anal. Mach. Intell., 2014, 37(8):1558-1570

Funding

National Natural Science Foundation of China(61876187)