Methodology and workflow for road lane recognition based on millimeter-wave radar point clouds

Abstract

Accurate road lane detection is critical for intelligent transportation, but existing camera- and LiDAR-based methods face challenges: LiDAR is ex- pensive, and cameras are sensitive to lighting and weather conditions. This study proposes a method using millimeter-wave radar data, which is cost- effective and robust under various conditions. This work applys an optical flow algorithm to compute point correspondences in radar point clouds, gen- erate lane line bitmaps, and fit polygonal lane regions. The approach effec- tively handles nonlinear lanes and noisy radar data. Experiments with data from multiple radar manufacturers at different intersections and traffic sce- narios demonstrate strong robustness and reliability. The results show that the method is practical for real-time traffic management, providing a reliable alternative to traditional sensors.

Keywords

Millimeter-wave radar technology / Lane detection and tracking / Optical flow analysis / Urban planning / Traffic flow optimization

References

[1] Li L, Sheng Xi, Du B, Wang Y, Ran B. A deep fusion model based on restricted boltzmann machines for traffic accident duration prediction. Eng Appl Artif Intell, 2020, 93: 103686.

[2] Dia H, Thomas K. Development and evaluation of arterial incident detection models using fusion of simulated probe vehicle and loop detector data. Inform Fusion, 2011, 12(1): 20-27. Special Issue on Intelligent Transportation Systems

[3] Fernandes B, Alam M, Gomes V, Ferreira J, Oliveira A. Automatic accident detection with multi-modal alert system implementation for ITS. Vehicular Commun, 2016, 3: 1-11.

[4] Liu J, Rizos C, Cai BG. A hybrid integrity monitoring method using vehicular wireless communication in difficult environments for gnss. Vehicular Commun, 2020, 23: 100229.

[5] Phoon KK, Zhang W. Future of machine learning in geotech- nics. Georisk, 2023, 17(1): 7-22

[6] Zhang W, Gu X, Tang L, Yin Y, Liu D, Zhang Y-M. Application of machine learning, deep learning and optimization algo- rithms in geoengineering and geoscience: comprehensive review and future challenge. Gondwana Res, 2022, 109: 1-17.

[7] Kutsov V, Badenko V, Ivanov S, Fedotov A. Millimeter wave radar for intelligent transportation systems: a case study of multi-target problem solution. In E3S Web of Conferences, 2020, 157: 05011. EDP Sciences

[8] Xu F, Wang H, Hu B, Ren M. Road boundaries detection based on modified occupancy grid map using millimeter-wave radar. Mob Netw Appl, 2020, 25: 1496-1503.

[9] Garnett N, Cohen R, Pe'er T, Lahav R, Levi D. 3d-lanenet: End- to-end 3d multiple lane detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019.

[10] Zhang X, Chen F. Lane line edge detection based on improved adaptive canny algorithm. J Physics Conference Series, 2020, 1549(2): 022131.

[11] Cao L, Wang T, Wang D, Kangning Du, Liu Y, Chong Fu. Lane determination of vehicles based on a novel clustering algorithm for intelligent traffic monitoring. IEEE Access, 2020, 8: 63004-63017.

[12] Du Bo, Cai S, Wu C. Object tracking in satellite videos based on a multiframe optical flow tracker. IEEE J Sel Top Appl Earth Obs Remote Sens, 2019, 12(8): 3043-3055.

[13] Zhang T, Zhang H, Li Y, Nakamura Y, Zhang L. Flow- fusion: Dynamic dense rgb-d slam based on optical flow. In 2020 IEEE International Conference on Robotics and Automation (ICRA), 2020:7322-7328.

[14] Horn BK, Schunck BG. Determining optical flow. artificial intelli- gence 17. Article in Artificial Intelligence, 1981.

[15] Chen Y, Zhu D, Shi W, Zhang G, Zhang T, Zhang X, Li J. Mfcflow: A motion feature compensated multi-frame recur- rent network for optical flow estimation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023:5068–5077.

[16] Lee KC, Jhih-Sian Ou, Fang MC. Application of svd noise-reduction technique to pca based radar target recognition. Progress In Electromagnetics Research, 2008, 81: 447-459.

[17] Azarafza M, Koçkar MK, Faramarzi L. Spacing and block volume estimation in discontinuous rock masses using image processing technique: a case study. Environ Earth Sci, 2021, 80(14): 471.

[18] Wang Y. Gauss–newton method. Wiley Interdisciplinary Reviews: Computational Statistics, 2012, 4(4): 415-420.

[19] Kautsky J, Golub GH. On the calculation of jacobi matrices. Linear Algebra and its Applications, 1983, 52–53: 439-455.

[20] Chen P. Hessian matrix vs. gauss–newton hessian matrix. SIAM J Numerical Analysis, 2011, 49(4): 1417-1435.

[21] Zille P, Corpetti T, Shao L, Chen Xu. Observation model based on scale interactions for optical flow estimation. IEEE Trans Image Process, 2014, 23(8): 3281-3293.

[22] Chen Y, Wang Z, Peng Y, Zhang Z, Yu G, Sun J. Cascaded pyramid network for multi-person pose estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.

[23] Eppstein D, van Kreveld M, Speckmann B, Staals F. Improved grid map layout by point set matching. Int J Comput Geom Appl, 2015, 25(02): 101-122.

[24] Azarafza M, Ghazifard A, Akgün H, Asghari-Kaljahi E. Development of a 2d and 3d computational algorithm for discontinuity structural geometry identification by artificial intelligence based on image processing techniques. Bull Eng Geol Environ, 2019, 78(5): 3371-3383.

[25] Mark De Berg. Computational geometry: algorithms and applications. Springer Science & Business Media, 2000.

[26] WHST. Whst official website. http://www.whst.com/.

[27] Calterah. Calterah official website. https://www.calterah.com/.

[28] Continental. Continental official website. https://www.continental-corporation. cn/zh-cn/.

[29] Li S, Wu S. Low-cost millimeter wave frequency scanning based syn- thesis aperture imaging system for concealed weapon detection. IEEE Trans Microw Theory Tech, 2022, 70(7): 3688-3699.

[30] Bică M, Koivunen V. Radar waveform optimization for target parame- ter estimation in cooperative radar-communications systems. IEEE Transactions on Aerospace and Electronic Systems, 2018, 55(5): 2314-2326.

[31] Oechslin R, Wellig P, Hinrichsen S, Wieland S, Aulenbacher U, Rech K. Cognitive radar parameter optimization in a congested spectrum environment. In 2018 IEEE Radar Conference (RadarConf18), 2018:0218-0223. IEEE.

[32] Waykole S, Shiwakoti N, Stasinopoulos P. Interpolation-based framework for generation of ground truth data for testing lane detection algorithm for automated vehicle. World Electr Veh J, 2023, 14(2): 48.

[33] Baur A. Lane Model Validation: Ground Truth Generation and Lane Model Evaluation. PhD thesis, 2022.

[34] Joardar BK, Doppa JR, Pande PP, Li H, Chakrabarty K. Accured: high accuracy training of cnns on reram/gpu heterogeneous 3-d architecture. IEEE Trans Comput Aided Des Integr Circuits Syst, 2021, 40(5): 971-984.

[35] Rock J, Toth M, Meissner P, Pernkopf F. Deep interference mitigation and denoising of real-world fmcw radar signals. 2020;624-629.

[36] Zhang X, Zhang J, Luo T, Huang T, Tang Z, Chen Y, Li J, Luo D. Radar signal intrapulse modulation recognition based on a denoising-guided disentangled network. Remote Sens, 2022.

[37] Tan S, Zhang X, Wang H, Le Yu, Yanlei Du, Yin J, Bingfang Wu. A cnn-based self-supervised synthetic aperture radar image denoising approach. IEEE Trans Geosci Remote Sens, 2022, 60: 1-15.

[38] Azarafza M, Nanehkaran YA, Akgun H, Mao Y. Applica- tion of an image processing-based algorithm for river-side granular sediment gradation distribution analysis. Adv Mater Res, 2021, 10(3): 229-244

[39] Luo Y, Cui S, Li Z. Dv-3dlane: End-to-end multi-modal 3d lane detection with dual-view representation. arXiv preprint arXiv:2406.16072, 2024.

[40] Rudra N Hota, Shahanaz Syed, Subhadip Bandyopadhyay, and P Radha Krishna. A simple and efficient lane detection using clustering and weighted regression. In COMAD, 2009.

[41] Tian W, Ren X, Yu X, Wu M, Zhao W, Li Q. Vision-based mapping of lane semantics and topology for intelligent vehicles. Int J Appl Earth Obs Geoinf, 2022, 111: 102851

[42] Bai M, Mattyus G, Homayounfar N, Wang S, Lakshmikanth SK, Urtasun R. Deep multi-sensor lane detection. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018:3102-3109. IEEE.

[43] Heng L. Automatic targetless extrinsic calibration of multiple 3d lidars and radars. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020:10669-10675.

[44] Yang J, Zhang L, Lu H. Lane detection with versatile atrous- former and local semantic guidance. Pattern Recognit, 2023, 133: 109053.

RIGHTS & PERMISSIONS

The Author(s)