Decoupled deep hough voting for point cloud registration

Abstract

Estimating rigid transformation using noisy correspondences is critical to feature-based point cloud registration. Recently, a series of studies have attempted to combine traditional robust model fitting with deep learning. Among them, DHVR proposed a hough voting-based method, achieving new state-of-the-art performance. However, we find voting on rotation and translation simultaneously hinders achieving better performance. Therefore, we proposed a new hough voting-based method, which decouples rotation and translation space. Specifically, we first utilize hough voting and a neural network to estimate rotation. Then based on good initialization on rotation, we can easily obtain accurate rigid transformation. Extensive experiments on 3DMatch and 3DLoMatch datasets show that our method achieves comparable performances over the state-of-the-art methods. We further demonstrate the generalization of our method by experimenting on KITTI dataset.

Graphical abstract

Keywords

point cloud registration / robust model fitting / deep learning / hough voting

Mingzhi Yuan received the BS degrees in communication engineering from the Harbin Institute of Technology (HIT), China in 2020. He is currently a PhD student in School of Basic Medical Science of Fudan University, China. His research interests include 3D vision and medical image processing

Kexue Fu received the BS degrees in Electronic and Information Engineering from Wuhan University of Technology, China in 2014. He is currently a PhD Candidate in School of Basic Medical Science of Fudan University, China. His research interest is computer vision and image-guided intervention

Zhihao Li received the BS degrees in electronic information engineering from Wuhan University of Technology, China in 2021. He is currently a graduate student in School of Basic Medical Science of Fudan University, China. His research interests include point cloud registration and 3D vision

Manning Wang received the BS and MS degrees in power electronics and power transmission from Shanghai Jiao Tong University, China in 1999 and 2002, respectively. He received PhD in biomedical engineering from Fudan University, China in 2011. He is currently a professor of biomedical engineering in School of Basic Medical Science of Fudan University, China. His research interests include medical image processing, image-guided intervention and computer vision

References

[1] Huang X, Mei G, Zhang J, Abbas R. A comprehensive survey on point cloud registration. 2021, arXiv preprint arXiv: 2103.02690

[2] Yang J, Xian K, Wang P, Zhang Y . A performance evaluation of correspondence grouping methods for 3D rigid data matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43( 6): 1859–1874

[3] Fu K, Liu S, Luo X, Wang M. Robust point cloud registration framework based on deep graph matching. In: Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, 8889–8898

[4] Tam G K L, Cheng Z Q, Lai Y K, Langbein F C, Liu Y, Marshall D, Martin R R, Sun X F, Rosin P L . Registration of 3D point clouds and meshes: a survey from rigid to nonrigid. IEEE Transactions on Visualization and Computer Graphics, 2013, 19( 7): 1199–1217

[5] Yang H, Shi J, Carlone L . TEASER: fast and certifiable point cloud registration. IEEE Transactions on Robotics, 2021, 37( 2): 314–333

[6] Luo J, Yuan M, Fu K, Wang M, Zhang C . Deep graph matching based dense correspondence learning between non-rigid point clouds. IEEE Robotics and Automation Letters, 2022, 7( 3): 5842–5849

[7] Yuan M, Huang X, Fu K, Li Z, Wang M. Boosting 3D point cloud registration by transferring multi-modality knowledge. 2023, arXiv preprint arXiv: 2302.05210

[8] Choy C, Park J, Koltun V. Fully convolutional geometric features. In: Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. 2019, 8957–8965

[9] Wang H, Liu Y, Dong Z, Wang W. You only hypothesize once: point cloud registration with rotation-equivariant descriptors. In: Proceedings of the 30th ACM International Conference on Multimedia. 2022, 1630–1641

[10] Huang S, Gojcic Z, Usvyatsov M, Wieser A, Schindler K. PREDATOR: registration of 3D point clouds with low overlap. In: Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, 4265–4274

[11] Bai X, Luo Z, Zhou L, Fu H, Quan L, Tai C L. D3Feat: joint learning of dense detection and description of 3D local features. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 6358–6366

[12] Fischler M A, Bolles R C . Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 1981, 24( 6): 381–395

[13] Leordeanu M, Hebert M. A spectral technique for correspondence problems using pairwise constraints. In: Proceedings of the 10th IEEE International Conference on Computer Vision. 2005, 1482–1489

[14] Barath D, Matas J. Graph-cut RANSAC. In: Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 6733–6741

[15] Zhou Q Y, Park J, Koltun V. Fast global registration. In: Proceedings of the 14th European Conference on Computer Vision. 2016, 766–782

[16] Yi K M, Trulls E, Ono Y, Lepetit V, Salzmann M, Fua P. Learning to find good correspondences. In: Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 2666–2674

[17] Choy C, Dong W, Koltun V. Deep global registration. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 2511–2520

[18] Lee J, Kim S, Cho M, Park J. Deep Hough voting for robust global registration. In: Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. 2021, 15974–15983

[19] Choy C, Gwak J, Savarese S. 4D spatio-temporal ConvNets: minkowski convolutional neural networks. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 3070–3079

[20] Zeng A, Song S, Nießner M, Fisher M, Xiao J, Funkhouser T. 3DMatch: learning local geometric descriptors from RGB-D reconstructions. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017, 199–208

[21] Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. 2012, 3354–3361

[22] Wang H, Huang D, Wang Y . GridNet: efficiently learning deep hierarchical representation for 3D point cloud understanding. Frontiers of Computer Science, 2022, 16( 1): 161301

[23] Xian Y, Xiao J, Wang Y . A fast registration algorithm of rock point cloud based on spherical projection and feature extraction. Frontiers of Computer Science, 2019, 13( 1): 170–182

[24] Xiao J, Liu S, Hu L, Wang Y . Filtering method of rock points based on BP neural network and principal component analysis. Frontiers of Computer Science, 2018, 12( 6): 1149–1159

[25] Yuan M, Li Z, Jin Q, Chen X, Wang M. PointCLM: a contrastive learning-based framework for multi-instance point cloud registration. In: Proceedings of the 17th European Conference on Computer Vision. 2022, 595–611

[26] Besl P J, McKay N D. Method for registration of 3-D shapes. In: Proceedings of SPIE 1611, Sensor Fusion IV: Control Paradigms and Data Structures. 1992, 586–606

[27] Granger S, Pennec X. Multi-scale EM-ICP: a fast and robust approach for surface registration. In: Proceedings of the 7th European Conference on Computer Vision. 2002, 418–432

[28] Chetverikov D, Stepanov D, Krsek P . Robust Euclidean alignment of 3D point sets: the trimmed iterative closest point algorithm. Image and Vision Computing, 2005, 23( 3): 299–309

[29] Yang J, Li H, Campbell D, Jia Y . Go-ICP: a globally optimal solution to 3D ICP point-set registration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38( 11): 2241–2254

[30] Aoki Y, Goforth H, Srivatsan R A, Lucey S. PointNetLK: robust & efficient point cloud registration using PointNet. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 7156–7165

[31] Huang X, Mei G, Zhang J. Feature-metric registration: a fast semi-supervised approach for robust point cloud registration without correspondences. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 11363–11371

[32] Rusu R B, Blodow N, Beetz M. Fast point feature histograms (FPFH) for 3D registration. In: Proceedings of 2009 IEEE International Conference on Robotics and Automation. 2009, 3212–3217

[33] Wang Y, Solomon J M. Deep closest point: learning representations for point cloud registration. In: Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. 2019, 3522–3531

[34] Yew Z J, Lee G H. RPM-Net: robust point matching using learned features. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 11821–11830

[35] Papadopoulo T, Lourakis M I A. Estimating the Jacobian of the singular value decomposition: theory and applications. In: Proceedings of the 6th European Conference on Computer Vision. 2000, 554–570

[36] Li Y, Harada T. Lepard: learning partial point cloud matching in rigid and deformable scenes. In: Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, 5544–5554

[37] Bustos Á P, Chin T J. Guaranteed outlier removal for rotation search. In: Proceedings of 2015 IEEE International Conference on Computer Vision. 2015, 2165–2173

[38] Parra Á, Chin T J, Neumann F, Friedrich T, Katzmann M. A practical maximum clique algorithm for matching with pairwise constraints. 2019, arXiv preprint arXiv: 1902.01534

[39] Bustos Á P, Chin T J . Guaranteed outlier removal for point cloud registration with correspondences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40( 12): 2868–2882

[40] Shi J, Yang H, Carlone L. ROBIN: a graph-theoretic approach to reject outliers in robust estimation using invariants. In: Proceedings of 2021 IEEE International Conference on Robotics and Automation. 2021, 13820–13827

[41] Le H M, Do T T, Hoang T, Cheung N M. SDRSAC: semidefinite-based randomized approach for robust point cloud registration without correspondences. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 124–133

[42] Yang J, Xian K, Xiao Y, Cao Z. Performance evaluation of 3D correspondence grouping algorithms. In: Proceedings of 2017 International Conference on 3D Vision. 2017, 467–476

[43] Charles R Q, Su H, Mo K, Guibas L J. PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017, 77–85

[44] Sun L, Deng L . TriVoC: efficient voting-based consensus maximization for robust point cloud registration with extreme outlier ratios. IEEE Robotics and Automation Letters, 2022, 7( 2): 4654–4661

[45] Gower J C . Generalized procrustes analysis. Psychometrika, 1975, 40( 1): 33–51

[46] Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. In: Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. 2015, 234–241

[47] Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A. Automatic differentiation in pytorch. In: Proceedings of the 31st Conference on Neural Information Processing Systems. 2017

[48] Zhou Q Y, Park J, Koltun V. Open3D: a modern library for 3D data processing. 2018, arXiv preprint arXiv: 1801.09847

[49] Mellado N, Aiger D, Mitra N J . Super 4PCS fast global pointcloud registration via smart indexing. Computer Graphics Forum, 2014, 33( 5): 205–215

[50] Straub J, Freifeld O, Rosman G, Leonard J J, Fisher J W . The manhattan frame model—manhattan world inference in the space of surface normals. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40( 1): 235–249

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No. 62076070) and the Science and Technology Innovation Action Plan of Shanghai (No. 23S41900400).

RIGHTS & PERMISSIONS

2024 Higher Education Press