FPSMix: data augmentation strategy for point cloud classification

Abstract

Data augmentation is a widely used regularization strategy in deep neural networks to mitigate overfitting and enhance generalization. In the context of point cloud data, mixing two samples to generate new training examples has proven to be effective. In this paper, we propose a novel and effective approach called Farthest Point Sampling Mix (FPSMix) for augmenting point cloud data. Our method leverages farthest point sampling, a technique used in point cloud processing, to generate new samples by mixing points from two original point clouds. Another key innovation of our approach is the introduction of a significance-based loss function, which assigns weights to the soft labels of the mixed samples based on the classification loss of each part of the new sample that is separated from the two original point clouds. This way, our method takes into account the importance of different parts of the mixed sample during the training process, allowing the model to learn better global features. Experimental results demonstrate that our FPSMix, combined with the significance-based loss function, improves the classification accuracy of point cloud models and achieves comparable performance with state-of-the-art data augmentation methods. Moreover, our approach is complementary to techniques that focus on local features, and their combined use further enhances the classification accuracy of the baseline model.

Graphical abstract

Keywords

point cloud classification / data augmentation / loss function / point cloud understanding / point cloud analysis

Taiyan Chen received the BE degree in electronic information engineer from University of Science and Technology of China, China in 2020. He is currently pursuing the PhD degree in computer science and technology with Peking University, China. His research interests include point cloud and image processing

Xianghua Ying received the PhD degree from the Institute of Automation, Chinese Academy of Sciences, China in 2004. He was a Visiting Professor at the University of Southern California, USA from September 2007 to August 2008. He is currently a Full Professor at the Key Laboratory of Machine Perception (Ministry of Education), School of Intelligence Science and Technology, Peking University, China. His major interests include 3D reconstruction, motion analysis, and computational photography

References

[1] Qi C R, Hao S, Mo K, Guibas L J. PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2017, 77−85

[2] Qi C R, Li Y, Hao S, Guibas L J. PointNet++: deep hierarchical feature learning on point sets in a metric space. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 5105−5114

[3] Wang Y, Sun Y, Liu Z, Sarma S E, Bronstein M M, Solomon J M . Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics, 2019, 38( 5): 146

[4] Liu Y, Fan B, Xiang S, Pan C. Relation-shape convolutional neural network for point cloud analysis. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 8887−8896

[5] Thomas H, Qi C R, Deschaud J E, Marcotegui B, Goulette F, Guibas L. KPConv: flexible and deformable convolution for point clouds. In: Proceedings of IEEE/CVF International Conference on Computer Vision. 2019, 6410−6419

[6] Deng J, Dong W, Socher R, Li L J, Li K, Fei-Fei L. ImageNet: a large-scale hierarchical image database. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2009, 248−255

[7] Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J. 3D ShapeNets: a deep representation for volumetric shapes. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2015, 1912−1920

[8] Li R, Li X, Heng P A, Fu C W. PointAugment: an auto-augmentation framework for point cloud classification. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 6377−6386

[9] Zhang H, Cisse M, Dauphin Y N, Lopez-Paz D. mixup: beyond empirical risk minimization. In: Proceedings of the 6th International Conference on Learning Representations. 2018

[10] Yun S, Han D, Chun S, Oh S J, Yoo Y, Choe J. CutMix: regularization strategy to train strong classifiers with localizable features. In: Proceedings of IEEE/CVF International Conference on Computer Vision. 2019, 6022−6031

[11] Lee D, Lee J, Lee J, Lee H, Lee M, Woo S, Lee S. Regularization strategy for point cloud via rigidly mixed sample. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, 15895−15904

[12] Zhang J, Chen L, Ouyang B, Liu B, Zhu J, Chen Y, Meng Y, Wu D . PointCutMix: regularization strategy for point cloud classification. Neurocomputing, 2022, 505: 58–67

[13] Chen Y, Hu V T, Gavves E, Mensink T, Mettes P, Yang P, Snoek C G M. PointMixup: augmentation for point clouds. In: Proceedings of the 16th European Conference on Computer Vision. 2020, 330−345

[14] Ding Z, Han X, Niethammer M. VoteNet: a deep learning label fusion method for multi-atlas segmentation. In: Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention. 2019, 202−210

[15] He Y, Sun W, Huang H, Liu J, Fan H, Sun J. PVN3D: a deep point-wise 3D keypoints voting network for 6DoF pose estimation. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 11629−11638

[16] Li J, Chen B M, Lee G H. SO-Net: self-organizing network for point cloud analysis. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 9397−9406

[17] Li Y, Bu R, Sun M, Wu W, Di X, Chen B. PointCNN: convolution on X-transformed points. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, 828−838

[18] Xu Y, Fan T, Xu M, Zeng L, Qiao Y. SpiderCNN: deep learning on point sets with parameterized convolutional filters. In: Proceedings of the 15th European Conference on Computer Vision. 2018, 90−105

[19] Liu Y, Fan B, Meng G, Lu J, Xiang S, Pan C. DensePoint: learning densely contextual representation for efficient point cloud processing. In: Proceedings of IEEE/CVF International Conference on Computer Vision. 2019, 5238−5247

[20] Wang C, Samari B, Siddiqi K. Local spectral graph convolution for point set feature learning. In: Proceedings of the 15th European Conference on Computer Vision. 2018, 56−71

[21] Shen Y, Feng C, Yang Y, Tian D. Mining point cloud local structures by kernel correlation and graph pooling. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 4548−4557

[22] Liu J, Ni B, Li C, Yang J, Tian Q. Dynamic points agglomeration for hierarchical point sets learning. In: Proceedings of IEEE/CVF International Conference on Computer Vision. 2019, 7545−7554

[23] Su H, Jampani V, Sun D, Maji S, Kalogerakis E, Yang M H, Kautz J. SPLATNet: sparse lattice networks for point cloud processing. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 2530−2539

[24] Wu W, Qi Z, Fuxin L. PointConv: deep convolutional networks on 3D point clouds. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 9613−9622

[25] Mao J, Wang X, Li H. Interpolated convolutional networks for 3D point cloud understanding. In: Proceedings of IEEE/CVF International Conference on Computer Vision. 2019, 1578−1587

[26] Xu M, Ding R, Zhao H, Qi X. PAConv: position adaptive convolution with dynamic kernel assembling on point clouds. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, 3172−3181

[27] Wang H, Huang D, Wang Y . GridNet: efficiently learning deep hierarchical representation for 3D point cloud understanding. Frontiers of Computer Science, 2022, 16( 1): 161301

[28] Xiang T, Zhang C, Song Y, Yu J, Cai W. Walk in the cloud: learning curves for point clouds shape analysis. In: Proceedings of IEEE/CVF International Conference on Computer Vision. 2021, 895−904

[29] Guo M H, Cai J X, Liu Z N, Mu T J, Martin R R, Hu S M . PCT: point cloud transformer. Computational Visual Media, 2021, 7( 2): 187–199

[30] Zhao H, Jiang L, Jia J, Torr P, Koltun V. Point transformer. In: Proceedings of IEEE/CVF International Conference on Computer Vision. 2021, 16239−16248

[31] Liu S, Luo X, Fu K, Wang M, Song Z . A learnable self-supervised task for unsupervised domain adaptation on point cloud classification and segmentation. Frontiers of Computer Science, 2023, 17( 6): 176708

[32] Xian Y, Xiao J, Wang Y . A fast registration algorithm of rock point cloud based on spherical projection and feature extraction. Frontiers of Computer Science, 2019, 13( 1): 170–182

[33] Li H, Liu Y, Xiong S, Wang L . Pedestrian detection algorithm based on video sequences and laser point cloud. Frontiers of Computer Science, 2015, 9( 3): 402–414

[34] Dabouei A, Soleymani S, Taherkhani F, Nasrabadi N M. SuperMix: supervising the mixing data augmentation. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, 13789−13798

[35] Verma V, Lamb A, Beckham C, Najafi A, Mitliagkas I, Lopez-Paz D, Bengio Y. Manifold mixup: better representations by interpolating hidden states. In: Proceedings of the 36th International Conference on Machine Learning. 2019, 6438−6447

[36] Guo H, Mao Y, Zhang R. MixUp as locally linear out-of-manifold regularization. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 3714−3722

[37] Harris E, Marcu A, Painter M, Niranjan M, Prügel-Bennett A, Hare J. FMix: enhancing mixed sample data augmentation. 2020, arXiv preprint arXiv: 2002.12047

[38] Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A. Automatic differentiation in PyTorch. In: Proceedings of the 31st Conference on Neural Information Processing Systems. 2017

[39] Kingma D P, Ba J. Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations. 2015

[40] Loshchilov I, Hutter F. SGDR: stochastic gradient descent with warm restarts. In: Proceedings of the 5th International Conference on Learning Representations. 2017

Acknowledgements

This work was supported by the National Key R&D Program of China (No. 2020YFB1708002) and the National Natural Science Foundation of China (Grant Nos. 62371009 and 61971008).

Competing interests

The authors declare that they have no competing interests or financial conflicts to disclose.

RIGHTS & PERMISSIONS

2025 Higher Education Press