Deep learning-based semantic segmentation of human features in bath scrubbing robots

Abstract

With the rise in the aging population, an increase in the number of semidisabled elderly individuals has been noted, leading to notable challenges in medical and healthcare, exacerbated by a shortage of nursing staff. This study aims to enhance the human feature recognition capabilities of bath scrubbing robots operating in a water fog environment. The investigation focuses on semantic segmentation of human features using deep learning methodologies. Initially, 3D point cloud data of human bodies with varying sizes are gathered through light detection and ranging to establish human models. Subsequently, a hybrid filtering algorithm was employed to address the impact of the water fog environment on the modeling and extraction of human regions. Finally, the network is refined by integrating the spatial feature extraction module and the channel attention module based on PointNet. The results indicate that the algorithm adeptly identifies feature information for 3D human models of diverse body sizes, achieving an overall accuracy of 95.7%. This represents a 4.5% improvement compared with the PointNet network and a 2.5% enhancement over mean intersection over union. In conclusion, this study substantially augments the human feature segmentation capabilities, facilitating effective collaboration with bath scrubbing robots for caregiving tasks, thereby possessing significant engineering application value.

Keywords

3D point cloud / Human model / LiDAR / Semantic segmentation

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgment

This work was supported by National Key R&D Program of China (2020YFC2007700).

References

[1] Panpan Guo, et al., A qualitative study of the interaction experiences between family caregivers and community nurses for disabled elderly people at home, BMC Geriatr. 23 (2023) 1, http://dx.doi.org/10.1186/s12877-023-03917.

[2] Zhou Zongwei, et al., Unet++: Anested u-net architecture for medical image segmentation, in: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Vol. 11045, 2018, http://dx.doi.org/10.1007/978-3-030-00889-5_1.

[3] Jé Simon, et al., The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation, in: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017, http://dx.doi.org/10.48550/arXiv.1611.09326, PP.

[4] Gao Huang, et al., Densely connected convolutional networks, in: Pro-ceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017, http://dx.doi.org/10.48550/arXiv.1608.06993, PP.

[5] Khened Mahendra, et al., Fully convolutional multi-scale residual DenseNets for cardiac segmentation and automated cardiac diagnosis using ensemble of classifiers, Med. Image Anal. 51 (2019) http://dx.doi.org/10.1016/j.media.2018.10.004.

[6] Huiying Li, et al., Color space transformation and multi-class weighted loss for adhesive white blood cell segmentation, IEEE Access 8 (2020) http://dx.doi.org/10.1109/ACCESS.2020.2970485.

[7] David Francis Maune, Digital Elevation Model Technologies and Applica-tions: The DEM Users Manual, The American Society for Photogrammetry & Remote Sensing, Bethesda, MD, USA, 2007, pp. 199-424.

[8] Shan Jie, Charles K. Toth,Topographic Laser Ranging and Scanning: Principles and Processing,CRC Press, Boca Raton, FL, USA, 2009, pp. 403-573.

[9] J. Beraldin, et al., Laser Scanning Technology:In Airborne and Terrestrial Laser Scanning, CRC Press, Boca Raton, FL, USA, 2010; pp. 1-42.

[10] Feihu Zhang, et al., Deep fusionnet for point cloud semantic segmentation,in: Computer Vision-ECCV 2020: 16th European Conference, 2 2020, http: //dx.doi.org/10.1007/978-3-030-585860_38, PP.

[11] Mingtao Feng, et al., Point attention network for semantic segmentation of 3D point clouds, Pattern Recognit. 107 (2020) http://dx.doi.org/10.1016/j.patcog.2020.107446.

[12] Na Zhao, et al., Few-shot 3D point cloud semantic segmentation,in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, http://dx.doi.org/10.48550/arXiv.2006.12052, PP.

[13] Gul Sania, et al., Clustering of spatial cues by semantic segmentation for anechoic binaural source separation, Appl. Acoust. 171 (2021) http://dx.doi.org/10.1016/j.apacoust.2020.107566.

[14] Benini Sergio, et al., FASSEG: A face semantic segmentation repository for face image analysis, Data Brief 24 (2019) 103881, http://dx.doi.org/10.1016/j.dib.2019.103881.

[15] Yichen Wu, et al., Extended depth-of-field in holographic imaging using deep learning based autofocusing and phase recovery, Optica 5 (2018) http://dx.doi.org/10.1364/OPTICA_5.000704.

[16] Yang Zhang, et al., Polarnet: An improved grid representation for online lidar point clouds semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020, http://dx.doi.org/10.48550/arXiv.2003.14032, PP.

[17] Xiaoxin Mi, et al., Automatic road structure detection and vectorization using mls PointClouds, ISPRS-Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XLII-2/W13 (2019) http://dx.doi.org/10.5194/isprs-archives-XLII-2-W13-1069-2019.

[18] Paszke Adam, et al., Enet: A deep neural network architecture for realtime semantic segmentation, CoRR 1606.2147, 2016, http://dx.doi.org/10.48550/arXiv.1606.02147.

[19] Dai Angela, et al., ScanComplete: Large-scale scene completion and se-mantic segmentation for 3D scans, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, http://dx.doi.org/10.48550/arXiv.1712.10215, PP.

[20] Weiwei Sun, Ruisheng Wang, Fully convolutional networks for semantic segmentation of very high resolution remotely sensed images combined with DSM, IEEE Geosci. Remote Sens. Lett. 15 (2018) 3, http://dx.doi.org/10.1109/LGRS.2018.2795531.

[21] Felix Järemo Lawin, et al., Deep projective 3D semantic segmentation,in: Proceedings of the 17th International Conference on Computer Analysis of Images and Patterns, 2017, http://dx.doi.org/10.1007/978-3-319-64689-3_8, PP.

[22] A. Boulch, et al., Unstructured point cloud semantic labeling using deep segmentation networks,3DOR@ Eurographics , 3(2017) http://dx.doi.org/10.2312/3dor.20171047.

[23] Audebert Nicolas, et al., Semantic segmentation of earth observation data using multimodal and multi-scale deep networks,in: Proceedings of the Asian Conference on Computer Vision, 2016, http://dx.doi.org/10.1007/978-3-319-54181-5_12, PP.

[24] Tatarchenko Maxim, et al., Tangent convolutions for dense prediction in 3D, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, http://dx.doi.org/10.48550/arXiv.1807.02443, PP.

[25] Charles R. Qi, et al., PointNet: Deep learning on point sets for 3D classification and segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, http://dx.doi.org/10.48550/arXiv.1612.00593, PP.

[26] Charles Ruizhongtai Qi, et al., PointNet++: Deep hierarchical feature learn-ing on point sets in a metric space, in: Proceedings of the Advances in Neural Information Processing Systems, 2017, http://dx.doi.org/10.48550/arXiv.1706.02413, PP.

[27] Thomas Hugues, et al., Kpconv: Flexible and deformable convolution for point clouds, in: Proceedings of the IEEE International Conference on Computer Vision, ICCV, 2019, http://dx.doi.org/10.48550/arXiv.1904.08889, PP.

[28] Zongyue Zhao, et al., DAR-Net: Dynamic aggregation network for semantic scene segmentation, IEICE Trans. Fundam. Electron. Commun. Comput. Sci. PP (2019) http://dx.doi.org/10.48550/arXiv.1907.12022.

[29] Li Jinyu, Luo Chenxu, Yang Xiaodong, PillarNeXt: Rethinking network designs for 3D object detection in LiDAR point clouds, Comput. Sci. Comput. Vis. Pattern Recognit. (CVPR) PP (2023) http://dx.doi.org/10.48550/arXiv.2305.04925.

[30] He Chenhang, Li Ruihuang, Li Shuai, et al., Voxel set transformer: A set-to-set approach to 3D object detection from point clouds, Comput. Sci. Comput. Vis. Pattern Recognit. (CVPR) PP (2022) http://dx.doi.org/10.48550/arXiv.2203.10314.

[31] Hu Tao, Xu Xiaogang, Chu Ruihang, Jia Jiaya, TriVol: Point cloud rendering via triple volumes, Comput. Sci. Comput. Vis. Pattern Recognit.TriVol: Point cloud rendering via triple volumes Comput. Sci. Comput. Vis. Pattern Recognit.(CVPR) PP (2023) http://dx.doi.org/10.48550/arXiv.2303.16485.

[32] Li Li, Hubert P.H. Shum, Toby P. Breckon, Less is more: Reducing task and model complexity for 3D point cloud semantic segmentation, Comput. Sci. Comput. Vis. Pattern Recogniti. (CVPR) PP (2023) http://dx.doi.org/10.48550/arXiv.2303.11203.

[33] Xu Yuexuan, Guo Xin, et al., Lmpedance iterative learning backstepping control for output-constrained multisection continuum arms based on PMA, Sci. Micromach. (2022) http://dx.doi.org/10.3390/mi13091532.