Scale-adaptive superpixels for medical images

Abstract

Background: Superpixel segmentation is a powerful preprocessing tool to reduce the complexity of image processing. Traditionally, size uniformity is one of the significant features of superpixels. However, in medical images, in which subjects scale varies greatly and background areas are often flat, size uniformity rarely conforms to the varying content. To obtain the fewest superpixels with retaining important details, the size of superpixel should be chosen carefully.

Methods: We propose a scale-adaptive superpixel algorithm relaxing the size-uniformity criterion for medical images, especially pathological images. A new path-based distance measure and superpixel region growing schema allow our algorithm to generate superpixels with different scales according to the complexity of image content, that is smaller (larger) superpixels in color-riching areas (flat areas).

Results: The proposed superpixel algorithm can generate superpixels with boundary adherence, insensitive to noise, and with extremely big sizes and extremely small sizes on one image. The number of superpixels is much smaller than size-uniformly superpixel algorithms while retaining more details of images.

Conclusion: With the proposed algorithm, the choice of superpixel size is automatic, which frees the user from the predicament of setting suitable superpixel size for a given application. The results on the nuclear dataset show that the proposed superpixel algorithm superior to the respective state-of-the-art algorithms on both quantitative and quantitative comparisons.

Author summary

Most superpixel algorithms help reduce the complexity of image processing by generating superpixels with uniform size and boundary adherence. For freeing the users from the predicament of setting suitable superpixel size for a given application, and taking account of the multi-scale of different tissues in medical images, we propose a scale-adaptive superpixel algorithm for medical images, generating superpixels with multi-scale according to the complexity of image content. As demonstrated by the experimental results, our method is superior to other superpixel methods for medical images segmentation.

Graphical abstract

Keywords

superpixels / scale adaptive / medical images / segmentation

References

[1] Ren, X. and Malik, J. (2003) Learning a classification model for segmentation. In: 9th IEEE International Conference on Computer Vision (ICCV 2003), pp. 10−17

[2] Bharath, H. N., Colleman, S., Sima, D. M. and Huffel, S. V. (2017) Tumor segmentation from multimodal MRI using random forest with superpixel and tensor based feature extraction. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries‒Third International Workshop, Crimi, A., Bakas, S., Kuijf, H. J., Menze, B. H. and Reyes, M. (Eds.), pp. 463−473

[3] Li, H., Wei, D., Cao, S., Ma, K., Wang, L. and Zheng, Y. (2020) Superpixel-guided label softening for medical image segmentation. In: Medical Image Computing and Computer Assisted Intervention‒MICCAI 2020‒23rd International Conference, Martel, A. L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M. A., Zhou, S. K., Racoceanu, D. and Joskowicz, L. (Eds.), pp. 227−237

[4] Huang Q., Huang, Y., Luo, Y., Yuan, F. , Li, X.. Segmentation of breast ultrasound image with semantic classification of superpixels. Med. Image Anal., 2020, 61 : 101657–

[5] Farag A., Lu, L., Roth, H. R., Liu, J., Turkbey, E. , Summers, R. M.. A bottom-up approach for pancreas segmentation using cascaded superpixels and (deep) image patch labeling. IEEE Trans. Image Process., 2017, 26 : 386– 399

[6] Leblond, A. and Kauffmann, C. (2016) Ultrafast superpixel segmentation of large 3d medical datasets. In: Medical Imaging 2016: Biomedical Applications in Molecular, Structural, and Functional Imaging, Gimi, B. and Kr’ol, A. (Eds.), pp. 97881N

[7] Tian Z., Liu, L., Zhang, Z. , Fei, B.. Superpixel-based segmentation for 3d prostate MR images. IEEE Trans. Med. Imaging, 2016, 35 : 791– 801

[8] da Silva, G. L. F., Diniz S., P. L., Ferreira V. F., J. C., França Paiva, J. C., Silva Cavalcanti, A. A. A. Superpixel-based deep convolutional neural networks and active contour model for automatic prostate segmentation on 3D MRI scans. Med. Biol. Eng. Comput., 2020, 58 : 1947– 1964

[9] Ouyang, C., Biffi, C., Chen, C., Kart, T., Qiu, H. and Rueckert, D. (2020) Self-supervision with superpixels: Training few-shot medical image segmentation without annotation. In: Computer Vision‒ECCV 2020‒16th European Conference, Vedaldi, A., Bischof, H., Brox, T. and Frahm, J. (Eds.), pp. 762–780

[10] Nguyen D. C. T., Benameur, S., Mignotte, M. , Lavoie, F.. Superpixel and multi-atlas based fusion entropic model for the segmentation of x-ray images. Med. Image Anal., 2018, 48 : 58– 74

[11] Achanta, R. and Su¨sstrunk, S. (2017) Superpixels and polygons using simple noniterative clustering. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, pp. 4895−4904

[12] Liu M., Tuzel, O., Ramalingam, S. , Chellappa, R.. ( 2011) Entropy rate superpixel segmentation. In: The 24th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2011, pp. 2097− 2104

[13] Janowczyk A. , Madabhushi, A.. Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases. J. Pathol. Inform., 2016, 7 : 29–

[14] Shi J. , Malik, J.. Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 2000, 22 : 888– 905

[15] Felzenszwalb P. F. , Huttenlocher, D. P.. Efficient graph-based image segmentation. Int. J. Comput., 2004, 59 : 167– 181

[16] Achanta R., Shaji, A., Smith, K., Lucchi, A., Fua, P. , Süsstrunk, S.. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell., 2012, 34 : 2274– 2282

[17] Levinshtein A., Stere, A., Kutulakos, K. N., Fleet, D. J., Dickinson, S. J. , Siddiqi, K.. TurboPixels: fast superpixels using geometric flows. IEEE Trans. Pattern Anal. Mach. Intell., 2009, 31 : 2290– 2297

[18] Zhou Y., Pan, X., Wang, W., Yin, Y. , Zhang, C.. Superpixels by bilateral geodesic distance. IEEE Trans. Circ. Syst. Video Tech., 2017, 27 : 2281– 2293

[19] den Bergh, M. V., Boix, X., Roig, G., de Capitani, B. and Gool, L. V. (2012) SEEDS: superpixels extracted via energy-driven sampling. In: Computer Vision‒ECCV 2012‒12th European Conference on Computer Vision, Fitzgibbon, A. W., Lazebnik, S., Perona, P., Sato, Y. and Schmid, C. (Eds.), pp. 13−26

[20] Vincent L. , Soille, P.. Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Trans. Pattern Anal. Mach. Intell., 1991, 13 : 583– 598

[21] Comaniciu D. , Meer, P.. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell., 2002, 24 : 603– 619

[22] Jampani, V., Sun, D., Liu, M., Yang, M. and Kautz, J. (2018) Superpixel sampling networks. In: Computer Vision‒ECCV 2018‒15th European Conference, Ferrari, V., Hebert, M., Sminchisescu, C. and Weiss, Y. (Eds.), pp. 363−380

[23] Yang, F., Sun, Q., Jin, H. and Zhou, Z. (2020) Superpixel segmentation with fully convolutional networks. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, pp. 13961−13970

[24] Stutz D., Hermans, A. , Leibe, B.. Superpixels: An evaluation of the stateof-the-art. Comput. Vis. Image Underst., 2018, 166 : 1– 27

[25] Achanta, R., Marquez Neila, P., Fua, P. and Süsstrunk, S. (2018) Scale-adaptive superpixels. In: 26th Color and Imaging Conference Final Program and Proceedings, pp. 1−6

[26] Uziel, R., Ronen, M. and Freifeld, O. (2019) Bayesian adaptive superpixel segmentation. In: 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, pp. 8469−8478

[27] Bauchet, J. and Lafarge, F. KIPPI: kinetic polygonal partitioning of images. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3146−3154

[28] Ma D., Zhou, Y., Xin, S. , Wang, W.. Convex and compact superpixels by edge-constrained centroidal power diagram. IEEE Trans. Image Process., 2021, 30 : 1825– 1839

[29] Kumar N., Verma, R., Sharma, S., Bhargava, S., Vahadane, A. , Sethi, A.. A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans. Med. Imaging, 2017, 36 : 1550– 1560

[30] Dataset. https://tianchi.aliyun.com/competition/, Accessed: January 1, 2021

[31] Shi F., Yap, P. T., Wu, G., Jia, H., Gilmore, J. H., Lin, W. , Shen, D.. Infant brain atlases from neonates to 1- and 2-year-olds. PLoS One, 2011, 6 : e18746–

[32] Wang J. , Wang, X.. VCells: simple and efficient superpixels using Edge-Weighted Centroidal Voronoi Tessellations. IEEE Trans. Pattern Anal. Mach. Intell., 2012, 34 : 1241– 1247

[33] Neubert P. , Protzel, P.. (2012) Superpixel benchmark and comparison. In: Proc. Forum Bildverarbeitung, Vol. 6

[34] Nowozin S., Gehler, P. V. , Lampert, C. H.. ( 2010) On parameter learning in crfbased approaches to object class image segmentation. In: Computer Vision‒ECCV 2010‒11th European Conference on Computer Vision, Daniilidis, K., Maragos, P. and Paragios N. (Eds.), pp. 98− 98

ACKNOWLEDGEMENTS

The work was supported by the NSFC-Zhejiang Joint Fund of the Integration of Informatization and Industrialization (No. U1909210), the National Natural Science Foundation of China (No. 61772312), and the Fundamental Research Funds of Shandong University (No. 2018JC030).

COMPLIANCE WITH ETHICS GUIDELINES

The authors Limin Sun, Dongyang Ma and Yuanfeng Zhou declare that they have no conflict of interest or financial conflicts to disclose. All procedures performed in studies involving animals were in accordance with the ethical standards of the institution or practice at which the studies were conducted, and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

OPEN ACCESS

This article is licensed by the CC By under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

RIGHTS & PERMISSIONS

2021 The Authors 2021. Published by Higher Education Press.