Lesion region segmentation via weakly supervised learning

Ran Yi, Rui Zeng, Yang Weng, Minjing Yu, Yu-Kun Lai, Yong-Jin Liu

PDF(6383 KB)
PDF(6383 KB)
Quant. Biol. ›› 2022, Vol. 10 ›› Issue (3) : 239-252. DOI: 10.15302/J-QB-021-0272
RESEARCH ARTICLE

Lesion region segmentation via weakly supervised learning

Author information +
History +

Abstract

Background: Image-based automatic diagnosis of field diseases can help increase crop yields and is of great importance. However, crop lesion regions tend to be scattered and of varying sizes, this along with substantial intra-class variation and small inter-class variation makes segmentation difficult.

Methods: We propose a novel end-to-end system that only requires weak supervision of image-level labels for lesion region segmentation. First, a two-branch network is designed for joint disease classification and seed region generation. The generated seed regions are then used as input to the next segmentation stage where we design to use an encoder-decoder network. Different from previous works that use an encoder in the segmentation network, the encoder-decoder network is critical for our system to successfully segment images with small and scattered regions, which is the major challenge in image-based diagnosis of field diseases. We further propose a novel weakly supervised training strategy for the encoder-decoder semantic segmentation network, making use of the extracted seed regions.

Results: Experimental results show that our system achieves better lesion region segmentation results than state of the arts. In addition to crop images, our method is also applicable to general scattered object segmentation. We demonstrate this by extending our framework to work on the PASCAL VOC dataset, which achieves comparable performance with the state-of-the-art DSRG (deep seeded region growing) method.

Conclusion: Our method not only outperforms state-of-the-art semantic segmentation methods by a large margin for the lesion segmentation task, but also shows its capability to perform well on more general tasks.

Author summary

Crop diseases seriously affect the quantity and quality of crop yields, causing huge economic losses and posing a serious threat to global food security. However, overuse of chemicals in traditional agriculture may be harmful to humans and livestock. Therefore, early diagnosis of crop diseases, which helps to avoid the heavy use of chemicals and provides feasible solutions, is much desired. We proposed a system to segment images with small and scattered regions. Experimental results show that our method not only outperforms state-of-the-art segmentation methods for the lesion segmentation task, but also shows its capability to perform well on more general tasks.

Graphical abstract

Keywords

weakly supervised learning / lesion segmentation / disease detection / semantic segmentation / agriculture

Cite this article

Download citation ▾
Ran Yi, Rui Zeng, Yang Weng, Minjing Yu, Yu-Kun Lai, Yong-Jin Liu. Lesion region segmentation via weakly supervised learning. Quant. Biol., 2022, 10(3): 239‒252 https://doi.org/10.15302/J-QB-021-0272

References

[1]
Strange, R. N. Scott, P. ( 2005). Plant disease: a threat to global food security. Annu. Rev. Phytopathol., 43 : 83– 116
CrossRef Pubmed Google scholar
[2]
Geiger, F., Bengtsson, J., Berendse, F., Weisser, W. W., Emmerson, M., Morales, M. B., Ceryngier, P., Liira, J., Tscharntke, T., Winqvist, C. . ( 2010). Persistent negative effects of pesticides on biodiversity and biological control potential on european farmland. Basic Appl. Ecol., 11 : 97– 105
CrossRef Google scholar
[3]
Aravind, K. R., Raja, P., Aniirudh, R., Mukesh, K. V., Ashiwin, R. and Vikas, G. (2018) Grape crop disease classification using transfer learning approach. In: Proc. ISMAC-CVB, pp. 1623–1633
[4]
DeChant, C., Wiesner-Hanks, T., Chen, S., Stewart, E. L., Yosinski, J., Gore, M. A., Nelson, R. J. ( 2017). Automated identification of northern leaf blight-infected maize plants from field imagery using deep learning. Phytopathology, 107 : 1426– 1432
CrossRef Pubmed Google scholar
[5]
Pound, M. P., Atkinson, J. A., Wells, D. M., Pridmore, T. P. and French, A. P. (2017) Deep learning for multi-task plant phenotyping. In: Proc. ICCV Workshops, pp. 2055–2063
[6]
Abdu, A. M., Mokji, M. M. and Sheikh, U. U. (2019) Deep learning for plant disease identification from disease region images. In: Proc. ICIRA, pp. 65–75
[7]
Mohanty, S. P., Hughes, D. P. ( 2016). Using deep learning for image-based plant disease detection. Front Plant Sci, 7 : 1419
CrossRef Pubmed Google scholar
[8]
Zabawa, L., Kicherer, A., Klingbeil, L., Milioto, A., Topfer, R., Kuhlmann, H. and Roscher, R. (2019) Detection of single grapevine berries in images using fully convolutional neural networks. In: Proc. CVPR Workshops
[9]
Zhang, S., You, Z. ( 2019). Plant disease leaf image segmentation based on superpixel clustering and EM algorithm. Neural Comput. Appl., 31 : 1225– 1232
CrossRef Google scholar
[10]
Krizhevsky, A., Sutskever, I. and Hinton, G. E. (2012) Imagenet classification with deep convolutional neural networks, In: Proc. NeurIPS, pp. 1097–1105
[11]
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V. and Rabinovich, A. (2015) Going deeper with convolutions. In: Proc. CVPR, pp. 1–9
[12]
Krause, J., Baek, K. and Lim, L. (2019) A guided multi-scale categorization of plant species in natural images. In: Proc. CVPR Workshops
[13]
Kumar, N., Belhumeur, P. N., Biswas, A., Jacobs, D. W., Kress, W. J., Lopez, I. C. and Soares, J. V. (2012) Leafsnap: A computer vision system for automatic plant species identification. In: Proc. ECCV, pp. 502–516
[14]
Fuentes, A., Yoon, S., Kim, S. C. Park, D. ( 2017). A robust deep-learning-based detector for real-time tomato plant diseases and pests recognition. Sensors (Basel), 17 : 2022
CrossRef Pubmed Google scholar
[15]
Simonyan, K. and Zisserman, A. (2015) Very deep convolutional networks for large-scale image recognition, In: Proc. ICLR
[16]
He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep residual learning for image recognition. In: Proc. CVPR, pp. 770–778
[17]
Chen, Y., Baireddy, S., Cai, E., Yang, C. and Delp, E. J. (2019) Leaf segmentation by functional modeling. In: Proc. CVPR Workshops
[18]
Phadikar, S., Sil, J. Das, A. ( 2013). Rice diseases classification using feature selection and rule generation techniques. Comput. Electron. Agric., 90 : 76– 85
CrossRef Google scholar
[19]
Johannes, A., Picon, A., Alvarez-Gila, A., Echazarra, J., Rodriguez-Vaamonde, S., Navajas, A. D. ( 2017). Automatic plant disease diagnosis using mobile capture devices, applied on a wheat use case. Comput. Electron. Agric., 138 : 200– 209
CrossRef Google scholar
[20]
Afridi, M. J., Liu, X. and McGrath, J. M. (2014) An automated system for plant-level disease rating in real fields. In: Proc. ICPR, pp. 148–153
[21]
Lin, K., Gong, L., Huang, Y., Liu, C. ( 2019). Deep learning-based segmentation and quantification of cucumber powdery mildew using convolutional neural network. Front Plant Sci., 10 : 155
CrossRef Pubmed Google scholar
[22]
Zhou, B., Khosla, A., Lapedriza, À., Oliva, A. and Torralba, A. (2015) Object detectors emerge in deep scene cnns. In: Proc. ICLR
[23]
Zhou, B., Khosla, A., Lapedriza, À., Oliva, A. and Torralba, A. 2016. Learning deep features for discriminative localization. In: Proc. CVPR, pp. 2921–2929
[24]
Yu, W., Zhu, F., Boushey, C. J. and Delp, E. J. (2017) Weakly supervised food image segmentation using class activation maps. In: Proc. ICIP, pp. 1277–1281
[25]
Bolaños, M. and Radeva, P. (2016) Simultaneous food localization and recognition. In: Proc. ICPR, pp. 3140–3145
[26]
Gondal, W. M., Kohler, J. M., Grzeszick, R., Fink, G. A. and Hirsch, M. (2017) Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images. In: Proc. ICIP, pp. 2069–2073
[27]
Kolesnikov, A. and Lampert, C. H. (2016) Seed, expand and constrain: Three principles for weakly-supervised image segmentation. In: Proc. ECCV, 695–711
[28]
Huang, Z., Wang, X., Wang, J., Liu, W. and Wang, J. (2018) Weakly-supervised semantic segmentation network with deep seeded region growing. In: Proc. CVPR, pp. 7014–7023
[29]
Wang, X., You, S., Li, X. and Ma, H. (2018) Weakly-supervised semantic segmentation by iteratively mining common object features. In: Proc. CVPR, pp. 1354–1362
[30]
Ahn, J. and Kwak, S. (2018) Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation. In: Proc. CVPR, pp. 4981–4990
[31]
Mottaghi, R., Chen, X., Liu, X., Cho, N. G., Lee, S. W., Fidler, S., Urtasun, R. and Yuille, A. (2014) The role of context for object detection and semantic segmentation in the wild. In: Proc. CVPR, pp. 891–898
[32]
krähenbühl, P. and Koltun, V. (2011) Efficient inference in fully connected crfs with gaussian edge potentials. In: Proc. NeurIPS, pp. 109–117
[33]
Lee, J., Kim, E., Lee, S., Lee, J. and Yoon, S. (2019) Ficklenet: Weakly and semi-supervised semantic image segmentation using stochastic inference. In: Proc. CVPR, pp. 5267–5276
[34]
Oquab, M., Bottou, L., Laptev, I. and Sivic, J. (2015) Is object localization for free? ‒ Weakly-supervised learning with convolutional neural networks. In: Proc. CVPR, pp. 685–694
[35]
Liu, B., Zhang, Y., He, D. ( 2018). Identification of apple leaf diseases based on deep convolutional neural networks. Symmetry (Basel), 10 : 11
CrossRef Google scholar
[36]
Chaudhry, A., Dokania, P. K. and Torr, P. H. (2017) Discovering class-specific pixels for weakly-supervised semantic segmentation. arXiv, 1707.05821
[37]
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R. B., Guadarrama, S. and Darrell, T. (2014) Caffe: Convolutional architecture for fast feature embedding. In: Proc. MM, pp. 675–678
[38]
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv, 1603.04467
[39]
Ronneberger, O., Fischer, P. and Brox, T. (2015) U-net: Convolutional networks for biomedical image segmentation. In: Proc. MICCAI, pp. 234–241
[40]
Chen, L. C., Zhu, Y., Papandreou, G., Schroff, F. and Adam, H. (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proc. ECCV, pp. 801–818
[41]
Hariharan, B., Arbeláez, P., Bourdev, L., Maji, S. and Malik, J. (2011) Semantic contours from inverse detectors. In: Proc. ICCV, pp. 991–998

ACKNOWLEDGEMENTS

This work was partially supported by the National Natural Science Foundation of China (Nos. 61725204 and 62002258) and a Grant from Science and Technology Department of Jiangsu Province, China.

COMPLIANCE WITH ETHICS GUIDELINES

The authors Ran Yi, Rui Zeng, Yang Weng, Minjing Yu, Yu-Kun Lai and Yong-Jin Liu declare that they have no conflict of interest or financial conflicts to disclose.

OPEN ACCESS

This article is licensed by the CC By under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

RIGHTS & PERMISSIONS

2021 The Author(s) 2021. Published by Higher Education Press.
AI Summary AI Mindmap
PDF(6383 KB)

Accesses

Citations

Detail

Sections
Recommended

/