Lesion region segmentation via weakly supervised learning

Abstract

Background: Image-based automatic diagnosis of field diseases can help increase crop yields and is of great importance. However, crop lesion regions tend to be scattered and of varying sizes, this along with substantial intra-class variation and small inter-class variation makes segmentation difficult.

Methods: We propose a novel end-to-end system that only requires weak supervision of image-level labels for lesion region segmentation. First, a two-branch network is designed for joint disease classification and seed region generation. The generated seed regions are then used as input to the next segmentation stage where we design to use an encoder-decoder network. Different from previous works that use an encoder in the segmentation network, the encoder-decoder network is critical for our system to successfully segment images with small and scattered regions, which is the major challenge in image-based diagnosis of field diseases. We further propose a novel weakly supervised training strategy for the encoder-decoder semantic segmentation network, making use of the extracted seed regions.

Results: Experimental results show that our system achieves better lesion region segmentation results than state of the arts. In addition to crop images, our method is also applicable to general scattered object segmentation. We demonstrate this by extending our framework to work on the PASCAL VOC dataset, which achieves comparable performance with the state-of-the-art DSRG (deep seeded region growing) method.

Conclusion: Our method not only outperforms state-of-the-art semantic segmentation methods by a large margin for the lesion segmentation task, but also shows its capability to perform well on more general tasks.

Graphical abstract

Keywords

weakly supervised learning / lesion segmentation / disease detection / semantic segmentation / agriculture

References

[1] Strange, R. N. Scott, P. ( 2005). Plant disease: a threat to global food security. Annu. Rev. Phytopathol., 43 : 83– 116

[2] Geiger, F., Bengtsson, J., Berendse, F., Weisser, W. W., Emmerson, M., Morales, M. B., Ceryngier, P., Liira, J., Tscharntke, T., Winqvist, C. . ( 2010). Persistent negative effects of pesticides on biodiversity and biological control potential on european farmland. Basic Appl. Ecol., 11 : 97– 105

[3] Aravind, K. R., Raja, P., Aniirudh, R., Mukesh, K. V., Ashiwin, R. and Vikas, G. (2018) Grape crop disease classification using transfer learning approach. In: Proc. ISMAC-CVB, pp. 1623–1633

[4] DeChant, C., Wiesner-Hanks, T., Chen, S., Stewart, E. L., Yosinski, J., Gore, M. A., Nelson, R. J. ( 2017). Automated identification of northern leaf blight-infected maize plants from field imagery using deep learning. Phytopathology, 107 : 1426– 1432

[5] Pound, M. P., Atkinson, J. A., Wells, D. M., Pridmore, T. P. and French, A. P. (2017) Deep learning for multi-task plant phenotyping. In: Proc. ICCV Workshops, pp. 2055–2063

[6] Abdu, A. M., Mokji, M. M. and Sheikh, U. U. (2019) Deep learning for plant disease identification from disease region images. In: Proc. ICIRA, pp. 65–75

[7] Mohanty, S. P., Hughes, D. P. ( 2016). Using deep learning for image-based plant disease detection. Front Plant Sci, 7 : 1419

[8] Zabawa, L., Kicherer, A., Klingbeil, L., Milioto, A., Topfer, R., Kuhlmann, H. and Roscher, R. (2019) Detection of single grapevine berries in images using fully convolutional neural networks. In: Proc. CVPR Workshops

[9] Zhang, S., You, Z. ( 2019). Plant disease leaf image segmentation based on superpixel clustering and EM algorithm. Neural Comput. Appl., 31 : 1225– 1232

[10] Krizhevsky, A., Sutskever, I. and Hinton, G. E. (2012) Imagenet classification with deep convolutional neural networks, In: Proc. NeurIPS, pp. 1097–1105

[11] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V. and Rabinovich, A. (2015) Going deeper with convolutions. In: Proc. CVPR, pp. 1–9

[12] Krause, J., Baek, K. and Lim, L. (2019) A guided multi-scale categorization of plant species in natural images. In: Proc. CVPR Workshops

[13] Kumar, N., Belhumeur, P. N., Biswas, A., Jacobs, D. W., Kress, W. J., Lopez, I. C. and Soares, J. V. (2012) Leafsnap: A computer vision system for automatic plant species identification. In: Proc. ECCV, pp. 502–516

[14] Fuentes, A., Yoon, S., Kim, S. C. Park, D. ( 2017). A robust deep-learning-based detector for real-time tomato plant diseases and pests recognition. Sensors (Basel), 17 : 2022

[15] Simonyan, K. and Zisserman, A. (2015) Very deep convolutional networks for large-scale image recognition, In: Proc. ICLR

[16] He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep residual learning for image recognition. In: Proc. CVPR, pp. 770–778

[17] Chen, Y., Baireddy, S., Cai, E., Yang, C. and Delp, E. J. (2019) Leaf segmentation by functional modeling. In: Proc. CVPR Workshops

[18] Phadikar, S., Sil, J. Das, A. ( 2013). Rice diseases classification using feature selection and rule generation techniques. Comput. Electron. Agric., 90 : 76– 85

[19] Johannes, A., Picon, A., Alvarez-Gila, A., Echazarra, J., Rodriguez-Vaamonde, S., Navajas, A. D. ( 2017). Automatic plant disease diagnosis using mobile capture devices, applied on a wheat use case. Comput. Electron. Agric., 138 : 200– 209

[20] Afridi, M. J., Liu, X. and McGrath, J. M. (2014) An automated system for plant-level disease rating in real fields. In: Proc. ICPR, pp. 148–153

[21] Lin, K., Gong, L., Huang, Y., Liu, C. ( 2019). Deep learning-based segmentation and quantification of cucumber powdery mildew using convolutional neural network. Front Plant Sci., 10 : 155

[22] Zhou, B., Khosla, A., Lapedriza, À., Oliva, A. and Torralba, A. (2015) Object detectors emerge in deep scene cnns. In: Proc. ICLR

[23] Zhou, B., Khosla, A., Lapedriza, À., Oliva, A. and Torralba, A. 2016. Learning deep features for discriminative localization. In: Proc. CVPR, pp. 2921–2929

[24] Yu, W., Zhu, F., Boushey, C. J. and Delp, E. J. (2017) Weakly supervised food image segmentation using class activation maps. In: Proc. ICIP, pp. 1277–1281

[25] Bolaños, M. and Radeva, P. (2016) Simultaneous food localization and recognition. In: Proc. ICPR, pp. 3140–3145

[26] Gondal, W. M., Kohler, J. M., Grzeszick, R., Fink, G. A. and Hirsch, M. (2017) Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images. In: Proc. ICIP, pp. 2069–2073

[27] Kolesnikov, A. and Lampert, C. H. (2016) Seed, expand and constrain: Three principles for weakly-supervised image segmentation. In: Proc. ECCV, 695–711

[28] Huang, Z., Wang, X., Wang, J., Liu, W. and Wang, J. (2018) Weakly-supervised semantic segmentation network with deep seeded region growing. In: Proc. CVPR, pp. 7014–7023

[29] Wang, X., You, S., Li, X. and Ma, H. (2018) Weakly-supervised semantic segmentation by iteratively mining common object features. In: Proc. CVPR, pp. 1354–1362

[30] Ahn, J. and Kwak, S. (2018) Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation. In: Proc. CVPR, pp. 4981–4990

[31] Mottaghi, R., Chen, X., Liu, X., Cho, N. G., Lee, S. W., Fidler, S., Urtasun, R. and Yuille, A. (2014) The role of context for object detection and semantic segmentation in the wild. In: Proc. CVPR, pp. 891–898

[32] krähenbühl, P. and Koltun, V. (2011) Efficient inference in fully connected crfs with gaussian edge potentials. In: Proc. NeurIPS, pp. 109–117

[33] Lee, J., Kim, E., Lee, S., Lee, J. and Yoon, S. (2019) Ficklenet: Weakly and semi-supervised semantic image segmentation using stochastic inference. In: Proc. CVPR, pp. 5267–5276

[34] Oquab, M., Bottou, L., Laptev, I. and Sivic, J. (2015) Is object localization for free? ‒ Weakly-supervised learning with convolutional neural networks. In: Proc. CVPR, pp. 685–694

[35] Liu, B., Zhang, Y., He, D. ( 2018). Identification of apple leaf diseases based on deep convolutional neural networks. Symmetry (Basel), 10 : 11

[36] Chaudhry, A., Dokania, P. K. and Torr, P. H. (2017) Discovering class-specific pixels for weakly-supervised semantic segmentation. arXiv, 1707.05821

[37] Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R. B., Guadarrama, S. and Darrell, T. (2014) Caffe: Convolutional architecture for fast feature embedding. In: Proc. MM, pp. 675–678

[38] Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv, 1603.04467

[39] Ronneberger, O., Fischer, P. and Brox, T. (2015) U-net: Convolutional networks for biomedical image segmentation. In: Proc. MICCAI, pp. 234–241

[40] Chen, L. C., Zhu, Y., Papandreou, G., Schroff, F. and Adam, H. (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proc. ECCV, pp. 801–818

[41] Hariharan, B., Arbeláez, P., Bourdev, L., Maji, S. and Malik, J. (2011) Semantic contours from inverse detectors. In: Proc. ICCV, pp. 991–998

RIGHTS & PERMISSIONS

The Author(s) 2021. Published by Higher Education Press.