Lesion region segmentation via weakly supervised learning
Ran Yi, Rui Zeng, Yang Weng, Minjing Yu, Yu-Kun Lai, Yong-Jin Liu
Lesion region segmentation via weakly supervised learning
Background: Image-based automatic diagnosis of field diseases can help increase crop yields and is of great importance. However, crop lesion regions tend to be scattered and of varying sizes, this along with substantial intra-class variation and small inter-class variation makes segmentation difficult.
Methods: We propose a novel end-to-end system that only requires weak supervision of image-level labels for lesion region segmentation. First, a two-branch network is designed for joint disease classification and seed region generation. The generated seed regions are then used as input to the next segmentation stage where we design to use an encoder-decoder network. Different from previous works that use an encoder in the segmentation network, the encoder-decoder network is critical for our system to successfully segment images with small and scattered regions, which is the major challenge in image-based diagnosis of field diseases. We further propose a novel weakly supervised training strategy for the encoder-decoder semantic segmentation network, making use of the extracted seed regions.
Results: Experimental results show that our system achieves better lesion region segmentation results than state of the arts. In addition to crop images, our method is also applicable to general scattered object segmentation. We demonstrate this by extending our framework to work on the PASCAL VOC dataset, which achieves comparable performance with the state-of-the-art DSRG (deep seeded region growing) method.
Conclusion: Our method not only outperforms state-of-the-art semantic segmentation methods by a large margin for the lesion segmentation task, but also shows its capability to perform well on more general tasks.
Crop diseases seriously affect the quantity and quality of crop yields, causing huge economic losses and posing a serious threat to global food security. However, overuse of chemicals in traditional agriculture may be harmful to humans and livestock. Therefore, early diagnosis of crop diseases, which helps to avoid the heavy use of chemicals and provides feasible solutions, is much desired. We proposed a system to segment images with small and scattered regions. Experimental results show that our method not only outperforms state-of-the-art segmentation methods for the lesion segmentation task, but also shows its capability to perform well on more general tasks.
weakly supervised learning / lesion segmentation / disease detection / semantic segmentation / agriculture
[1] |
Strange, R. N. Scott, P. ( 2005). Plant disease: a threat to global food security. Annu. Rev. Phytopathol., 43 : 83– 116
CrossRef
Pubmed
Google scholar
|
[2] |
Geiger, F., Bengtsson, J., Berendse, F., Weisser, W. W., Emmerson, M., Morales, M. B., Ceryngier, P., Liira, J., Tscharntke, T., Winqvist, C.
CrossRef
Google scholar
|
[3] |
Aravind, K. R., Raja, P., Aniirudh, R., Mukesh, K. V., Ashiwin, R. and Vikas, G. (2018) Grape crop disease classification using transfer learning approach. In: Proc. ISMAC-CVB, pp. 1623–1633
|
[4] |
DeChant, C., Wiesner-Hanks, T., Chen, S., Stewart, E. L., Yosinski, J., Gore, M. A., Nelson, R. J. ( 2017). Automated identification of northern leaf blight-infected maize plants from field imagery using deep learning. Phytopathology, 107 : 1426– 1432
CrossRef
Pubmed
Google scholar
|
[5] |
Pound, M. P., Atkinson, J. A., Wells, D. M., Pridmore, T. P. and French, A. P. (2017) Deep learning for multi-task plant phenotyping. In: Proc. ICCV Workshops, pp. 2055–2063
|
[6] |
Abdu, A. M., Mokji, M. M. and Sheikh, U. U. (2019) Deep learning for plant disease identification from disease region images. In: Proc. ICIRA, pp. 65–75
|
[7] |
Mohanty, S. P., Hughes, D. P. ( 2016). Using deep learning for image-based plant disease detection. Front Plant Sci, 7 : 1419
CrossRef
Pubmed
Google scholar
|
[8] |
Zabawa, L., Kicherer, A., Klingbeil, L., Milioto, A., Topfer, R., Kuhlmann, H. and Roscher, R. (2019) Detection of single grapevine berries in images using fully convolutional neural networks. In: Proc. CVPR Workshops
|
[9] |
Zhang, S., You, Z. ( 2019). Plant disease leaf image segmentation based on superpixel clustering and EM algorithm. Neural Comput. Appl., 31 : 1225– 1232
CrossRef
Google scholar
|
[10] |
Krizhevsky, A., Sutskever, I. and Hinton, G. E. (2012) Imagenet classification with deep convolutional neural networks, In: Proc. NeurIPS, pp. 1097–1105
|
[11] |
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V. and Rabinovich, A. (2015) Going deeper with convolutions. In: Proc. CVPR, pp. 1–9
|
[12] |
Krause, J., Baek, K. and Lim, L. (2019) A guided multi-scale categorization of plant species in natural images. In: Proc. CVPR Workshops
|
[13] |
Kumar, N., Belhumeur, P. N., Biswas, A., Jacobs, D. W., Kress, W. J., Lopez, I. C. and Soares, J. V. (2012) Leafsnap: A computer vision system for automatic plant species identification. In: Proc. ECCV, pp. 502–516
|
[14] |
Fuentes, A., Yoon, S., Kim, S. C. Park, D. ( 2017). A robust deep-learning-based detector for real-time tomato plant diseases and pests recognition. Sensors (Basel), 17 : 2022
CrossRef
Pubmed
Google scholar
|
[15] |
Simonyan, K. and Zisserman, A. (2015) Very deep convolutional networks for large-scale image recognition, In: Proc. ICLR
|
[16] |
He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep residual learning for image recognition. In: Proc. CVPR, pp. 770–778
|
[17] |
Chen, Y., Baireddy, S., Cai, E., Yang, C. and Delp, E. J. (2019) Leaf segmentation by functional modeling. In: Proc. CVPR Workshops
|
[18] |
Phadikar, S., Sil, J. Das, A. ( 2013). Rice diseases classification using feature selection and rule generation techniques. Comput. Electron. Agric., 90 : 76– 85
CrossRef
Google scholar
|
[19] |
Johannes, A., Picon, A., Alvarez-Gila, A., Echazarra, J., Rodriguez-Vaamonde, S., Navajas, A. D. ( 2017). Automatic plant disease diagnosis using mobile capture devices, applied on a wheat use case. Comput. Electron. Agric., 138 : 200– 209
CrossRef
Google scholar
|
[20] |
Afridi, M. J., Liu, X. and McGrath, J. M. (2014) An automated system for plant-level disease rating in real fields. In: Proc. ICPR, pp. 148–153
|
[21] |
Lin, K., Gong, L., Huang, Y., Liu, C. ( 2019). Deep learning-based segmentation and quantification of cucumber powdery mildew using convolutional neural network. Front Plant Sci., 10 : 155
CrossRef
Pubmed
Google scholar
|
[22] |
Zhou, B., Khosla, A., Lapedriza, À., Oliva, A. and Torralba, A. (2015) Object detectors emerge in deep scene cnns. In: Proc. ICLR
|
[23] |
Zhou, B., Khosla, A., Lapedriza, À., Oliva, A. and Torralba, A. 2016. Learning deep features for discriminative localization. In: Proc. CVPR, pp. 2921–2929
|
[24] |
Yu, W., Zhu, F., Boushey, C. J. and Delp, E. J. (2017) Weakly supervised food image segmentation using class activation maps. In: Proc. ICIP, pp. 1277–1281
|
[25] |
Bolaños, M. and Radeva, P. (2016) Simultaneous food localization and recognition. In: Proc. ICPR, pp. 3140–3145
|
[26] |
Gondal, W. M., Kohler, J. M., Grzeszick, R., Fink, G. A. and Hirsch, M. (2017) Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images. In: Proc. ICIP, pp. 2069–2073
|
[27] |
Kolesnikov, A. and Lampert, C. H. (2016) Seed, expand and constrain: Three principles for weakly-supervised image segmentation. In: Proc. ECCV, 695–711
|
[28] |
Huang, Z., Wang, X., Wang, J., Liu, W. and Wang, J. (2018) Weakly-supervised semantic segmentation network with deep seeded region growing. In: Proc. CVPR, pp. 7014–7023
|
[29] |
Wang, X., You, S., Li, X. and Ma, H. (2018) Weakly-supervised semantic segmentation by iteratively mining common object features. In: Proc. CVPR, pp. 1354–1362
|
[30] |
Ahn, J. and Kwak, S. (2018) Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation. In: Proc. CVPR, pp. 4981–4990
|
[31] |
Mottaghi, R., Chen, X., Liu, X., Cho, N. G., Lee, S. W., Fidler, S., Urtasun, R. and Yuille, A. (2014) The role of context for object detection and semantic segmentation in the wild. In: Proc. CVPR, pp. 891–898
|
[32] |
krähenbühl, P. and Koltun, V. (2011) Efficient inference in fully connected crfs with gaussian edge potentials. In: Proc. NeurIPS, pp. 109–117
|
[33] |
Lee, J., Kim, E., Lee, S., Lee, J. and Yoon, S. (2019) Ficklenet: Weakly and semi-supervised semantic image segmentation using stochastic inference. In: Proc. CVPR, pp. 5267–5276
|
[34] |
Oquab, M., Bottou, L., Laptev, I. and Sivic, J. (2015) Is object localization for free? ‒ Weakly-supervised learning with convolutional neural networks. In: Proc. CVPR, pp. 685–694
|
[35] |
Liu, B., Zhang, Y., He, D. ( 2018). Identification of apple leaf diseases based on deep convolutional neural networks. Symmetry (Basel), 10 : 11
CrossRef
Google scholar
|
[36] |
Chaudhry, A., Dokania, P. K. and Torr, P. H. (2017) Discovering class-specific pixels for weakly-supervised semantic segmentation. arXiv, 1707.05821
|
[37] |
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R. B., Guadarrama, S. and Darrell, T. (2014) Caffe: Convolutional architecture for fast feature embedding. In: Proc. MM, pp. 675–678
|
[38] |
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv, 1603.04467
|
[39] |
Ronneberger, O., Fischer, P. and Brox, T. (2015) U-net: Convolutional networks for biomedical image segmentation. In: Proc. MICCAI, pp. 234–241
|
[40] |
Chen, L. C., Zhu, Y., Papandreou, G., Schroff, F. and Adam, H. (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proc. ECCV, pp. 801–818
|
[41] |
Hariharan, B., Arbeláez, P., Bourdev, L., Maji, S. and Malik, J. (2011) Semantic contours from inverse detectors. In: Proc. ICCV, pp. 991–998
|
/
〈 | 〉 |