Survey on deep learning for pulmonary medical imaging

Abstract

As a promising method in artificial intelligence, deep learning has been proven successful in several domains ranging from acoustics and images to natural language processing. With medical imaging becoming an important part of disease screening and diagnosis, deep learning-based approaches have emerged as powerful techniques in medical image areas. In this process, feature representations are learned directly and automatically from data, leading to remarkable breakthroughs in the medical field. Deep learning has been widely applied in medical imaging for improved image analysis. This paper reviews the major deep learning techniques in this time of rapid evolution and summarizes some of its key contributions and state-of-the-art outcomes. The topics include classification, detection, and segmentation tasks on medical image analysis with respect to pulmonary medical images, datasets, and benchmarks. A comprehensive overview of these methods implemented on various lung diseases consisting of pulmonary nodule diseases, pulmonary embolism, pneumonia, and interstitial lung disease is also provided. Lastly, the application of deep learning techniques to the medical image and an analysis of their future challenges and potential directions are discussed.

Keywords

deep learning / neural networks / pulmonary medical image / survey

References

[1] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521(7553): 436–444

[2] Schmidhuber J. Deep learning in neural networks: an overview. Neural Netw 2015; 61: 85–117

[3] Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJWL. Artificial intelligence in radiology. Nat Rev Cancer 2018; 18(8): 500–510

[4] Camarlinghi N. Automatic detection of lung nodules in computed tomography images: training and validation of algorithms using public research databases. Eur Phys J Plus 2013; 128(9): 110

[5] Siegel RL, Miller KD, Jemal A. Cancer Statistics, 2017. CA Cancer J Clin 2017; 67(1): 7–30

[6] AbuBaker AA, Qahwaji RS, Aqel MJ, Saleh M. Average row thresholding method for mammogram segmentation. Conf Proc IEEE Eng Med Biol Soc 2005; 3: 3288–3291

[7] Haider W, Sharif M, Raza M. Achieving accuracy in early stage tumor identification systems based on image segmentation and 3D structure analysis. Comput Eng Intell Syst 2011; 2(6): 96–102

[8] Lo SCB, Lin JS, Freedman MT, . Computer-assisted diagnosis of lung nodule detection using artificial convoultion neural network[C]//Medical Imaging 1993: Image Processing. International Society for Optics and Photonics. 1993. 1898: 859–869

[9] Sahiner B, Chan HP, Petrick N, Wei D, Helvie MA, Adler DD, Goodsitt MM. Classification of mass and normal breast tissue: a convolution neural network classifier with spatial domain and texture images. IEEE Trans Med Imaging 1996; 15(5): 598–610

[10] Wang X, Han T X, Yan S. An HOG-LBP human detector with partial occlusion handling[C]//2009 IEEE 12th international conference on computer vision. IEEE. 2009. 32–39

[11] Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI. A survey on deep learning in medical image analysis. Med Image Anal 2017; 42: 60–88

[12] Zhou H, Yuan Y, Shi C. Object tracking using sift features and mean shift. Comput Vis Image Underst 2009; 113(3): 345–352

[13] Mori K, Hahn HK. Computer-aided diagnosis[C]//Proc. of SPIE Vol. 2019. 10950: 1095001–1

[14] Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L. ImageNet large scale visual recognition challenge. Int J Comput Vis 2015; 115(3): 211–252 (IJCV)

[15] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks[C]//Advances in neural information processing systems. 2012. 1097–1105

[16] Szegedy C, Liu W, Jia Y, . Going deeper with convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015. 1–9

[17] He K, Zhang X, Ren S, . Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. 770–778

[18] Ren S, He K, Girshick R, . Faster r-cnn: towards real-time object detection with region proposal networks[C]//Advances in neural information processing systems. 2015. 91–99

[19] Girshick R. Fast r-cnn[C]//Proceedings of the IEEE international conference on computer vision. 2015. 1440–1448

[20] Liu W, Anguelov D, Erhan D, . Ssd: single shot multibox detector[C]//European conference on computer vision. Springer, Cham. 2016. 21–37

[21] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015. 3431–3440

[22] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C]//International Conference on Medical image computing and computer-assisted intervention. Springer, Cham. 2015. 234–241

[23] Lederlin M, Revel MP, Khalil A, Ferretti G, Milleron B, Laurent F. Management strategy of pulmonary nodule in 2013. Diagn Interv Imaging 2013; 94(11): 1081–1094

[24] Ozekes S, Osman O, Ucan ON. Nodule detection in a lung region that’s segmented with using genetic cellular neural networks and 3D template matching with fuzzy rule based thresholding. Korean J Radiol 2008; 9(1): 1–9

[25] Shen D, Wu G, Suk HI. Deep learning in medical image analysis. Annu Rev Biomed Eng 2017; 19(1): 221–248

[26] Monkam P, Qi S, Ma H, Gao W, Yao Y, Qian W. Detection and classification of pulmonary nodules using convolutional neural networks: a survey. IEEE Access 2019; 7: 78075–78091

[27] Wall B, Hart D. Revised radiation doses for typical X-ray examinations. Report on a recent review of doses to patients from medical X-ray examinations in the UK by NRPB. National Radiological Protection Board.. Br J Radiol 1997; 70(833): 437–439

[28] Ashby WR. An introduction to cybernetics. Chapman & Hall Ltd, 1961

[29] Wiener N. Cybernetics. Bull Am Acad Arts Sci 1950; 3(7): 2–4

[30] Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 1958; 65(6): 386–408

[31] Hubel DH, Wiesel TN. Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J Physiol 1962; 160(1): 106–154

[32] Rodieck RW, Stone J. Analysis of receptive fields of cat retinal ganglion cells. J Neurophysiol 1965; 28(5): 833–849

[33] Blakemore C. The working brain. Nature 1972; 239(5373): 473

[34] Fukushima K. Neocognitron: a self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybern 1980; 36(4): 193–202

[35] Fukushima K, Hirota M, Terasaki PI, Wakisaka A, Togashi H, Chia D, Suyama N, Fukushi Y, Nudelman E, Hakomori S. Characterization of sialosylated Lewisx as a new tumor-associated antigen. Cancer Res 1984; 44(11): 5279–5285

[36] Fukushima K, Miyake S, Ito T. Neocognitron: a neural network model for a mechanism of visual pattern recognition. IEEE Trans Syst Man Cybern 1983 (5): 826–834

[37] Rumelhart DE, Hinton GE, Williams RJ. Learning representations by back-propagating errors. Nature 1988; 323(6088): 696–699

[38] Nitta T. Solving the XOR problem and the detection of symmetry using a single complex-valued neuron. Neural Netw 2003; 16(8): 1101–1105

[39] Pineda FJ. Generalization of back-propagation to recurrent neural networks. Phys Rev Lett 1987; 59(19): 2229–2232

[40] Wigner EP. The problem of measurement. Am J Phys 1963; 31(1): 6–15

[41] Hecht-Nielsen R. Theory of the backpropagation neural network. Neural Netw 1988; 1: 445–448

[42] Rumelhart DE, Hinton GE, Williams RJ. Learning representations by back-propagating errors. Nature 1988; 323(6088): 696–699

[43] LeCun Y. Generalization and network design strategies. Connectionism in perspective. Amsterdam: Elsevier, 1989.Vol. 19

[44] LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE 1998; 86(11): 2278–2324

[45] Hinton GE, Osindero S, Teh YW. A fast learning algorithm for deep belief nets. Neural Comput 2006; 18(7): 1527–1554

[46] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems. 2012. 1097–1105

[47] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint. 2014. arXiv: 1409.1556

[48] Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017. 4700–4708

[49] Krogh A, Hertz JA. Dynamics of generalization in linear perceptrons. In: Advances in Neural Information Processing Systems. 1991. 897–903

[50] LeCun Y, Boser B E, Denker JS, Henderson D, Howard RE, Hubbard WE, Jackel LD. Handwritten digit recognition with a back-propagation network. In: Advances in neural information processing systems. 1990. 396–404

[51] Haskell BG, Howard PG, LeCun YA, Puri A, Ostermann J, Civanlar MR, Rabiner L, Bottou L, Haffner P. Image and video coding-emerging standards and beyond. IEEE Trans Circ Syst Video Tech 1998; 8(7): 814–837

[52] Hochreiter S, Bengio Y, Frasconi P, Schmidhuber J. Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. 2001

[53] Pedrazzi M, Patrone M, Passalacqua M, Ranzato E, Colamassaro D, Sparatore B, Pontremoli S, Melloni E. Selective proinflammatory activation of astrocytes by high-mobility group box 1 protein signaling. J Immunol 2007; 179(12): 8525–8532

[54] Deng J, Dong W, Socher R, Li LJ, Li K, Li FF. Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. 2009. 248–255

[55] Anthimopoulos M, Christodoulidis S, Ebner L, Christe A, Mougiakakou S. Lung pattern classification for interstitial lung diseases using a deep convolutional neural network. IEEE Trans Med Imaging 2016; 35(5): 1207–1216

[56] Kawahara J, BenTaieb A, Hamarneh G. Deep features to classify skin lesions. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI). 2016. 1397–1400

[57] Setio AAA, Ciompi F, Litjens G, Gerke P, Jacobs C, van Riel SJ, Wille MMW, Naqibullah M, Sanchez CI, van Ginneken B. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging 2016; 35(5): 1160–1169

[58] Yang D, Zhang S, Yan Z, Tan C, Li K, Metaxas D. Automated anatomical landmark detection ondistal femur surface using convolutional neural network. In: Biomedical Imaging (ISBI), 2015 IEEE 12th International Symposium on. 2015. 17–21

[59] de Vos BD, Wolterink JM, de Jong PA, Viergever MA, Išgum I. 2D image classification for 3D anatomy localization: employing deep convolutional neural networks. In: Medical Imaging 2016: Image Processing. 2016. vol. 9784, p. 97841Y

[60] Zheng Y, Liu D, Georgescu B, Nguyen H, Comaniciu D. 3D deep learning for efficient and robust landmark detection in volumetric data. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer. 2015. 565–572

[61] Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. Springer. 2015. 234–241

[62] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2015. 3431–3440

[63] Cicek O, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O. 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer. 2016. 424–432

[64] Milletari F, Navab N, Ahmadi SA. V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV). 565–571

[65] Siegel RL, Miller KD, Jemal A. Cancer statistics, 2018. CA Cancer J Clin 2018; 68(1): 7–30

[66] Awai K, Murao K, Ozawa A, Komi M, Hayakawa H, Hori S, Nishimura Y. Pulmonary nodules at chest CT: effect of computer-aided diagnosis on radiologists’ detection performance. Radiology 2004; 230(2): 347–352

[67] Ciompi F, Chung K, van Riel SJ, Setio AAA, Gerke PK, Jacobs C, Scholten ET, Schaefer-Prokop C, Wille MMW, Marchianò A, Pastorino U, Prokop M, van Ginneken B. Towards automatic pulmonary nodule management in lung cancer screening with deep learning. Sci Rep 2017; 7(1): 46479

[68] Liu S, Xie Y, Jirapatnakul A, Reeves AP. Pulmonary nodule classification in lung cancer screening with three-dimensional convolutional neural networks. J Med Imaging (Bellingham) 2017; 4(4): 041308

[69] Hua KL, Hsu CH, Hidayati SC, Cheng WH, Chen YJ. Computer-aided classification of lung nodules on computed tomography images via deep learning technique. Onco Targets Ther 2015; 8: 2015–2022

[70] Li W, Cao P, Zhao D, Wang J. Pulmonary nodule classification with deep convolutional neural networks on computed tomography images. Comput Math Methods Med 2016; 2016: 6215085

[71] Magalhães Barros Netto S, Corrêa Silva A, Acatauassú Nunes R, Gattass M. Automatic segmentation of lung nodules with growing neural gas and support vector machine. Comput Biol Med 2012; 42(11): 1110–1121

[72] Pei X, Guo H, Dai J. Computerized detection of lung nodules in CT images by use of multiscale filters and geometrical constraint region growing[C]//2010 4th International Conference on Bioinformatics and Biomedical Engineering. IEEE. 2010: 1–4

[73] Suzuki K, Li F, Sone S, Doi K. Computer-aided diagnostic scheme for distinction between benign and malignant nodules in thoracic low-dose CT by use of massive training artificial neural network. IEEE Trans Med Imaging 2005; 24(9): 1138–1150

[74] Suzuki K, Doi K. Computerized scheme for distinction between benign and malignant nodules in thoracic low-dose CT: U.S. Patent Application 11/181,884[P]. 2006-1-26

[75] Causey JL, Zhang J, Ma S, Jiang B, Qualls JA, Politte DG, Prior F, Zhang S, Huang X. Highly accurate model for prediction of lung nodule malignancy with CT scans. Sci Rep 2018; 8(1): 9286

[76] Zhao X, Liu L, Qi S, Teng Y, Li J, Qian W. Agile convolutional neural network for pulmonary nodule classification using CT images. Int J CARS 2018; 13(4): 585–595

[77] Xie Y, Xia Y, Zhang J, . Transferable multi-model ensemble for benign-malignant lung nodule classification on chest CT[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2017. 656–664

[78] Shen W, Zhou M, Yang F, Yu D, Dong D, Yang C, Zang Y, Tian J. Multi-crop convolutional neural networks for lung nodule malignancy suspiciousness classification. Pattern Recognit 2017; 61: 663–673

[79] Liu L, Dou Q, Chen H, Olatunji IE, Qin J, Heng PA. Mtmr-net: Multi-task deep learning with margin ranking loss for lung nodule analysis. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Springer. 2018. 74–82

[80] Heng PA. Mtmr-net: Multi-task deep learning with margin ranking loss for lung nodule analysis. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings. 2018. vol. 11045, p. 74

[81] Liao F, Liang M, Li Z, Hu X, Song S. Evaluate the malignancy of pulmonary nodules using the 3D deep leaky noisy-or network. IEEE Trans Neural Netw Learn Syst 2019; 1–12

[82] Ardila D, Kiraly AP, Bharadwaj S, Choi B, Reicher JJ, Peng L, Tse D, Etemadi M, Ye W, Corrado G, Naidich DP, Shetty S. End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nat Med 2019; 25(6): 954–961

[83] Wu B, Zhou Z, Wang J, Wang Y. Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). 2018. 1109–1113

[84] Shen S, Han SX, Aberle DR, Bui AAT, Hsu W. An interpretable deep hierarchical semantic convolutional neural network for lung nodule malignancy classification. Expert Syst Appl 2019; 128: 84–95

[85] Ding J, Li A, Hu Z, Wang L. Accurate pulmonary nodule detection in computed tomography images using deep convolutional neural networks. Medical Image Computing and Computer Assisted Intervention − MICCAI 2017. 2017. Springer. 559–567

[86] Winkels M, Cohen T S. 3D g-cnns for pulmonary nodule detection. arXiv preprint. 2018. arXiv:1804.04656

[87] Zhu W, Liu C, Fan W, Xie X. Deeplung: 3D deep convolutional nets for automated pulmonary nodule detection and classification. arXiv preprint. 2017. arXiv:1709.05538

[88] Tang H, Kim DR, Xie X. Automated pulmonary nodule detection using 3D deep convolutional neural networks. International Symposium on Biomedical Imaging. 2018. 523–526

[89] Tang H, Liu XW, Xie XH. An end-to-end framework for integrated pulmonary nodule detection and false positive reduction. arXiv preprint. 2019. arXiv:1903.09880

[90] Xie Z. Towards single-phase single-stage detection of pulmonary nodules in chest CT imaging. arXiv preprint. 2018. arXiv: 1807.05972

[91] Ma JC, . Group-Attention Single-Shot Detector (GA-SSD): finding pulmonary nodules in large-scale CT images. arXiv preprint. 2018. arXiv:1812.07166

[92] Feng X, Yang J, Laine AF, Angelini ED. Discriminative localization in cnns for weakly-supervised segmentation of pulmonary nodules. Medical image computing and computer assisted intervention. 2017. 568–576

[93] Messay T, Hardie RC, Tuinstra TR. Segmentation of pulmonary nodules in computed tomography using a regression neural network approach and its application to the Lung Image Database Consortium and Image Database Resource Initiative dataset. Med Image Anal 2015; 22(1): 48–62

[94] Liu K, Li Q, Ma J, Zhou Z, Sun M, Deng Y, Xiao Y. Evaluating a fully automated pulmonary nodule detection approach and its impact on radiologist performance. Radiol Artif Intell 2019. 1(3): e180084

[95] Rucco M, Sousa-Rodrigues D, Merelli E, Johnson JH, Falsetti L, Nitti C, Salvi A. Neural hypernetwork approach for pulmonary embolism diagnosis. BMC Res Notes 2015; 8(1): 617

[96] Bi J, Liang J. Multiple instance learning of pulmonary embolism detection with geodesic distance along vascular structure. 2007 IEEE Conference on Computer Vision and Pattern Recognition. 2007. 1–8

[97] Agharezaei L, Agharezaei Z, Nemati A, Bahaadinbeigy K, Keynia F, Baneshi MR, Iranpour A, Agharezaei M. The prediction of the risk level of pulmonary embolism and deep vein thrombosis through artificial neural network. Acta Inform Med 2016; 24(5): 354–359

[98] Serpen G, Tekkedil DK, Orra M. A knowledge-based artificial neural network classifier for pulmonary embolism diagnosis. Comput Biol Med 2008; 38(2): 204–220

[99] Tsai H, Chin C, Cheng Y. Intelligent pulmonary embolsim detection system. Biomed Eng (Singapore) 2012; 24(6): 471–483

[100] Tajbakhsh N, Gotway MB, Liang J. Computer-aided pulmonary embolism detection using a novel vessel-aligned multi-planar image representation and convolutional neural networks. MICCAI 2015: Medical Image Computing and Computer-Assisted Intervention. 2015. 62–69

[101] Chen MC, Ball RL, Yang L, Moradzadeh N, Chapman BE, Larson DB, Langlotz CP, Amrhein TJ, Lungren MP. Deep learning to classify radiology free-text reports. Radiology 2017; 286(3): 845–852

[102] Rumelhart DE, Hinton GE, Williams RJ. Learning representations by back-propagating errors. Nature 1988; 323(6088): 696–699

[103] Messay T, Hardie RC, Tuinstra TR. Segmentation of pulmonary nodules in computed tomography using a regression neural network approach and its application to the Lung Image Database Consortium and Image Database Resource Initiative dataset. Med Image Anal 2015; 22(1): 48–62

[104] Breiman L. Bagging predictors. Mach Learn 1996; 24(2): 123–140

[105] Blackmon KN, Florin C, Bogoni L, McCain JW, Koonce JD, Lee H, Bastarrika G, Thilo C, Costello P, Salganicoff M, Joseph Schoepf U. Computer-aided detection of pulmonary embolism at CT pulmonary angiography: can it improve performance of inexperienced readers? Eur Radiol 2011; 21(6): 1214–1223

[106] Wang X, Song X F, Chapman B E, . Improving performance of computer-aided detection of pulmonary embolisms by incorporating a new pulmonary vascular-tree segmentation algorithm[C]//Medical Imaging 2012: Computer-Aided Diagnosis. International Society for Optics and Photonics. 2012. 8315: 83152U

[107] Loud PA, Katz DS, Bruce DA, Klippenstein DL, Grossman ZD. Deep venous thrombosis with suspected pulmonary embolism: detection with combined CT venography and pulmonary angiography. Radiology 2001; 219(2): 498–502

[108] Özkan H, Osman O, Şahin S, Boz AF. A novel method for pulmonary embolism detection in CTA images. Comput Methods Programs Biomed 2014; 113(3): 757–766

[109] Schoepf UJ, Costello P. CT angiography for diagnosis of pulmonary embolism: state of the art. Radiology 2004; 230(2): 329–337

[110] Liang J, Bi J. Computer aided detection of pulmonary embolism with tobogganing and mutiple instance classification in CT pulmonary angiography. In: Biennial International Conference on Information Processing in Medical Imaging. Springer. 2007. 630–641

[111] Engelke C, Schmidt S, Bakai A, Auer F, Marten K. Computer-assisted detection of pulmonary embolism: performance evaluation in consensus with experienced and inexperienced chest radiologists. Eur Radiol 2008; 18(2): 298–307

[112] Liang J, Bi J. Local characteristic features for computer-aided detection of pulmonary embolism in CT angiography. In: Proceedings of the First MICCAI Workshop on Pulmonary Image Analysis. 2008. 263–272

[113] Park SC, Chapman BE, Zheng B. A multistage approach to improve performance of computer-aided detection of pulmonary embolisms depicted on CT images: preliminary investigation. IEEE Trans Biomed Eng 2011; 58(6): 1519–1527

[114] Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang JM. Convolutional neural networks for medical image analysis: Full training or fine tuning? IEEE Trans Med Imaging 2016; 35(5): 1299–1312

[115] Tang L, Wang L, Pan S, Su Y, Chen Y. A neural network to pulmonary embolism aided diagnosis with a feature selection approach. 2010 3rd International Conference on Biomedical Engineering and Informatics. IEEE. 2010. 2255–2260

[116] Ebrahimdoost Y, Dehmeshki J, Ellis TS, Firoozbakht M, Youannic A, Qanadli SD. Medical image segmentation using active contours and a level set model: application to pulmonary embolism (PE) segmentation. 2010 Fourth International Conference on Digital Society. IEEE. 2010. 269–273

[117] Scott JA, Palmer EL, Fischman AJ. How well can radiologists using neural network software diagnose pulmonary embolism? AJR Am J Roentgenol 2000; 175(2): 399–405

[118] Tajbakhsh N, Gotway MB, Liang J. Computer-aided pulmonary embolism detection using a novel vesselaligned multi-planar image representation and convolutional neural networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer. 2015. 62–69

[119] Lee Y, Hara T, Fujita H, Itoh S, Ishigaki T. Automated detection of pulmonary nodules in helical CT images based on an improved template-matching technique. IEEE Trans Med Imaging 2001; 20(7): 595–604

[120] Abdullah AA, Posdzi NM, Nishio Y. Preliminary study of pneumonia symptoms detection method using cellular neural network. In: International Conference on Electrical, Control and Computer Engineering 2011 (InECCE). 2011. 497–500

[121] Correa M, Zimic M, Barrientos F, Barrientos R, Román-Gonzalez A, Pajuelo MJ, Anticona C, Mayta H, Alva A, Solis-Vasquez L, Figueroa DA, Chavez MA, Lavarello R, Castañeda B, Paz-Soldán VA, Checkley W, Gilman RH, Oberhelman R. Automatic classification of pediatric pneumonia based on lung ultrasound pattern recognition. PLoS One 2018; 13(12): e0206410

[122] Cisnerosvelarde P, Correa M, Mayta H, Anticona C, Pajuelo M, Oberhelman RA, Checkley W, Gilman RH, Figueroa D, Zimic M, . Automatic pneumonia detection based on ultrasound video analysis. 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE. 2016. 4117–4120

[123] Sharma A, Raju D, Ranjan S. Detection of pneumonia clouds in chest X-ray using image processing approach[C]//2017 Nirma University International Conference on Engineering (NUiCONE). IEEE. 2017. 1–4

[124] de Melo G, Macedo S O, Vieira S L, . Classification of images and enhancement of performance using parallel algorithm to detection of pneumonia. 2018 IEEE International Conference on Automation/XXIII Congress of the Chilean Association of Automatic Control (ICA-ACCA). IEEE. 2018. 1–5

[125] Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM. Chestx-ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017. 3462–3471

[126] Franquet T. Imaging of community-acquired pneumonia. J Thorac Imaging 2018; 33(5): 282–294

[127] Lee Y, Hara T, Fujita H, Itoh S, Ishigaki T. Automated detection of pulmonary nodules in helical CT images based on an improved template-matching technique. IEEE Trans Med Imaging 2001; 20(7): 595–604

[128] Nanni L, Lumini A, Brahnam S. Local binary patterns variants as texture descriptors for medical image analysis. Artif Intell Med 2010; 49(2): 117–125

[129] Dalal N, Triggs B. Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). 2005. 886–893

[130] Barrientos R, Roman-Gonzalez A, Barrientos F, . Automatic detection of pneumonia analyzing ultrasound digital images. 2016 IEEE 36th Central American and Panama Convention. 2016. 1–4

[131] Nvidia C. Nvidia cuda c programming guide. Nvidia Corporation 2011; 120(18): 8

[132] Dye C. Global epidemiology of tuberculosis. Lancet 2006; 367(9514): 938–940

[133] Sudre P, ten Dam G, Kochi A. Tuberculosis: a global overview of the situation today. Bull World Health Organ 1992; 70(2): 149–159

[134] Ponnudurai N, Denkinger C M, Van Gemert W, . New TB tools need to be affordable in the private sector: The case study of Xpert MTB/RIF. J Epidemiol Glob Health 2018; 8(3–4): 103–105

[135] Pande T, Cohen C, Pai M, Ahmad Khan F. Computer aided diagnosis of tuberculosis using digital chest radiographs: a systematic review. Chest 2015; 148(4 Suppl): 135A

[136] Rohilla A, Hooda R, Mittal A. Tb detection in chest radiograph using deep learning architecture. ICETETSM-17. 2017. 136–147

[137] Lakhani P, Sundaram B. Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology 2017; 284(2): 574–582

[138] Melendez J, Sánchez CI, Philipsen RHHM, Maduskar P, Dawson R, Theron G, Dheda K, van Ginneken B. An automated tuberculosis screening strategy combining X-ray-based computer-aided detection and clinical information. Sci Rep 2016; 6(1): 25265

[139] Melendez J, Sánchez CI, Philipsen RHHM, . Multiple-instance learning for computer-aided detection of tuberculosis. Computer-Aided Diagnosis. International Society for Optics and Photonics. 2014. 9035: 90351J

[140] Shin HC, Roberts K, Lu L, Demner-Fushman D, Yao J, Summers RM. Learning to read chest X-rays: recurrent neural cascade model for automated image annotation. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. 2497–2506

[141] Murphy K, Habib S S, Zaidi S M A, . Computer aided detection of tuberculosis on chest radiographs: an evaluation of the CAD4TB v6 system. arXiv preprint. 2019. arXiv:1903.03349

[142] Zheng Y, Liu D, Georgescu B, Nguyen H, Comaniciu D. 3D deep learning for efficient and robust landmark detection in volumetric data. International Conference on Medical Image Computing and Computer-Assisted Intervention. 2015. 565–572

[143] Bar Y, Diamant I, Wolf L, Lieberman S, Konen E, Greenspan H. Chest pathology detection using deep learning with non-medical training. 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI). 2015. 294–297

[144] Feng X, Yang J, Laine AF, Angelini ED. Discriminative Localization in CNNs for Weakly-Supervised Segmentation of Pulmonary Nodules. Medical Image Computing and Computer Assisted Intervention − MICCAI 2017. Springer. 2017. 568–576

[145] Melendez J, van Ginneken B, Maduskar P, Philipsen RHHM, Reither K, Breuninger M, Adetifa IMO, Maane R, Ayles H, Sánchez CI. A novel multiple-instance learning-based approach to computer-aided detection of tuberculosis on chest X-rays. IEEE Trans Med Imaging 2015; 34(1): 179–192

[146] Melendez J, Sánchez CI, Philipsen RH, Maduskar P, Dawson R, Theron G, Dheda K, van Ginneken B. An automated tuberculosis screening strategy combining X-ray-based computer-aided detection and clinical information. Sci Rep 2016; 6(1): 25265

[147] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. Computer Vision and Pattern Recognition. arXiv preprint. 2014. arXiv:1409.1556

[148] Li Q, Cai W, Feng DD. Lung image patch classification with automatic feature learning. 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE. 2013. 6079–6082

[149] Li Q, Cai W, Wang X, Zhou Y, Feng DD, Chen M. Medical image classification with convolutional neural network. 13th International Conference on Control Automation Robotics & Vision (ICARCV). IEEE. 2014. 844–848

[150] Gao M, Xu Z, Lu L, . Multi-label deep regression and unordered pooling for holistic interstitial lung disease pattern detection[C]//International Workshop on Machine Learning in Medical Imaging. Springer, Cham. 2016.147–155

[151] Christodoulidis S, Anthimopoulos M, Ebner L, Christe A, Mougiakakou S. Multisource transfer learning with convolutional neural networks for lung pattern analysis. IEEE J Biomed Health Inform 2017; 21(1): 76–84

[152] Gao M, Bagci U, Lu L, Wu A, Buty M, Shin HC, Roth H, Papadakis GZ, Depeursinge A, Summers RM, Xu Z, Mollura DJ. Holistic classification of CT attenuation patterns for interstitial lung diseases via deep convolutional neural networks. Comput Methods Biomech Biomed Eng Imaging Vis 2018; 6(1): 1–6

[153] Cengil E, Çinar A. A deep learning based approach to lung cancer identification[C]//2018 International Conference on Artificial Intelligence and Data Processing (IDAP). IEEE. 2018. 1–5

[154] Chamberlain D, Kodgule R, Ganelin D, Miglani V, Fletcher R.Application of semi-supervised deep learning to lung sound analysis. 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE. 2016. 804–807

[155] Hashemi A, Arabalibiek H, Agin K. Classification of wheeze sounds using wavelets and neural networks. In: 2011 International Conference on Biomedical Engineering and Technology. Singapore: IACSIT Press. 2011. vol. 11. 127–131

[156] Aykanat M, Kılıç Ö, Kurt B, Saryal S. Classification of lung sounds using convolutional neural networks. EURASIP J Image Video Processing 2017; 2017: 65

[157] Tan T, Li Z, Liu H, Zanjani FG, Ouyang Q, Tang Y, Hu Z, Li Q. Optimize transfer learning for lung diseases in bronchoscopy using a new concept: sequential fine-tuning. IEEE J Transl Eng Health Med 2018; 6: 1800808

[158] Tang C, Plasek JM, Zhang H, Xiong Y, Bates DW, Zhou L. A deep learning approach to handling temporal variation in chronic obstructive pulmonary disease progression. 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE. 2018. 502–509

[159] Campo MI, Pascau J, Estepar RSJ. Emphysema quantification on simulated X-rays through deep learning techniques. 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). IEEE. 2018. 273–276

[160] Armato III SG, McLennan G, Bidaut L, . The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med Phys 2011; 38(2): 915–931

[161] Armato SG 3rd, Giger ML, Moran CJ, Blackburn JT, Doi K, MacMahon H. Computerized detection of pulmonary nodules on CT scans. Radiographics 1999; 19(5): 1303–1311

[162] Huidrom R, Chanu YJ, Singh KM. Pulmonary nodule detection on computed tomography using neuroevolutionary scheme. Signal Image Video Process 2019; 13(1): 53–60

[163] Shaukat F, Raja G, Ashraf R, Khalid S, Ahmad M, Ali A. Artificial neural network based classification of lung nodules in CT images using intensity, shape and texture features. J Ambient Intell Humaniz Comput 2019; 10(10): 4135–4149

[164] Zhang W, Wang X, Li X, Chen J. 3D skeletonization feature based computer-aided detection system for pulmonary nodules in CT datasets. Comput Biol Med 2018; 92: 64–72

[165] Naqi S, Sharif M, Yasmin M, Fernandes SL. Lung nodule detection using polygon approximation and hybrid features from CT images. Curr Med Imaging Rev 2018; 14(1): 108–117

[166] Liu JK, Jiang HY, Gao MD, He CG, Wang Y, Wang P, Ma H, Li Y. An assisted diagnosis system for detection of early pulmonary nodule in computed tomography images. J Med Syst 2017; 41(2): 30

[167] Javaid M, Javid M, Rehman MZU, Shah SIA. A novel approach to CAD system for the detection of lung nodules in CT images. Comput Methods Programs Biomed 2016; 135: 125–139

[168] Akram S, Javed MY, Akram MU, Qamar U, Hassan A. Pulmonary nodules detection and classification using hybrid features from computerized tomographic images. J Med Imaging Health Inform 2016; 6(1): 252–259

[169] Özkan H, Osman O, Şahin S, Boz AF. A novel method for pulmonary embolism detection in CTA images. Comput Methods Programs Biomed 2014; 113(3): 757–766

[170] Mehre S A, Mukhopadhyay S, Dutta A, . An automated lung nodule detection system for CT images using synthetic minority oversampling[C]//Medical Imaging 2016: Computer-Aided Diagnosis. International Society for Optics and Photonics. 2016. 9785: 97850H

[171] Naqi SM, Sharif M, Lali IU. A 3D nodule candidate detection method supported by hybrid features to reduce false positives in lung nodule detection. Multimedia Tools Appl 2019; 78(18): 26287–26311

[172] Huidrom R, Chanu YJ, Singh KM. Pulmonary nodule detection on computed tomography using neuroevolutionary scheme. Signal Image Video Process 2019; 13(1): 53–60

[173] Anthimopoulos M, Christodoulidis S, Christe A, Mougiakako S.Classification of interstitial lung disease patterns using local DCT features and random forest. 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 2014. 6040–6043

[174] Gangeh MJ, Sorensen L, Shaker SB, Kamel MS, de Bruijne M, Loog M. A texton-based approach for the classification of lung parenchyma in CT images. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2010. Springer. 2010. 595–602

[175] Dou Q, Chen H, Yu L, Qin J, Heng PA. Multilevel contextual 3-D CNNs for false positive reduction in pulmonary nodule detection. IEEE Trans Biomed Eng 2017; 64(7): 1558–1567

[176] Torres EL, Fiorina E, Pennazio F, Peroni C, Saletta M, Camarlinghi N, Fantacci ME, Cerello P. Large scale validation of the M5L lung CAD on heterogeneous CT datasets. Med Phys 2015; 42(4): 1477–1489

[177] van Ginneken B, Armato SG 3rd, de Hoop B, van Amelsvoort-van de Vorst S, Duindam T, Niemeijer M, Murphy K, Schilham A, Retico A, Fantacci ME, Camarlinghi N, Bagagli F, Gori I, Hara T, Fujita H, Gargano G, Bellotti R, Tangaro S, Bolaños L, de Carlo F, Cerello P, Cristian Cheran S, Lopez Torres E, Prokop M. Comparing and combining algorithms for computer-aided detection of pulmonary nodules in computed tomography scans: The ANODE09 study. Med Image Anal 2010; 14(6): 707–722

[178] Rajpurkar P, Irvin J, Zhu K, . CheXNet: radiologist-level pneumonia detection on chest X-rays with deep learning. arXiv: Computer Vision and Pattern Recognition. arXiv preprint. 2017. arXiv:1711.05225

[179] Yao L, Poblenz E, Dagunts D, . Learning to diagnose from scratch by exploiting dependencies among labels. arXiv: Computer Vision and Pattern Recognition. arXiv preprint. 2018 arXiv: 1710.10501

[180] Jaeger S, Candemir S, Antani S, Wáng YXJ, Lu PX, Thoma G. Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. Quant Imaging Med Surg 2014; 4(6): 475–477

[181] Bi WL, Hosny A, Schabath MB, Giger ML, Birkbak NJ, Mehrtash A, Allison T, Arnaout O, Abbosh C, Dunn IF, Mak RH, Tamimi RM, Tempany CM, Swanton C, Hoffmann U, Schwartz LH, Gillies RJ, Huang RY, Aerts HJWL. Artificial intelligence in cancer imaging: clinical challenges and applications. CA Cancer J Clin 2019; 69(2): 127–157

Compliance with ethics guidelines

Jiechao Ma, Yang Song, Xi Tian, Yiting Hua, Rongguo Zhang, and Jianlin Wu declare that they have no conflicts of interest. This manuscript is a review article that does not need a research protocol requiring approval by the relevant institutional review board or ethics committee.

Open Access

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

RIGHTS & PERMISSIONS

2019 The Author(s) 2019. This article is published with open access at link.springer.com and journal.hep.com.cn