Deep learning in fluorescence imaging and analysis

Abstract

Fluorescence imaging (FI) has been instrumental in advancing biological research and enhancing biomedical diagnostics. Despite its widespread applications, FI faces challenges such as efficiently acquiring high signal-to-noise ratio (SNR) images, improving spatiotemporal resolution, and conducting precise quantitative analysis. Deep learning (DL), which emulates the neural network structure of the human brain, excels at learning from complex data patterns, extracting subtle features, and enhancing the SNR and spatiotemporal resolution of fluorescence images. These advancements significantly elevate the quality and usability of imaging data. Additionally, DL technology is adept at handling large datasets efficiently, which is crucial for improving the accuracy and efficiency of image analysis. This article reviews the latest advances in the application of DL to FI methodologies and their subsequent impact on biology and biomedicine. It also explores the future possibilities for DL in FI research, and providing insights and prospects could shape the field’s trajectory.

Keywords

biomedicine / deep learning / fluorescence / imaging / machine learning

References

[1] Xia T, Li N, Fang X. Single-molecule fluorescence imaging in living cells. Annu Rev Phys Chem. 2013; 64(1): 459-480.

[2] Refaat A, Yap ML, Pietersz G, et al. In vivo fluorescence imaging: success in preclinical imaging paves the way for clinical applications. J Nanobiotechnol. 2022; 20(1): 450.

[3] Rao J, Dragulescu-Andrasi A. Yao H. Fluorescence imaging in vivo: recent advances. Curr Opin Biotechnol. 2007; 18(1): 17-25.

[4] Huang B, Bates M, Zhuang X. Super-resolution fluorescence microscopy. Annu Rev Biochem. 2009; 78(1): 993-1016.

[5] He H, Chen X, Feng Z, Liu L, Wang Q, Bi S. Nanoscopic imaging of nucleolar stress enabled by protein-mimicking carbon dots. Nano Lett. 2021; 21(13): 5689-5696.

[6] Sun N, Jia Y, Bai SW, Li Q, Dai LR, Li JB. The power of super-resolution microscopy in modern biomedical science. Adv Colloid Interface Sci. 2023; 314: 28.

[7] Sigal YM, Zhou RB, Zhuang XW. Visualizing and discovering cellular structures with super-resolution microscopy. Science. 2018; 361(6405): 880-887.

[8] Li M, Huang Z-L. Rethinking resolution estimation in fluorescence microscopy: from theoretical resolution criteria to super-resolution microscopy. Sci China Life Sci. 2020; 63(12): 1776-1785.

[9] Bond C, Santiago-Ruiz AN. Tang Q, Lakadamyali M. Technological advances in super-resolution microscopy to study cellular processes. Mol Cell. 2022; 82(2): 315-332.

[10] Lanzanò L, Coto Hernández I, Castello M, Gratton E, Diaspro A, Vicidomini G. Encoding and decoding spatio-temporal information for super-resolution microscopy. Nat Commun. 2015; 6(1): 6701.

[11] Mao J, Xue M, Guan X, et al. Near-Infrared blinking carbon dots designed for quantitative nanoscopy. Nano Lett. 2023; 23(1): 124-131.

[12] Acuna-Rodriguez JP, Mena-Vega JP. Arguello-Miranda O. Live-cell fluorescence spectral imaging as a data science challenge. Biophys Rev. 2022; 14(2): 579-597.

[13] Tian L, Hunt B, Bell MAL, et al. Deep learning in biomedical optics. Laser Surg Med. 2021; 53(6): 748-775.

[14] Nilsson NJ. Principles of Artificial Intelligence. Springer Science and Business Media; 1982.

[15] Jordan MI, Mitchell TM. Machine learning: trends, perspectives, and prospects. Science. 2015; 349(6245): 255-260.

[16] Ray S. A quick review of machine learning algorithms. In: International conference on machine learning, big data, cloud and parallel computing (COMITCon). IEEE; 2019: 35-39.

[17] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015; 521(7553): 436-444.

[18] Wainberg M, Merico D, Delong A, Frey BJ. Deep learning in biomedicine. Nat Biotechnol. 2018; 36(9): 829-838.

[19] Moen E, Bannon D, Kudo T, Graf W, Covert M, Van Valen D. Deep learning for cellular image analysis. Nat Methods. 2019; 16(12): 1233-1246.

[20] Cao C, Liu F, Tan H, et al. Deep learning and its applications in biomedicine. Genom Proteom Bioinf. 2018; 16(1): 17-32.

[21] Qiao, C; Li, D; Liu, Y, et al. Rationalized deep learning super-resolution microscopy for sustained live imaging of rapid subcellular processes. Nat Biotechnol 2023, 41 (3), 367-+. DOI: 377,

[22] Belthangady C, Royer LA. Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction. Nat Methods. 2019; 16(12): 1215-1225.

[23] Xing F, Xie Y, Su H, Liu F, Yang L. Deep learning in microscopy image analysis: a survey. IEEE Transact Neural Networks Learn Syst. 2018; 29(10): 4550-4568.

[24] He H, Liu L, Chen X, et al. Carbon dot blinking enables accurate molecular counting at nanoscale resolution. Anal Chem. 2021; 93(8): 3968-3975.

[25] Möckl L, Roy AR, Petrov PN, Moerner WE. Accurate and rapid background estimation in single-molecule localization microscopy using the deep neural network BGnet. Proc Natl Acad Sci USA. 2020; 117(1): 60-67.

[26] Benfenati A. upU-Net approaches for background emission removal in fluorescence microscopy. J Imaging. 2022; 8(5): 142.

[27] von Chamier L, Laine RF, Jukkala J, et al. Democratising deep learning for microscopy with ZeroCostDL4Mic. Nat Commun. 2021; 12(1): 2276.

[28] Speiser A, Müller LR, Hoess P, et al. Deep learning enables fast and dense single-molecule localization with high accuracy. Nat Methods. 2021; 18(9): 1082-1090.

[29] Spilger R, Lee J-Y, Chagin VO, et al. Deep probabilistic tracking of particles in fluorescence microscopy images. Med Image Anal. 2021; 72: 102128.

[30] Spilger R, Wollmann T, Qiang Y, et al. Deep particle tracker: automatic tracking of particles in fluorescence microscopy images using deep learning. In: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4. Springer; 2018: 128-136.

[31] Meng F, Yoo J, Chung HS. Single-molecule fluorescence imaging and deep learning reveal highly heterogeneous aggregation of amyloid-β 42. Proc Natl Acad Sci USA. 2022; 119(12): e2116736119.

[32] Williamson DJ, Burn GL, Simoncelli S, et al. Machine learning for cluster analysis of localization microscopy data. Nat Commun. 2020; 11(1): 1493.

[33] Caicedo JC, Roth J, Goodman A, et al. Evaluation of deep learning strategies for nucleus segmentation in fluorescence images. Cytometry. 2019; 95(9): 952-965.

[34] Moen E, Bannon D, Kudo T, Graf W, Covert M, Van Valen D. Deep learning for cellular image analysis. Nat Methods. 2019; 16(12): 1233-1246.

[35] Rocco B, Sighinolfi MC, Sandri M, et al. Digital biopsy with fluorescence confocal microscope for effective real-time diagnosis of prostate cancer: a prospective, comparative study. Eur.Urol. Oncol. 2021; 4(5): 784-791.

[36] Schmidt A, Gao G, Little SR, Jalihal AP, Walter NG. Following the messenger: recent innovations in live cell single molecule fluorescence imaging. Wiley Interdiscip. Rev. RNA. 2020; 11(4): e1587.

[37] Hwang D-W, Maekiniemi A, Singer RH, Sato H. Real-time single-molecule imaging of transcriptional regulatory networks in living cells. Nat Rev Genet. 2024; 25(4): 272-285.

[38] Lerner, E; Cordes, T; Ingargiola, A, et al. Toward dynamic structural biology: two decades of single-molecule Forster resonance energy transfer. Science 2018, 359 (6373), 288-+.

[39] Liu S, Hoess P, Ries J. Super-resolution microscopy for structural cell biology. Annu Rev Biophys. 2022; 51(1): 301-326.

[40] Gong JH, Jin ZQ, Chen HD, He J, Zhang YR, Yang XF. Super-resolution fluorescence microscopic imaging in pathogenesis and drug treatment of neurological disease. Adv. Drug Deliv. 2023; 196: 114791.

[41] Datta R, Heaster TM, Sharick JT, Gillette AA, Skala MC. Fluorescence lifetime imaging microscopy: fundamentals and advances in instrumentation, analysis, and applications. J Biomed Opt. 2020; 25(7): 071203.

[42] Ebrecht R, Don Paul C, Wouters FS. Fluorescence lifetime imaging microscopy in the medical sciences. Protoplasma. 2014; 251(2): 293-305.

[43] Stelzer EH, Strobl F, Chang B-J, et al. Light sheet fluorescence microscopy. Nat. Rev. Methods Primers. 2021; 1(1): 73.

[44] Hillman EM, Voleti V, Li W, Yu H. Light-sheet microscopy in neuroscience. Annu Rev Neurosci. 2019; 42(1): 295-313.

[45] Zong, WJ; Wu, RL; Chen, SY, et al. Miniature two-photon microscopy for enlarged field-of-view, multi-plane an. long-term brain imaging. Nat Methods 2021, 18 (1), 46-+. DOI: 49,

[46] Benninger RK, Piston DW. Two-photon excitation microscopy for the study of living cells and tissues. Curr Protoc Cell Biol. 2013; 59(1): 4-11.

[47] Refaat A, Yap ML, Pietersz G, et al. In vivo fluorescence imaging: success in preclinical imaging paves the way for clinical applications. J Nanobiotechnol. 2022; 20(1): 450.

[48] Lauwerends LJ, van Driel P, Baatenburg de Jong RJ, et al. Real-time fluorescence imaging in intraoperative decision making for cancer surgery. Lancet Oncol. 2021; 22(5): e186-e195.

[49] Ren F, Wang F, Baghdasaryan A, et al. Shortwave-infrared-light-emitting probes for the in vivo tracking of cancer vaccines and the elicited immune responses. Nat Biomed Eng. 2023; 8(6): 726-739.

[50] Baghdasaryan A, Wang F, Ren F, et al. Phosphorylcholine-conjugated gold-molecular clusters improve signal for Lymph Node NIR-II fluorescence imaging in preclinical cancer models. Nat Commun. 2022; 13(1): 5613.

[51] Möckl L, Moerner W. Super-resolution microscopy with single molecules in biology and beyond–essentials, current trends, and future challenges. J Am Chem Soc. 2020; 142(42): 17828-17844.

[52] Jingyu W, Yongdeng Z. Adaptive optics in super-resolution microscopy. Biophysics Reports. 2021; 7(4): 267-279.

[53] Betzig E, Patterson GH, Sougrat R, et al. Imaging intracellular fluorescent proteins at nanometer resolution. Science. 2006; 313(5793): 1642-1645.

[54] Nehme E, Weiss LE, Michaeli T, Shechtman Y. Deep-STORM: super-resolution single-molecule microscopy by deep learning. Optica. 2018; 5(4): 458.

[55] Liu Z, Lavis LD, Betzig E. Imaging live-cell dynamics and structure at the single-molecule level. Mol Cell. 2015; 58(4): 644-659.

[56] He H, Qin G, Xue M, et al. Mitochondrial nanoprobe for precise cellular and drug analysis via graph Neural network. Chem Eng J. 2024; 493: 152709.

[57] Pineda J, Midtvedt B, Bachimanchi H, et al. Geometric deep learning reveals the spatiotemporal features of microscopic motion. Nat Mach Intell. 2023; 5(1): 71-82.

[58] Falk T, Mai D, Bensch R, et al. U-Net: deep learning for cell counting, detection, and morphometry. Nat Methods. 2019; 16(1): 67-70.

[59] Ledig C, Theis L, Huszár F, et al. Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2017: 4681-4690.

[60] Solomon O, Eldar YC, Mutzafi M, Segev M. SPARCOM: sparsity based super-resolution correlation microscopy. SIAM J Imag Sci. 2019; 12(1): 392-419.

[61] Mannam V, Zhang Y, Yuan X, Ravasio C, Howard SS. Machine learning for faster and smarter fluorescence lifetime imaging microscopy. J Phys Photonics. 2020; 2(4): 042005.

[62] Xiao D, Sapermsap N, Chen Y, Li DDU. Deep learning enhanced fast fluorescence lifetime imaging with a few photons. Optica. 2023; 10(7): 944-951.

[63] Xiao D, Zang Z, Xie W, Sapermsap N, Chen Y, Uei Li DD. Spatial resolution improved fluorescence lifetime imaging via deep learning. Opt Express. 2022; 30(7): 11479-11494.

[64] Bai C, Liu C, Yu X, et al. Imaging enhancement of light-sheet fluorescence microscopy via deep learning. IEEE Photonics Technol Lett. 2019; 31(22): 1803-1806.

[65] Prince MNH, Garcia B, Henn C, et al. Signal improved ultra-fast light-sheet microscope for large tissue imaging. Commun Eng. 2024; 3(1): 59.

[66] Cai B, Xu X, Jia K, Qing C, Tao D. Dehazenet: an end-to-end system for single image haze removal. IEEE Trans Image Process. 2016; 25(11): 5187-5198.

[67] Lee S, Negishi M, Urakubo H, Kasai H, Ishii S. Mu-net: multi-scale U-net for two-photon microscopy image denoising and restoration. Neural Network. 2020; 125: 92-103.

[68] Soltanian-Zadeh S, Sahingur K, Blau S, Gong Y, Farsiu S. Fast and robust active neuron segmentation in two-photon calcium imaging using spatiotemporal deep learning. Proc Natl Acad Sci USA. 2019; 116(17): 8554-8563.

[69] Ma Z, Wang F, Wang W, Zhong Y, Dai H. Deep learning for in vivo near-infrared imaging. Proc Natl Acad Sci USA. 2020; 118(1): e2021446118.

[70] Zhang H, Qie Y. Applying deep learning to medical imaging: a review. Appl Sci. 2023; 13(18): 10521.

[71] Rivenson Y, de Haan K, Wallace WD, Ozcan A. Emerging advances to transform histopathology using virtual staining. BME Front. 2020; 2020.

[72] Thomsen J, Sletfjerding MB, Jensen SB, et al. DeepFRET, a software for rapid and automated single-molecule FRET data classification using deep learning. Elife. 2020; 9: e60404.

[73] Li J, Zhang L, Johnson-Buck A. Walter NG. Automatic classification and segmentation of single-molecule fluorescence time traces with deep learning. Nat Commun. 2020; 11(1): 5833.

[74] Cortes C, Vapnik V. Support-vector networks. Mach Learn. 1995; 20(3): 273-297.

[75] Breiman L. Random forests. Mach Learn. 2001; 45(1): 5-32.

[76] Flach P. Machine Learning: The Art and Science of Algorithms that Make Sense of Data. Cambridge university press; 2012.

[77] McCulloch WS, Pitts W. A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys. 1943; 5(4): 115-133.

[78] Rosenblatt F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev. 1958; 65(6): 386-408.

[79] Rumelhart DE, Hinton GE, Williams RJ. Learning representations by back-propagating errors. Nature. 1986; 323(6088): 533-536.

[80] Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997; 9(8): 1735-1780.

[81] LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE. 1998; 86(11): 2278-2324.

[82] Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G. The graph neural network model. IEEE Trans Neural Network. 2008; 20(1): 61-80.

[83] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst. 2012; 25.

[84] Goodfellow I, Pouget-Abadie J. Mirza M, et al. Generative adversarial nets. Adv Neural Inf Process Syst. 2014; 27.

[85] Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, Lecture Notes in Computer Science; 2015: 234-241.

[86] Wei Ouyang AA, Mickaël L, Xian H, Zimmer C, Zimmer C. Deep learning massively accelerates super-resolution localization microscopy. Nat Biotechnol. 2018; 36(5): 460-468.

[87] Hershko E, Weiss LE, Michaeli T, Shechtman Y. Multicolor localization microscopy and point-spread-function engineering by deep learning. Opt Express. 2019; 27(5): 6158-6183.

[88] Nehme E, Freedman D, Gordon R, et al. DeepSTORM3D: dense 3D localization microscopy and PSF design by deep learning. Nat Methods. 2020; 17(7): 734-740.

[89] Speiser A, Muller LR, Hoess P, et al. Deep learning enables fast and dense single-molecule localization with high accuracy. Nat Methods. 2021; 18(9): 1082-1090.

[90] Hu Q, Hailstone M, Wang J, et al. Universal adaptive optics for microscopy through embedded neural network control. Light Sci Appl. 2023; 12(1): 270.

[91] Li Z, Liu F, Yang W, Peng S, Zhou J. A survey of convolutional neural networks: analysis, applications, and prospects. IEEE Transact Neural Networks Learn Syst. 2022; 33(12): 6999-7019.

[92] Sarvamangala DR, Kulkarni RV. Convolutional neural networks in medical image understanding: a survey. Evol Intell. 2022; 15(1): 1-22.

[93] Kim T, Moon S, Xu K. Information-rich localization microscopy through machine learning. Nat Commun. 2019; 10(1): 1996.

[94] Hiroshima M, Yasui M, Ueda M. Large-scale single-molecule imaging aided by artificial intelligence. Microscopy (Lond). 2020; 69(2): 69-78.

[95] Hussain S, Anwar SM, Majid M. Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing. 2018; 282: 248-261.

[96] Li Q, Cai W, Wang X, Zhou Y, Feng DD, Chen M. Medical image classification with convolutional neural network. In: 2014 13th international conference on control automation robotics and vision (ICARCV); 2014: 844-848.

[97] Sirinukunwattana K, Ahmed Raza SE, Yee-Wah T. Snead DR, Cree IA, Rajpoot NM. Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans Med Imag. 2016; 35(5): 1196-1206.

[98] Anthimopoulos M, Christodoulidis S, Ebner L, Christe A, Mougiakakou S. Lung pattern classification for interstitial lung diseases using a deep convolutional neural network. IEEE Trans Med Imag. 2016; 35(5): 1207-1216.

[99] Kazeminia S, Baur C, Kuijper A, et al. GANs for medical image analysis. Artif Intell Med. 2020; 109: 101938.

[100] Schmidhuber J. Deep learning. Scholarpedia. 2015; 10(11): 32832.

[101] Lechner, M; Hasani, R Learning long-term dependencies in irregularly-sampled time series. arXiv preprint arXiv:2006.04418 2020,

[102] Yu Y, Si X, Hu C, Zhang J. A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput. 2019; 31(7): 1235-1270.

[103] Angermueller C, Pärnamaa T, Parts L, Stegle O. Deep learning for computational biology. Mol Syst Biol. 2016; 12(7): 878.

[104] Cireşan DC, Meier U, Gambardella LM, Schmidhuber J. Deep, big, simple neural nets for handwritten digit recognition. Neural Comput. 2010; 22(12): 3207-3220.

[105] Schindelin J, Arganda-Carreras I. Frise E, et al. Fiji: an open-source platform for biological-image analysis. Nat Methods. 2012; 9(7): 676–682.

[106] Carpenter AE, Jones TR, Lamprecht MR, et al. CellProfiler: image analysis software for identifying and quantifying cell phenotypes. Genome Biol. 2006; 7(10): R100.

[107] Chen J, Ding L, Viana MP, et al. The Allen Cell and Structure Segmenter: a new open source toolkit for segmenting 3D intracellular structures in fluorescence microscopy images. bioRxiv. 2018:491035.

[108] Sammut C, Webb GI. Encyclopedia of Machine Learning and Data Mining. Springer Publishing Company; 2017.

[109] Kohavi R. A study of cross-validation and bootstrap for accuracy estimation and model selection. Ijcai. 1995; 14: 1137-1145. Montreal, Canada.

[110] Deschout H, Zanacchi FC, Mlodzianoski M, et al. Precisely and accurately localizing single emitters in fluorescence microscopy. Nat Methods. 2014; 11(3): 253-266.

[111] Monici M. Cell and tissue autofluorescence research and diagnostic applications. Biotechnol Annu Rev. 2005; 11: 227-256.

[112] Waters JC. Accuracy and precision in quantitative fluorescence microscopy. J Cell Biol. 2009; 18(7): 1135-1148.

[113] Quirin S, Pavani SRP, Piestun R. Optimal 3D single-molecule localization for superresolution microscopy with aberrations and engineered point spread functions. Proc Natl Acad Sci USA. 2012; 109(3): 675-679.

[114] Mockl L, Roy AR, Petrov PN, Moerner WE. Accurate and rapid background estimation in single-molecule localization microscopy using the deep neural network BGnet. Proc Natl Acad Sci USA. 2020; 117(1): 60-67.

[115] Sha H, Li H, Zhang Y, Hou S. Deep learning-enhanced single-molecule spectrum imaging. bioRxiv. 2023: 2005-2023.

[116] Zhang G, Li X, Zhang Y, et al. Bio-friendly long-term subcellular dynamic recording by self-supervised image enhancement microscopy. Nat Methods. 2023; 20(12): 1957-1970.

[117] Qiao C, Li D, Guo Y, et al. Evaluation and development of deep neural networks for image super-resolution in optical microscopy. Nat Methods. 2021; 18(2): 194-202.

[118] Zhao W, Zhao S, Li L, et al. Sparse deconvolution improves the resolution of live-cell super-resolution fluorescence microscopy. Nat Biotechnol. 2022; 40(4): 606-617.

[119] Wang H, Rivenson Y, Jin Y, et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat Methods. 2019; 16(1): 103-110.

[120] Wang H, Rivenson Y, Jin Y, et al. Deep learning achieves super-resolution in fluorescence microscopy. bioRxiv. 2018.

[121] Zhang P, Liu S, Chaurasia A, et al. Analyzing complex single-molecule emission patterns with deep learning. Nat Methods. 2018; 15(11): 913-916.

[122] Alon Saguy OA, Opatovski N, Jang S, Heilemann M, Shechtman Y, Shechtman Y. DBlink: dynamic localization microscopy in super spatiotemporal resolution via deep learning. Nat Methods. 2023; 20(12): 1939-1948.

[123] Chen R, Tang X, Zhao Y, et al. Single-frame deep-learning super-resolution microscopy for intracellular dynamics imaging. Nat Commun. 2023; 14(1): 2854.

[124] Chen J, Fang Q, Huang L, et al. Deep-learning accelerated super-resolution radial fluctuations (SRRF) enables real-time live cell imaging. Opt Laser Eng. 2024; 172: 107840.

[125] Boutros M, Heigwer F, Laufer C. Microscopy-based high-content screening. Cell. 2015; 163(6): 1314-1325.

[126] Godin AG, Lounis B, Cognet L. Super-resolution microscopy approaches for live cell imaging. Biophys J. 2014; 107(8): 1777-1784.

[127] Dertinger T, Colyer R, Iyer G, Weiss S, Enderlein J. Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI). Proc Natl Acad Sci USA. 2009; 106(52): 22287-22292.

[128] Gustafsson N, Culley S, Ashdown G, Owen DM, Pereira PM, Henriques R. Fast live-cell conventional fluorophore nanoscopy with ImageJ through super-resolution radial fluctuations. Nat Commun. 2016; 7(1): 12471.

[129] Zhao W, Zhao S, Han Z, et al. Enhanced detection of fluorescence fluctuations for high-throughput super-resolution imaging. Nat Photonics. 2023; 17(9): 806-813.

[130] Dardikman-Yoffe G, Eldar YC. Learned SPARCOM: unfolded deep super-resolution microscopy. Opt Express. 2020; 28(19): 27736-27763.

[131] Yoon S, Kim M, Jang M, et al. Deep optical imaging within complex scattering media. Nat Rev Phys. 2020; 2(3): 141-158.

[132] Gigan S, Katz O, De Aguiar HB, et al. Roadmap on wavefront shaping and deep imaging in complex media. J Phys Photonics. 2022; 4(4): 042501.

[133] Puxiang L, Qi Z, Zhou Y, et al. Deep-tissue optics: technological development and applications (invited). Chin J Lasers. 2024; 51(1).

[134] Cheng S, Li H, Luo Y, Zheng Y, Lai P. Artificial intelligence-assisted light control and computational imaging through scattering media. J Innov Opt Health Sci. 2019; 12(04): 1930006.

[135] Horisaki R, Takagi R, Tanida J. Learning-based focusing through scattering media. Appl Opt. 2017; 56(15): 4358-4362.

[136] May MA, Barre N, Kummer KK, Kress M, Ritsch-Marte M. Jesacher A. Fast holographic scattering compensation for deep tissue biological imaging. Nat Commun. 2021; 12(1): 4340.

[137] Li Y, Xue Y, Tian L. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media. Optica. 2018; 5(10): 1181.

[138] Zhan X, Gao J, Gan Y, et al. Roles of scattered and ballistic photons in imaging through scattering media: a deep learning-based study. arXiv preprint arXiv:2207.10263. 2022.

[139] Boyd N, Jonas E, Babcock H, Recht B. DeepLoco:Fast 3D localization microscopy using neural networks. bioRxiv. 2018:267096.

[140] Zelger P, Kaser K, Rossboth B, Velas L, Schutz GJ, Jesacher A. Three-dimensional localization microscopy using deep learning. Opt Express. 2018; 26(25): 33166-33179.

[141] Li Y, Mund M, Hoess P, et al. Real-time 3D single-molecule localization using experimental point spread functions. Nat Methods. 2018; 15(5): 367-369.

[142] Zhang Y, de Haan K, Rivenson Y, Li J, Delis A, Ozcan A. Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue. Light Sci Appl. 2020; 9(1): 78.

[143] Nygate YN, Levi M, Mirsky SK, et al. Holographic virtual staining of individual biological cells. Proc Natl Acad Sci USA. 2020; 117(17): 9223-9231.

[144] Rivenson Y, Wang H, Wei Z, et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat Biomed Eng. 2019; 3(6): 466-477.

[145] Rivenson Y, Zhang Y, Günaydın H, Teng D, Ozcan A. Phase recovery and holographic image reconstruction using deep learning in neural networks. Light Sci Appl. 2018; 7(2): 17141.

[146] Wu Y, Rivenson Y, Wang H, et al. Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning. Nat Methods. 2019; 16(12): 1323-1331.

[147] Guo M, Wu Y, Su Y, et al. Deep learning-based aberration compensation improves contrast and resolution in fluorescence microscopy. bioRxiv. 2023.

[148] Xu J, Qin G, Luo F, et al. Automated stoichiometry analysis of single-molecule fluorescence imaging traces via deep learning. J Am Chem Soc. 2019; 141(17): 6976-6985.

[149] Wang Q, Zhang Q, He H, et al. Carbon dot blinking fingerprint uncovers native membrane receptor organizations via deep learning. Anal Chem. 2022; 94(9): 3914-3921.

[150] He H, Qin G, Bi S, et al. Deep-learning-enhanced diffusion imaging assay for resolving local-density effects on membrane receptors. Anal Chem. 2023; 95(6): 3300-3308.

[151] Williamson DJ, Burn GL, Simoncelli S, et al. Machine learning for cluster analysis of localization microscopy data. Nat Commun. 2020; 11(1): 1493.

[152] Yuan J, Zhao R, Xu J, et al. Analyzing protein dynamics from fluorescence intensity traces using unsupervised deep learning network. Commun Biol. 2020; 3(1): 669.

[153] Wang Q, He H, Zhang Q, et al. Deep-learning-Assisted single-molecule tracking on a live cell membrane. Anal Chem. 2021; 93(25): 8810-8816.

[154] Newby JM, Schaefer AM, Lee PT, Forest MG, Lai SK. Convolutional neural networks automate detection for tracking of submicron-scale particles in 2D and 3D. Proc Natl Acad Sci USA. 2018; 115(36): 9026-9031.

[155] Parthasarathy RR. Accurate particle tracking by calculation of radial symmetry centers. Nat Methods. 2012; 9(7): 724-726.

[156] Greenwald NF, Miller G, Moen E, et al. Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning. Nat Biotechnol. 2022; 40(4): 555-565.

[157] Heckenbach I, Mkrtchyan GV, Ezra MB, et al. Nuclear morphology is a deep learning biomarker of cellular senescence. Nat Aging. 2022; 2(8): 742-755.

[158] Morone D, Marazza A, Bergmann TJ, Molinari M, Gruenberg J. Deep learning approach for quantification of organelles and misfolded polypeptide delivery within degradative compartments. Mol Biol Cell. 2020; 31(14): 1512-1524.

[159] Iqbal MS, Luo B, Mehmood R, Alrige MA, Alharbey R. Mitochondrial organelle movement classification (fission and fusion) via convolutional neural network approach. IEEE Access. 2019; 7: 86570-86577.

[160] Yapp C, Novikov E, Jang W-D, et al. UnMICST: deep learning with real augmentation for robust segmentation of highly multiplexed images of human tissues. bioRxiv. 2022; 2021.2004.2002.438285.

[161] Lefebvre A, Ma D, Kessenbrock K, Lawson DA, Digman MA. Automated segmentation and tracking of mitochondria in live-cell time-lapse images. Nat Methods. 2021; 18(9): 1091-1102.

[162] Mosquera-Lopez C, Agaian S, Velez-Hoyos A. Thompson I. Computer-aided prostate cancer diagnosis from digitized histopathology: a review on texture-based systems. IEEE Rev Biomed Eng. 2014; 8: 98-113.

[163] Torrents-Barrena J, Lazar P, Jayapathy R, Rathnam M, Mohandhas B, Puig D. Complex wavelet algorithm for computer-aided diagnosis of Alzheimer’s disease. Electron Lett. 2015; 51(20): 1566-1568.

[164] Saha M, Mukherjee R, Chakraborty C. Computer-aided diagnosis of breast cancer using cytological images: a systematic review. Tissue Cell. 2016; 48(5): 461-474.

[165] You S, Sun Y, Yang L, et al. Real-time intraoperative diagnosis by deep neural network driven multiphoton virtual histology. npj Precis Oncol. 2019; 3(1): 33.

[166] Matsumoto T, Niioka H, Kumamoto Y, et al. Deep-UV excitation fluorescence microscopy for detection of lymph node metastasis using deep neural network. Sci Rep. 2019; 9(1): 16912.

[167] Lin H, Wei C, Wang G, et al. Automated classification of hepatocellular carcinoma differentiation using multiphoton microscopy and deep learning. J Biophot. 2019; 12(7): e201800435.

[168] Kitamura S, Takahashi K, Sang Y, Fukushima K, Tsuji K, Wada J. Deep learning could diagnose diabetic nephropathy with renal pathological immunofluorescent images. Diagnostics. 2020; 10(7): 466.

[169] Lee YS, Choi JW, Kang T, Chung BG. Deep learning-assisted droplet digital PCR for quantitative detection of human coronavirus. BioChip Journal. 2023; 17(1): 112-119.

[170] Shiaelis N, Tometzki A, Peto L, et al. Virus detection and identification in minutes using single-particle imaging and deep learning. ACS Nano. 2023; 17(1): 697-710.

[171] Wong F, de la Fuente-Nunez C, Collins JJ. Leveraging artificial intelligence in the fight against infectious diseases. Science. 2023; 381(6654): 164-170.

[172] Tao C, Du J, Wang J, Hu B, Zhang Z. Rapid identification of infectious pathogens at the single-cell level via combining hyperspectral microscopic images and deep learning. Cells. 2023; 12(3): 379.

[173] Ning K, Zhang X, Gao X, et al. Deep-learning-based whole-brain imaging at single-neuron resolution. Biomed Opt Express. 2020; 11(7): 3567-3584.

[174] Wang H, Zhang D, Song Y, et al. Multiscale kernels for enhanced U-shaped network to improve 3D neuron tracing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops; 2019. 0-0.

[175] Jiang Y, Chen W, Liu M, Wang Y, Meijering E. 3D neuron microscopy image segmentation via the ray-shooting model and a DC-BLSTM network. IEEE Trans Med Imag. 2021; 40(1): 26-37.

[176] Banerjee S, Magee L, Wang D, et al. Semantic segmentation of microscopic neuroanatomical data by combining topological priors with encoder-decoder deep networks. Nat Mach Intell. 2020; 2(10): 585-594.

[177] Forman R. A user’s guide to discrete Morse theory. Séminaire Lotharingien de Combinatoire [electronic only]. 2002; 48: B35-B48c.

[178] Buslaev A, Seferbekov S, Iglovikov V, Shvets A. Fully convolutional network for automatic road extraction from satellite imagery. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops; 2018: 207-210.

RIGHTS & PERMISSIONS

2024 The Author(s). Journal of Intelligent Medicine published by John Wiley & Sons Australia, Ltd on behalf of Tianjin University.