An interpretable multi-task whole-slide histopathology AI model for non-small cell lung cancer: Cross-cohort generalisation, spatial attention–transcriptomic integration, and molecular–immune profiling

Abstract

Background: Tumour-node-metastasis staging does not fully explain prognostic heterogeneity in non-small cell lung cancer. We evaluated whether haematoxylin-and-eosin whole-slide images could estimate histological subtype, pathological stage probabilities, survival risk and spatially grounded biological associations.

Methods: SparseAGE-MTL, a weakly supervised multi-task multiple-instance learning model with a shared projection-topology encoder and endpoint-specific heads, was trained and benchmarked in 954 The Cancer Genome Atlas cases using seven pathology feature spaces and 18 comparator models. External evaluation used 948 tissue-microarray and 324 whole-slide cases. Attention maps were co-registered with 10x Visium spatial transcriptomics and integrated with bulk transcriptomics, immune-infiltration estimates and ESTIMATE scores. Analyses included paired model comparisons, false-discovery-rate correction, Cox models, calibration assessment and decision curve analysis.

Results: In the CONCH feature space, SparseAGE-MTL achieved 93.73% accuracy, 98.19% area under the receiver-operating-characteristic curve and 93.08% F1-score for adenocarcinoma/squamous cell carcinoma classification in internal benchmarking; external area-under-the-curve values were approximately .91 and .82. Stage estimation had lower discrimination, with external overall area under the curve approximately .70 and cohort-dependent calibration. Risk-score-defined groups differed in overall survival in both histological subtypes and showed similar external trends. High-attention regions were enriched at tumour–stroma or tumour–immune interfaces and were associated with B-cell, fibroblast, C1QC, COL1A1, epithelial–mesenchymal transition, metastasis and hypoxia signals. Higher risk cases showed malignant pathway activation, lower immune/stromal scores, higher tumour purity and subtype-specific immune/stromal differences. Adding the risk score to the clinical model increased external pooled concordance index from approximately .620 to .672.

Conclusions: In retrospective cohorts, SparseAGE-MTL generated subtype-classification, stage-probability and survival-risk outputs from routine pathology images. Subtype classification had higher numerical performance than stage estimation. Survival-risk and attention outputs were associated with outcome and spatial/transcriptomic features, but prospective, treatment-annotated validation is required before clinical use.

Keywords

discrete-time survival prediction / non–small cell lung cancer / spatial-feature topology / spatial transcriptomics / tumour immune microenvironment / weakly supervised multi-task MIL / whole-slide imaging

References

[1] Chen P, Liu Y, Wen Y, Zhou C. Non-small cell lung cancer in China. Cancer Commun (Lond). 2022; 42: 937-970.

[2] Ganti AK, Klein AB, Cotarla I, Seal B, Chou E. Update of incidence, prevalence, survival, and initial treatment in patients with non-small cell lung cancer in the US. JAMA Oncol. 2021; 7: 1824-1832.

[3] Saji H, Okada M, Tsuboi M, et al. Segmentectomy versus lobectomy in small-sized peripheral non-small cell lung cancer (JCOG0802/WJOG4607L): a multicentre, open-label, phase 3, randomised, controlled, non-inferiority trial. Lancet. 2022; 399: 1607-1617.

[4] Cai J-S, Wang X, Yang F, Li Y, Qiu M-T. Lymphovascular invasion: a non-sized T descriptor for stage IA non-small cell lung cancer. Thorac Cancer. 2022; 13: 2413-2420.

[5] Wang C, Yu Q, Song T, et al. The heterogeneous immune landscape between lung adenocarcinoma and squamous carcinoma revealed by single-cell RNA sequencing. Signal Transduct Target Ther. 2022; 7: 289.

[6] Moding EJ, Shahrokh Esfahani M, Jin C, et al. Integrating ctDNA analysis and radiomics for dynamic risk assessment in localized lung cancer. Cancer Discov. 2025; 15: 1609-1629.

[7] Koyama J, Morise M, Furukawa T, et al. Artificial intelligence-based personalized survival prediction using clinical and radiomics features in patients with advanced non-small cell lung cancer. BMC Cancer. 2024; 24: 1417.

[8] Han K, Wang J, Qian K, Zhao T, Zhang Y. Establishment of non-small cell lung cancer risk prediction model based on prognosis-associated ADME genes. Biosci Rep. 2021; 41:BSR20211433.

[9] Mino-Kenudson M, Schalper K, Cooper W, et al. Predictive biomarkers for immunotherapy in lung cancer: perspective from the International Association for the Study of Lung Cancer Pathology Committee. J Thorac Oncol. 2022; 17: 1335-1354.

[10] Coudray N, Ocampo PS, Sakellaropoulos T, et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med. 2018; 24: 1559-1567.

[11] Lu MY, Williamson DFK, Chen TY, Chen RJ, Barbieri M, Mahmood F. Data-efficient and weakly supervised computational pathology on whole-slide images. Nat Biomed Eng. 2021; 5: 555-570.

[12] Zheng Y, Sadée C, Ozawa M, Howitt BE, Gevaert O. Single-cell multimodal analysis reveals tumor microenvironment predictive of treatment response in non-small cell lung cancer. Sci Adv. 2025; 11:eadu2151.

[13] Wang Y, Chen R, Wa Y, et al. Tumor immune microenvironment and immunotherapy in brain metastasis from non-small cell lung cancer. Front Immunol. 2022; 13:829451.

[14] Lehtiö J, Arslan T, Siavelis I, et al. Proteogenomics of non-small cell lung cancer reveals molecular subtypes associated with specific therapeutic targets and immune evasion mechanisms. Nat Cancer. 2021; 2: 1224-1242.

[15] Zhao J, Wang L, Zhou A, et al. Decision model for durable clinical benefit from front- or late-line immunotherapy alone or with chemotherapy in non-small cell lung cancer. Med. 2024; 5: 981-997.e4.

[16] Jiang M, Jia K, Wang L, et al. Alterations of DNA damage response pathway: biomarker and therapeutic strategy for cancer immunotherapy. Acta Pharm Sin B. 2021; 11: 2983-2994.

[17] Ou Z, Lin S, Qiu J, et al. Single-nucleus RNA sequencing and spatial transcriptomics reveal the immunological microenvironment of cervical squamous cell carcinoma. Adv Sci (Weinh). 2022; 9:e2203040.

[18] He B, Bergenstråhle L, Stenbeck L, et al. Integrating spatial gene expression and breast tumour morphology via deep learning. Nat Biomed Eng. 2020; 4: 827-834.

[19] Cancer Genome Atlas Research Network. Comprehensive molecular profiling of lung adenocarcinoma. Nature. 2014; 511: 543-550.

[20] Cancer Genome Atlas Research Network. Comprehensive genomic characterization of squamous cell lung cancers. Nature. 2012; 489: 519-525.

[21] WHO Classification of Tumours Editorial Board. WHO classification of tumours. In: Thoracic Tumours. IARC Publications; 2021: 20-29.

[22] Kanavati F, Toyokawa G, Momosaki S, et al. Weakly-supervised learning for lung carcinoma classification using deep learning. Sci Rep. 2020; 10: 9297.

[23] Shao Z, Bian H, Chen Y, et al. TransMIL: transformer based correlated multiple instance learning for whole slide image classification. Adv Neural Inf Process Syst. 2021; 34: 2136-2147.

[24] Zhang H, Meng Y, Zhao Y, et al. DTFD-MIL: double-tier feature distillation multiple instance learning for histopathology whole slide image classification. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2022: 18802-18812.

[25] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2016: 770-778.

[26] Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. ImageNET: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, IEEE; 2009: 248-255.

[27] Huang Z, Bianchi F, Yuksekgonul M, Montine TJ, Zou J. A visual–language foundation model for pathology image analysis using medical twitter. Nat Med. 2023; 29: 2307-2316.

[28] Chen RJ, Ding T, Lu MY, et al. Towards a general-purpose foundation model for computational pathology. Nat Med. 2024; 30: 850-862.

[29] Lu MY, Chen B, Williamson DF, et al. A visual-language foundation model for computational pathology. Nat Med. 2024; 30: 863-874.

[30] Zimmermann E, Vorontsov E, Viret J, et al. Virchow2: scaling self-supervised mixed magnification models in pathology. arXiv. Preprint last updated online November 6, 2024.

[31] Scalbert M, Saillard C, Peeters T, et al. Abstract LB174: H-optimus-1: a foundation model for computational histopathology. Cancer Res. 2026; 86:LB174.

[32] Chen RJ, Ding T, Lu MY, et al. Towards a general-purpose foundation model for computational pathology. Nat Med. 2024; 30: 850-862.

[33] Zadeh SG, Schmid M. Bias in cross-entropy-based training of deep survival networks. IEEE Trans Pattern Anal Mach Intell. 2021; 43: 3126-3137.

[34] Kendall A, Gal Y, Cipolla R. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. arXiv. Preprint last updated online April 24, 2018.

[35] Ilse M, Tomczak J, Welling M. Attention-based deep multiple instance learning. In: International Conference on Machine Learning. PMLR; 2018: 2127-2136.

[36] Li B, Li Y, Eliceiri KW. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. Conf Comput Vis Pattern Recognit Workshops. 2021; 2021: 14318-14328.

[37] Lin T, Yu Z, Hu H, Xu Y, Chen C-W. Interventional bag multi-instance learning on whole-slide pathological images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE; 2023: 19830-19839.

[38] Tang W, Huang S, Zhang X, Zhou F, Zhang Y, Liu B. Multiple instance learning framework with masked hard instance mining for whole slide image classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE; 2023: 4078-4087.

[39] Guo Z, Zhao W, Wang S, Yu L. HIGT: hierarchical interaction graph-transformer for whole slide image analysis. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2023: 755-764.

[40] Tang W, Zhou F, Huang S, Zhu X, Zhang Y, Liu B. Feature re-embedding: towards foundation model-level performance in computational pathology. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE; 2024: 11343-11352.

[41] Yang S, Wang Y, Chen H. MambaMIL: enhancing long sequence modeling with sequence reordering in computational pathology. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2024: 296-306.

[42] Yu S, Chen X, Cao F, Tian T. TKA-MIL: top-K attention multiple instance learning for whole slide image classification and instance probability derivation. In: ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2025: 1-5.

[43] Li J, Chen Y, Chu H, et al. Dynamic graph representation with knowledge-aware attention for histopathology whole slide image analysis. In: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE; 2024: 11323-11332.

[44] Xu B, Zhou Y, Song B, et al. Hierarchical multi-scale graph learning with knowledge-guided attention for whole-slide image survival analysis. arXiv. 2026.

[45] Zhang Y, Li H, Sun Y, Zheng S, Zhu C, Yang L. Attention-challenging multiple instance learning for whole slide image classification. In: A Leonardis, E Ricci, S Roth, O Russakovsky, T Sattler, G Varol, eds. Computer Vision – ECCV 2024, Springer Nature Switzerland; 2025: 125-143.

[46] Chen RJ, Lu MY, Shaban M, et al. Whole slide images are 2D point clouds: context-aware survival prediction using patch-based graph convolutional networks. In: M Bruijne, PC Cattin, S Cotin, et al., eds. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. Springer International Publishing; 2021: 339-349.

[47] Fourkioti O, Vries MD, Bakal C. CAMIL: context-aware multiple instance learning for cancer detection and subtyping in whole slide images. CAMIL; 2023.

[48] Yang Z, Liu H, Wang X. SCMIL: sparse context-aware multiple instance learning for predicting cancer survival probability distribution in whole slide images. In: MG Linguraru, Q Dou, A Feragen, et al., eds. Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. Springer Nature Switzerland; 2024: 448-458.

[49] Zhang X, Huang S, Zhang Y, Zhang X, Gao M, Liu C. Dual space multiple instance representative learning for medical image classification. In: BMVC. Vol. 768. BMVC; 2022.

[50] Zhou F, Chen H. Cross-modal translation and alignment for survival analysis. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE; 2023: 21485-21494.

[51] van der Maaten L, Hinton G. Visualizing data using t-SNE. J Mach Learn Res. 2008; 9: 2579-2605.

[52] Salcher S, Sturm G, Horvath L, et al. High-resolution single-cell atlas reveals diversity and plasticity of tissue-resident neutrophils in non-small cell lung cancer. Cancer Cell. 2022; 40: 1503-1520.

[53] Yuan H, Yan M, Zhang G, et al. CancerSEA: a cancer single-cell state atlas. Nucleic Acids Res. 2019; 47: D900-D908.

[54] Liu Z, Gao Z, Li B, et al. Lipid-associated macrophages in the tumor-adipose microenvironment facilitate breast cancer progression. Oncoimmunology. 2022; 11:2085432.

[55] Bai J, Liu T, Tu B, et al. Autophagy loss impedes cancer-associated fibroblast activation via downregulating proline biosynthesis. Autophagy. 2023; 19: 632-643.

[56] Robinson MD, McCarthy DJ, Smyth GK. edgeR: a Bioconductor package for differential expression analysis of digital gene expression data. Bioinformatics. 2010; 26: 139-140.

[57] Zhang J, Li H, Tao W, Zhou J. GseaVis: an R package for enhanced visualization of gene set enrichment analysis in biomedicine. Med Res. 2025; 1: 131-135.

[58] Fang Y, Kong Y, Rong G, Luo Q, Liao W, Zeng D. Systematic investigation of tumor microenvironment and antitumor immunity with IOBR. Med Res. 2025; 1: 136-140.

[59] Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc B: Stat Methodol. 1995; 57(1): 289-300.

[60] Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006; 26: 565-574.

[61] Guo C, Pleiss G, Sun Y, Weinberger KQ. On calibration of modern neural networks. In: Proceedings of the 34th International Conference on Machine Learning. Vol 70. JMLR.org; 2017: 1321-1330.

[62] Van Calster B, McLernon DJ, van Smeden M, et al. Calibration: the Achilles heel of predictive analytics. BMC Med. 2019; 17: 230.

[63] Efron B. Bootstrap methods: another look at the Jackknife. In: S Kotz, NL Johnson, eds. Breakthroughs in Statistics: Methodology and Distribution. Springer; 1992: 569-593.

[64] Molodianovitch K, Faraggi D, Reiser B. Comparing the areas under two correlated ROC curves: parametric and non-parametric approaches. Biom J. 2006; 48: 745-757.

[65] Deng Y, Chen Q, Guo C, et al. Comprehensive single-cell atlas of colorectal neuroendocrine tumors with liver metastases: unraveling tumor microenvironment heterogeneity between primary lesions and metastases. Mol Cancer. 2025; 24: 28.

[66] Moorman A, Benitez EK, Cambulli F, et al. Progressive plasticity during colorectal cancer metastasis. Nature. 2025; 637: 947-954.

[67] Liu H, Chen M, Hong B, Xiao Y, Chen Q, Qian Y. Single-nucleus RNA sequencing and spatial transcriptomics reveal an immunosuppressive tumor microenvironment related to metastatic dissemination during pancreatic cancer liver metastasis. Theranostics. 2025; 15: 5337-5357.

[68] Muratoğlu B, Özdemir C, Eylem CC, et al. Circadian rhythm and aryl hydrocarbon receptor crosstalk in bone marrow adipose tissue and implications in leukemia. Sci Rep. 2025; 15:16387.

[69] Shao F, Gao Y, Wang W, et al. Silencing EGFR-upregulated expression of CD55 and CD59 activates the complement system and sensitizes lung cancer to checkpoint blockade. Nat Cancer. 2022; 3: 1192-1210.

[70] Fard D, Testa E, Panzeri V, et al. SEMA6C: a novel adhesion-independent FAK and YAP activator, required for cancer cell viability and growth. Cell Mol Life Sci. 2023; 80: 111.

[71] Simonik EA, Cai Y, Kimmelshue KN, et al. LIM-only protein 4 (LMO4) and LIM domain binding protein 1 (LDB1) promote growth and metastasis of human head and neck cancer (LMO4 and LDB1 in head and neck cancer). PLoS One. 2016; 11:e0164804.

[72] Qiang N, Ao J, Nakamura M, et al. MEGF6 knockdown ameliorates lenvatinib-induced muscle differentiation suppression and enhances the antitumor effect of lenvatinib on hepatocellular carcinoma. Biochem Pharmacol. 2025; 235:116829.

[73] Huang K, Liu Z, Xie Z, et al. HIGD2A silencing impairs hepatocellular carcinoma growth via inhibiting mitochondrial function and the MAPK/ERK pathway. J Transl Med. 2023; 21: 253.

[74] Yu J, Zhong B, Zhao L, et al. Fighting drug-resistant lung cancer by induction of NAD(P)H:quinone oxidoreductase 1 (NQO1)-mediated ferroptosis. Drug Resist Updat. 2023; 70:100977.

[75] Sen U, Coleman C, Sen T. Stearoyl coenzyme A desaturase-1: multitasker in cancer, metabolism, and ferroptosis. Trends Cancer. 2023; 9: 480-489.

[76] Wang Z, Liu C, Zheng S, et al. Molecular subtypes of neuroendocrine carcinomas: a cross-tissue classification framework based on five transcriptional regulators. Cancer Cell. 2024; 42: 1106-1125.e8.

[77] Mortazavi F, An J, Dubinett S, Rettig M. p120-catenin is transcriptionally downregulated by FOXC2 in non-small cell lung cancer cells. Mol Cancer Res. 2010; 8: 762-774.

[78] Xie M, Qin H, Liu L, et al. GABA regulates metabolic reprogramming to mediate the development of brain metastasis in non-small cell lung cancer. J Exp Clin Cancer Res. 2025; 44: 61.

[79] Tavora B, Mederer T, Wessel KJ, et al. Tumoural activation of TLR3-SLIT2 axis in endothelium drives metastasis. Nature. 2020; 586: 299-304.

[80] Friedman JR, Richbart SD, Merritt JC, et al. Acetylcholine signaling system in progression of lung cancers. Pharmacol Ther. 2019; 194: 222-254.

[81] Zheng Y, Li L, Shen Z, et al. Mechanisms of neural infiltration-mediated tumor metabolic reprogramming impacting immunotherapy efficacy in non-small cell lung cancer. J Exp Clin Cancer Res. 2024; 43: 284.

[82] Pu T, Sun J, Ren G, Li H. Neuro-immune crosstalk in cancer: mechanisms and therapeutic implications. Signal Transduct Target Ther. 2025; 10: 176.

[83] Schneider L, Laiouar-Pedari S, Kuntz S, et al. Integration of deep learning-based image analysis and genomic data in cancer pathology: a systematic review. Eur J Cancer. 2022; 160: 80-91.

[84] Yofe I, Shami T, Cohen N, et al. Spatial and temporal mapping of breast cancer lung metastases identify TREM2 macrophages as regulators of the metastatic boundary. Cancer Discov. 2023; 13: 2610-2631.

[85] Icard P, Simula L, Fournel L, et al. The strategic roles of four enzymes in the interconnection between metabolism and oncogene activation in non-small cell lung cancer: therapeutic implications. Drug Resist Updat. 2022; 63:100852.

[86] Luo F, Lu F-T, Cao J-X, et al. HIF-1α inhibition promotes the efficacy of immune checkpoint blockade in the treatment of non-small cell lung cancer. Cancer Lett. 2022; 531: 39-56.

[87] Semenza GL. Targeting intratumoral hypoxia to enhance anti-tumor immunity. Semin Cancer Biol. 2023; 96: 5-10.

[88] Wei R, Zhao Y, Wang J, et al. Tagitinin C induces ferroptosis through PERK-Nrf2-HO-1 signaling pathway in colorectal cancer cells. Int J Biol Sci. 2021; 17: 2703-2717.

[89] Avila Cobos F, Alquicira-Hernandez J, Powell JE, Mestdagh P, De Preter K. Benchmarking of cell type deconvolution pipelines for transcriptomics data. Nat Commun. 2020; 11(1): 5650.

[90] Mariathasan S, Turley SJ, Nickles D, et al. TGFβ attenuates tumour response to PD-L1 blockade by contributing to exclusion of T cells. Nature. 2018; 554: 544-548.

[91] Khosravi G-R, Mostafavi S, Bastan S, Ebrahimi N, Gharibvand RS, Eskandari N. Immunologic tumor microenvironment modulators for turning cold tumors hot. Cancer Commun (Lond). 2024; 44: 521-553.

[92] Vanguri RS, Luo J, Aukerman AT, et al. Multimodal integration of radiology, pathology and genomics for prediction of response to PD-(L)1 blockade in patients with non-small cell lung cancer. Nat Cancer. 2022; 3: 1151-1164.

[93] Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024; 385:e078378.

[94] Moons KGM, Damen JAA, Kaul T, et al. PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods. BMJ. 2025; 388:e082505.

[95] Cruz Rivera S, Liu X, Chan AW, et al. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nat Med. 2020; 26(9): 1351-1363.

[96] Liu X, Cruz Rivera S, Moher D, Calvert MJ, Denniston AK, SPIRIT-AI and CONSORT-AI Working Group. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat Med. 2020; 26(9): 1364-1374.

Rights & permissions

2026 The Author(s). Clinical and Translational Medicine published by John Wiley & Sons Australia, Ltd on behalf of Shanghai Institute of Clinical Bioinformatics.