Towards artificial intelligence-assisted digital pathology: A systematic evaluation of multimodal generative artificial intelligence in clear cell renal cell carcinoma assessment

Abstract

Clear cell renal cell carcinoma (ccRCC), the most common subtype of RCC, requires accurate pathological grading for effective prognosis. However, current grading methods rely heavily on subjective pathologist assessment, leading to variability. While generative artificial intelligence (GenAI) has shown promise in medical imaging, its application in digital pathology remains underexplored. This study evaluates the performance of three multimodal GenAI models—GPT-4o, Claude-3.5-Sonnet, and Gemini-1.5-Pro—in ccRCC grading and prognosis prediction. A total of 499 ccRCC slides from The Cancer Genome Atlas and 349 external samples from two independent cohorts were analyzed. A standardized prompt repetition mechanism and variance-based stability validation method guided GenAI models in extracting 17 pathological features. Feature stability was assessed using intraclass correlation coefficient (ICC). These features, combined with 3 clinical variables, were used to build grading and prognostic models via logistic regression and 113 machine learning algorithms. Performance was benchmarked against CellProfiler, ResNet-50, DenseNet-121, attention-based multiple instance learning (MIL) and Pathology Language and Image Pre-training, using the concordance index (C-index) and area under the receiver operating characteristic curve (AUC). Claude-3.5-Sonnet outperformed the other two GenAI models (ICC = 0.76; micro-average AUC = 0.87), exceeding ResNet-50 (AUC = 0.78) and attention-based MIL (AUC = 0.70). Its top prognostic models achieved an average C-index of 0.739, effectively stratifying high- and low-risk patients. Key predictors included stage, calcification, sarcomatoid differentiation, and vascular networks. GenAI, particularly Claude-3.5-Sonnet, enhances accuracy and consistency in ccRCC pathology, showing strong potential for clinical use, especially in resource-limited settings.

Keywords

clear cell renal cell carcinoma / Gen-AI / large language models / pathological grading / prognostic prediction

References

[1] H. Moch, A. L. Cubilla, P. A. Humphrey, V. E. Reuter, T. M. Ulbright, Eur. Urol. 2016, 70, 93.

[2] A. Jiang, J. Li, Z. He, Y. Liu, K. Qiao, Y. Fang, L. Qu, P. Luo, L. Wang, MedComm 2024, 5, e676.

[3] A. Nishie, Y. Takayama, Y. Asayama, K. Ishigami, Y. Ushijima, D. Okamoto, N. Fujita, D. Tsurumaru, O. Togao, T. Manabe, E. Oki, Y. Kubo, T. Hida, M. Hirahashi-Fujiwara, J. Keupp, H. Honda, Magn. Reson. Imaging 2018, 51, 96.

[4] A. Crane, C. Suk-Ouichai, J. A. Campbell, E. R. Caraballo, D. Aguilar Palacios, H. Tanaka, S. C. Campbell, Urology 2018, 117, 22.

[5] S. Malik, S. Zaheer, Pathol. Res. Pract. 2024, 253, 154989.

[6] A. Lin, L. Zhu, W. Mou, Z. Yuan, Q. Cheng, A. Jiang, P. Luo, Int. J. Surg. 2024, 110, 4547.

[7] M. Omar, V. Ullanat, M. Loda, L. Marchionni, R. Umeton, Lancet Digital Health 2024, 6, e595.

[8] L. Zhu, Y. Lai, W. Mou, H. Zhang, A. Lin, C. Qi, T. Yang, L. Xu, J. Zhang, P. Luo, J. Hematol. Oncol. 2024, 17, 27.

[9] L. Zhu, W. Mou, C. Hong, T. Yang, Y. Lai, C. Qi, A. Lin, J. Zhang, P. Luo, JMIR MHealth UHealth 2024, 12, e57978.

[10] A. Lin, C. Qi, M. Li, R. Guan, E. N. Imyanitov, N. V. Mitiushkina, Q. Cheng, Z. Liu, X. Wang, Q. Lyu, J. Zhang, P. Luo, Front. Nutr. 2022, 9, 869263.

[11] L. Zhu, W. Mou, K. Wu, Y. Lai, A. Lin, T. Yang, J. Zhang, P. Luo, J. Med. Internet Res. 2024, 26, e54607.

[12] L. Zhu, W. Mou, Y. Lai, J. Chen, S. Lin, L. Xu, J. Lin, Z. Guo, T. Yang, A. Lin, C. Qi, L. Gan, J. Zhang, P. Luo, Int. J. Surg. 2024, 110, 4096.

[13] J. Chen, L. Zhu, W. Mou, D. Zeng, C. Qi, Z. Liu, A. Jiang, B. Tang, W. Shi, U. D. Kahlert, J. Zhou, S. Guo, X. Lu, X. Sun, T. Ngo, Z. Pu, B. Jia, C. O. Jeon, Y. He, H. Wu, S. Gu, W. Cheungpasitporn, H. Huang, W. Mao, S. Wang, X. Chen, L. Cabannes, G. S. G. Ren, I. S. Whitaker, S. Ali, Q. Cheng, K. Miao, S. Yuan, P. Luo, iMetaOmics 2024, 1, e7.

[14] Z. B. Akhtar, J. Electr. Syst. Inf. Technol. 2024, 11, 22.

[15] D. Horiuchi, H. Tatekawa, T. Oura, T. Shimono, S. L. Walston, H. Takita, S. Matsushita, Y. Mitsuyama, Y. Miki, D. Ueda, Eur. Radiol. 2024, 35, 506.

[16] Y. Zhou, H. Ong, P. Kennedy, C. C. Wu, J. Kazam, K. Hentel, A. Flanders, G. Shih, Y. Peng, Radiology 2024, 311, e233270.

[17] J. Huang, D. M. Yang, R. Rong, K. Nezafati, C. Treager, Z. Chi, S. Wang, X. Cheng, Y. Guo, L. J. Klesse, G. Xiao, E. D. Peterson, X. Zhan, Y. Xie, npj Digital Med. 2024, 7, 106.

[18] S. Apornvirat, W. Thinpanja, K. Damrongkiet, N. Benjakul, T. Laohawetwanit, Ann. Diagn. Pathol. 2024, 73, 152359.

[19] E. Steimetz, J. Minkowitz, E. C. Gabutan, J. Ngichabe, H. Attia, M. Hershkop, F. Ozay, M. G. Hanna, R. Gupta, JAMA Network Open 2024, 7, e2412767.

[20] R. Kurokawa, Y. Ohizumi, J. Kanzawa, M. Kurokawa, Y. Sonoda, Y. Nakamura, T. Kiguchi, W. Gonoi, O. Abe, Jpn. J. Radiol. 2024, 42, 1399.

[21] Gemini Team Google, v5, 2024.

[22] S. Shahriar, B. D. Lund, N. R. Mannuru, M. A. Arshad, K. Hayawi, R. V. K. Bevara, A. Mannuru, L. Batool, Appl. Sci. 2024, 14, 7782.

[23] B. C. Andreiana, A. E. Stepan, C. Mărgăritescu, A. M. Al Khatib, M. M. Florescu, R. N. Ciurea, C. E. Simionescu, Curr. Health Sci. J. 2018, 44, 201.

[24] P. Schnuelle, J. Clin. Med. 2023, 12, 6424.

[25] R. J. Motzer, K. Penkov, J. Haanen, B. Rini, L. Albiges, M. T. Campbell, B. Venugopal, C. Kollmannsberger, S. Negrier, M. Uemura, J. L. Lee, A. Vasiliev, W. H. Miller, H. Gurney, M. Schmidinger, J. Larkin, M. B. Atkins, J. Bedke, B. Alekseev, J. Wang, M. Mariani, P. B. Robbins, A. Chudnovsky, C. Fowst, S. Hariharan, B. Huang, A. di Pietro, T. K. Choueiri, N. Engl. J. Med. 2019, 380, 1103.

[26] K. Zarrabi, E. Walzer, M. Zibelman, Cancers 2021, 13, 3652.

[27] G. Kristiansen, B. Delahunt, J. R. Srigley, C. Lüders, J. M. Lunkenheimer, H. Gevensleben, T. Thiesler, R. Montironi, L. Egevad, Pathology 2015, 36, 310.

[28] M. Ilse, J. Tomczak, M. Welling, in Proc. 35th Int. Conf. Mach. Learn., PMLR 2018, pp. 2127–2136.

[29] Z. Huang, F. Bianchi, M. Yuksekgonul, T. J. Montine, J. Zou, Nat. Med. 2023, 29, 2307.

[30] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, F.-F. Li, in 2009 IEEE Conf. Comput. Vis. Pattern Recogn., IEEE, Miami, FL, USA 2009, pp. 248–255.

[31] K. He, X. Zhang, S. Ren, J. Sun, in 2016 IEEE Conf. Comput. Vis. Pattern Recogn., IEEE, Las Vegas, NV, USA 2016, pp. 770–778.

[32] G. Huang, Z. Liu, L. van der Maaten, K. Q. Weinberger, in 2017 IEEE Conf. Comput. Vis. Pattern Recogn., IEEE, Honolulu, HI, USA 2017, pp. 2261–2269.

[33] D. Chen, M. Fu, L. Chi, L. Lin, J. Cheng, W. Xue, C. Long, W. Jiang, X. Dong, J. Sui, D. Lin, J. Lu, S. Zhuo, S. Liu, G. Li, G. Chen, J. Yan, Nat. Commun. 2022, 13, 6903.

[34] J. J. Hsieh, M. P. Purdue, S. Signoretti, C. Swanton, L. Albiges, M. Schmidinger, D. Y. Heng, J. Larkin, V. Ficarra, Nat. Rev. Dis. Primer 2017, 3, 17009.

[35] C.-L. Yin, Y.-J. Ma, Curr. Mol. Pharmacol. 2024, 17, e18761429266116.

[36] J. Goveia, K. Rohlenova, F. Taverna, L. Treps, L. C. Conradi, A. Pircher, V. Geldhof, L. P. de Rooij, J. Kalucka, L. Sokol, M. García-Caballero, Y. Zheng, J. Qian, L. A. Teuwen, S. Khan, B. Boeckx, E. Wauters, H. Decaluwé, P. De Leyn, J. Vansteenkiste, B. Weynand, X. Sagaert, E. Verbeken, A. Wolthuis, B. Topal, W. Everaerts, H. Bohnenberger, A. Emmert, D. Panovska, F. De Smet, F. J. Staal, R. J. Mclaughlin, F. Impens, V. Lagani, S. Vinckier, M. Mazzone, L. Schoonjans, M. Dewerchin, G. Eelen, T. K. Karakach, H. Yang, J. Wang, L. Bolund, L. Lin, B. Thienpont, X. Li, D. Lambrechts, Y. Luo, P. Carmeliet, Cancer Cell 2020, 37, 21.

[37] S. Chevrier, J. H. Levine, V. R. T. Zanotelli, K. Silina, D. Schulz, M. Bacac, C. H. Ries, L. Ailles, M. A. S. Jewett, H. Moch, M. van den Broek, C. Beisel, M. B. Stadler, C. Gedye, B. Reis, D. Pe’er, B. Bodenmiller, Cell 2017, 169, 736.

[38] C. D. Bahadir, M. Omar, J. Rosenthal, L. Marchionni, B. Liechty, D. J. Pisapia, M. R. Sabuncu, Nat. Rev. Electr. Eng. 2024, 1, 93.

[39] Y. Zhang, H. Liu, B. Sheng, Y. C. Tham, H. Ji, J. Hepatol. 2024, 80, e279.

[40] J. Liu, Y. Du, K. Yang, Y. Wang, X. Hu, Z. Wang, Y. Liu, P. Sun, A. Boukerche, V. C. M. Leung, v2, 2025.

[41] Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, P. Fung, ACM Comput. Surv. 2023, 55, 1.

[42] D. F. Steiner, P.-H. C. Chen, C. H. Mermel, BBA Rev. Cancer 2021, 1875, 188452.

[43] B. Mittelstadt, C. Russell, S. Wachter, in Proc. Conf. Fairness, Accountability, and Transparency, Association for Computing Machinery, New York, NY, USA 2019, pp. 279–288.

RIGHTS & PERMISSIONS

2025 The Author(s). Interdisciplinary Medicine published by Wiley-VCH GmbH on behalf of Nanfang Hospital, Southern Medical University.