A perspective on developing foundation models for analyzing spatial transcriptomic data

Abstract

Do we need a foundation model (FM) for spatial transcriptomic analysis? To answer this question, we prepared this perspective as a primer. We first review the current progress of developing FMs for modeling spatial transcriptomic data and then discuss possible tasks that can be addressed by FMs. Finally, we explore future directions of developing such models for understanding spatial transcriptomics by describing both opportunities and challenges. In particular, we expect that a successful FM should boost research productivity, increase novel biological discoveries, and provide user-friendly access.

Keywords

artificial intelligence / foundation models / perspective / spatial transcriptomics data

References

[1] Bommasani R, Hudson DA, Adeli E, Altman R, Arora S, von Arx S, et al. On the opportunities and risks of foundation models. 2021. Preprint at arXiv:210807258.

[2] Zhao WX, Zhou K, Li J, Tang T, Wang X, Hou Y, et al. A survey of large language models. 2023. Preprint at arXiv:230318223.

[3] Wu J, Gan W, Chen Z, Wan S, Philip SY. Multimodal large language models: a survey. In: 2023 IEEE international conference on big data (BigData). IEEE. 2023. p. 2247- 56.

[4] Moor M, Banerjee O, Abad ZSH, Krumholz HM, Leskovec J, Topol EJ, et al. Foundation models for generalist medical artificial intelligence. Nature. 2023; 616 (7956): 259- 65.

[5] Vaswani A. Attention is all you need. Adv Neural Inf Process Syst. 2017.

[6] Li M, Zhang Y, Li Z, Chen J, Chen L, Cheng N, et al. From quantity to quality: boosting llm performance with self-guided data selection for instruction tuning. 2023. Preprint at arXiv:230812032.

[7] Wang J, Zhang B, Du Q, Zhang J, Chu D. A survey on data selection for LLM instruction tuning. 2024. Preprint arXiv:240205123.

[8] Jia C, Hu Y, Kelly D, Kim J, Li M, Zhang NR. Accounting for technical noise in differential expression analysis of single-cell RNA sequencing data. Nucleic Acids Res. 2017; 45 (19): 10978- 88.

[9] Liu X, Zeira R, Raphael BJ. Partial alignment of multislice spatially resolved transcriptomics data. Genome Res. 2023; 33 (7): 1124- 32.

[10] Cui H, Wang C, Maan H, Pang K, Luo F, Duan N, et al. scGPT: toward building a foundation model for single-cell multiomics using generative AI. Nat Methods. 2024; 21 (8): 1- 11.

[11] Hao M, Gong J, Zeng X, Liu C, Guo Y, Cheng X, et al. Large-scale foundation model on single-cell transcriptomics. Nat Methods. 2024; 21 (8): 1- 11.

[12] Theodoris CV, Xiao L, Chopra A, Chaffin MD, Al Sayed ZR, Hill MC, et al. Transfer learning enables predictions in network biology. Nature. 2023; 618 (7965): 616- 24.

[13] Liu T, Li K, Wang Y, Li H, Zhao H. Evaluating the utilities of foundation models in single-cell data analysis. 2023. Preprint at bioRxiv:2023.09.08.555192.

[14] Kedzierska KZ, Crawford L, Amini AP, Lu AX. Assessing the limits of zero-shot foundation models in single-cell biology. 2023. Preprint at bioRxiv:2023.10.16.561085.

[15] Boiarsky R, Singh NM, Buendia A, Getz G, Sontag D. A deep dive into single-cell RNA sequencing foundation models. 2023. Preprint at bioRxiv:2023.10.19.563100.

[16] Ahlmann-Eltze C, Huber W, Anders S. Deep learning-based predictions of gene perturbation effects do not yet outperform simple linear methods. 2024. Preprint at bioRxiv:2024.09.16.6 13342.

[17] Zhao P, Zhu J, Ma Y, Zhou X. Modeling zero inflation is not necessary for spatial transcriptomics. Genome Biol. 2022; 23 (1): 118.

[18] Dong M, Su D, Kluger H, Fan R, Kluger Y. SIMVI reveals intrinsic and spatial-induced states in spatial omics data. 2024. Preprint at bioRxiv:2023.08.28.554970.

[19] Smith KD, Prince DK, MacDonald JW, Bammler TK, Akilesh S. Challenges and opportunities for the clinical translation of spatial transcriptomics technologies. Glomerular Dis. 2024; 4 (1): 49- 63.

[20] Liu X, Zhang F, Hou Z, Mian L, Wang Z, Zhang J, et al. Self-supervised learning: generative or contrastive. IEEE Trans Knowl Data Eng. 2021; 35 (1): 857- 76.

[21] Birk S, Bonafonte-Pardàs I, Feriz AM, Boxall A, Agirre E, Memi F, et al. Large-scale characterization of cell niches in spatial atlases using bio-inspired graph learning. 2024. Preprint at bioRxiv:2024.02.21.581428.

[22] Blampey Q, Benkirane H, Bercovici N, Andre F, Cournede PH. Novae: a graph-based foundation model for spatial transcriptomics data. 2024. Preprint at bioRxiv:2024.09.09.612009.

[23] Schaar AC, Tejada-Lapuerta A, Palla G, Gutgesell R, Halle L, Minaeva M, et al. Nicheformer: a foundation model for single-cell and spatial omics. 2024. Preprint at bioRxiv:2024.04.15.589472.

[24] Cao S, Yuan Y. A framework for gene representation on spatial transcriptomics. 2024. Preprint at bioRxiv:2024.09.27.615337.

[25] Wen H, Tang W, Jin W, Ding J, Liu R, Dai X, et al. Single cells are spatial tokens: transformers for spatial transcriptomic data imputation. 2023. Preprint at arXiv:230203038.

[26] Wen H, Tang W, Dai X, Ding J, Jin W, Xie Y, et al. CellPLM: pre-training of cell language model beyond single cells. In: The twelfth international conference on learning representations; 2024.

[27] Singhal K, Azizi S, Tu T, Mahdavi SS, Wei J, Chung HW, et al. Large language models encode clinical knowledge. Nature. 2023; 620 (7972): 172- 80.

[28] Zhang K, Zhou R, Adhikarla E, Yan Z, Liu Y, Yu J, et al. A generalist vision-language foundation model for diverse biomedical tasks. Nat Med. 2024; 30 (11): 1- 13.

[29] Huang CH. QuST-LLM: integrating Large Language Models for comprehensive spatial transcriptomics analysis. 2024. Preprint at arXiv:240614307.

[30] Liu T, Xiao Y, Luo X, Xu H, Zheng W, Zhao H. Geneverse: a collection of open-source multimodal Large Language Models for genomic and proteomic research. In: Al-Onaizan Y, Bansal M, Chen YN, editors. Findings of the association for computational linguistics: EMNLP 2024. Association for Computational Linguistics; 2024. p. 4819- 36.

[31] Liu T, Huang T, Ying R, Zhao H. spEMO: exploring the capacity of foundation models for analyzing spatial multi-omic data. 2025. Preprint at bioRxiv:2025.01.13.632818.

[32] Wang CX, Cui H, Zhang AH, Xie R, Goodarzi H, Wang B. scGPT-spatial: continual pretraining of single-cell foundation model for spatial transcriptomics. 2025. Preprint at bioRxiv:2025.02.05. 636714.

[33] Theodoris CV. Perspectives on benchmarking foundation models for network biology. Quantitative Biology. 2024; 12 (4): 335- 8.

[34] Hao M, Wei L, Yang F, Yao J, Theodoris CV, Wang B, et al. Current opinions on large cellular models. Quantitative Biology. 2024; 12 (4): 433- 43.

[35] Yuan Z, Zhao F, Lin S, Zhao Y, Yao J, Cui Y, et al. Benchmarking spatial clustering methods with spatially resolved transcriptomics data. Nat Methods. 2024; 21 (4): 712- 22.

[36] Ospina O, Soupir A, Fridley BL. A primer on preprocessing, visualization, clustering, and phenotyping of barcode-based spatial transcriptomics data. In: Statistical genomics. Springer; 2023. p. 115- 40.

[37] Piñeiro AJ, Houser AE, Ji AL. Research techniques made simple: spatial transcriptomics. J Invest Dermatol. 2022; 142 (4): 993- 1001.

[38] Dries R, Chen J, Del Rossi N, Khan MM, Sistig A, Yuan GC. Advances in spatial transcriptomic data analysis. Genome Res. 2021; 31 (10): 1706- 18.

[39] Chen Y, Zou J. GenePT: a simple but effective foundation model for genes and cells built from ChatGPT. 2023. Preprint at bioRxiv:2023.10.16.562533.

[40] Liu T, Chen T, Zheng W, Luo X, Zhao H. scELMo: embeddings from Language Models are good learners for single-cell data analysis. 2023. Preprint at bioRxiv:2023.12.07.569910.

[41] Halmos P, Liu X, Gold J, Chen F, Ding L, Raphael BJ. DeST-OT: alignment of spatiotemporal transcriptomics data. In: International conference on research in computational molecular biology. Springer; 2024. p. 434- 7.

[42] Chitra U, Arnold BJ, Sarkar H, Ma C, Lopez-Darwin S, Sanno K, et al. Mapping the topography of spatial gene expression with interpretable deep learning. In: International conference on research in computational molecular biology. Springer; 2024. p. 368- 71.

[43] Luecken MD, Büttner M, Chaichoompu K, Danese A, Interlandi M, Müller MF, et al. Benchmarking atlas-level data integration in single-cell genomics. Nat Methods. 2022; 19 (1): 41- 50.

[44] Li B, Zhang W, Guo C, Xu H, Li L, Fang M, et al. Benchmarking spatial and single-cell transcriptomics integration methods for transcript distribution prediction and cell type deconvolution. Nat Methods. 2022; 19 (6): 662- 70.

[45] Bunne C, Roohani Y, Rosen Y, Gupta A, Zhang X, Roed M, et al. How to build the virtual cell with artificial intelligence: priorities and opportunities. 2024. Preprint at arXiv:240911654.

[46] M Bran A, Cox S, Schilter O, Baldassari C, White AD, Schwaller P. Augmenting large language models with chemistry tools. Nat Mach Intell. 2024; 6 (5): 1- 11.

[47] Rosen Y, Roohani Y, Agrawal A, Samotorcan L, Consortium TS, Quake SR, et al. Universal cell embeddings: a foundation model for cell biology. 2023. Preprint at bioRxiv:2023.11.28.568918.

[48] Touvron H, Lavril T, Izacard G, Martinet X, Lachaux MA, Lacroix T, et al. Llama: open and efficient foundation language models. 2023. Preprint at arXiv:230213971.

[49] Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, Aleman FL, et al. Gpt-4 technical report. 2023. Preprint at arXiv:230308774.

[50] Team G, Anil R, Borgeaud S, Wu Y, Alayrac JB, Yu J, et al. Gemini: a family of highly capable multimodal models. 2023. Preprint at arXiv:231211805.

[51] The Claude 3 Model Family: Opus, Sonnet, Haiku. Available from SEMANTICS SCHOLAR website (CorpusID:268232499).

RIGHTS & PERMISSIONS

The Author(s). Quantitative Biology published by John Wiley & Sons Australia, Ltd on behalf of Higher Education Press.