1 Introduction
Health equity is crucial for improving population health outcomes [
1]. However, due to the uneven distribution of healthcare resources worldwide, significant disparities exist in access to high-quality healthcare services across different regions [
2–
5]. The remote diagnosis and intelligent analysis capabilities enabled by artificial intelligence (AI) models hold promise for narrowing these disparities [
6]. In single-task applications such as radiology, dermatology, and ophthalmology, deep learning convolutional neural network (CNN) models have demonstrated diagnostic capabilities comparable to those of clinicians [
7]. However, these models heavily rely on large volumes of expert-annotated data [
8]. This not only incurs substantial annotation costs but also makes models prone to overfitting due to limited data scale, resulting in poor generalization performance [
9]. More importantly, clinical disease diagnosis typically requires multimodal text and image data, whereas CNN models generally utilize only single-modality data. Consequently, their outputs may lack clinical explainability and logical coherence [
10,
11].
To address these challenges, foundation models have emerged. These models typically employ self-supervised learning (SSL) paradigms, enabling them to learn general representations and knowledge directly from massive datasets without requiring costly human annotation. This makes them powerful foundational platforms capable of rapidly adapting to diverse downstream tasks through fine-tuning [
12]. Their mainstream architectures primarily include encoders, decoders, and encoder-decoder structures [
13]. Among these, the Transformer architecture has been widely adopted by foundation models due to its self-attention mechanism, which flexibly handles diverse input types and jointly learns input representations from datasets of varying origins. This cross-modal learning and data processing capability facilitates the evolution of foundation models from single-modal to multi-modal systems [
14,
15]. Furthermore, foundation models exhibit “emergent capabilities,” meaning they can perform tasks not explicitly intended during training. This ability strengthens as model scale increases or data quality improves, endowing models with the potential for zero-shot or few-shot prediction on new tasks [
16] (Figure 1).
In the field of ophthalmology, the application of foundation models has advanced primarily across three major categories: large language models (LLMs), large vision models (LVMs), and large multimodal models (LMMs) [
19]. While LLMs have demonstrated capabilities in guiding patient consultations, streamlining medical documentation, and enhancing ophthalmic education [
20], ophthalmic clinical practice often relies on multiple imaging modalities, such as optical coherence tomography (OCT) and color fundus photography (CFP). LLMs, which only support text-based interactions, struggle to perform eye disease examinations and diagnoses. To address this, Moorfields Eye Hospital collaborated with a research team from University College London (UCL) to develop the first ophthalmology-specific large vision model, RETFound [
19]. RETFound comprises two independent versions pre-trained on CFP and OCT images, respectively. Employing masked autoencoder-based SSL, the model demonstrates high accuracy in diagnosing ocular diseases, assessing prognosis, and even predicting systemic diseases [
21]. However, the model has yet to achieve multimodal information fusion between CFP and OCT images, nor does it possess text processing capabilities. In actual clinical practice, ophthalmologists must integrate multiple imaging modalities and medical history texts to diagnose various diseases and perform numerous clinical tasks. Against this backdrop, LMMs have emerged. Integrating the technical strengths of LLMs and LVMs, these systems leverage contrastive language-image pre-training (CLIP) to not only process and generate text but also simultaneously understand and generate diverse modalities including images, videos, and audio [
22]. For instance, VisionFM employs a single decoder independent of input modality to process images from diverse ophthalmic imaging modalities [
23]. Moreover, EyeCLIP, VisionFM, and MIRAGE further combine images and text [
23–
25]. These advancements collectively provide a viable technical pathway for enhancing the efficiency of clinical ophthalmic diagnosis and treatment.
Although ophthalmic AI foundation models have emerged in recent years, there remains a lack of systematic reviews and comparative analyses examining their architectural designs, performance characteristics, and application prospects. This review aims to fill this gap by systematically cataloging and dissecting current representative ophthalmic foundation models, clarifying their respective strengths and limitations, and projecting future application pathways and development directions based on this analysis.
2 Model Selection and Eligibility Criteria
We searched the IEEE and PubMed databases for literature on foundation models in ophthalmology and identified a total of 12 models (Table S1): RETFound [
21], FLAIR [
26], VisionFM [
23], EyeFound [
27], FMUE [
28], URFound [
29], RetiZero [
30], VisionUnite [
31], RET-CLIP [
32], EyeCLIP [
24], EyeFM [
33], and MIRAGE [
25]. To ensure the reliability, clinical relevance, and methodological rigor of this review, we selected ophthalmic foundation models that meet three core criteria:
1.Independent development: Models should represent an original pre-training effort, rather than serving as a downstream adaptation of existing ophthalmic foundation models;
2.Dataset scale and quality: Pre-training datasets must contain more than 200,000 images and incorporate private clinical data. Accordingly, models trained exclusively on public datasets were excluded, as they may fail to capture the full complexity and diversity of real-world clinical practice [
34];
3.Rigorous external validation: Models are required to undergo validation on at least one independent dataset rather than relying solely on internal test splits.
Based on these criteria, five models were selected: RETFound, VisionFM, EyeCLIP, EyeFM, and MIRAGE, representing robust, independently developed, and clinically oriented foundation models in ophthalmology.
3 Model Construction: From Single-Modal to Multi-Modal Fusion
In terms of model architecture, most existing ophthalmic foundation models rely on masked autoencoders (MAE) for self-supervised pre-training, with core components comprising encoders and decoders. However, notable differences still exist: (1) Encoder-decoder design across models: some employ modality-specific encoders or decoders to capture unique features of distinct imaging modalities, while others adopt modality-agnostic shared designs to enhance architectural efficiency and consistency. (2) From single modality to multimodal: RETFound processes OCT and CFP separately; while the other four models: VisionFM, EyeCLIP, EyeFM and MIRAGE, all support multimodal image processing. Among them, MIRAGE integrates two types of images, and VisionFM further expands processing to eight modalities. EyeFM and EyeCLIP integrate 5–11 modalities alongside clinical metadata, establishing a new paradigm for vision-language multimodal models in ophthalmic studies (Table 1).
3.1 Initial Single-Modal Foundation Model: RETFound
RETFound is a retinal imaging foundation model jointly developed by Moorfields Eye Hospital and UCL, with SSL as its core developmental framework. As an emerging machine learning paradigm, SSL can directly extract supervisory signals from within the data itself rather than relying on external annotations, thereby leveraging vast amounts of unlabeled data to acquire general knowledge representations [
35]. Zhou et al. [
21] trained two separate RETFound models based on a large-scale unlabeled retinal imaging dataset, containing 904,170 CFP and 736,442 OCT images. Before SSL, all images were preprocessed: for CFP images, AutoMorph (an automated retinal image analysis tool) was employed to remove image backgrounds and preserve retinal regions; for OCT images, only the middle slices were retained. All images were uniformly resized to 256 × 256 and subsequently randomly cropped to 224 × 224. Research confirms that this image resizing in pre-processing steps not only preserves most visual information [
36] but also demonstrates good compatibility with existing advanced deep learning architectures such as the Vision Transformer [
37–
39].
The pre-training process comprises two key stages: firstly, to ensure the capacity for capturing general visual feature representations, the model was pre-trained on the ImageNet-1k dataset, which contains approximately 1.4 million natural images, using a masked autoencoder architecture as an SSL method. Subsequently, the model underwent further SSL on approximately 1.6 million unlabeled retinal images sourced from the Moorfields Diabetic Retinopathy Image Database and multiple public datasets. Notably, the masked autoencoder employed during pre-training represents an advanced SSL technique, featuring an architecture composed of a large Vision Transformer encoder and a small Vision Transformer decoder. The masking ratios for CFP and OCT images were 0.75 and 0.85, respectively. The encoder divided input images into 16 × 16-pixel unmasked patches and extracted high-level features through 24 Transformer blocks. While the decoder reconstructed the masked portions of the original image based on these features, thereby learning deep image representations. By reconstructing randomly and highly masked input images, the model effectively learns anatomy-specific contextual information unique to the retina, including structural features such as the optic disc, large vessels, nerve fiber layer, and retinal pigment epithelium. It is through this technical approach that RETFound achieves strong generalization capabilities, maintaining excellent performance even when confronted with novel, unseen clinical data [
18].
3.2 First Model Enables Modality-Agnostic Diagnosis: VisionFM
VisionFM was jointly developed by institutions including the Chinese University of Hong Kong, Beijing Tongren Hospital, and King's College London using an SSL paradigm similar to RETFound. However, unlike RETFound, VisionFM places greater emphasis on multimodal and multi-task integration. Initially, Qiu et al. [
23] constructed a massive pre-training dataset comprising 3.4 million diverse ophthalmic images from 560,457 participants. This dataset exhibits broad representativeness, including eight major ophthalmic imaging modalities: fundus photography (FP), OCT, fundus fluorescein angiography (FFA), B-scan ultrasound (B-Ultrasound), slit-lamp photography (slit-lamp), ultrasound biomicroscopy (UBM), magnetic resonance imaging (MRI) and external eye images. These images were aggregated from diverse devices, cover multiple ophthalmic disease types, and incorporate demographic information from 26 countries and regions worldwide. Another key distinction lies in the decoder architecture: RETFound employs modality-specific decoders to process a single modality, while VisionFM utilizes a modality-agnostic decoder, processing and interpreting images from multiple input modalities independently to enable modality-agnostic diagnosis.
Notably, to mitigate the scarcity of real data in certain modalities (e.g., MRI), VisionFM innovatively incorporates high-fidelity synthetic ophthalmic images generated by large generative models into its pre-training strategy. These synthetic datasets underwent rigorous visual Turing tests organized by clinicians to ensure visual authenticity and medical validity. Incorporating such synthetic data enhances the representational learning capabilities of the model across corresponding modalities and improves its performance in downstream tasks [
40]. Compared to RETFound, VisionFM not only underwent pre-training on larger datasets with high-fidelity synthetic data but also employs a modality-agnostic decoder, enabling unified analysis across eight major ophthalmic imaging modalities.
3.3 A Vision-Language Model With Contrastive Learning: EyeCLIP
EyeCLIP is a multimodal vision-language foundation model developed by the Hong Kong Polytechnic University specifically for ophthalmology. Built upon SSL while incorporating contrastive learning methods, its core objective is to achieve multimodal alignment. Contrastive learning plays a pivotal role in this framework. By constructing positive and negative sample pairs, it enables models to aggregate similar samples and reject dissimilar ones in the feature space, thereby enhancing the ability to distinguish multimodal features [
41].
The construction of EyeCLIP primarily involves three key components: a large-scale dataset, a CLIP-based architecture extension, and a unique three-stage pre-training strategy. Shi et al. [
24] integrated data from 128,554 patients across multiple regions and hospitals in China, comprising 2,777,593 multimodal ophthalmic images and 11,180 clinical text reports. This dataset, encompassing both unlabeled and partially labeled samples, forms the foundation for model pre-training. CLIP is a concise and efficient pre-training paradigm that successfully introduces text-supervised signals into visual model training, demonstrating outstanding performance across multiple tasks [
42]. Therefore, EyeCLIP adopts CLIP as its base framework, featuring independent image and text encoders to process visual and textual inputs respectively. Building upon the CLIP architecture, the researchers introduced an additional image encoder based on an MAE. This enables the model to perform masked image reconstruction while simultaneously leveraging image-text pairs for contrastive learning. Unlike RETFound and VisionFM, EyeCLIP utilizes a shared visual encoder to align data from 11 distinct modalities, thereby streamlining its architecture. This design enables EyeCLIP to effectively integrate MAE and contrastive learning strategies during pre-training.
The pre-training strategy consists of three stages. Firstly, EyeCLIP employs self-supervised reconstruction learning, in line with RETFound. With the help of the MAE, it reconstructs masked image regions to learn robust feature representations. Secondly, the model introduces multimodal image contrastive learning. This aligns different images from the same patient (e.g., CFP and OCT) in the embedding space, promoting the complementarity and fusion of cross-modal information. Finally, image-text contrastive learning further aligns ophthalmic images with corresponding clinical descriptions, establishing deep semantic connections between visual content and medical concepts. This multi-strategy fusion endows EyeCLIP with both the robust generalization of an MAE and the multimodal understanding and zero-shot potential of contrastive learning.
3.4 An Eyecare Foundation Model for Clinical Assistance: EyeFM
The eyecare foundation model (EyeFM) is an ophthalmic clinical assistance system jointly developed by Shanghai Jiao Tong University and Tsinghua University, built upon the multimodal vision-language foundation model paradigm. The system primarily consists of a visual (image) module and a language module, with its pre-training process divided into two independent stages: image module pre-training and joint vision-language pre-training.
During the first stage, a multimodal, multi-task MAE was employed using a dataset integrating multi-source data from around the globe. This includes four community-derived cohorts and six hospital-derived cohorts from China, one hospital-derived cohort each from Thailand and India, and four open-source datasets from the United States, Brazil, France, and Spain. In total, the dataset contains 14.5 million eye images covering five imaging modalities: CFP, OCT, ultra-wide-field imaging (UWF), FFA, and external eye photography (EEP). In line with EyeCLIP, EyeFM uses a unified encoder to process images across all five modalities to avoid the complexity of developing multiple modality-specific models, significantly reducing computational resource requirements. Meanwhile, EyeFM incorporates five modality-specific decoders to reconstruct masked image patches within their respective modalities, which enables the model to capture fine-grained, modality-unique features more comprehensively, thereby achieving superior performance [
43]. Notably, these modality-specific decoders are discarded after pre-training when the model is transitioned to downstream tasks.
During the vision-language pre-training stage, the researchers integrated the Llama large language model [
44] for joint vision-language pre-training, using data from internal cohorts at four Chinese hospitals. Training commenced with modality alignment pre-training using image-text pairs (e.g., medical images and corresponding reports), aiming to precisely establish semantic associations between retinal image features and ophthalmic terminology. Subsequently, the model underwent instruction fine-tuning using generated instruction-following data pairs to enhance its capability in handling complex clinical dialog tasks and generating compliant clinical text. Compared to previous ophthalmic models, EyeFM, by integrating a visual encoder with a large language model, not only processes multimodal image inputs but also enables textual content output such as generating structured imaging reports and performing clinical visual question-answering.
3.5 Foundation Model With Multi-Modal Mask Autoencoder: MIRAGE
The MIRAGE model, developed by the Medical University of Vienna, is a multimodal foundation model in ophthalmology capable of simultaneously processing and fusing information from OCT and scanning laser ophthalmoscopy (SLO). The core architecture of MIRAGE is a multimodal MAE pre-training strategy, similar to EyeFM. What sets MIRAGE apart is its paired multimodal MAE training approach. Specifically, this is characterized by fully paired pre-training datasets and interdependencies between modalities during the reconstruction process. The pre-training dataset originates from the Vienna Imaging Biomarker Eye Study (VIBES) registry, comprising approximately 261,184 paired OCT and SLO images supplemented with algorithmically generated retinal layer pseudo-labels. Incorporating these retinal layer pseudo-labels during pre-training significantly enhances the model's performance in subsequent image classification and segmentation tasks [
25]. During pre-training, input images across all modalities are randomly and heavily masked. Leveraging the dataset's fully paired nature, when regions in one modality's image are masked, the model learns reconstruction by utilizing corresponding unmasked information from the other modality as supervisory signals. This reconstruction task compels the model to learn subtle intermodal correlations and complementarities. This cross-modal information complementarity relies primarily upon the self-attention mechanism within the Vision Transformer architecture. Compared to EyeCLIP, methods based on the CLIP paradigm emphasize global feature alignment, potentially overlooking pixel-level correspondences between modalities [
45]. This leads to inherent limitations in tasks requiring high spatial precision, such as segmentation. The paired multimodal MAE approach adopted by MIRAGE offers a novel perspective for overcoming the limitations of current models that handle only single or loosely mixed modalities by explicitly modeling tightly paired cross-modal dependencies.
4 Applications in Ophthalmology
4.1 RETFound
After fine-tuning, RETFound demonstrates significant advantages in disease diagnosis, achieving high accuracy in systematic validation across multiple critical ocular diseases. This includes multi-category ocular disease differentiation [
21], diabetic retinopathy (DR) grading, glaucoma diagnosis, choroidal melanoma versus pigmented nevus discrimination [
46], macular degeneration classification [
47], optic neuropathy detection [
48], and identification of epiretinal membrane (ERM) versus retinal pigment epithelial detachment [
49].
In internal validation, Zhou et al. [
21] demonstrated that RETFound achieved superior area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPR) in diagnosing DR, glaucoma, and multi-category eye diseases based on multiple public DR datasets, including Kaggle APTOS-2019, IDRID, and MESSIDOR-2, consistently outperforming baseline models such as SL-ImageNet, SSL-ImageNet, and SSL-Retinal (Figure 2). Furthermore, RETFound achieved top cross-validation performance across multiple datasets. After fine-tuning on Kaggle APTOS-2019, the AUROC reached 0.822 (95% CI: 0.815–0.829) on the IDRID dataset, significantly higher than SL-ImageNet.
The superiority of RETFound was further confirmed via comparisons with traditional models such as CNNs. When the training data was limited (sample size < 1000), RETFound achieved a higher AUROC for referral-grade glaucoma detection than CNN models [
50]. Moreover, RETFound demonstrated outstanding performance in drusen classification, significantly outperforming ResNet-50 both on internal datasets (AUROC: 0.66 vs. 0.56) and external datasets (AUROC: 0.84 vs. 0.44) [
49]. A subsequent study [
51] verified that CNN models exhibit significant diagnostic performance degradation on datasets with different distributions, whereas RETFound demonstrates superior robustness with relatively stable performance.
For downstream tasks, the performance of RETFound remains robust. In cross-regional validation, the model achieved an AUROC exceeding 0.90 in more than half of the eye disease classification tasks [
52]. Moreover, in cross-age validation, the model was able to identify pathological features of prematurity despite being pre-trained on adult data, thereby demonstrating robust cross-age transferability [
52]. Regarding cross-racial validation, while unimodal models exhibited bias in tasks involving different genders and races, multimodal fusion models showed no significant differences in AUROC for glaucoma diagnosis across different racial subgroups [
53,
54]. In cross-disease validation, the application scenarios of RETFound have been extended to systemic disease prediction [
21], retinal age prediction [
55], explainable AI-assisted biomarker discovery [
56], and multi-disease screening using low-quality fundus images in resource-limited settings [
51], all of which have yielded promising results (Table 2).
4.2 VisionFM
VisionFM is capable of accurately performing various complex ophthalmic tasks, including disease diagnosis and grading, prognosis prediction, medical image segmentation, and anatomical landmark detection. The model has been applied to the identification and auxiliary diagnosis of a series of common eye diseases, including DR, glaucoma, age-related macular degeneration (AMD), cataracts, hypertensive retinopathy, retinal vein occlusion (RVO), pathological myopia, and retinal detachment [
23].
Notably, VisionFM employs a modality-agnostic decoder capable of simultaneously identifying all eight diseases, achieving an average AUROC of 0.993. Researchers further compared the diagnostic capabilities of VisionFM against nine ophthalmologists with varying levels of experience. Results showed that in the combined diagnosis task for 12 ophthalmic diseases, subcategories of the eight primary diseases, VisionFM outperformed both junior and intermediate ophthalmologists. Its F1 score was nearly double that of intermediate-level doctors (82.8% vs. 42.6%), indicating robust diagnostic performance across different imaging modalities for various eye conditions. Furthermore, VisionFM demonstrated strong performance when diagnosing diseases using imaging modalities unseen during pre-training. For instance, when grading DR using optical coherence tomography angiography (OCTA) (unseen during pre-training) on the public DRAC dataset, it achieved an average AUROC of 0.935. Moreover, when diagnosing ocular albinism, a condition underrepresented in the pre-training data, VisionFM attained an AUROC of 0.993 with just one labeled sample, demonstrating exceptional few-shot learning capabilities.
For downstream tasks, Fecso et al. [
57] proposed the RetFiner vision-language refinement scheme, which further fine-tuned VisionFM. This scheme employs a special feature pooling strategy that concatenates the CLS token with the average pool of patch tokens to integrate global features with local details. After fine-tuning, VisionFM achieves comprehensive improvements in linear detection performance across seven retinal disease classification datasets: balanced accuracy (BAcc) increases by 2.1%, AUROC increases by 1.6%, and the average precision (AP) increases by 1.9% [
57] (Table 2).
4.3 EyeCLIP
EyeCLIP's performance has been systematically validated across 14 benchmark datasets encompassing 11 ophthalmic imaging modalities, demonstrating outstanding capabilities in diverse downstream tasks, including disease classification, visual question answering, and cross-modal retrieval [
24]. The model's disease diagnosis scope is extensive, covering common eye diseases like DR and glaucoma, multi-disease co-diagnosis, and strong diagnostic potential for 17 rare eye diseases (such as choroidal melanoma and retinitis pigmentosa).
EyeCLIP demonstrates superior zero-shot classification performance across nine public ophthalmic datasets. For instance, when diagnosing DR on the MESSIDOR-2 dataset using CFP images, EyeCLIP achieved an AUROC of 0.739, significantly higher than RETFound's 0.471 (Figure 3). For OCT images, EyeCLIP achieved AUROCs of 0.800 and 0.776 on the OCTID and OCTDL datasets for multi-disease classification, respectively, again significantly outperforming all benchmark models (all p < 0.001). In few-shot classification tasks, evaluation on a subset of the Retina Image Bank dataset containing 17 rare diseases (each with over 16 samples) demonstrated that EyeCLIP significantly outperformed all baseline models in both AUROC and AUPR (p < 0.05) (Figure 4). Under full-data supervised training, EyeCLIP comprehensively outperformed all comparison models in multimodal tasks. For instance, on a complex retinal image library dataset encompassing 14 modalities and 84 disease types, EyeCLIP achieved an AUROC of 0.561 for rare diseases detection, surpassing RETFound (0.545) with a statistically significant difference (p < 0.001). In summary, EyeCLIP demonstrates exceptional robustness in handling diverse imaging modalities and complex pathological scenarios.
4.4 EyeFM
EyeFM can diagnose multiple ophthalmic conditions by integrating data from various imaging modalities. Its input formats include OCT, CFP, external eye photographs, and even ultra-widefield OCTA images not encountered during pre-training. The model covers a broad spectrum of diseases, including DR, glaucoma, AMD, myopic macular degeneration, central foveal diabetic macular edema (ciDME), and anterior segment diseases such as cataracts.
Retrospective validation demonstrated that EyeFM achieved high AUROC scores in both single-modality disease detection and integrated multimodal diagnosis tasks [
33]. Notably, in diagnosing AMD (AUROC: 0.932, 95% CI: 0.927–0.937), ciDME (AUROC: 0.845, 95% CI: 0.833–0.857), and glaucoma (AUROC: 0.821, 95% CI: 0.810–0.833), the model using integrated inputs demonstrated significantly superior performance compared to those using single modality inputs (
p < 0.001). Furthermore, EyeFM demonstrated excellent cross-modal disease detection capabilities. In diagnosing ciDME, a task typically requiring OCT images, EyeFM achieved the highest AUROC (0.883, 95% CI: 0.872–0.893) using only CFP images, significantly outperforming models based on ImageNet and RETFound (
p < 0.001) [
33].
EyeFM has also been validated in multi-country reader studies and shown to effectively improve diagnostic performance [
33]. Compared to independent physician diagnosis, ophthalmologists using EyeFM demonstrated significantly improved sensitivity for common eye diseases, including DR requiring referral, suspected glaucoma, and suspected AMD, while maintaining specificity. Sensitivity for diagnosing DR requiring referral increased from 0.745 to 0.867. In cross-modal detection of ciDME, sensitivity also rose from 45.3% to 59.0% (
p < 0.001). More critically, a double-blind randomized controlled trial (RCT) demonstrated that ophthalmologists using EyeFM achieved significantly higher diagnostic accuracy (92.2% vs. 75.4%,
p < 0.001) and referral accuracy (92.2% vs. 80.5%,
p < 0.001) compared to standard care. Particularly for macular degeneration requiring referral typically confirmed by OCT, EyeFM's cross-modal capabilities substantially improved clinicians' diagnostic accuracy based solely on CFP (intervention group: 85.9%; control group: 57.5%;
p < 0.001).
4.5 MIRAGE
The MIRAGE model underwent comprehensive evaluation across eight public datasets and one internal dataset, covering multiple ophthalmic diseases and diagnostic tasks. Its validation scope extensively covered the following: diagnosis and staging of AMD; diagnosis and staging of glaucoma; differentiation and detection of DR and DME; and identification and classification of key features of multiple complex fundus pathologies, including RVO, retinal artery occlusion (RAO), vitreomacular interface disease (VID), macular hole (MH), central serous chorioretinopathy (CSR), ERM, and drusen.
In the classification task based on OCT images, MIRAGE achieved superior performance, with an AUROC, AP, and BAcc of 0.956%, 0.930%, and 84.0% respectively, conspicuously higher than RETFound (AUROC higher by 1.15% points, p < 0.001). In the classification task based on SLO images, MIRAGE achieved an average AUROC of 0.837, also exceeding self-supervised learning-ImageNet (SL-IN) by 1.15% points (p < 0.05). In the more challenging cross-dataset generalization test, MIRAGE demonstrated outstanding performance. For instance, in the cross-center OCT classification task on the Noor Eye Hospital dataset, MIRAGE achieved an AUROC of 0.948, significantly higher than RETFound's 0.816 (p < 0.01). These results collectively demonstrate that MIRAGE delivers leading performance in OCT and SLO image analysis, particularly in disease diagnosis and staging tasks.
5 Technical Support for Generalization and Explainability
Among various technical approaches for enhancing model generalization, the MAE has demonstrated substantial value. Trained through staged self-supervised pre-training with an MAE, the RETFound model outperforms baseline models across multiple cross-dataset evaluations, demonstrating exceptional out-of-domain generalization performance [
21]. Similarly, EyeCLIP and MIRAGE adopt the MAE as their pre-training core, though each introduces distinct architectural innovations: EyeCLIP employs a unified encoder to process multimodal ophthalmic images and incorporates multimodal image contrastive learning and image-text contrast learning to enhance semantic alignment between visual and linguistic representations [
24]; MIRAGE proposes a paired multimodal masked autoencoder, which fuses complementary information across modalities through cross-attention mechanisms, thereby improving both model robustness and cross-modal generalization [
25]. Notably, the use of synthetic data also provides an effective pathway for improving model performance [
40]. MIRAGE automatically generates pseudo-labels for retinal layers, enabling the model to learn discriminative structural features even with limited expert annotations [
25]. VisionFM incorporates high-quality synthetic ophthalmic images and adopts a modality-agnostic design, demonstrating exceptional adaptability in few-shot scenarios and tasks involving unseen modalities [
23].
In terms of explainability, the complexity, high-dimensional parameters, and autonomous learning characteristics of AI models collectively contribute to a significant “black box” problem: while one can observe model inputs and outputs and evaluate performance metrics, the internal process by which a model derives its final decisions from the input data remains opaque [
58]. In healthcare domains, this lack of explainability severely hinders the clinical translation and application of such models [
59]. Currently, a common explainability enhancement technique involves using visualization methods to identify the image regions a model focuses on when diagnosing specific diseases (Figure 5). Typical foundation models employing this strategy include VisionFM, EyeCLIP, and EyeFM. Generally, these models leverage attention mechanisms to visually highlight heatmap regions of interest during ophthalmic image processing. When all attention heads are combined, these models consistently focus on the foreground regions of the image and clinically relevant targets. However, while this technique reveals areas of focus, it fails to establish an explicit causal link between multimodal interactions and model performance. A model's “focus” on a region does not imply that that region is the “cause” of the disease. It may merely represent a discriminative feature.
6 Discussion
In summary, foundation models are trained on massive multimodal data and employ diverse architectural designs. Leveraging techniques such as MAE, contrastive learning, and synthetic data augmentation, these models demonstrate strong generalization capabilities across datasets, cross-center validation, and even in zero-shot or few-shot tasks. In clinical applications, certain models achieve high diagnostic accuracy for common ophthalmic diseases, while others are able to recognize rare disease features through vision-language contrastive learning or cross-modal paired data training. Additionally, systems integrating LLMs serve as clinical aids, effectively enhancing physician diagnostic accuracy and assisting in the generation of standardized clinical metadata.
6.1 Model Synthesis and Selection
Foundation models are undergoing a critical shift from monomodal architectures to a multimodal paradigm. Multimodal fusion not only improves model performance but also mitigates bias. The underlying reason lies in the inherently multimodal nature of clinical diagnosis. In ophthalmic practice, diagnostic decisions require the integration of various sources of information, including CFP, OCT, OCTA, visual field testing, medical history, and symptoms [
60]. While RETFound [
21] can process CFP and OCT images separately and achieves good performance in the diagnosis and prognosis of common eye diseases, as well as systemic disease prediction, its architecture does not support cross-modal information fusion. Consequently, it cannot leverage complementary information and may even exacerbate the risk of bias [
53]. To address this limitation, multimodal foundation models, through cross-modal joint learning, can extract richer features and demonstrate superior performance in few-shot and zero-shot scenarios. For multimodal image fusion, VisionFM [
23] employs a modality-agnostic decoder to expand its supported modalities to eight types (CFP, OCT, FFA, B-Ultrasound, slit-lamp, UBM, MRI, and external eye images). It not only achieves multi-disease recognition but also demonstrates the potential to predict systemic biomarkers and even intracranial tumors from fundus images. MIRAGE focuses on paired OCT and SLO data, achieving pixel-level segmentation capabilities through a paired learning paradigm [
25]. In addition to expanding image modalities, the incorporation of clinical text provides a critical bridge for foundation models to transition from the laboratory to clinical practice. EyeCLIP processes 11 image modalities using a unified image encoder and combines them with clinical text for joint learning. It can classify rare diseases in few-shot scenarios and supports image retrieval based on text descriptions, as well as cross-modal pathological correlation [
24]. More importantly, multimodal foundation models are beginning to be integrated into real-world clinical workflows. EyeFM integrates five imaging modalities (CFP, OCT, UWF, FFA, and external eye photographs) with corresponding clinical text. It increased physician diagnostic accuracy from 75.4% to 92.2% in a RCT, while also improving patient compliance [
33]. This model can assist with disease screening in resource-limited settings and support multimodal diagnosis and report generation in specialized clinical environments.
In summary, the technical architectures and key focus areas of these five models determine their respective application scenarios. RETFound is suitable for large-scale monomodal screening tasks with limited computational resources, enabling efficient classification of common eye diseases at low annotation costs. VisionFM is well-suited for comprehensive assessment scenarios requiring multi-disease diagnosis and systematic health monitoring due to its broad task coverage. MIRAGE excels at pixel-level segmentation tasks and can serve as a general-purpose retinal feature extractor integrated into downstream AI systems. EyeCLIP is particularly suitable for multi-center studies or resource-limited regions where labeled data is scarce and cross-modal retrieval is required. EyeFM, meanwhile, is positioned for integration into real-world clinical workflows and decision support. The authors argue that the core value of current multimodal foundation models has shifted from “technological performance breakthroughs” to “clinical ecosystem integration.” The key challenge for the future lies not in simply adding more modalities, but in achieving organic synergy between modalities in a cost-effective and interpretable manner.
6.2 Potential of Foundation Models
The widespread application of AI in ophthalmology is transforming traditional diagnostic and treatment models. The evolution of foundation models signals the emergence of universal AI systems capable of simultaneously diagnosing multiple ocular diseases, decoding multimodal images, generating reports, and answering medical queries. A prospective study revealed that physicians with AI assistance achieved a 16.8% higher diagnostic accuracy compared to independent physicians, reduced the time required to write each imaging report by an average of 63.3 s, and received higher overall satisfaction ratings [
33]. Specific supportive technologies include: the Transformer architecture, which endows models with generalizability [
61], allowing a single model to address multiple eye diseases and overcoming the previous limitations of task-specific models [
21]. Shared visual encoders and embedding fusion techniques from LLMs enable the concurrent analysis of multimodal images (e.g., CFP and OCT) and clinical text [
23,
24,
33]. This mechanism mimics the diagnostic thinking patterns of physicians by integrating image and text information, significantly enhancing applicability in real clinical scenarios.
The application of foundation models also expands the disease diagnosis scope to include systemic disease identification. As the only structure enabling non-invasive in vivo observation of microcirculation, the retina offers potential for non-invasive early screening and risk stratification of systemic diseases [
62]. Research has confirmed that fundus imaging models can predict risks for systemic pathologies such as cardiovascular disease and neurodegenerative disorders [
21]. For instance, when using CFP as input, the RETFound model demonstrated stable performance in predicting myocardial infarction, heart failure, and ischemic stroke during internal validation, achieving AUROC values of 0.737, 0.794, and 0.754, respectively. Even in complex neurological disease prediction tasks such as Parkinson's disease, the AUROC reached 0.669. Furthermore, a method based on a multilayer perceptron (MLP) enabled the estimation of 38 systemic biomarkers from ophthalmic images. In this task, the VisionFM model achieved a high average accuracy of 78.6% [
23]. This high-precision systemic biomarker estimation offers new insights into early disease identification. This approach could reduce reliance on invasive diagnostics such as blood draws or costly imaging tests, significantly enhancing the convenience and accessibility of early diagnosis. In the future, preliminary health assessments might be conducted through simple eye photography. Additionally, long-term vascular remodeling captured in images could reflect true cardiovascular risk more accurately than single blood pressure readings [
63]. Biomarker detection based on fundus images may outperform certain transient metric values in disease diagnosis and prediction.
6.3 Accessibility and Equity in Healthcare
Remote ophthalmic diagnosis and treatment supported by AI are increasingly integrated into clinical practice and have the potential to transform clinical practice patterns. By incorporating LLMs, these systems can provide patients with disease-related information and enable online Q&A, thereby streamlining the medical consultation process and optimizing tiered healthcare delivery [
64]. Although foundation models cannot yet replace trained optometrists or ophthalmologists, their personalized conversational capabilities enhance doctor-patient interaction and improve patient adherence [
33,
64]. Besides, AI models can assist clinicians in selecting the most appropriate anti-vascular endothelial growth factor (VEGF) drug for each patient by predicting treatment outcomes specific to anti-VEGF therapy in neovascular AMD [
65]. Therefore, leveraging their robust generalization capabilities, foundation models could support the development of personalized drug treatment plans. The benefits of AI-assisted healthcare extend beyond individual diagnosis to the transformation of the broader healthcare landscape.
Regarding healthcare resource allocation, significant disparities exist in access to high-quality medical services between developed and developing regions. The promotion of online platforms, open-source foundation models, and shared datasets holds promise for mitigating this inequality [
66]. For instance, RETFound enhanced model demonstrates robust sensitivity and specificity in eye disease screening even when processing lower-resolution images captured by community healthcare facilities [
51]. Foundation models empower primary healthcare, optimize resource allocation, and promote the efficient use of medical resources.
At the level of health economics, AI-powered telemedicine screening demonstrates significant cost-effectiveness. A study on screening for blinding eye diseases revealed that compared to conventional non-telemedicine screening and non-AI telemedicine screening, the AI telemedicine screening approach yielded the lowest incremental cost-utility ratios (ICUR) [
67]. With a one-year screening interval, the ICUR was $7243 per year in rural areas and $15,189 per year in urban areas. The willingness-to-pay thresholds for each quality-adjusted life year were set at $25,751 for rural areas and $37,259 for urban areas. All incremental cost-effectiveness ratio (ICER) values fall well below these thresholds, indicating that AI telemedicine screening is highly cost-effective. Additionally, within the digital myopia prevention and control initiative, its ICER consistently remains within three times the per capita gross domestic product, underscoring the sustainability of AI-powered telemedicine screening in terms of cost-effectiveness advantages [
68].
6.4 Limitations and Future Prospects
Despite demonstrating potential in multimodal, multi-task analysis for ophthalmology, foundation models still face several critical challenges in practical clinical deployment. One major hurdle is data homogeneity and limited universality. Model performance heavily relies on the demographic diversity and quality of training data, yet existing training datasets often fail to meet this requirement [
69]. To address this concern, Sun et al. [
70] used controllable generative AI to produce large-scale synthetic retinal images covering 23 single-disease and 17 multi-disease categories. The RETFound-DE model pre-trained on this synthetic data outperformed the original RETFound on six public datasets (
p < 0.05) and performed comparably on three others. High-quality synthetic data can effectively mitigate data homogeneity and limited universality, and it may represent a key future direction.
Another challenge lies in insufficient task coverage. Existing models have yet to fully encompass the broad range of imaging techniques in ophthalmology and exhibit suboptimal performance in certain specialized diagnostic tasks [
46,
50]. To address this concern, RetFiner integrates image-text contrastive learning, matching, masked language modeling, and generative modeling, boosting the average AUROC of RETFound and VisionFM by 1.7% and 1.6%, respectively, on retinal disease classification [
57]. Future models must also preserve complete 3D spatial context to capture anisotropic structural features in OCT, moving beyond the limited 2D information used by current models like EyeCLIP [
24].
The black-box phenomenon remains a major obstacle hindering clinical deployment. Current explainability research largely relies on
post hoc methods such as heatmaps or attribution maps, which visually indicate contributing input regions but do not reveal internal mechanisms [
71]. To address this, novel approaches like sparse autoencoders and low-rank sparse attention have emerged. They decouple attention superposition effects in multi-head attention, enhancing Transformer explainability [
72]. Beyond interpretability, most medical foundation models lack rigorous prospective, multicenter clinical validation. Encouragingly, the EyeFM team's real-world study and RCTs have demonstrated clinical translatability and set a benchmark [
33]. The authors believe that the challenge of clinical translation of foundation models will soon be a thing of the past.
7 Conclusion
Foundation models demonstrate significant application value in ophthalmology, offering new solutions to improve clinical efficiency and promote equitable access to healthcare resources. However, before their widespread clinical adoption, concerns regarding diagnostic accuracy and explainability must be adequately addressed. Despite rapid technological advancements in this field, additional prospective, multicenter clinical trials are required to systematically evaluate their effectiveness and reliability in real-world healthcare settings.
2026 The Author(s). Eye & ENT Research published by John Wiley & Sons Australia, Ltd on behalf of Higher Education Press.