Multimodal AI model combines eye scans for early disease classification

A dual-modality artificial-intelligence model that analyzed both fundus photographs and optical coherence tomography (OCT) scans reported high performance when classifying diabetic retinopathy, macular edema, age-related macular degeneration and glaucoma in a combined benchmark dataset. The optimized hierarchical dual-modality feature pyramid network, or OHDM-FPN, achieved 98.2% accuracy, 97.9% sensitivity, 98.7% specificity, an area under the receiver operating characteristic curve of 0.993 and an F1 score of 98.1%. The study addressed a limitation of single-modality imaging.

Fundus photography captures surface-level vascular and optic-disc information, whereas OCT provides high-resolution cross-sectional detail of retinal layers but does not include the same surface-level cues. OHDM-FPN was designed to combine these complementary representations for eye-disease detection, including early-stage disease detection. For fundus images, the framework used EfficientNetV2-Small as its encoder.

OCT images were processed by GhostNet with dilated residual blocks. Each modality passed through a feature pyramid network using atrous spatial pyramid pooling to develop multi-scale representations before the two streams were brought together. The fusion stage used a cross-modality multi-scale fusion block to align and combine modality-specific features.

This block incorporated channel-spatial attention and deformable convolutions. A lightweight classification head then predicted the disease category, while hyperparameter tuning with the Salmon Salar optimization algorithm was used to address class imbalance and improve generalization. Experiments were conducted on a combined fundus–OCT benchmark dataset assembled from EyePACS and Duke OCT.

Against the study’s comparison models, the reported OHDM-FPN results were described as averaging 6.3% better in accuracy and 5.8% better in AUC than single-modality models. The researchers also reported a 9.1% improvement in early-stage disease detection rates compared with OCT-only baselines. Attention heat maps were used to examine the regions influencing the model’s classifications.

The reported maps showed that OHDM-FPN focused on clinically relevant retinal regions, providing an interpretability aid for ophthalmologists. This visualization was presented as a way to inspect where the network placed emphasis when making predictions. The findings are computational results from benchmark datasets rather than results from a prospective clinical trial.

Consequently, the reported performance does not establish how OHDM-FPN would perform in routine clinical use. Within the study’s stated scope, the results indicate that fusing fundus and OCT features improved classification performance relative to the reported single-modality baselines. OHDM-FPN combined modality-specific feature extraction, multi-scale processing, attention, deformable convolutions and optimized classification in one framework for early eye-disease detection.

Subject of Research: Evaluation of a deep-learning model for classifying eye diseases from combined fundus and OCT images

Article Title: Optimized multi-scale dual-modality feature pyramid network for early eye disease classification using fundus and OCT imaging

Article References: Kiruthiga, R., & Kumar, A. S. (2026). Optimized multi-scale dual-modality feature pyramid network for early eye disease classification using fundus and OCT imaging. Complex & Intelligent Systems. https://doi.org/10.1007/s40747-026-02507-w

Image Credits: AI Generated

DOI: 10.1007/s40747-026-02507-w

Keywords: multimodal AI, eye disease detection, fundus photography, optical coherence tomography, deep learning, feature fusion, diabetic retinopathy, macular edema, glaucoma, interpretable AI, Optimized, multi-scale