Medical artificial intelligence has reached a point where being right is no longer enough. In hospitals, clinicians must also understand why an algorithm produced a diagnosis, particularly when the decision could influence surgery, cancer treatment, or emergency care. A new study introduces ConceptCLIP, a biomedical foundation model designed to combine high diagnostic performance with explanations that clinicians can inspect and understand. Reported in Nature Biomedical Engineering, the system links medical images not only to text descriptions, but also to recognizable clinical concepts, creating a pathway between an algorithm’s prediction and the visual evidence supporting it.
The model was developed by researchers led by Y. Nie, S. He, and Y. Bie, who argue that many existing multimodal biomedical models remain difficult to interpret. These systems are commonly trained to associate images with captions, reports, or diagnostic labels. Although such training can produce powerful image-recognition capabilities, it does not necessarily reveal which anatomical structures, lesions, or visual patterns led to a conclusion. ConceptCLIP seeks to address this limitation by making medical concepts a central part of the learning process. Instead of treating an image as an opaque collection of pixels, it attempts to connect specific regions within that image to clinically meaningful descriptions.
At the foundation of the project is MedConcept-23M, a dataset containing 23 million biomedical image–text–concept triplets. Each triplet is designed to provide three complementary forms of information: the medical image itself, associated language, and a concept that captures an interpretable feature or finding. In principle, the text can describe the broader clinical context, while the concept identifies a more focused element, such as a visible abnormality or anatomical characteristic. This structure gives the model more supervision than a conventional image–caption pair and helps it learn how clinical language maps onto visual evidence.
ConceptCLIP uses joint image–text and region–concept alignment during pretraining. The image–text component teaches the system to associate entire images with their corresponding descriptions, a strategy related to contrastive language–image learning. In contrastive training, matching image and text pairs are pulled closer together in a shared mathematical representation, while mismatched pairs are pushed apart. The region–concept component operates at a more localized level. It encourages selected regions of an image to align with specific concepts, allowing the model to identify where an interpretable finding is located rather than merely assigning a label to the complete scan.
This distinction is technically important in medical imaging. A diagnosis may depend on a small nodule in a chest scan, a subtle retinal change, a tissue pattern in a pathology slide, or a localized abnormality in an ultrasound image. A model that only learns global image-level associations may recognize that an image resembles a disease category without demonstrating which feature matters. By incorporating region-level alignment, ConceptCLIP is intended to produce explanations grounded in visible image content. The resulting concept-based outputs can potentially show clinicians which findings the model considered relevant and how those findings contributed to its prediction.
The researchers evaluated ConceptCLIP across a large benchmark comprising 78 datasets and 10 imaging modalities. The benchmark was designed to test whether the model’s capabilities could transfer across different types of biomedical images rather than remain restricted to a single clinical application. Medical imaging datasets vary widely in resolution, anatomy, acquisition technology, and diagnostic terminology, making broad generalization a demanding test for any foundation model. According to the study, ConceptCLIP achieved superior diagnostic performance across this evaluation while also producing human-understandable explanations.
The model’s claimed advantage is therefore not simply a higher score on image classification. Its developers present it as a system that combines diagnostic accuracy with a more transparent reasoning interface. This is particularly relevant because foundation models are typically trained on broad datasets and then adapted to specialized tasks. In medicine, such flexibility can be valuable, but it can also make failures difficult to detect. An algorithm may perform well on average while relying on irrelevant correlations, variations in equipment, or artifacts associated with a particular dataset. Concept-based explanations could give clinicians an opportunity to challenge those shortcuts before they influence patient care.
To examine whether the explanations were useful in practice, the researchers conducted a clinician user study covering three imaging modalities. In the study, concept-based explanations helped clinicians verify model predictions and identify potential errors. This finding addresses a central question in explainable AI: whether an explanation is merely visually attractive or actually improves human oversight. If a clinician can quickly compare the model’s highlighted concept with the image and recognize that the reasoning is medically plausible, the explanation may support safer collaboration. Conversely, if the highlighted concept appears irrelevant or misleading, it may reveal that the model’s prediction requires further review.
The arrival of ConceptCLIP reflects a broader shift in biomedical AI, from systems that generate predictions to systems that must also communicate evidence. Yet interpretability alone does not guarantee clinical reliability. Explanations can be incomplete, and a model may describe a real image feature without proving that the feature is causally responsible for disease. MedConcept-23M and the model’s benchmark results represent a substantial effort to address these concerns, but deployment in hospitals would still require prospective validation, careful monitoring, assessment across patient populations, and integration with clinical workflows. Even so, by combining large-scale vision–language pretraining with region-level concept alignment, ConceptCLIP offers a potentially influential blueprint for building medical AI that is not only capable of recognizing disease, but also better able to show clinicians what it sees.
Subject of Research: Explainable biomedical artificial intelligence for medical imaging
Article Title: An explainable biomedical foundation model via large-scale concept-enhanced vision–language pretraining
Article References: Nie, Y., He, S., Bie, Y. et al. “An explainable biomedical foundation model via large-scale concept-enhanced vision–language pretraining.” Nature Biomedical Engineering (2026). https://doi.org/10.1038/s41551-026-01764-x
Image Credits: AI Generated
DOI: https://doi.org/10.1038/s41551-026-01764-x
Keywords: ConceptCLIP, biomedical foundation models, explainable AI, medical imaging, vision–language pretraining, region–concept alignment, clinician decision support, trustworthy artificial intelligence, MedConcept-23M
Tags: combining diagnostic accuracy with explainability in AIConceptCLIP for medical image interpretabilitydeep learning for clinical concept recognitionenhancing transparency in medical AI systemsexplainable AI for surgical and cancer treatment decision supportExplainable biomedical foundation modelsinterpretable artificial intelligence in medicinelarge-scale vision-language pretraining in healthcarelinking medical images to clinical conceptsmultimodal biomedical AI modelsvisual evidence-based medical diagnosisvisual-textual medical data integration
