The relentless surge in abdominal computed tomography (CT) scans performed globally has inundated radiology departments, exacerbating a workforce shortage and placing immense strain on radiologists. This escalating demand for rapid, accurate imaging interpretation has intensified the quest for sophisticated automated tools capable of assisting medical professionals. Addressing these challenges, researchers have unveiled Merlin, a groundbreaking three-dimensional vision–language model (VLM) designed specifically for volumetric abdominal CT analysis. Unlike prior models constrained to two-dimensional imaging and brief textual contexts, Merlin integrates volumetric data, extensive electronic health records, and comprehensive radiology reports, heralding a transformative leap in automated medical imaging.
At the core of Merlin’s innovation lies a rigorous multistage pretraining strategy that circumvents the need for additional manual annotations, a major bottleneck in medical AI development. By leveraging an unprecedentedly rich clinical dataset comprising over 6 million CT images from 15,331 scans, complemented by 1.8 million diagnostic codes and more than 6 million tokens extracted from radiology narratives, Merlin capitalizes on the synergy of multimodal data. This vast trove enables the model to internalize complex spatial and linguistic relationships critical for nuanced medical interpretation, far surpassing the constraints of prior 2D models.
The evaluation of Merlin is notable for its breadth and depth, encompassing six distinct task categories and an astounding 752 subtasks that span diagnostic, prognostic, and quality assurance objectives. These cover zero-shot classification of 30 clinically pertinent findings, phenotype classification across 692 distinct phenotypes, and sophisticated zero-shot image-to-text and image-to-impression retrieval tasks. Model adaptation further extends Merlin’s capabilities to long-term chronic disease prediction over a five-year horizon for six diseases, generation of detailed radiology reports, and three-dimensional semantic segmentation of twenty abdominal organs. This wide-ranging functionality speaks to Merlin’s potential as a truly generalist tool in radiological workflows.
Robust validation was conducted both internally, on a test set of 5,137 CT scans, and externally across 44,098 scans originating from three disparate healthcare systems and two publicly available datasets. Such rigorous cross-institutional and cross-anatomical testing demonstrated Merlin’s extraordinary generalizability—a crucial characteristic for deploying AI in heterogeneous clinical environments. In these evaluations, Merlin consistently outperformed leading-edge 2D VLMs, foundation models tailored specifically for CT, and off-the-shelf radiology AI tools, underscoring its unprecedented capability to comprehend and analyze volumetric medical imagery.
The technical advancements embodied by Merlin extend beyond raw performance metrics. The model incorporates a novel approach toward aligning volumetric image data with dense textual reports, enabling richer semantic understanding. This methodology effectively bridges the modality gap, fostering more accurate cross-modal interpretation—a longstanding challenge in medical AI research. Moreover, through scaling laws and meticulous ablation studies, the team elucidated optimal training regimes, revealing insightful correlations between dataset scale, training duration, and model efficacy, thereby paving the way for future refinement and broader adoption.
In terms of clinical impact, Merlin’s ability to augment radiologists’ workflows promises to alleviate the diagnostic bottleneck exacerbated by the global radiologist shortage. Automated classification and nuanced report generation expedite case handling while maintaining, or even enhancing, diagnostic accuracy. Furthermore, Merlin’s incorporation of prognosis and disease risk stratification heralds a new era of predictive radiology, where imaging can inform long-term patient management with unprecedented precision. This suggests transformative utility not only in diagnostics but also in preventative medicine and personalized care strategies.
Merlin’s open release of its trained models, source code, and a curated dataset comprising 25,494 abdominal CT scans paired with corresponding radiology reports epitomizes a commitment to open science and reproducibility. By democratizing access, the developers invite the global research community to validate, extend, and apply Merlin’s capabilities, fostering innovation and accelerating clinical translation. This resource is poised to catalyze advances across diagnostic AI, radiomics, and bioinformatics domains.
The emergent paradigm embodied by Merlin exemplifies a broader shift within medical AI toward foundation models that leverage vast, multimodal datasets to achieve generalized, scalable intelligence. Unlike narrowly engineered tools, such foundation models offer versatility across tasks and institutions, mitigating biases and performance drops caused by varying clinical practices. Merlin’s success underscores the feasibility and preference for 3D volumetric data integration within vision-language frameworks, a frontier ripe for exploration across other imaging modalities and anatomical regions.
Despite the promising advancements, challenges remain in integrating Merlin seamlessly into routine clinical practice. Ethical considerations surrounding data privacy, interpretability of AI decisions, and clinician trust must be meticulously addressed. Furthermore, ongoing efforts are essential to ensure that Merlin and models of its ilk remain robust against domain shifts, artifacts, and rare pathologies. Continuous refinement, coupled with prospective clinical trials, will be pivotal in establishing their ultimate role as indispensable tools in precision radiology.
In summary, Merlin stands as a landmark accomplishment in medical imaging AI, marrying complex volumetric CT data with rich linguistic contexts within a sophisticated vision–language architecture. Its expansive dataset, extensive validation, and superior performance position it as a vital enabler for overcoming radiology workforce challenges, enhancing diagnostic accuracy, and pioneering predictive radiology applications. As the medical community navigates an era of data deluge and growing health demands, innovations like Merlin illuminate the path toward intelligent, efficient, and patient-centric care.
The advent of Merlin demonstrates the transformative potential of combining 3D imaging with natural language processing to deliver holistic, automated insights that resonate with clinical reasoning. This integrative approach not only accelerates image interpretation but also enriches understanding by embedding radiological findings within broader health narratives. Such fusion is pivotal for the next generation of AI-driven diagnostics, ensuring rapid and reliable clinical decisions that improve patient outcomes.
Looking forward, the architecture and training protocols introduced with Merlin are expected to inspire a new wave of multimodal foundation models across radiology and beyond. Expanding these frameworks to other imaging techniques like MRI or PET, and incorporating richer clinical records such as laboratory results or genomic data, could yield even more powerful diagnostic ecosystems. Merlin thus represents both a culmination of prior efforts and a springboard for future innovation in AI-empowered healthcare.
Subject of Research: Automated interpretation of abdominal computed tomography scans using a 3D vision–language foundation model.
Article Title: Merlin: a computed tomography vision–language foundation model and dataset.
Article References:
Blankemeier, L., Kumar, A., Cohen, J.P. et al. Merlin: a computed tomography vision–language foundation model and dataset. Nature (2026). https://doi.org/10.1038/s41586-026-10181-8
DOI: https://doi.org/10.1038/s41586-026-10181-8
Tags: 3D vision-language model in radiologyadvanced diagnostic coding integrationautomated medical imaging interpretationdeep learning for volumetric CT scansenhancing radiologist workflow with AIintegrating electronic health records with imaginglarge-scale CT imaging datasetmultimodal medical AIovercoming annotation bottlenecks in healthcare AIpretraining strategies in medical AIradiology report natural language processingvolumetric abdominal CT analysis
