Manually tracing the exact outline of a rectal tumor on an MRI scan is one of those deceptively simple tasks that quietly consumes enormous amounts of clinical time. Radiation oncologists rely on these segmentations to define the target volume for radiotherapy planning, and researchers need them to extract quantitative features from imaging studies. The work is almost always done by hand, preferably by experienced radiologists, slice by slice, because whole-volume delineation is considered the most accurate and reproducible approach. A new multicenter study now reports that a fully automated deep learning network can perform this task almost as well as two expert radiologists agree with each other, even when fed the messy, heterogeneous data of real-world clinical practice rather than a carefully curated research dataset.
The study, published in European Radiology Abdomen, analyzed baseline staging MRI scans from 603 rectal cancer patients scanned at six institutions between 2012 and 2017. The cohort included 376 men, with a mean age of 65 years. Patients came from four large regional teaching centers, one university hospital, and one comprehensive cancer center, and their scans were acquired on 18 different MRI scanners using a wide variety of protocols. T2-weighted slice thickness ranged from 3 to 5 millimeters, diffusion-weighted imaging slice thickness from 3 to 8 millimeters, and the highest available b-value ranged from 600 to 1200 s/mm2. This deliberate diversity was the point: most previous deep learning segmentation studies have relied on single-center, highly curated datasets that bear little resemblance to the variable image quality encountered in routine care.
The researchers trained networks using the nnU-Net framework, a system that automates hyperparameter tuning and data augmentation and has achieved state-of-the-art performance across medical imaging tasks. Two sets of networks were trained with five-fold cross-validation on the training set of 482 cases: one using a combination of T2-weighted MRI, diffusion-weighted imaging, and apparent diffusion coefficient maps, and another using T2-weighted images alone. An expert radiologist with more than ten years of experience in rectal cancer MRI manually delineated all tumors on high b-value diffusion-weighted images, using the T2-weighted sequence for anatomical correlation. These segmentations served both as the training input and as the ground truth for testing. A second expert radiologist independently segmented all 121 cases in the test set, allowing the researchers to measure how closely humans agree with each other and to hold the network to that same standard.
The results showed that the combined-input network performed well against this human benchmark. Measured with the Dice similarity coefficient, a standard overlap metric where 1.0 indicates perfect agreement, the network achieved scores of 0.726 when compared to the first expert’s segmentations and 0.669 when compared to the second expert’s in the test set. The agreement between the two expert radiologists themselves was 0.757. When the network was compared to the first expert’s segmentations, its performance was statistically indistinguishable from the level of agreement between the two human experts. Against the second expert’s segmentations, performance was significantly poorer, a difference the authors attribute in part to reader-specific segmentation preferences, since the network was trained on annotations from a single reader.
A striking finding emerged when the researchers stripped away the diffusion-weighted data. Networks trained on T2-weighted images alone achieved Dice scores of only 0.586 and 0.590 against the two experts in the test set, significantly lower than the combined model. This indicates a strong positive effect from adding diffusion-weighted imaging and ADC maps, despite the fact that diffusion sequences are more prone to artifacts and quality variations. The authors suggest the advantage stems from the intrinsically higher tumor-to-background contrast on diffusion-weighted images, which previous work has shown reduces interreader variation when delineating rectal tumors compared to T2-weighted sequences alone.
Perhaps the most counterintuitive result concerned image quality. The team graded every diffusion-weighted scan on five-point Likert scales for signal-to-noise ratio and for artifacts such as signal pile-up and geometric distortion at the tumor site. Neither the network’s performance nor the agreement between the human experts was significantly affected by either measure. In other words, both human and artificial readers appear able to work around limitations in diffusion-weighted image quality, at least within the range of quality present in this dataset. This challenges the assumption that deep learning models trained on heterogeneous data will falter when confronted with imperfect scans.
Tumor morphology told a different story. The first expert rated each tumor’s complexity on a three-point scale, from easy to segment, meaning homogeneous, regular, and well-demarcated, to difficult, meaning heterogeneous, irregularly shaped, or poorly demarcated. More complex tumors significantly reduced both the network’s performance and the agreement between the human experts, with the effect reaching statistical significance across the range of comparisons. These difficult cases, which made up roughly 15 to 20 percent of the dataset, are likely to remain a persistent bottleneck for segmentation, whether the work is done by a radiologist or an algorithm. In one illustrated example, a tumor with heterogeneous signal and partly ill-defined margins was only moderately agreed upon by the two radiologists, while the network largely under-segmented it because of poor tumor-to-background contrast on the diffusion-weighted image.
The study had limitations the authors acknowledge candidly. In the absence of a true gold standard, expert agreement served as the target benchmark rather than an objective truth. Only the test set was independently segmented by two readers, given how time-consuming manual segmentation is. The quality and complexity scores were assigned subjectively by a single expert reader, and the tumor complexity scale, developed for this study, has not been formally validated. Although the test set was independent at the patient level, cases from all six centers appeared in both training and test sets, a design chosen to maximize heterogeneity and reduce center-specific bias. The authors note that external validation using data from entirely independent institutions remains warranted. A small portion of cases, fewer than 10 percent, included diffusion acquisitions with maximum b-values below 800, which do not meet current ESGAR recommendations, though this did not significantly affect performance.
Compared with earlier work, the findings sit in an interesting middle ground. A previous bi-institutional study trained a 2D U-Net on T2-weighted input and achieved a Dice score of 0.77 internally, but external validation dropped to 0.59, comparable to the T2W-only performance seen here. Another study using a U-Net with attention-based fusion of input sequences reached a Dice score of 0.821, slightly above the current results, but that model was developed and evaluated on data from a single center, which is likely more homogeneous and may limit generalizability. Among the studies the authors reviewed, only the bi-institutional one performed external validation, and it too showed significantly poorer results outside its training data.
The practical implications are straightforward. Once further optimized, networks of this kind could significantly reduce the segmentation workload for radiotherapy planning and for future quantitative imaging studies, and could ultimately bring segmentation support tools into routine clinical practice. The authors are careful to frame the current performance as a starting point rather than a finished clinical product: the segmentations are good enough to serve as a basis for further research and to lighten manual workloads, not yet a replacement for expert judgment. The code and model weights have been released publicly, which may accelerate the multi-reader annotation studies and external validations the authors say are needed next. For a field that has long been criticized for demonstrating impressive results only on pristine, single-center data, this study offers a more sober and more useful benchmark: a measure of what deep learning can actually do when the scans come from the real world, artifacts, complexity, and all.
Subject of Research: Deep learning segmentation of rectal cancer on multicenter MRI and the effects of image quality and tumor complexity
Article Title: Impact of DWI quality and tumor complexity on deep learning-based segmentation of rectal cancer using multicenter MRI data
Article References: van Griethuysen, J. J. M., Lambregts, D. M. J., Schurink, N. W., Lahaye, M. J., Trebeschi, S., Bakers, F. C. H., Vliegen, R. F. A., Geenen, R. W. F., Cappendijk, V., de Bie, S. H., Aerts, H. J. W. L., & Beets-Tan, R. G. H. (2026). Impact of DWI quality and tumor complexity on deep learning-based segmentation of rectal cancer using multicenter MRI data. European Radiology Abdomen, 1(1), Article 6. https://doi.org/10.1007/s44501-026-00003-7
Image Credits: AI Generated
DOI: 10.1007/s44501-026-00003-7
Keywords: deep learning, rectal cancer, MRI segmentation, nnU-Net, diffusion-weighted imaging, radiotherapy planning, interreader agreement, Dice similarity coefficient, tumor complexity, multicenter study, medical imaging, radiology
News Source: Nathaniel Bowman. (October 27, 2026). Deep learning matches expert radiologists at outlining rectal tumors on MRI. Scienmag.
