AI prostate MRI risk score edges PI-RADS thresholds but advantage stays unproven

A commercial artificial intelligence tool for prostate MRI has been put to one of its most direct tests yet: researchers compared the software’s own risk score against the familiar PI-RADS categories, using biopsy results from every single patient as the ground truth. The study, published in Cancer Imaging, examined the AI-Rad Companion Prostate MR software, version VA60, developed by Siemens Healthineers, and found that its quasi-continuous Level-of-Suspicion score produced a numerically higher area under the ROC curve for detecting clinically significant prostate cancer than the algorithm’s PI-RADS classification, 0.882 versus 0.833. That gap, however, did not reach statistical significance, with a DeLong test p-value of 0.106, and the authors themselves caution that the result should be read as a promising trend rather than a demonstrated advantage.

The retrospective single-center study, conducted at Friedrich-Alexander-Universität Erlangen-Nürnberg, drew on 123 consecutive multiparametric prostate MRI examinations performed for suspected prostate cancer between June 2014 and July 2017. One patient was excluded because a required diffusion-weighted imaging series was missing, leaving 122 examinations in the final analysis. Every MRI was followed by biopsy within one month, either a standard 12-core systematic sampling or a combination of systematic cores plus MRI-ultrasound fusion-targeted cores when a visible lesion was present. Histopathology, read by subspecialized uropathologists, served as the reference standard for both any prostate cancer, defined as Gleason score 6 or higher, and clinically significant prostate cancer, defined as Gleason score 7 or higher.

What makes this evaluation distinctive is the dual output of the algorithm itself. The software ingests biparametric MRI inputs, T2-weighted imaging and diffusion-weighted imaging with ADC maps, and works fully automatically, without user interaction and without access to clinical patient information or histopathological results. It first segments the prostate gland, then localizes suspicious lesions, and assigns each detected lesion both a PI-RADS-compatible category and a Level-of-Suspicion score ranging from 60 to 100. An LoS between 60 and 79 maps to an AI-assigned PI-RADS 3, while a score of 80 to 100 yields an AI-assigned PI-RADS 4 or 5, depending on whether the lesion’s segmented diameter is smaller or at least 15 millimeters. Examinations in which the algorithm detected no lesion, 31 of the 122 cases or 25.4 percent, were assigned a fixed LoS value of 30 and counted as negative at every threshold.

For clinically significant prostate cancer, the PI-RADS classification achieved an AUC of 0.833 with a 95 percent confidence interval of 0.764 to 0.901, while the LoS score reached 0.882 with a confidence interval of 0.820 to 0.943. The difference between the two classifiers was 0.049, with a 95 percent confidence interval stretching from minus 0.010 to 0.108, meaning the data remain compatible with anything from a negligible difference in favor of PI-RADS to an advantage of roughly one tenth of an AUC unit for the LoS score. For detecting any prostate cancer, the two outputs performed almost identically, with AUCs of 0.832 for PI-RADS and 0.844 for LoS, a difference that was clearly not statistically significant at p = 0.653.

The most practically interesting result emerged at a specific operating point. Using the Youden criterion, which maximizes the sum of sensitivity and specificity, the researchers identified an optimal LoS cutoff of 81. No examination in the cohort carried a score of exactly 81; the nearest occupied values were 80 and 82. At this exploratory threshold, sensitivity for clinically significant cancer remained 94.7 percent, identical to the sensitivity at the conventional PI-RADS 4-or-higher cutoff, but specificity rose from 63.1 percent to 70.8 percent. Positive and negative predictive values at the LoS cutoff were 74.0 percent and 93.9 percent, respectively. The two decision rules classified every examination identically except for five csPCa-negative examinations with a LoS of exactly 80, and an exact McNemar test on those discordant pairs yielded p = 0.063, just short of conventional significance.

Because the Youden-optimal threshold was both derived and evaluated in the same cohort, the researchers quantified its optimism through bootstrap internal validation with 2,000 resamples. In each resample the optimal threshold was re-derived and then applied to the original cohort. The procedure indicated modest optimism of 1.8 percentage points for sensitivity and 1.7 percentage points for specificity, yielding optimism-corrected estimates of 92.9 percent and 69.1 percent. Discrimination was unaffected, with a corrected AUC of 0.882, since no model was fitted and the optimism arose solely from the data-driven choice of threshold. Across resamples, the re-derived threshold ranged from 75.5 to 86.5 at the 2.5th to 97.5th percentiles, a spread that hints at how much the ideal cutoff could wobble in a different population.

Calibration of the LoS score was assessed after logistic recalibration, a necessary step because the LoS is a bounded score rather than a probability. Since the mapping was fitted in the same cohort, the apparent calibration slope and intercept are 1 and 0 by construction and carry no information. The bootstrap-corrected values were a slope of 0.98 and an intercept of minus 0.01, indicating no material over- or under-spreading of risk. The Brier score was 0.141, compared with 0.249 for an uninformative model at the observed prevalence. The authors stress that the score should not be read as a probability of malignancy without recalibration in the target setting.

To place the algorithm in broader context, the researchers compared its performance against two large contemporary meta-analyses of PI-RADS v2.1 performance by expert human readers, Park et al. and Oerther et al., which together summarize more than 14,000 patients, though the underlying primary studies may overlap. At the optimal LoS cutoff, the algorithm achieved a sensitivity of 94.7 percent, with a 95 percent confidence interval of 85.4 to 98.9 percent, numerically higher than the sensitivities reported at PI-RADS 4 or higher by Park et al. at 81 percent and Oerther et al. at 89 percent. Specificity at the LoS cutoff, 70.8 percent with a confidence interval of 58.2 to 81.4 percent, lay between the two reference estimates of 82 percent and 66 percent. The authors emphasize that these comparisons are descriptive only, since the meta-analytic cohorts differ in patient population, imaging protocol and clinical setting, and no non-inferiority design was pre-specified.

Three representative clinical cases illustrate the algorithm’s behavior across the diagnostic spectrum. In one true-positive case, a 59-year-old man received an AI-assigned PI-RADS 4 with a maximal LoS of 100 for a left posterolateral middle peripheral zone lesion, and targeted biopsy confirmed moderately differentiated adenocarcinoma with a maximum Gleason score of 7, involving up to 90 percent of an individual core. A false-positive case involved a 67-year-old man whose left posteromedial middle peripheral zone lesion also drew PI-RADS 4 and LoS 100, yet histopathology revealed no malignancy, underscoring the inherent specificity limitations of imaging-based detection. A third case showed a sensitivity-favoring tendency: a 73-year-old man’s partially encapsulated nodule with restricted diffusion would formally qualify as PI-RADS 3 under strict v2.1 criteria, but the algorithm assigned PI-RADS 4 with an intermediate LoS of 80, and biopsy confirmed acinar adenocarcinoma with Gleason score 6, a cancer below the threshold for clinical significance applied in this study.

The limitations are substantial and the authors enumerate them candidly. As a retrospective, single-center analysis, generalizability may be limited, and larger, prospective, multi-center studies are required. With 57 examinations showing clinically significant cancer and 65 without, the study had limited power to detect a difference of the observed magnitude, and the absence of a significant difference should not be interpreted as evidence of equivalence. The analysis was at the examination level and did not evaluate lesion-level detection. The higher specificity of the LoS cutoff rests on just five examinations with a score of exactly 80, all free of clinically significant cancer, a margin the authors describe as fragile and unlikely to persist unchanged in another cohort. Biopsy itself carries sampling error, and verification was not fully independent of the index test, since targeted cores were directed at MRI-visible lesions, meaning cancers invisible on MRI entered the reference standard less often and may bias sensitivity upward. The Youden-optimal threshold requires confirmation in an independent cohort before any clinical use. The study team also notes that future work should integrate clinical covariates, particularly PSA density, with the LoS score to build composite risk models, and that reader-in-the-loop studies measuring AI’s impact on radiologist accuracy and inter-observer agreement will be crucial. For now, the message is measured: a tunable, quasi-continuous risk score is an intriguing complement to fixed PI-RADS thresholds, but prospective validation across diverse prevalence environments must come first.

Subject of Research: Diagnostic performance of a commercial AI algorithm for prostate MRI using a quasi-continuous risk score compared with fixed PI-RADS thresholds against biopsy ground truth

Article Title: Diagnostic performance of AI-powered prostate MRI against biopsy ground truth: quasi-continuous risk scoring versus fixed PI-RADS thresholds

Article References: Bayerl, N., Appel, J., Schmidt, M., Cavallaro, A., Grimm, R., von Busch, H., Wullich, B., Hartmann, A., Schlicht, M., Uder, M., May, M. S., & Ellmann, S. (2026). Diagnostic performance of AI-powered prostate MRI against biopsy ground truth: quasi-continuous risk scoring versus fixed PI-RADS thresholds. Cancer Imaging, 26(1), Article 130. https://doi.org/10.1186/s40644-026-01126-5

Image Credits: AI Generated

DOI: 10.1186/s40644-026-01126-5

Keywords: prostate cancer, prostate MRI, artificial intelligence, PI-RADS, Level-of-Suspicion score, biopsy, clinically significant prostate cancer, ROC analysis, AI-Rad Companion, diagnostic accuracy, histopathology, retrospective study