Inside MEMTAB 2025: New Methods Put Medical Tests and Biomarkers on Trial

When more than 200 researchers, methodologists, clinicians and patient advocates gathered at the University of Birmingham for the seventh international Methods for Evaluating Models, Tests and Biomarkers conference, the theme they chose was deliberately pointed: “Methodology that stands the test”. The meeting, whose abstracts are now published as a supplement in Diagnostic and Prognostic Research, is the leading international conference devoted to the methods used to research, evaluate and implement the models, tests and biomarkers that underpin screening, diagnosis, prognosis and monitoring in clinical practice. The collection of invited talks, oral presentations and posters offers an unusually candid snapshot of a field wrestling with a hard question: how do we know whether the tests and prediction tools reaching patients are actually as good as their creators claim?

One of the most consequential announcements came from the team updating QUADAS-2, the standard tool used since 2011 to assess the risk of bias and applicability of diagnostic test accuracy studies. The original publication has been cited more than 12,000 times and remains the recommended instrument for major health technology assessment organisations, yet feedback has flagged problematic signalling questions and newer tools have introduced features worth borrowing. Penny Whiting of the University of Bristol and colleagues described how a core group of methodological experts, supported by a wider steering group, summarised modifications made for the Cochrane Handbook, surveyed reviewers who had used QUADAS-2, reviewed methodological studies of the tool, and examined 50 Cochrane diagnostic accuracy reviews to expose challenges in applicability assessment. The resulting draft, QUADAS-3, keeps a similar structure but introduces major changes: separate synthesis questions rather than a single review question, a new section defining the ideal test accuracy trial for each question, assessment of bias and applicability at the level of individual accuracy estimates rather than whole studies, revised answer options including “probably yes” and “probably no”, and the replacement of the old Flow and Timing domain with a new Analysis domain. A section for judging overall risk of bias across domains has also been added, and the tool is currently being piloted.

If QUADAS-3 addresses how accuracy evidence is appraised, another Birmingham presentation asked a more uncomfortable question about how much of that evidence exists at all. Beth Hillier and colleagues investigated the evidence base behind rapid self-tests sold in UK supermarkets, pharmacies and health shops within ten miles of the university in 2023. They identified 35 self-tests and obtained 30, spanning seven sample types and twenty different biomarkers. Instructions for use made accuracy claims for 24 of the 30 tests, with performance claims of at least 98 percent appearing for just over half of the accuracy claims, 41 percent of sensitivity claims and 63 percent of specificity claims. No statements were made about predictive values, and where a reference standard was reported, 28 percent of studies had used an inappropriate one, comparing the self-test against another similar rapid test. Most strikingly, the team could obtain the clinical and layperson study reports that regulators rely on for only 40 percent of the tests; manufacturers refused seven requests and never replied to eleven. Among the nine unique documents retrieved, participant descriptions were missing in 78 percent, most studies showed unclear or high risk of bias across all QUADAS-2 domains, and there was no mention of blinding in 89 percent of layperson usability studies. The authors conclude that much of the evidence behind self-test performance claims is either not publicly available or of poor methodological quality, undermining the safety, credibility and reliability of products sold directly to consumers.

Screening programmes themselves came under the microscope in a study that exploited a rare opportunity in the data. England’s breast screening programme currently requires two readers per mammogram, and earlier research suggested two-reader screens detect more cancers with fewer false positives. But those studies inferred single-reader behaviour from the decisions of individual readers within pairs. Breanna Morrison, Alice Sitch and colleagues instead used 1994 to 1999 screening data covering six million mammograms from nearly four million women, which included screens genuinely read by a single reader. Across both incident and prevalent screens, two-reader screens were less likely to recall women for further tests, less likely to generate false positive recalls, but also less likely to find cancer, with adjusted odds ratios for cancer detection of 0.901 for incident and 0.802 for prevalent screens. Comparisons within readers who worked both alone and in pairs showed similar trends, though the difference in cancer detection was not significant in that analysis. The finding that single readers tend to recall more women, catching more cancers but also more false positives, reframes the trade-off at the heart of screening policy.

Several presentations probed what happens when tests are deployed at population scale without clear evidence of benefit. Katie Charlwood and colleagues at the University of Bristol emulated a randomised trial in routinely collected primary care data to compare regular liver function testing against no regular testing in people newly diagnosed with type 2 diabetes. Of 221,000 patients, about 32,000 were eligible for trial recruitment, with 12,165 following the control strategy and 19,951 the intervention strategy of approximately yearly testing. Over a maximum follow-up of 23.3 months, the team found an increased risk of hospitalisation among regularly tested patients, with an odds ratio of 1.08, which they caution may reflect unmeasured confounding, but no difference in all-cause mortality. The authors stress that, given significant limitations, these results should not be used in isolation to change monitoring practice, but the demonstration that target trial emulation can be applied to monitoring decisions opens a path to strengthening an evidence base that currently rests largely on expert opinion.

Diagnostic overuse also featured in a Dutch retrospective cohort study of children with fatigue. Iris Baars and colleagues analysed the AHON primary care database, identifying 6,859 children aged 4 to 18 who consulted their GP for persistent tiredness between 2015 and 2022. Somatic diagnoses were recorded in 26.3 percent of patients, yet only eight cases were linked by the GP to the fatigue episode, meaning potentially explanatory disease was found in fewer than one percent. Nevertheless, blood tests were ordered at the first consultation for 66.6 percent of children, generating 3,343 unique test combinations, and only 7.1 percent followed the Dutch guideline-recommended panel, always embedded within broader testing. Abnormal results appeared in 47.1 percent of children, illustrating how widespread testing in a low-prevalence setting generates findings of uncertain significance.

Artificial intelligence featured prominently, often with a sober assessment attached. A systematic review of AI-based grading of meibography images for Meibomian gland dysfunction, a leading cause of dry eye, identified eight studies involving 3,886 predominantly middle-aged participants. Estimated summary sensitivity and specificity for single-algorithm models reached 92.5 and 92.9 percent respectively, yet all studies carried a high risk of bias in at least one domain, 87 percent raised high applicability concerns, and only 17 percent of algorithms were validated using cross-validation techniques. The authors judge the current evidence to be of low certainty. A similar pattern emerged for exhaled breath biomarkers of tuberculosis: Emily MacLean and colleagues found 43 studies spanning early discovery to later-phase validation, but accuracy tended to decline as development advanced, and only two exclusively paediatric studies, covering 34 participants combined, had been conducted.

Methodological innovation extended to how evidence itself is synthesised. Eric Lam and colleagues plan an individual participant data meta-analysis of 53 studies to determine whether ancillary features in the LI-RADS liver imaging system contribute to diagnosis, building on single-centre findings that 13 of 21 such features could be removed without harming performance. Martha Elwenspoek’s team is assembling an international consortium for an IPD meta-analysis of coeliac disease serology, aiming to define threshold and test combinations accurate enough to spare patients confirmatory biopsy; twelve studies covering 7,086 participants have so far agreed in principle to share data. A systematic review of the Canadian C-Spine Rule pooled ten validation studies in adults with trauma, finding excellent sensitivity of 0.999 but low specificity of 0.188, effective for ruling out cervical fracture yet limited in its ability to spare imaging. Jacqueline Murphy and Thomas Fanshawe, analysing 46 Cochrane diagnostic accuracy reviews, found temporal trends in accuracy estimates in 87 percent of them, a finding with direct implications for postmarket surveillance of diagnostic devices.

Woven through the conference were themes of equity, transparency and implementation. A modified Delphi survey prioritised 40 unmet diagnostic needs in care homes, with urinary and respiratory infection diagnostics taken forward for rapid review. A Delphi-derived list of 18 criteria now exists to prioritise which tests validated in secondary care deserve evaluation in primary care. New networks such as CLEARED, a UK centre of excellence for in vitro diagnostic regulatory science, and frameworks like IMPACT and the Biomarker Toolkit for Alzheimer’s disease, signal a field determined to close the gap between promising technology and reliable clinical practice. As the Birmingham meeting made clear, the future of diagnostics depends less on inventing new tests than on proving, with rigorous and transparent methods, whether the ones we already have truly stand the test.

Subject of Research: Methods for evaluating diagnostic models, tests and biomarkers presented at the MEMTAB 2025 conference

Article Title: Methods for Evaluating Models, Tests And Biomarkers (MEMTAB): Abstracts from the 7th International Conference, MEMTAB 2025

Article References: Methods for Evaluating Models, Tests And Biomarkers (MEMTAB): Abstracts from the 7th International Conference, MEMTAB 2025. (2026). Diagnostic and Prognostic Research, 10(1), 13. https://doi.org/10.1186/s41512-026-00227-w (Author names are not included in the publisher’s citation export.)

Image Credits: AI Generated

DOI: 10.1186/s41512-026-00227-w

Keywords: MEMTAB 2025, diagnostic accuracy, QUADAS-3, self-tests, breast screening, risk prediction models, artificial intelligence, biomarkers, meta-analysis, regulation, screening, clinical prediction rules