Query audit finds most named-pathway searches unready for evidence review

Only three of 26 named signaling-pathway searches in extracellular-vesicle research returned literature sets that were actually ready for screening, according to a query audit published in Journal of Translational Medicine by Michaela Liegertová and Jan Malý of Charles University. The study, released as an open-access research article on 24 October 2026, examined whether PubMed searches built around the names of KEGG signaling pathways reliably retrieve records that study those pathways in extracellular-vesicle work, rather than merely mentioning them. Extracellular vesicles transfer proteins, lipids and nucleic acids between cells with context-dependent effects, and publication counts from named-pathway searches are often read as evidence of where the field is active or ready for systematic synthesis. The audit showed that such counts can be misleading: every query passed the technical rules of search construction, yet most retrievals contained records that referenced a pathway without studying it, or that had little to do with extracellular vesicles at all.

The primary universe of the audit consisted of all 26 human maps in the KEGG Signal-transduction subcategory, supplemented by a prespecified secondary layer of 13 additional maps. Before running any definitive retrieval, raters judged up to ten records per map separately for two qualities: whether the record had genuine pathway substance, meaning the pathway was actually studied rather than name-dropped, and whether it was relevant to extracellular vesicles in the cardiac repair, neurodegeneration, or tumor metastasis contexts under examination. In total, 234 queries were evaluated. All of them satisfied the formal technical criteria for a reproducible search, but only three primary maps, Notch, NF-kappa B and mTOR, yielded bounded source sets suitable for screening. Ras produced a non-empty set that failed the audit, three maps returned empty results, 18 required redesign, and one was a duplicate. The secondary layer of 13 maps reproduced the same pattern, indicating the problem was not confined to a handful of pathways.

The failure rates across the audited material were substantial. Of 139 audited units, 43.2 percent failed pathway substantiveness and 40.3 percent failed extracellular-vesicle or context relevance. In other words, a large fraction of what a technically valid named-pathway search retrieves consists of records in which the pathway name appears without the pathway being investigated, or records unrelated to vesicle biology in the intended sense. The audit also tested whether the ranking of pathways by unaudited PubMed counts could have identified the ready retrievals, and found that it could not. The three bounded source sets ranked 4th, 14th and 24th of 26 when pathways were ordered by raw counts, so the most productive searches sat at positions that no count-based triage would have flagged as ready.

The authors further asked whether Medical Subject Headings, the controlled vocabulary used to index PubMed records, could have substituted for the manual audit, and concluded that they could not. Only four of the pathways in the universe have a MeSH descriptor at all, 26.7 percent of the examined records carried no indexing headings, and a relevant heading showed sensitivity of 0.56 with specificity of 0.97 for identifying substantive indexed units. That combination means a heading-based filter would miss nearly half of the substantive records it should capture, even while excluding most irrelevant ones with high confidence. Controlled vocabulary, in this setting, cannot replace direct inspection of a sample of retrieved records.

The most frequent failure mechanism the audit recorded was what the authors call annotation echo: pathway names entering extracellular-vesicle records only as enrichment or predicted-target annotations, for example when a gene-set analysis labels a vesicle cargo as belonging to a named pathway. Units containing enrichment language were 36 percent pathway-substantive, against 70 percent of units without such language, so enrichment annotations were a strong marker of records that mention a pathway without studying it. The practical upshot is that query auditing assigns each retrieval a readiness status and a concrete action, and here the main target for redesign was annotation echo. Before treating retrieved records as pathway evidence, the authors conclude, a review team should rate a sample for pathway substance and extracellular-vesicle or context relevance rather than relying on publication counts, which alone did not show which of the 26 retrievals were fit for screening.

Subject of Research: Query auditing of named KEGG signaling-pathway PubMed searches in extracellular-vesicle research

Article Title: From PubMed counts to evidence-review readiness: a query audit of named KEGG signaling pathways in extracellular-vesicle research

Article References: Liegertová, M., & Malý, J. (2026). From PubMed counts to evidence-review readiness: a query audit of named KEGG signaling pathways in extracellular-vesicle research. Journal of Translational Medicine. https://doi.org/10.1186/s12967-026-09124-8

Image Credits: AI Generated

DOI: 10.1186/s12967-026-09124-8

Keywords: extracellular vesicles, KEGG pathways, PubMed, query audit, search reproducibility, MeSH indexing, annotation echo, Notch signaling, NF-kappa B, mTOR, evidence synthesis, Journal of Translational Medicine