Pathway enrichment analysis is one of the most widely used techniques in bioinformatics, turning long lists of genes from RNA-sequencing, proteomics or metabolomics experiments into interpretable biological stories. Yet the method carries a structural flaw that researchers have long handled by hand: pathway databases such as Gene Ontology, KEGG and Reactome organize genes into hierarchies of nested terms that partially overlap one another, and standard enrichment tools test each pathway as if it were statistically independent of all the others. A new open-source method called PathwayDenester, described in BMC Methods, tackles this problem directly by re-evaluating every enriched pathway in light of its overlaps with more significant terms.
The consequence of ignoring overlap structure is what the developers call hitchhiking. Because large pathways contain smaller ones and many terms share substantial fractions of their genes, a pathway can appear highly significant simply because it inherits the genes that enriched a more significant, overlapping term. In current methods, each pathway is tested independently with a hypergeometric test that considers the total number of measured genes, the number of differentially expressed genes, the size of the term and how many of those genes belong to the term. This independence assumption breaks down when many nested pathways are defined by shared genes, so multiple similar terms frequently show almost identical results and essentially reflect the same biological signal. The authors note that some co-enriched pathways can share more than 90 percent of their genes.
To illustrate how misleading this can be, the team performed enrichment analysis on the 225 genes that make up Reactome’s HIV Infection pathway, a scenario representing a theoretically perfect experiment in which every differentially expressed gene belongs to a single pathway. The analysis returned 960 pathways with varying degrees of similarity to the target. The original pathway ranked as most significant, as expected, but the much larger Infectious disease pathway, containing 1,000 genes, achieved a very similar p-value even though only 22.5 percent of its genes were in the input list. Meanwhile the smaller Late Phase of HIV life cycle pathway, with 132 genes entirely contained within the HIV Infection set, also scored as highly significant. A reader scanning such results could easily conclude that other diseases or other phases of the HIV life cycle were represented in the data when they were not.
PathwayDenester works by asking a different question of each pathway: was it enriched by its own genes, or by genes it shares with a more significant pathway? For each pathway A that intersects a more significant pathway B, the method adapts the hypergeometric formula, replacing the usual quantities with the size of pathway A, its number of selected genes, the size of the intersection with B, and the number of selected genes falling in that intersection. If the number of selected genes in the intersection is smaller than expected by chance, the less significant pathway is judged to be dependent on the more significant one and flagged for removal. Although the developers describe the logic through a gene-shuffling thought experiment, the calculations are performed analytically.
The method also includes a reciprocity rule for an important edge case. Sometimes testing a pathway pair in reverse shows that both directions are significant, meaning the two pathways are reciprocally co-enriched rather than one merely hitchhiking on the other. In such cases neither pathway is removed, because co-enriched, biologically related pathways that jointly participate in interacting processes deserve closer examination. As a hypothetical example, the authors suggest that co-enrichment of neutrophil degranulation and lysosome terms could indicate an overrepresentation of genes in the neutrophil lysosome that detect antigen and trigger degranulation. PathwayDenester reports these reciprocal pairs, with their p-values and shared gene counts, as an additional interpretive layer.
Benchmarking showed substantial pruning power. On simulated datasets built from Gene Ontology, Reactome and WikiPathways gene sets, in which gene lists contained varying fractions of genes from a chosen pathway plus random outside genes, PathwayDenester filtered out between roughly 40 and 90 percent of imputed pathways. Filtering was least effective, and least needed, in the extreme cases where only 10 percent of selected genes belonged to the main pathway and fewer than 100 pathways were enriched in total. When the gene list became more representative of a principal pathway, the algorithm converged on eliminating about 90 percent of enriched pathways in the most favorable scenarios, while the authors report no loss of important biological insights.
Validation on two public transcriptomic datasets reinforced the picture. In an RNA-sequencing study of liver tissue from transplant patients with or without inflammation related to nonalcoholic steatohepatitis, an unpaired case-control comparison of eight samples produced 448 differentially expressed genes enriching 286 significant pathways. PathwayDenester excluded 211 of them, or 74 percent. Notably, every one of the five subjectively defined categories used by the original authors still contained at least one pathway approved by the tool, but the reanalysis exposed a mislabeling: a cluster the original study called leukocyte locomotion contained only one immune-specific term, leukocyte migration, which itself would be excluded due to overlap with the more general locomotion term, indicating the cluster should be classified as locomotion-related.
The same NASH analysis illustrated the value of the reciprocity rule. PathwayDenester detected that the terms transmembrane receptor protein kinase activity and regulation of cell migration were reciprocally enriched, suggesting a larger than expected proportion of differentially expressed genes involved in cell migration regulated specifically through transmembrane protein kinases. Among the most differentially expressed genes in the kinase term were FLT4 and KDR, transmembrane kinases involved in the migration of epithelial cells during angiogenesis. In the second validation dataset, RNA-sequencing of JEG-3 cells infected with Zika virus, 174 differentially expressed genes 24 hours after infection enriched 158 Gene Ontology pathways, and PathwayDenester excluded 73 percent of them. Crucially, the tool removed the type I IFN signaling pathway, which was co-enriched with its more significant superset, negative regulation of viral genome replication. Closer inspection showed only two significantly differentially expressed interferon genes, both from the type III IFN family, supporting the idea that viral genome replication was being negatively regulated by other means.
The developers emphasize that PathwayDenester is agnostic to the enrichment tool or pathway database used, requiring only the list of selected genes, pathway p-values and gene membership lists, so it can be applied as a post-enrichment filtering step to the output of any algorithm that reports pathway p-values, including DAVID, clusterProfiler and Enrichr. It is available as an open-source Python command-line program on GitHub and as a web service that requires no installation and generates a Cytoscape network showing kept and excluded pathways linked to their more significant partners. By reducing lists that can run to hundreds of significant terms, the tool aims to reduce the subjective, outcome-biased handpicking of pathways for publication, where researchers tend to choose results conforming to prior expectations, and to make enriched pathway lists more concise, reproducible and biologically meaningful.
Subject of Research: A computational method that removes redundant, overlapping pathways from pathway enrichment analysis results
Article Title: PathwayDenester: a novel method for improving pathway enrichment analyses by taking overlap structures into account
Article References: Pellegrina, D., El-Haj, N., Chaudhari, N., & Helmy, M. (2026). PathwayDenester: a novel method for improving pathway enrichment analyses by taking overlap structures into account. BMC Methods, 3(1), Article 24. https://doi.org/10.1186/s44330-026-00076-5
Image Credits: AI Generated
DOI: 10.1186/s44330-026-00076-5
Keywords: pathway enrichment analysis, bioinformatics, PathwayDenester, Gene Ontology, Reactome, hypergeometric test, RNA-seq, differentially expressed genes, redundancy, open-source software, Zika virus, nonalcoholic steatohepatitis
News Source: Juliet Wilcox. (October 29, 2026). New tool prunes misleading redundant pathways from gene enrichment results. Scienmag.
