Benchmarking Study Reveals Best Statistical Tools for Detecting Variable DNA Methylation

DNA methylation, the chemical addition of methyl groups to cytosines at CpG dinucleotides, is one of the most studied epigenetic modifications in human biology. It regulates gene expression without altering the underlying DNA sequence and is implicated in development, X-chromosome inactivation, genomic imprinting, suppression of transposable elements, and diseases ranging from cancer to Alzheimer’s. Most epigenome-wide association studies, however, have concentrated on differential methylation analysis, which compares average methylation levels between groups. A newer and increasingly recognized approach, variable methylation analysis, focuses instead on differences in methylation variability, and a new comprehensive evaluation published in Heredity now offers the most detailed empirical guidance yet on which computational tools researchers should trust for this task.

The study, led by Siddhant Meshram and colleagues, systematically benchmarked eight statistical models that can be used to identify variably methylated CpG sites in DNA methylation microarray data. The motivation is clear: while tools for differential methylation have proliferated over the past decade, methods for detecting variability-based signals remain scarce, leaving researchers with limited evidence on which to base their choice. Variable methylation matters because shifts in the spread of methylation values, rather than their averages, can reveal subtle epigenetic changes that signal early disease stages or complex regulatory mechanisms. In cancer, for example, methylation variability has been shown to increase in normal tissues adjacent to tumors and to progress with cancer development, with links to disruptions in WNT and FGF signaling pathways in colorectal and breast cancers.

The eight models evaluated span three broad methodological categories. Classical variance tests include Bartlett’s test, Levene’s test, and the Brown-Forsythe test, which assess whether variances differ across groups but vary in their sensitivity to departures from normality. Generalized linear models with varying dispersion, or DGLMs, extend traditional regression by modeling both the mean and the dispersion of a distribution as functions of covariates. Methylation-specific tools include DiffVar, which adapts Levene’s test with moderated t-statistics for large-scale genomic data; EVORA and its regularized successor iEVORA, which use Bartlett’s test to flag outlier methylation patterns; the penalized Exponential Tilt Model, or pETM, which jointly models mean and variance while accounting for correlations between nearby CpG sites; and two joint location-and-scale methods, JLSp and JLSsc, which integrate tests of both average differences and variability differences while adjusting for covariates. EVORA itself was excluded from the benchmark because it provides no clear guidance on how many top-ranked CpG sites should be selected for downstream analysis.

To compare these methods rigorously, the team generated synthetic DNA methylation datasets using two distinct simulation frameworks. The first produced beta-distributed data, which closely mirrors the properties of raw methylation beta values that are naturally bounded between zero and one and often skewed. The second followed a normally distributed framework based on M-values, the logit-transformed methylation measurements commonly used in the field. Each synthetic dataset contained 10,000 CpG sites, with the proportion of variably methylated sites set at zero percent for null simulations and ten percent for power simulations. Variance effects were calibrated so that one group’s variability was either three or five times greater in the beta framework, or 1.5 or 2.5 times greater in the normal framework. Sample sizes ranged from 40 to 150, under both balanced and unbalanced group configurations, and outliers were introduced at controlled levels. Each scenario was replicated 100 times, and performance was assessed using type-I error rate, false discovery rate, statistical power, area under the ROC curve, Matthews correlation coefficient, and F1 score.

The results revealed a nuanced picture of strengths and weaknesses. Under null conditions without outliers, most models controlled type-I error at the nominal five percent level, with Brown-Forsythe and JLSsc showing close adherence, Bartlett’s test moderately conservative, and DGLM and pETM highly conservative. iEVORA, DiffVar, and JLSp failed to control type-I error in these scenarios. When ten percent outliers were introduced, the picture shifted: Brown-Forsythe and JLSsc, previously reliable, no longer maintained type-I error control, while iEVORA, which had struggled without outliers, successfully controlled the error rate in their presence. False discovery rate control followed a similar pattern. All models controlled FDR at the five percent threshold under balanced groups without outliers, but when outliers were added, only Bartlett’s test and pETM maintained effective control. Under a stronger variance effect of five-fold, all models controlled FDR except pETM in the unequal group split with outliers.

Statistical power increased gradually for all models as sample size grew from 40 to 150, reflecting improved sensitivity with more data. Under unequal group sizes of seventy percent controls and thirty percent cases, JLSsc and JLSp showed a slight increase in power while the remaining models declined, suggesting that some methods are more robust to group imbalance than others. An apparent power increase for DGLM under outlier contamination turned out to be misleading: because DGLM’s dispersion estimates rely on deviance-based quantities from the fitted mean model, extreme observations inflate the number of CpGs flagged as differentially variable, and this gain in power was accompanied by an elevated false discovery rate that was no longer controlled at the nominal level. Classification performance, measured by area under the ROC curve, told another striking story: pETM yielded an AuROC score of approximately 0.5, equivalent to random guessing, and remained static across all sample sizes, while every other model improved steadily with increasing data.

Beyond simulations, the researchers applied four models—DiffVar, JLSsc, JLSp, and iEVORA, selected for achieving at least forty percent power in most simulated scenarios—to two publicly available DNA methylation microarray datasets. The first was the Early-Onset Pre-eclampsia dataset, comprising forty placental samples profiled on the Illumina HumanMethylation450 BeadChip with over 430,000 probes remaining after quality filtering. Here, JLSp identified the highest number of variably methylated CpG sites at 65,575, of which 31,681 were uniquely detected by that method alone. JLSsc identified 31,402 sites with 1,041 unique, iEVORA found 6,506 with 1,723 unique, and DiffVar detected only 26 sites with just one unique. Strikingly, only three CpG sites were shared across all four models, indicating minimal consensus among methods. A similar lack of concordance emerged in the second dataset, a breast cancer study of peripheral blood mononuclear cells, underscoring a critical challenge for reproducibility in variable methylation research.

The findings also carry important methodological warnings. Models that assume normally distributed data, such as Bartlett’s test and DGLM, performed well on normal-distributed simulations but exhibited inflated type-I error rates and reduced power when applied to beta-distributed data. The authors argue against the routine transformation of beta values into M-values solely to satisfy model assumptions, noting that such transformations can distort the original data, create artificial differences in means or variances where none exist, and reduce biological interpretability. Instead, they recommend methods inherently compatible with raw methylation values, such as Brown-Forsythe and JLSsc, which maintained robust performance across both simulation frameworks. Sample size emerged as one of the most influential factors, with JLSsc, JLSp, and Brown-Forsythe showing marked improvements as datasets grew, while pETM performed near random even at higher sample sizes.

The study also identifies practical gaps in existing variable methylation pipelines. Most methods do not report effect size estimates, standard errors, or test statistics, quantities essential for prioritizing findings, performing pathway enrichment analyses, and enabling meta-analysis across studies. The authors propose a minimum standardized reporting format that would include a feature identifier, genomic coordinates, the reference genome, the estimated variability effect, its standard error, test statistic, confidence interval, nominal and adjusted p-values, and platform-specific probe identifiers for microarray studies. They further recommend extending variable methylation analysis to the regional level, detecting variably methylated regions as counterparts to differentially methylated regions, and developing hybrid models capable of jointly assessing both mean and variance differences at sites and regions. Limitations remain: all simulations and real-data analyses were restricted to binary case-control designs, real-data analyses did not adjust for estimated cell-type composition, and the evaluation was limited to array-based data, leaving extensions to sequencing-based technologies for future work. No single tool dominated every scenario, but JLSsc emerged as the most consistently reliable performer, offering researchers a clearer empirical foundation for choosing among methods in this rapidly evolving field.

Subject of Research: Computational evaluation of statistical models for detecting variable DNA methylation in microarray data

Article Title: A comprehensive evaluation of computational models for identifying methylation variability in DNA methylation arrays

Article References: Meshram, S., Fadikar, A., Arunkumar, G., & Chatterjee, S. (2026). A comprehensive evaluation of computational models for identifying methylation variability in DNA methylation arrays. Heredity. https://doi.org/10.1038/s41437-026-00892-z

Image Credits: AI Generated

DOI: 10.1038/s41437-026-00892-z

Keywords: DNA methylation, variable methylation, epigenetics, JLSsc, false discovery rate, statistical power, simulation study, methylation arrays, EWAS, cancer epigenetics, benchmarking, bioinformatics