A new study introduces a background-free framework for detecting out-of-distribution text by leveraging the theoretical probability distributions of Mahalanobis distance under multivariate normality assumptions.
Machine learning models often degrade in reliability when exposed to out-of-distribution data points that diverge from the general profile of the task-specific training data. In natural language processing, out-of-domain detection models serve dual functions: sieving anomalous inputs before they enter a downstream pipeline and scrutinising outputs from generative language models in a selective generation setup. Traditional anomaly detection literature frequently employs Mahalanobis distance to measure how far an incoming embedding vector lies from an in-distribution population mean relative to its covariance structure.
Distributional shifts in text data are broadly divided into background shifts and semantic shifts, which correspond to near-shift and far-shift conditions. A background shift, also characterized as a covariate shift, occurs when the overall domain changes, whereas a semantic shift occurs when unseen classes or entities appear. Conventional techniques often rely on both in-distribution and background datasets to construct scoring classifiers or calibration metrics. However, securing high-quality, representative background text collections presents practical challenges, and shifts in background distributions can unpredictably alter the resulting decision boundaries.
Prior benchmark methods, most notably the Relative Mahalanobis Distance formulation, calculate the difference between the distance from an in-domain distribution and the distance from an auxiliary background class. In that formulation, a query is designated as in-distribution if the Relative Mahalanobis Distance value is less than or equal to zero, and out-of-distribution otherwise. The researchers noted that setting this decision boundary induces an asymmetric, outward-inclined ellipsoid relative to the in-domain center, which inflates false positive rates where in-domain queries are rejected and false negative rates where out-of-domain queries are misclassified as in-domain.
To overcome these limitations, the authors developed an approach derived solely from in-domain text embeddings, removing background data dependency entirely. Under the assumption that in-domain text embeddings approximate a p-dimensional multivariate normal distribution with mean vector mu and covariance matrix Sigma, the squared Mahalanobis distance of a new independent query vector follows known distribution laws. Specifically, when parameters are estimated using sample mean and unbiased sample covariance from n training samples, the scaled Mahalanobis distance follows a Hotelling’s T-squared distribution with dimension p and degrees of freedom n minus one.
The study formally established in Lemma 1 that multiplying the Mahalanobis distance statistic by a scaling factor of one plus one over n raised to the power of negative one-half produces a Hotelling T-squared distribution. By further relating the Hotelling T-squared statistic to the Snedecor F-distribution with degrees of freedom p and n minus p minus one, the framework allows exact cumulative probabilities and percentile thresholds to be calculated directly. Under this transformation, setting an alpha percentile cutoff establishes an explicit out-of-distribution detection decision rule with an analytical false positive rate of one minus alpha.
As an alternative approximation for settings where sample size n is sufficiently large, the authors investigated a Chi-squared distribution formulation. As the sample size grows, the sample mean and covariance matrix converge toward the true population parameters, enabling the Mahalanobis distance to be approximated by a Chi-squared distribution with p degrees of freedom. In this simplified formulation, the threshold corresponding to a target cumulative probability alpha corresponds directly to the critical percentile point of the Chi-squared distribution.
The experimental setup evaluated the proposed algorithms against several standard machine learning baselines across benchmark text datasets, selecting approximately 10,000 documents from each source. The in-distribution dataset was derived from the xsum corpus of BBC articles, while English web text from the Colossal Clean Crawled Corpus served as the background dataset for baseline training. Near-shift out-of-distribution evaluation utilized news articles from cnn dailymail, while far-shift out-of-distribution evaluation incorporated dialogue transcripts from samsum and online forum submissions from reddit tifu.
Text documents were partitioned into sentence chunks constrained by token limits and processed through the pre-trained mxbai-embed-large-v1 embedding model without fine-tuning, avoiding feature distortion. Performance across a seventy-thirty train-test split was evaluated using the area under the receiver operating characteristic curve between the in-distribution test split and each respective out-of-distribution test set. Baseline comparisons included Relative Mahalanobis Distance, logistic regression, support vector machines, AdaBoost, and Random Forest models trained on combined in-distribution and background data.
The evaluation demonstrated that both the Hotelling’s T-squared approach and the Chi-squared approach outperformed Relative Mahalanobis Distance and logistic regression baselines, while remaining completely background-free. Hotelling’s T-squared method further improved upon the Chi-squared formulation across near-shift and far-shift benchmarks, showing that exact finite-sample covariance modeling enhances detection accuracy. For the Chi-squared variant, covariance parameters estimated via Maximum Likelihood Estimation yielded superior area under the curve metrics compared to robust covariance estimation.
All evaluated models achieved higher area under the receiver operating characteristic curve scores on far-shift datasets than on near-shift datasets. Despite having access to background training corpora, traditional supervised classifiers including support vector machines, Random Forest, and AdaBoost failed to exceed the performance of the proposed statistical distance methods. The authors highlighted that because the new method requires only calculating the Mahalanobis distance of incoming query embeddings, it avoids secondary classifier training and scales efficiently to high-dimensional representations and large document collections.
The authors discussed operational boundaries of the method, identifying the multivariate normality assumption of embedding vectors as a central consideration. The researchers suggested that modeling embedding spaces using Gaussian mixture distributions offers an important direction for future extensions while preserving the background-free statistical foundation.
Subject of Research: Background-free out-of-distribution text detection using statistical distance distributions
Article Title: Out-of-distribution detection in text using statistical techniques
Article References: Raghavan, D., Bhandari, S., Banerjee, J., & Bandyopadhyay, S. (2025). Out-of-distribution detection in text using statistical techniques. Discover Data, 3(1), Article 46. https://doi.org/10.1007/s44248-025-00091-x
Image Credits: AI Generated
DOI: 10.1007/s44248-025-00091-x
Keywords: Out-of-distribution detection, Mahalanobis distance, Hotelling's T-squared distribution, Chi-squared distribution, Natural language processing, Text embeddings, Selective generation, Covariate shift, Semantic shift, Relative Mahalanobis distance, mxbai-embed-large-v1, AUROC evaluation, Multivariate normal distribution, xsum dataset, cnn dailymail dataset, samsum dataset, reddit tifu dataset
News Source: Denise Maddox. (November 16, 2026). Statistical Distance Methods Accurately Detect Out-of-Distribution Text Without Background Data. Scienmag.



