New Methods Tame Bayesian Inference When Prior Covariances Lose Rank

Bayesian inverse problems sit at the heart of modern science, from weather forecasting to structural engineering, allowing researchers to update uncertain guesses about a system’s state using real-world measurements. But when the unknown parameter is enormously high-dimensional, computing the posterior distribution — the refined probability distribution that combines prior knowledge with observed data — can become prohibitively expensive. A new study published in Computational Science and Engineering on 31 August 2026 proposes fresh mathematical machinery that makes these computations cheaper and more robust in a setting that has long been overlooked: problems where the prior covariance matrix is rank-deficient, meaning it is not invertible.

The paper, authored by Josie König, Elizabeth Qian and Melina A. Freitag, addresses a gap in existing methodology. Likelihood-informed subspace (LIS) dimension reduction and related inference-oriented model reduction approaches had previously assumed that the prior covariance matrix is invertible, both in the definition of the approximations and in the derivation of their optimality guarantees. Yet rank-deficient prior covariances arise naturally in many practical settings. In particle filtering algorithms, for example, the rank of the covariance is restricted by the size of the ensemble. In large-scale applications such as weather forecasting, prior covariances may be poorly conditioned, very close to rank-deficient, or approximated by low-rank factors.

The core idea behind dimension reduction is elegant. Inverse problems often exhibit low-dimensional structure because the available data are only informative in a low-dimensional subspace of the parameter space. Dimension reduction approaches exploit this by restricting inference to the subspace informed by the data, which can be explored far more efficiently. The leading generalized eigenvectors of the Hessian of the negative log-likelihood and the prior precision matrix yield optimal posterior approximations for linear Gaussian Bayesian inverse problems, as established in earlier work. However, these approaches still require evaluation of the original high-dimensional forward model, which may involve evolving a large-scale dynamical system.

Model reduction methods attack the problem from a different angle: instead of shrinking the parameter space, they replace the expensive high-dimensional forward model with a cheaper low-dimensional reduced model. Balanced truncation, a system-theoretic technique, projects the system onto state directions that are simultaneously highly observable and easily reachable. Earlier work showed that balanced truncation can be adapted to the inference problem of recovering the initial condition of a linear dynamical system from noisy measurements taken after the initial time, yielding near-optimal posterior approximations in certain limits. The new paper extends this framework to rank-deficient priors by defining the likelihood-informed subspace through a square-root balancing procedure, which leads to a nonsymmetric eigenvalue problem that remains well-defined even when the prior covariance is singular.

With this reformulation, the authors prove that the resulting posterior approximations inherit optimality guarantees in a lower-dimensional space associated with the range of the prior covariance. Because the posterior covariance for a rank-deficient prior is itself rank-deficient, the classical metrics used to measure optimality — the Förstner distance between covariance matrices and the Mahalanobis-distance Bayes risk for the mean — are only well-defined in this restricted space. The paper shows that the d-dimensional approximations are equivalent to optimal approximations in the s-dimensional restricted problem, where s is the rank of the prior covariance. Notably, the oblique projector defined by square-root balancing is the natural choice here: using an ordinary orthogonal projector instead would produce approximations that lack this optimality interpretation.

The study’s second major contribution is an entirely new model reduction method called prior-driven balancing, or PD-BT, applicable to inferring the initial condition of a linear dynamical system from linear measurements taken after the initial time. The key insight is to interpret the unknown initial condition as an impulse input to a linear time-invariant control system whose input port matrix is given by a square root factor of the prior covariance matrix. The response of this so-called prior-driven system to the impulse input reproduces the noise-weighted output of the original dynamical system, so accurately approximating the input–output map of the prior-driven system enables accurate approximation of the measurements. The method naturally handles both full-rank and rank-deficient prior covariances through the use of an appropriate square-root factor.

The authors also establish theoretical guarantees for the new method. The reduced prior-driven system inherits the stability of the full system, and the approach inherits error bounds from standard system-theoretic balanced truncation analyses. One bound relates the expected output error to the truncated Hankel singular values of the prior-driven system, meaning the expected quality of the approximation depends on how quickly those singular values decay. The authors caution that the constant appearing in the bound can become large when observations are close together in time, making the bound pessimistic in magnitude; its primary utility is to describe trends in the reduced model error rather than exact error sizes.

Numerical experiments tested the three approaches — optimal LIS dimension reduction, LIS balanced truncation, and the new PD-BT — on two benchmark problems. The first is the ISS1R benchmark modeling structural dynamics of the Russian service module of the International Space Station, with a state dimension of 270, using roll, pitch and yaw jets as inputs and gyroscope measurements as outputs. Measurements were simulated over eight seconds with Gaussian noise at roughly ten percent of the maximum output magnitude, and results were averaged over 100 independent replicates. Two prior covariances were tested: an incompatible one of rank 89, and a compatible one of rank 236 constructed using a modification procedure.

In the ISS1R experiments, LIS dimension reduction achieved the lowest approximation error in the restricted space for both prior types. For a compatible prior covariance, the two balanced truncation methods performed identically, and neither could match the optimal dimension reduction. With an incompatible prior, however, LIS-BT produced the largest errors, while PD-BT nearly matched the optimal dimension reduction approach for the mean approximation. In the full space, PD-BT maintained relative errors of about ten to the minus six for the mean and ten to the minus four for the covariance approximation, even though it could not reach the error rates of LIS dimension reduction at higher ranks. The output error bound proved tight in this setting.

The second benchmark, a two-dimensional heat equation with a state dimension of 2500, revealed a different picture. There, observations were sparse relative to the high state dimension, so the Fisher information matrix had a maximum rank of only 25, and the restricted posterior covariance had a rank of just 5. In the restricted space, neither balancing method yielded good approximations, partly because LIS-BT’s optimality guarantees apply under frequent observations, which the sparse measurements violated. In the full space, however, the OLR approach significantly underestimated the posterior covariance rank, while both balancing methods performed well, with PD-BT slightly better — an expected outcome given the incompatibility of the prior. The experiments demonstrate that PD-BT is advantageous for incompatible, rank-deficient prior covariances, delivering accurate and inexpensive forward models and posterior approximations in cases where LIS-BT fails to provide robust results. The authors note that future work includes analyzing the posterior approximation error of PD-BT, exploring other system-theoretic reduction methods, and extending the approach to nonlinear forward models and non-Gaussian distributions.

Subject of Research: Dimension and model reduction methods for linear Bayesian inverse problems with rank-deficient prior covariances

Article Title: Dimension and model reduction approaches for linear Bayesian inverse problems with rank-deficient prior covariances

Article References: König, J., Qian, E., & Freitag, M. A. (2026). Dimension and model reduction approaches for linear Bayesian inverse problems with rank-deficient prior covariances. Computational Science and Engineering, 3(1), Article 7. https://doi.org/10.1007/s44207-026-00016-9

Image Credits: AI Generated

DOI: 10.1007/s44207-026-00016-9

Keywords: Bayesian inverse problems, rank-deficient prior covariance, dimension reduction, model reduction, balanced truncation, likelihood-informed subspace, prior-driven balancing, linear dynamical systems, posterior approximation, Hankel singular values, data assimilation, computational mathematics