Persistence Model Outperforms Complex Machine Learning in Indonesian Rice Price Forecasting

In national agricultural policy and food security monitoring across Indonesia, accurate short-horizon commodity price predictions play an essential role in stabilizing staple food supplies and guiding government interventions for approximately 270 million consumers. A comprehensive benchmarking study published in the Journal of Agriculture and Food Research has evaluated nine forecasting pipelines alongside five post-hoc explainable artificial intelligence methods under a leakage-aware temporal framework. The empirical results demonstrate that a basic naïve persistence forecast, which simply projects the current month’s observed price into the subsequent month, achieved the lowest bootstrap mean absolute percentage error of 1.847 percent across 48 expanding-window forecast origins spanning from January 2021 through December 2024. This simple baseline outperformed an extensive array of sophisticated linear, tree-based ensemble, and kernel-based machine learning architectures. Simultaneously, the investigation revealed that predictive accuracy rankings and model explanation quality rankings diverge substantially, demonstrating that algorithms offering superior numerical precision do not automatically deliver faithful, consistent, or stable diagnostics for human auditing.

Rice price forecasting occupies a foundational position within Indonesian agricultural governance because medium-grade rice serves as the primary staple food across the archipelago and represents a major weighting factor within the national Consumer Price Index. Reliable forward estimates enable institutional bodies, including the National Food Agency and state logistics agency Badan Urusan Logistik, to orchestrate operational measures such as regional buffer-stock releases, targeted consumer subsidies, and external import or export regulations. Although historical statistical modeling in the region has predominantly depended on linear autoregressive integrated moving average formulations, these classical tools cannot readily accommodate non-linear regime shifts, high-dimensional input spaces, or complex interactions among exogenous market determinants. Consequently, researchers have progressively turned toward machine learning architectures, such as Random Forest, XGBoost, and LightGBM, to capture intricate predictor behaviors. However, the inherent black-box opacity of advanced machine learning models introduces significant deployment barriers, limiting institutional trust and impeding compliance with emerging regulatory standards that demand verifiable transparency across critical food supply chains.

To rigorously address these computational challenges without generating overly optimistic forecasts, the researchers constructed a reproducible, leakage-aware protocol utilizing 84 monthly national records collected from Badan Pusat Statistik between January 2018 and December 2024. To reflect genuine administrative constraints, exogenous production indicators—including harvested area, paddy output, milled production, and agricultural productivity—were lagged by exactly two months, accounting for inherent data-gathering and reporting delays. Furthermore, seventeen total predictors were engineered, incorporating five direct price lags alongside eight moving averages and rolling standard deviations calculated over three, six, and twelve-month lookback windows. Critically, every rolling statistic was systematically shifted by one month to prevent target leakage, ensuring that no future target information contaminated the historical feature space. Because the maximum feature lag required twelve preceding months of historical values, the initial twelve observations served as the burn-in period, leaving 72 feature-complete monthly observations from January 2019 to December 2024 for formal empirical investigation.

The experimental architecture systematically separated the evaluation of forecasting accuracy from the quantitative assessment of explainability through two distinct analytical designs. Design A evaluated out-of-sample forecasting performance using an expanding-window protocol spanning 48 consecutive one-step-ahead origins from January 2021 to December 2024, beginning with an initial training window of 24 feature-complete observations covering January 2019 to December 2020. At each monthly origin, all candidate models underwent complete re-estimation using only historical data available before the forecast target month. In contrast, Design B established a fixed-model environment dedicated to explainability benchmarking, where machine learning architectures were trained on the 60 feature-complete observations available prior to 2024 and subsequently generated predictions for the twelve held-out monthly observations of 2024. This methodological bifurcation prevented monthly parameter updates from confounding post-hoc explanation metrics, allowing the research team to evaluate how attribution methods behaved when interrogating a fixed model across an entire unseen calendar year.

Under the expanding-window forecasting protocol of Design A, the benchmark revealed striking differences across algorithmic families. The standard naïve benchmark secured the lowest overall bootstrap mean absolute percentage error at 1.847 percent with a 95 percent bootstrap confidence interval of 1.29 to 2.53 percent, accompanied by a bootstrap mean absolute error of 211.542 Indonesian Rupiah and a root mean squared error of 342.242 Rupiah. Ordinary least squares Linear Regression ranked second overall with a mean absolute percentage error of 2.522 percent. Among non-linear machine learning configurations, XGBoost performed best with an error of 2.906 percent, followed by Random Forest at 3.155 percent, while LightGBM produced a substantially higher error of 5.941 percent. Support Vector Regression with a radial basis function kernel deteriorated markedly, recording an error of 10.804 percent, as incoming prices expanded beyond the historical ranges captured in its standardized training window.

Statistical evaluation further highlighted the subtle dynamics governing these predictive outcomes across the 48 evaluation origins. A prespecified two-sided Diebold-Mariano test comparing squared-error loss differentials between XGBoost and the naïve baseline returned a test statistic of 1.873 with an unadjusted p-value of 0.061. Because this value exceeded the five percent significance threshold, the empirical evidence proved insufficient to reject the null hypothesis of equal forecast accuracy, indicating that XGBoost’s non-linear capacity provided no statistically significant error reduction over simple persistence. Meanwhile, Mean Absolute Scaled Error calculations yielded values of 2.203 for the naïve forecast and 3.401 for XGBoost relative to the pre-evaluation training baseline. Both ARIMAX and seasonal ARIMAX formulations encountered numerical singular matrix exceptions at all 48 evaluation origins under the L-BFGS-B optimization routine, triggering an automated operational fallback to the naïve forecast that yielded near-identical error rates by construction without demonstrating true autoregressive convergence.

To investigate underlying market conditions, the authors attempted an endogenous Zivot-Andrews structural break test on the medium-grade price series, which similarly failed due to a singular matrix error. A descriptive examination of the empirical series revealed that the mean monthly price across the twelve observations of 2024 reached 12,800.80 Rupiah per kilogram, representing an upward level shift of 30.0 percent compared to the pre-2024 historical mean of 9846.41 Rupiah. Despite this substantial macroeconomic elevation, month-to-month price changes remained orderly and predictable from immediate prior observations. This sustained continuity explained why the zero-parameter naïve persistence forecast retained superior accuracy throughout the evaluation window, as the most recent monthly observation continued to provide an exceptionally strong predictive anchor that tree-based splitting algorithms and kernel projections struggled to surpass when faced with previously unobserved price domains.

In the post-hoc explainability assessment conducted under Design B, five prominent diagnostic frameworks—SHAP, LIME, Tree Interpreter, Permutation Importance, and Partial Dependence Plots—were evaluated across nineteen method-model pairings using four quantitative dimensions. The researchers formulated a composite Degree of Explainability metric synthesizing the Feature Sensitivity Index, neighborhood consistency, perturbation stability, and sparsity-based simplicity under a baseline weighting scheme of 0.35, 0.25, 0.25, and 0.15, respectively. Across all nineteen configurations, the global pairing of Random Forest with Partial Dependence Plots achieved the highest composite score of 0.854 and captured a 70.6 percent rank-one probability across 500 bootstrap iterations. This configuration demonstrated exceptional neighborhood consistency of 0.995 and perturbation stability of 0.992, alongside a Feature Sensitivity Index of 0.811, illustrating that global response surface profiling generated highly reproducible structural summaries of model mechanics.

Among local instance-level explanation techniques, Random Forest paired with SHAP achieved the premier standing, yielding a composite Degree of Explainability score of 0.825 and recording the narrowest leave-one-month-out sensitivity range of 0.815 to 0.829 across the entire study. Local SHAP diagnostics revealed that the fixed Random Forest model allocated primary predictive importance to recent price history, led by the one-month price lag and short-term rolling moving averages, while two-month-lagged national production and harvest area indicators received minor predictive weight. Conversely, Support Vector Regression paired with SHAP ranked lowest among all nineteen candidates with a score of 0.454, crippled by an unstable perturbation score of 0.218 and consistency of 0.370. Permutation-based methods also exhibited wide uncertainty intervals due to severe collinearity among temporal lags, confirming that naive feature shuffling introduces artificial predictor combinations that destabilize variance-based importance rankings.

The benchmark established that model performance and explanation quality do not correlate linearly, as XGBoost delivered the strongest machine learning forecast accuracy while its local SHAP diagnostic ranked eleventh overall in explainability. Sensitivity testing across six contrasting weighting schemes demonstrated substantial broad alignment with a mean pairwise Kendall rank correlation of 0.733, yet the optimal method shifted according to analytical priorities: Random Forest with Partial Dependence Plots led under sensitivity-focused criteria, LightGBM with Partial Dependence Plots prevailed under stability and consistency schemes, and XGBoost with Partial Dependence Plots topped simplicity-oriented assessments. Crucially, the authors emphasize that post-hoc feature attributions strictly delineate predictive associations within fitted computational parameters rather than causal economic relationships. Predictive reliance on recent price momentum reflects data-generating patterns during the evaluation period rather than the irrelevance of agricultural fundamentals, demonstrating that machine learning explanations provide vital operational auditing tools without serving as causal policy simulation instruments.

Subject of Research: Benchmarking machine learning and explainable artificial intelligence for Indonesian rice price forecasting

Article Title: Time-aware explainable AI benchmarking for Indonesian rice price forecasting: A leakage-aware quantitative evaluation framework

Article References: Hidayat, R., Jati, T. A., Yeniay, A. E., & Wicaksono, H. (2026). Time-aware explainable AI benchmarking for Indonesian rice price forecasting: A leakage-aware quantitative evaluation framework. Journal of Agriculture and Food Research, 32, Article 103404. https://doi.org/10.1016/j.jafr.2026.103404

Image Credits: AI Generated

DOI: 10.1016/j.jafr.2026.103404

Keywords: rice price forecasting, explainable artificial intelligence, Indonesian agriculture, machine learning benchmarking, Degree of Explainability, SHAP, Partial Dependence Plots, LIME, Tree Interpreter, Random Forest, XGBoost, LightGBM, Support Vector Regression, time series forecasting, data leakage prevention, expanding window evaluation, Feature Sensitivity Index, price persistence, agricultural economics