Algorithm Selection Influences Soil Prediction Accuracy More Than Remote Sensing Covariates

A comprehensive Bayesian hierarchical evaluation of agricultural soils across the Republic of Cyprus has demonstrated that the choice of machine learning algorithm influences the predictive accuracy of soil pedotransfer functions far more than augmenting baseline soil measurements with satellite and environmental covariates. Researchers evaluated four critical topsoil properties across Mediterranean croplands, specifically total nitrogen, available phosphorus, exchangeable potassium, and soil organic carbon, to determine where analytical effort yields the greatest performance return. While earth observation indices from satellite platforms are increasingly integrated into digital agricultural workflows, the experimental findings established that advanced learning algorithms, particularly ensemble combinations and decision tree models, produced the dominant reductions in prediction error. Augmenting laboratory soil data with remotely sensed environmental covariates provided secondary and highly variable performance shifts that differed substantially by soil property, with true environmental signals remaining constrained by the physical mismatch between point-scale soil extractions and area-averaged satellite pixels.

Understanding topsoil condition has gained renewed regulatory urgency across Europe following the formal agreement in October 2025 on the European Union Directive on Soil Monitoring and Resilience. This policy framework establishes concrete monitoring standards across member states to ensure that all European soils attain healthy status by the year 2050, explicitly instructing authorities to monitor vital soil health indicators including nutrient availability and organic matter concentrations. Soil constitutes a complex physicochemical matrix underpinning critical terrestrial ecosystem services such as agricultural food supply, biomass energy, and industrial fiber production. However, because natural soil formation operates on extremely protracted geological timescales, topsoil remains an inherently finite natural capital subject to degradation from climate change, intensive tillage, and erosion. Essential macronutrients including nitrogen, phosphorus, and potassium govern physiological plant processes such as amino acid synthesis, cellular energy transfer through adenosine triphosphate, and osmotic stomatal regulation, while organic carbon enhances moisture retention capacity, regulates nutrient cycling, and serves as an active sink or source for atmospheric carbon dioxide.

Despite the recognized agronomic necessity of tracking these chemical components, traditional empirical characterization relies on field soil sampling, physical transport, chemical preparation, and exhaustive wet-chemistry laboratory analysis. These conventional protocols demand trained personnel, impose substantial labor burdens, and incur recurring financial expenses that can become prohibitive for agricultural producers and environmental agencies. Consequently, soil scientists introduced pedotransfer functions as predictive mathematical models that approximate difficult-to-measure chemical properties from routine, less demanding physical measurements. Traditionally, pedotransfer functions utilized multiple linear regression frameworks to summarize linear relationships between input predictors and target soil attributes. Modern agricultural research has sought to enhance these mathematical approximations by deploying machine learning algorithms capable of capturing complex non-linear associations, alongside remote sensing inputs gathered from satellite and aerial sensors. However, published literature has presented contradictory findings regarding the actual utility of remote sensing data in pedotransfer functions, reporting outcomes ranging from substantial predictive accuracy gains to negligible or even negative performance contributions.

To rigorously resolve these contradictory findings, the research team examined a published national georeferenced topsoil database encompassing 951 agricultural sampling sites distributed across the government-controlled southern regions of Cyprus. The island represents a semi-arid Mediterranean agroecosystem characterized by calcareous soils and escalating climatic vulnerabilities. Field crews collected composite topsoil samples from the upper zero to twenty centimeter layer between July 2019 and June 2023, capturing all seasonal conditions with sampling shares distributed across autumn, spring, summer, and winter. In the source laboratory analyses, total nitrogen was quantified using the Kjeldahl digestion procedure, available phosphorus through the spectrophotometric Olsen extraction, exchangeable potassium via ammonium acetate extraction with flame photometry, and organic carbon through the Walkley-Black wet-oxidation technique. Ancillary laboratory measurements provided baseline predictors, including soil pH and electrical conductivity determined in one-to-two-point-five soil-to-water suspensions, calcium carbonate concentrations measured with a Bernard calcimeter, and soil particle-size fractions of sand, silt, and clay assessed using the hydrometer method.

To construct the multi-source remote sensing predictor suite, the researchers accessed satellite archives through the Google Earth Engine cloud computing environment using the rgee interface in the R statistical environment. Multispectral observations from the Sentinel-2 constellation provided surface reflectance data from Level-2A products, screened to retain scenes with less than five percent cloud cover and filtered for atmospheric artefacts using the Scene Classification Layer. To account for dynamic surface conditions, satellite observations were temporally matched to each individual soil sample within a plus-or-minus one-month window centered on its specific field collection date. Valid cloud-free pixels were summarized via median compositing, and three standardized spectral indices were derived: the Normalized Difference Vegetation Index, the Normalized Difference Water Index, and the Bare Soil Index. These spectral metrics were combined with daily daytime land surface temperature from the MODIS MOD11A2 composite, rainfall totals from the CHIRPS quasi-global precipitation dataset, six topographic parameters derived from the SRTM digital elevation model, and categorical land cover from the Copernicus Proba-V dataset. Continuous layers were harmonized to a common ten-meter spatial grid using bilinear interpolation, while nearest-neighbor interpolation preserved the integrity of the categorical land cover class.

Data preprocessing addressed quality assurance across the 951 records, eliminating incomplete rows to establish a clean baseline of 928 entries. The researchers mitigated univariate extremity by trimming target distributions to the first through ninety-ninth percentiles, followed by a residual-based outlier filter that trained random forest and isolation forest models to remove extreme residual deviations exceeding the ninety-seventh percentile. The resulting distributions showed that organic carbon approached a symmetrical distribution with skewness of 0.31 and kurtosis of 2.85, whereas phosphorus displayed strong right-skewness with skewness of 1.87 and kurtosis of 6.46. Predictor variables were standardized to zero mean and unit variance for scale-sensitive algorithms, while tree-based models retained native scaling. The team evaluated twelve distinct regression architectures representing varying computational complexity: multiple linear regression, least absolute shrinkage and selection operator regression, partial least squares regression, K-nearest neighbors, Cubist rule-based ensembles, support vector machines fitted with linear, polynomial, and radial kernels, random forests, Extreme Gradient Boosting, deep feedforward neural networks built with Keras and TensorFlow, and a weighted multi-model ensemble combining the three best-performing algorithms.

To evaluate methodological choices alongside data inputs, the study applied four feature selection strategies: a full predictor baseline without reduction, the Boruta algorithm based on random forest shadow features, Recursive Feature Elimination using repeated cross-validation in the caret package, and Forward Recursive Feature Selection implemented through stepwise model optimization. Crucially, the experimental framework introduced three distinct database structures to separate real environmental signals from dimensionality artifacts. The baseline Soil database contained purely measured laboratory soil parameters. The Soil RS database combined measured soil properties with the twelve remote-sensing-derived environmental covariates. The third variant, designated Soil RS Pseudo, was constructed by jointly permuting the remote-sensing vectors across observations within each replicate using distinct random seeds. This block-permutation preserved the marginal distributions and mutual correlations of the environmental covariates while completely severing their empirical association with the target soil attributes, establishing a rigorous control to evaluate whether remote-sensing gains represented genuine predictive signals or merely mathematical capitalization on added degrees of freedom.

Cross-validated model performance was quantified using root mean squared error, mean error bias, the coefficient of determination, and Lin’s concordance correlation coefficient evaluated across repeated Monte Carlo cross-validation runs with an eighty-twenty training-test split. To account for the hierarchical structure of the factorial experiment and prevent aggregation bias, the resulting cross-validated errors were analyzed using a Bayesian hierarchical linear model implemented via the brms package and cmdstanr interface. The statistical formulation modeled percentage differences in root mean squared error relative to established factorial baselines, using Gamma likelihoods with log links for nitrogen, phosphorus, and potassium, and a lognormal likelihood with a three-way interaction for residual dispersion to accommodate heteroskedasticity in organic carbon. Analysis incorporated varying intercept and slope terms across replicates, while evaluating informative, weakly informative, and diffuse prior specifications. Model stability was corroborated across five Markov chains running six thousand iterations each, with convergence confirmed by potential scale reduction factors of 1.00 and Bayesian coefficients of determination exceeding 0.73 across all soil properties.

The posterior parameter distributions provided definitive empirical answers to the four core research hypotheses. Evaluating the first hypothesis confirmed that algorithm selection exerted a vastly stronger influence on predictive error than remote sensing integration, with posterior probabilities exceeding 0.999 across all four soil properties and mean difference contrasts reaching minus 53.84 percent for organic carbon and minus 13.56 percent for nitrogen. Random forest and ensemble frameworks consistently achieved the lowest prediction errors, with ensembles reducing root mean squared error by 22.6 percent for nitrogen, 31.3 percent for phosphorus, 23.5 percent for potassium, and 41.4 percent for organic carbon relative to linear baselines. Addressing the second hypothesis, feature selection procedures demonstrated secondary but consistent performance improvements, generating the greatest error reductions for organic carbon at minus 6.66 percent and nitrogen at minus 5.91 percent when using Forward Recursive Feature Selection or Recursive Feature Elimination. In contrast, Boruta selected nearly all candidate variables and offered negligible error improvements over the full predictor set.

Testing the third and fourth hypotheses clarified the nuanced contribution of remote sensing covariates. Remote-sensing integration yielded distinct, property-dependent effects, supporting the third hypothesis: organic carbon and nitrogen demonstrated meaningful gains from the environmental predictor suite, whereas phosphorus and potassium showed minimal shifts. Furthermore, the fourth hypothesis was verified by contrast analysis against the permuted control, where true remote-sensing predictors consistently outperformed pseudo-covariates with posterior probabilities exceeding 0.999. In nitrogen models, true covariates lowered root mean squared error by 5.1 percent while permuted covariates increased error by 10.6 percent. However, the organic carbon results revealed a critical methodological caveat: while adding satellite covariates reduced nominal error by 22.3 percent, the permuted pseudo-covariates also generated an apparent error reduction of 19.0 percent. This contrast proved that roughly sixteen percentage points of the apparent performance enhancement stemmed entirely from the expanded dimensionality of the predictor pool, leaving a genuine environmental signal of approximately three percent.

Observation-level residual diagnostics of the high-performing random forest models revealed that while mean residual errors remained close to zero, test-set residuals correlated positively with measured values, exhibiting Pearson correlation coefficients between 0.53 and 0.92. This conditional bias caused tree-ensemble regressors to systematically over-predict low nutrient concentrations and under-predict high values, pulling extreme observations toward the sample mean. The dispersion of residuals expanded considerably at higher measurement magnitudes, particularly for strongly skewed properties like phosphorus and potassium. Feature selection with Forward Recursive Feature Selection tightened the limits of agreement from plus-or-minus 0.80 percent to plus-or-minus 0.70 percent for organic carbon and reduced residual slopes for nitrogen and phosphorus. In contrast, adding the satellite predictor suite exerted only marginal effects on residual spread, confirming that parsimonious feature pruning had a greater stabilizing influence on observation-level predictive uncertainty than adding external geospatial layers.

The underlying mechanistic explanation for the bounded contribution of remote sensing centers on the severe mismatch in spatial support between predictors and laboratory responses. The physical soil analyses examined composite samples collected from soil volumes smaller than one hundred cubic centimeters, whereas the environmental covariates reflected pixel footprints spanning from one hundred square meters for ten-meter Sentinel-2 pixels to several square kilometers for coarse-resolution CHIRPS precipitation grids. Resampling these datasets to a shared ten-meter raster provided point-to-pixel co-registration but could not reconstruct sub-pixel soil heterogeneity. Consequently, satellite covariates provided valuable predictive utility primarily for soil properties governed by broad landscape-scale gradients of climate and topography, such as organic carbon. Conversely, nutrients such as available phosphorus and exchangeable potassium remain heavily controlled by short-range mineralogy, soil chemistry, and localized agronomic management practices including fertilizer application and tillage history, rendering coarse-support environmental covariates largely uninformative and partially redundant relative to direct laboratory soil measurements.

Subject of Research: Bayesian assessment of machine learning algorithms and remote sensing covariates for soil property prediction

Article Title: Quantifying the added value of remote-sensing covariates in soil pedotransfer functions: A Bayesian hierarchical assessment with permutation controls for Mediterranean croplands in Cyprus

Article References: Themistokleous, G., Varvaris, I., Pittaki-Chrysodonta, Z., & Hadjimitsis, D. (2026). Quantifying the added value of remote-sensing covariates in soil pedotransfer functions: A Bayesian hierarchical assessment with permutation controls for Mediterranean croplands in Cyprus. Smart Agricultural Technology, 15, Article 102696. https://doi.org/10.1016/j.atech.2026.102696

Image Credits: AI Generated

DOI: 10.1016/j.atech.2026.102696

Keywords: pedotransfer functions, soil organic carbon, soil nutrients, remote sensing, Sentinel-2, Bayesian hierarchical modeling, machine learning, random forest, ensemble models, feature selection, Cyprus, Mediterranean croplands, soil monitoring, permutation controls, root mean squared error, digital soil mapping, topsoil assessment