Hidden data leakage inflates AI model accuracy in cement plant records

A machine-learning audit of a working cement plant has revealed how badly industrial data can mislead even sophisticated predictive models. Researchers analyzing a cement clinker production line found that a model trained on raw PLC-SCADA records achieved a near-perfect coefficient of determination of R² = 0.9999 for predicting firing calorie, a figure that collapsed to R² = 0.8109 once leaked variables were removed from the training set. The dramatic gap, reported in Scientific Reports, illustrates how variables that secretly encode the answer, rather than genuinely predict it, can make industrial AI systems look far more capable than they are.

The problem begins with the data itself. Modern cement plants log thousands of PLC-SCADA variables, but the tags and labels attached to those signals are often incomplete, inconsistent, or outright deceptive. When data scientists build models from such records, they may include variables that were never actually manipulated by operators, or accidentally admit target-related variables that leak information about the quantity being predicted. This contaminates both control-advisory systems and data-driven models, producing inflated performance metrics that evaporate under real operating conditions.

To tackle this, the research team proposed a two-evidence variable ontology and leakage-auditing approach designed specifically for industrial PLC-SCADA data. The method combines two independent lines of evidence: name-based evidence drawn from the variable labels themselves, and behavior-based signal evidence drawn from how the variables actually behave over time. Expert validation is reserved for exceptional cases that the automated evidence cannot resolve. The whole procedure is packaged into a unified pre-modelling workflow that screens variables before any model is trained.

At the heart of the behavioral screening lies the staircase score, a lightweight descriptor that distinguishes piecewise-constant, setpoint-like signals from continuously varying measurements. Setpoints, the target values entered by operators or control systems, tend to hold steady for stretches and then jump, while physical measurements drift smoothly. Separating the two matters because a variable labeled as a setpoint may in fact be a continuously changing signal, an anomaly the team explicitly documented in the plant data.

The approach was applied to a real industrial dataset from a cement clinker production line comprising 2.46 million samples, 375 variables, and 11 logged operational days. Among 26 documented control parameters, eight exhibited actively changing setpoint behavior during the analyzed period, while one additional operational control was confirmed through process-engineering validation. In total, nine control variables were identified for the ontology.

The leakage audit went beyond simple label errors. The researchers considered three distinct leakage types relevant to industrial multivariate regression: definitional or mass balance leakage, where a variable is mathematically tied to the target by a physical balance; component leakage, where constituent quantities reveal the aggregate; and computed feature leakage, where derived features inadvertently encode the outcome. Removing variables implicated in these categories was what drove the firing-calorie model’s performance down from R² = 0.9999 to R² = 0.8109, a more honest estimate of what the remaining variables can genuinely support.

Robustness checks strengthened the case for the behavioral method. The separation between setpoint-like and continuous signals remained unchanged when the threshold was selected automatically by the unsupervised Otsu method rather than by hand. The result was also reproduced in two sub-periods of the data that had different logging rates, suggesting the approach does not depend on a particular sampling regime. Computational cost proved modest: screening all 375 variables took about 21 seconds, with runtime scaling linearly with data size, making the audit practical for routine use on large industrial archives.

Perhaps most importantly, the ontology held up to independent human scrutiny. A second, independent process expert performed a blinded annotation of all 375 variables, without knowing the automated classification. That expert agreed with the final ontology for 89.3 percent of the variables, yielding a Cohen’s kappa of 0.755, a level conventionally interpreted as substantial agreement. All nine control variables were assigned to the CONTROL category by the blinded annotator, matching the pipeline’s conclusions.

For industrial operators and data scientists, the findings carry a cautionary message: a model’s apparent accuracy can be an artifact of hidden leakage rather than genuine predictive power, and the semantics of raw plant data cannot be taken at face value. By combining what variables are called with how they behave, and by auditing for multiple leakage pathways before modelling begins, the workflow offers a practical route to trustworthy machine learning in process industries, where the difference between R² = 0.9999 and R² = 0.8109 is the difference between a mirage and a usable model.

Subject of Research: Variable ontology and target-leakage screening in industrial PLC-SCADA data

Article Title: Variable ontology and target-leakage screening in industrial PLC-SCADA data using name-based and behavioral evidence

Article References: Şahin, M., Talu, M. F., Kürker, F., Onat, C., Öztürk, M., & Dağ, A. (2026). Variable ontology and target-leakage screening in industrial PLC-SCADA data using name-based and behavioral evidence. Scientific Reports. https://doi.org/10.1038/s41598-026-74767-y

Image Credits: AI Generated

DOI: 10.1038/s41598-026-74767-y

Keywords: PLC-SCADA, target leakage, cement clinker production, variable ontology, machine learning, industrial data, staircase score, setpoint detection, multivariate regression, process industries, Variable, ontology