Spatiotemporal Transformer forecasts citrus pests within one orchard

The model, called ST-CitrusFormer, was evaluated on a chronologically held-out 2024 test set using field observations collected from 2019 through 2024. It outperformed established time-series baselines, including LSTM, Informer, PatchTST and TimeXer, but the findings apply only to the studied orchard and do not yet establish an operationally validated system for other farms, varieties or climates. Researchers Changning Ji and Hong Xie designed the model to address two features of orchard forecasting: pests and diseases change over time, while conditions in neighbouring plots may influence one another.

The study covered Huanglongbing, citrus red mite, aphid and sooty mould, with occurrence severity recorded on a five-level scale from zero to four. Rather than predicting only whether a category was present, ST-CitrusFormer was trained to forecast both the severity class and a continuous occurrence-intensity value for every plot and day in a seven-day horizon. This dual-task design reflects the information needed for plot-level early warning over one to seven days.

The evidence came from an approximately 18-hectare demonstration orchard near the boundary between southern Jiangxi and northern Guangxi, where navel orange and Wokan mandarin were the main subtropical citrus varieties. Twelve sampling plots were differentiated by topography, variety and cultivation management. Each plot had an automatic weather station, a soil-moisture sensor and yellow sticky-trap monitoring devices, while professional plant-protection technicians conducted weekly field surveys.

Integrating these measurements at daily granularity produced approximately 18,720 samples with 14 variables: six meteorological features, three phenological features, soil moisture, elevation, slope aspect, pesticide-application timing and citrus variety. Data preparation included missing-value imputation, outlier detection, normalisation and label encoding. Continuous variables with gaps were filled with a five-nearest-neighbour procedure, while categorical variables used the modal value.

Outliers identified by the three-sigma rule were manually checked, and continuous features were standardised using training-set means and standard deviations. The records were divided in chronological rather than random order: 2019–2022 supplied about 12,540 training samples, 2023 supplied about 3,090 validation samples and 2024 supplied about 3,090 test samples. This split was intended to reproduce forecasting in which future conditions are predicted from historical observations and to reduce leakage from future information.

ST-CitrusFormer uses an encoder–decoder architecture with separate temporal and spatial attention streams. Each plot’s 14 raw variables are first projected into a 128-dimensional latent representation and combined with sinusoidal positional encoding so that temporal order remains available to the network. Three parallel one-dimensional convolutions, with kernel widths of three, five and seven days, extract short-, medium- and longer-window local patterns.

Their outputs are merged with a point-wise convolution, and a residual connection retains the shallow-layer signal before the representations enter the attention branches. The temporal branch applies eight-head self-attention independently to each plot’s history. Its sliding window allows a time step to attend only to nearby positions, reducing the stated computational complexity from quadratic in the full sequence length to linear in sequence length multiplied by window size.

The spatial branch represents the 12 sampling plots as nodes in a directed, weighted graph. Edge weights combine distance between plot centroids with management similarity derived from cultivar, terrain aspect and overlap in pesticide-application records. The distance bandwidth was fixed at the median pairwise plot distance, about 150 metres for this orchard, while the three management components were weighted equally because the authors had no prior evidence favouring one of them.

Multi-head graph attention aggregated information from the five nearest graph neighbours at each of four layers. A gated fusion module then combined the two streams. Its learned gate determined, element by element, how much information to retain from temporal and spatial attention.

Fused features passed through a two-layer Transformer decoder and a dual prediction head: a five-class softmax output represented severity level, while a linear output estimated occurrence intensity. To address the strongly imbalanced severity distribution, the classification component used Focal Loss, which reduces the contribution of abundant easy examples and increases attention to rare high-severity cases. Intensity regression used Huber Loss, with residuals up to 1.0 in normalised severity units penalised quadratically and larger residuals penalised linearly.

The combined objective weighted Focal and Huber losses at 0.6 and 0.4, respectively, and was optimised with AdamW, cosine learning-rate annealing and early stopping based on the 2023 validation set. On the 2024 test set, ST-CitrusFormer produced the best average classification F1-score across the four targets. Its score of 0.812 exceeded those of LSTM, GRU, TCN, Transformer, Informer, PatchTST and TimeXer by 7.9, 6.8, 5.7, 4.1, 3.5, 2.0 and 1.1 percentage points, respectively.

The model also recorded the lowest RMSE and MAE among evaluated baselines at one-, three- and seven-day horizons. At seven days, its RMSE was 11.6% lower than PatchTST’s and 5.2% lower than TimeXer’s. In a representative high-occurrence summer period, its seven-day trajectory followed the timing and magnitude of two observed peaks more closely than the comparison models, while LSTM and Informer showed visible lag and oscillation.

Across five random seeds and four tasks, Wilcoxon signed-rank tests found the improvements over every evaluated baseline statistically significant at p<0.05, with p<0.001 against the classical baselines. Ablation experiments indicated that several components made distinct contributions. Removing the spatial graph-attention branch caused the largest fall in average F1, a decline of 3.1 percentage points.

Removing gated fusion and Focal Loss reduced the score by 2.0 and 2.7 points, respectively, while removing the multi-scale convolutions had a smaller effect. The largest category-specific decline was for Huanglongbing, whose F1-score fell by 4.9 points without spatial attention. Attention visualisations showed temporal heads concentrating on intervals containing concentrated rainfall, abrupt humidity increases and sharp temperature changes, while within-block connections received greater spatial attention than cross-block connections.

The researchers interpreted these patterns cautiously: they are consistent with the meteorological sensitivity of red mite and aphid dynamics and with the vector-mediated spread of Huanglongbing, but attention weights alone do not establish a biological mechanism. Robustness analyses were encouraging within the orchard but limited in scope. Four management blocks—Slope–Navel, Flat–Navel, Slope–Wokan and Flat–Wokan—were each held out in turn, and 2022, 2023 and 2024 were each excluded in separate leave-one-year analyses.

Average F1 remained at or above 0.77 in all these settings, with maximum ranges of 2.1 percentage points across blocks and 1.2 points across years. These folds differed from the main chronological split, so their results are not directly comparable with the 0.812 headline score; the leave-one-year analysis also relaxed the restriction against using future observations and should not be interpreted as a leakage-free operational evaluation. The findings support stability among plots and years within the same site, not transfer to an independent orchard.

Computational cost was measurable but not negligible. ST-CitrusFormer contained 2.05 million parameters and required 2.1 milliseconds for single-sample inference on an Nvidia RTX 4090 under PyTorch 2.2, with peak GPU memory consumption of 2.7 gigabytes. It was heavier than recurrent and convolutional baselines but broadly comparable to Informer and TimeXer and used fewer floating-point operations than Informer.

No CPU-only or embedded-accelerator benchmark was conducted, so the study does not demonstrate operation on hardware such as a Raspberry Pi. Quantisation, structured pruning, knowledge distillation and device-specific profiling were identified as future requirements. The authors instead described lightweight edge sensors transmitting observations to a local gateway or cloud server as a more realistic near-term configuration.

Several limitations constrain how the results should be understood. The dataset came from one orchard with relatively homogeneous varieties, management and climate, leaving generalisation to pomelo, lemon, other citrus types, different agro-climatic zones and extreme-weather seasons unestablished. Neighbouring training and test plots may also retain spatial correlations that could make within-orchard estimates mildly optimistic.

The pipeline’s sensitivity to sensor failure, prolonged missingness and imperfect inputs was not systematically quantified, and the model did not incorporate imagery from unmanned aerial vehicles, thermal cameras or hyperspectral remote sensing. Training also requires labelled field surveys, while forecasts beyond seven days remain untested. MC Dropout and deep ensembles were evaluated for uncertainty, but calibrated uncertainty estimates were not yet integrated into operational decisions.

The dataset and code are not publicly available but may be obtained from the corresponding author on reasonable request. Within those constraints, the study shows that combining temporal attention with a geographically and managerially informed graph can improve one- to seven-day forecasts of citrus pest and disease occurrence within a finely monitored orchard. The model is best understood as a candidate plot-level forecasting component for an existing IoT-based management platform, complementing rather than replacing field inspection and specialist judgement.

Multi-site, multi-variety and multi-climate validation, spatially disjoint external testing, missing-data stress tests and explicit integration of uncertainty into decision workflows will be needed before large-scale deployment can be justified.

Subject of Research: Spatiotemporal Transformer modelling for seven-day citrus pest and disease forecasting within a single orchard

Article Title: A hybrid transformer with spatiotemporal attention for citrus pest and disease forecasting

Article References: Ji, C., & Xie, H. (2026). A hybrid transformer with spatiotemporal attention for citrus pest and disease forecasting. Discover Artificial Intelligence, 6(1), Article 1202. https://doi.org/10.1007/s44163-026-02274-0

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02274-0

Keywords: citrus forecasting, spatiotemporal modelling, hybrid Transformer, graph attention, temporal attention, precision agriculture, plant pathology, orchard sensors, multi-step forecasting, pest management, hybrid, transformer