Multimodal AI Reads Leaves Three Ways to Catch Hidden Citrus Greening Infection

Huanglongbing, one of the most destructive citrus diseases worldwide, may be detectable before it ever shows itself, according to a new study from researchers in China. The team built a multimodal deep learning framework that fuses ordinary RGB photographs, hyperspectral reflectance curves, and a handheld chlorophyll index to identify infection even when a leaf looks perfectly healthy. In field-collected samples, the model reached a test accuracy of 97.62 percent across five disease classes, and in the hardest task of all—separating healthy leaves from molecularly confirmed but symptomless infections—it still achieved 91.25 percent accuracy. The work addresses a stubborn problem in citrus greening control: by the time the disease shows itself in blotchy mottling and yellowed veins, infection has often already spread to neighboring plants, and no cure exists other than removing and destroying diseased trees.

The biology behind the challenge is unforgiving. External symptoms—blotchy mottling, chlorosis, leaf curling, deformation, premature leaf drop, and the misshapen ‘red-nose’ fruit—appear relatively late in infection. Crucially, the disease disrupts photosynthetic systems, leaf pigment content, and water status internally long before those disruptions become visible, producing subtle but characteristic spectral responses in specific wavelength regions. Because hyperspectral sensing is sensitive to these physiological variations, the Guangxi team reasoned that spectral data could catch the disease in its latent window, when intervention still prevents large-scale spread. Previous hyperspectral studies, however, have struggled with generalization across scenes and acquisition conditions, while RGB-based deep learning detectors depend on visible morphology that symptomless infection simply does not display.

To build a dataset worthy of the problem, the researchers sampled at two citrus practice bases in Lingchuan County, Guilin City, in subtropical Guangxi, working under the guidance of two citrus disease experts from the Guangxi Academy of Specialty Crops. Two cultivars, Mianju and Murcott, were collected in November 2024 and November 2025 respectively, spanning 52 trees. Field screening targeted both suspected infected and visually healthy plants, excluding leaves with mechanical damage, severe insect injury, or non-target diseases. A total of 2,700 individual leaves were gathered, and each leaf was photographed, spectrally scanned, and SPAD-measured in strict one-to-one correspondence, then linked by unique identification number to a molecular test result. Importantly, the prediction unit was defined as the individual leaf rather than the whole tree, preserving within-tree variation in symptom severity that tree-level labels would have averaged away.

Label reliability was anchored in molecular confirmation. Two experts first independently assigned candidate leaves to five preliminary classes—healthy, asymptomatic infection, mild, moderate, and severe—and samples where no consensus could be reached were excluded. Every leaf with complete multimodal data was then sent for TR-PCDR detection, a tandem repeat-based polymerase chain displacement reaction that served as the molecular basis for confirming infection status. The final five-class labels therefore combined TR-PCDR-based infection confirmation with expert phenotypic severity grading. Acquisition hardware was deliberately practical: RGB images came from an iPhone 13 front camera with leaves on white photographic cloth under standard laboratory lighting at 30 to 50 centimeters, while hyperspectral reflectance was recorded with a portable ASD FieldSpec 4 spectroradiometer covering 350 to 2500 nanometers at 1-nanometer intervals—2,151 bands in all, with ten spectra per leaf averaged to suppress noise.

Physiological status was captured nondestructively with a SPAD-502 chlorophyll meter, whose relative optical index from red and near-infrared leaf responses correlates positively with laboratory-extracted chlorophyll content. Three vein-free measurement points per leaf were averaged into a single SPAD index. Rigorous preprocessing guarded against the information leakage that inflates many agricultural AI results: because leaves from the same tree are biologically correlated, dataset partitioning was performed at the tree level in a 7:1:2 training-validation-test ratio, so no leaf from one tree ever appeared in different subsets. All distribution-dependent preprocessing—Monte Carlo partial least squares outlier screening for spectra and interquartile-range screening for SPAD values—was derived exclusively from training data after each split. Ten independent tree-level partitions were generated with different random seeds, and the same ten predefined splits were used for every compared model to ensure paired, fair evaluation.

Because one-dimensional spectral sequences lack explicit structural information, the team converted them into two-dimensional images using four encoding schemes: Gramian Angular Summation Field, Gramian Angular Difference Field, Markov Transition Field, and Recurrence Plot. GASF preserves the overall spectral trend through angular encoding, mapping normalized values into polar coordinates and constructing a matrix of angular relationships between any two bands while retaining native band ordering. Tested with an improved CoAtNet classifier, GASF achieved 91.43 percent accuracy, beating GADF at 89.47, RP at 88.78, and MTF at 80.43, and was selected as the spectral representation. RGB images in the training set alone received augmentation—random brightness, contrast, sharpness adjustments and flips—to simulate the illumination and angle variation of real orchards, while validation and test sets remained untouched to reflect genuine generalization performance.

The architecture itself is a trio of modality-specific networks joined by a novel fusion strategy. For RGB images, the improved SwinFPA algorithm modifies the Swin Transformer, whose shifted-window multi-head self-attention restricts computation to local windows for efficiency, then adds atrous spatial pyramid pooling before Stage 1 to aggregate multiscale context through parallel dilated convolutions, and a feature pyramid network to fuse multiresolution features via lateral connections. For GASF-encoded spectra, SSM-CoatCA modifies CoAtNet by replacing channel attention in MBConv blocks with coordinate attention, which aggregates along height and width directions to inject positional information, and by introducing a Global SSM Block that uses learnable state transition structures to propagate information through dynamic state evolution. A lightweight multilayer perceptron with LayerNorm and GELU activation encodes the single SPAD scalar into one token embedding dimensionally aligned with the other branches.

The heart of the system is the hierarchical progressive fusion module, THPF, which converts all modalities into token sequences—49 image tokens, 32 spectral tokens, and one SPAD token—and fuses them in three stages rather than by naive concatenation. Stage I couples the two biochemically consistent modalities: bidirectional cross-attention lets the physiological token query spectral information relevant to current physiological state while spectral tokens impose physiological constraints, and a sigmoid-gated mechanism dynamically reweights the two enhanced representations into 33 biochemical tokens. Stage II interacts these biochemical tokens with RGB tokens through further bidirectional cross-attention, self-attention refinement within each modality, and a softmax-based dynamic modality-weight allocation that adapts to disease stage. Stage III concatenates a learnable classification token and passes the 82-token sequence through stacked transformer layers, taking the CLS output as the global discriminative representation for the classifier head.

The controlled comparisons show that the fusion strategy, not just the extra data, drives the gains. Single-modality SwinFPA reached 87.64 percent accuracy on five classes but showed a strong positive correlation of 0.83 between healthy and asymptomatic features in Pearson analysis, with t-SNE visualization revealing overlap that RGB alone cannot resolve. SSM-CoatCA achieved 91.43 percent, with ablations confirming both coordinate attention and the SSM module are essential. SPAD-only classifiers topped out near 57 percent, confirming the physiological index works only as a complement. Late fusion of the three branches scored 91.26 percent, while direct feature concatenation actually fell to 80.42 percent—below the RGB model—because heterogeneous representations of differing dimensionality and semantic granularity conflict without explicit alignment. THPF, by contrast, hit 97.62 percent, and ablations showed removing Stage 2 RGB fusion caused the largest degradation, to 86.34 percent, while single-stage fusion, no gating, or mean pooling instead of CLS aggregation each cost several points.

Shapley-value analysis revealed the model’s internal logic: for healthy samples RGB and hyperspectral branches contributed similarly, but for asymptomatic and mildly infected leaves the spectral and SPAD contributions rose, reflecting their sensitivity to biochemical change when visible cues are absent, while RGB contribution climbed again in moderate and severe stages as symptoms emerged. In a three-class subset of healthy, asymptomatic, and mildly infected leaves, the trimodal model achieved 91.25 percent accuracy with recall of 91.7, 86.1, and 96.9 percent respectively, and the SPAD branch cut asymptomatic samples misclassified as healthy from 8.7 to 5.2 percent. Performance was consistent across the Mianju-2024 group at 96.79 percent and the Murcott-2025 group at 97.43 percent. The authors acknowledge limits: cultivar and year are confounded in the dataset, experiments were laboratory-controlled, and leaf-level predictions cannot directly grade an entire tree. Future work will pursue multi-leaf canopy sampling, additional physiological indicators such as water status, field validation, and integration with UAV remote sensing and lightweight edge-computing architectures for real-time orchard monitoring—steps toward turning a laboratory result into a deployable early-warning system for one of the world’s most destructive citrus diseases.

Subject of Research: Multimodal AI detection of asymptomatic citrus huanglongbing infection and severity grading using RGB, hyperspectral, and physiological leaf data

Article Title: Hierarchical multimodal fusion for detection of asymptomatic citrus huanglongbing infection and severity grading using RGB, hyperspectral, and physiological information

Article References: Tao, J., Dou, S., Du, J., Wu, G., Mei, Z., Song, Y., & Yan, J. (2026). Hierarchical multimodal fusion for detection of asymptomatic citrus huanglongbing infection and severity grading using RGB, hyperspectral, and physiological information. Smart Agricultural Technology, 15, Article 102642. https://doi.org/10.1016/j.atech.2026.102642

Image Credits: AI Generated

DOI: 10.1016/j.atech.2026.102642

Keywords: citrus huanglongbing, multimodal fusion, hyperspectral imaging, deep learning, asymptomatic infection, SPAD chlorophyll index, severity grading, TR-PCDR molecular confirmation, Swin Transformer, plant disease detection, precision agriculture, Guangxi citrus