Drone-built 3D point clouds let AI segment every maize plant without labels

Researchers have built an end-to-end pipeline that flies a consumer drone over a maize field, reconstructs the crop as a three-dimensional point cloud, and then splits that cloud into individual plants without any manual labeling. The key ingredient is a new unsupervised segmentation algorithm called SPGrowing, which exploits the way maize grows: every plant radiates outward from a central stem. In field trials spanning three varieties, six growth stages, and three planting spacings, the method reached an overall point-level accuracy of 0.950 across all stages, outperforming three existing unsupervised segmentation methods. The work, published in Artificial Intelligence in Agriculture, was carried out by Ziang Niu, Ting Huang, Chen Yang, Xinyue Sun, Yong He, and Zhengjun Qiu of Zhejiang University.

The motivation lies in a bottleneck that has long frustrated field phenotyping. Maize is one of the world’s most vital grain crops, and accurate measurement of plant-level traits such as height, canopy width, and canopy volume underpins breeding selection and precision farm management. Conventional phenotyping depends on manual measurement and subjective assessment, which is labor-intensive and hard to scale. Imaging from satellites, drones, and ground sensors has automated data collection, but standard two-dimensional imaging is inherently projective and planar, making it difficult to resolve overlapping leaves, intertwined stems, and organs distributed in three dimensions. That limitation obscures the topology of individual plants and restricts how far phenotypic data can be interpreted biologically.

Three-dimensional sensing addresses occlusion directly, but each existing technology carries trade-offs. Laboratory depth cameras and structured-light scanners deliver high precision yet cannot serve large field populations. Terrestrial laser scanning produces high-resolution field data but requires supplementary registration equipment and manual operation, which is time-consuming and risks soil compaction. UAV photogrammetry with oblique multi-view stereo has emerged as an efficient alternative, particularly using the cross-circling oblique flight route introduced by Xiao and colleagues, in which the drone circles the target at low altitude with tilted cameras. The new study applies that flight strategy to a densely planted maize experiment at the Agricultural Science and Technology Innovation Experiment Center of Zhejiang University in Hangzhou.

The experimental design was deliberately demanding. The team grew 231 maize plants across three architecturally distinct varieties: Zhengdan 958, a compact type with upright leaves; Suketian 1506, a semi-compact sweet maize with slightly curling leaf margins; and Sanbao Black Rose, a semi-compact waxy variety. Each variety occupied its own ridge, and within rows the researchers applied three plant spacings of 0.8, 0.6, and 0.4 meters, creating a gradient of canopy density. A DJI Air 3S drone with a 1-inch CMOS camera captured 12-megapixel images while circling the field at 5 meters altitude with a 45-degree camera tilt, following the cross-circling oblique route with up to 20 images per circle. Checkerboard calibration panels on the ground provided scale references for reconstruction. Flights occurred on six dates from May 6 to June 12, 2025, covering the V3, V5, V8, V12, R1, and R2 growth stages, and all 231 plants were tracked throughout with none lost or excluded.

Reconstruction and preprocessing turned raw images into a clean plant-only point cloud. The imagery was processed with structure from motion and multi-view stereo in Agisoft Metashape, and the resulting scene cloud was scaled against the checkerboards to correct drift from the drone’s onboard positioning. Preprocessing then unfolded in three steps. A cloth simulation filter classified ground and non-ground points by simulating a cloth draping over the inverted surface, with the cloth resolution set to 0.05 meters to match typical maize stem diameters so the cloth would not drape around individual stems. An HSV color filter trained with linear discriminant analysis separated green and yellow plant tissue from brown soil, applied only to points below 0.4 meters to avoid misclassifying upper plant parts such as ears. Finally, statistical outlier removal eliminated noise, with parameters tuned to the roughly 2-millimeter point spacing of the reconstructed cloud so that fine structures like leaf edges and narrow stems survived.

SPGrowing itself operates in three steps that mirror the plant’s own growth logic. First, stem cores are extracted by density clustering: a two-dimensional k-d tree indexes the horizontal projection of the cloud, kernel density estimation finds points whose neighborhoods are densely packed, and an iterative high-to-low density process groups components that pass linearity and size thresholds into stem cores. The approach rests on an ecological fact of maize stands: the erect growth habit and spatial competition among leaves ensure that no plant part other than the stem region exhibits comparable local density on the ground projection. Second, the raw cloud is abstracted into superpoints, semantically homogeneous patches such as a short stem segment or part of a leaf, using graph-based segmentation with an L0-cut pursuit algorithm. Each point contributes geometric features including linearity, planarity, scattering, and verticality derived from the eigenvalues of its local covariance matrix. Notably, even leaf-overlapping regions were divided into distinct superpoints rather than merged, and adherent superpoints from different plants retained distinct verticality values, laying the groundwork for separating them.

The third step assigns every remaining superpoint to a plant through multi-source shortest path optimization. Stem cores become seed superpoints via majority voting, and a multi-source Dijkstra algorithm progressively merges unlabeled superpoints into the nearest seed by minimizing an accumulated cost. The cost function combines four terms: lateral centroid distance, difference in verticality, directional deviation that keeps the aggregation path aligned with the natural leaf growth orientation, and boundary tightness between adjacent superpoints. Weighting parameters were chosen by grid search on a subset of 15 adjacent plants tracked across three growth stages, with lateral distance and directional deviation together accounting for 60 percent of the total cost. The aggregation sequence visibly recapitulates maize development: stem superpoints merge first, then leaf bases attach, and finally the algorithm extends outward along leaf orientation, resolving even adherent leaf regions where neighboring plants touch.

Performance was strong but not uniform, and the authors report the declines candidly. Overall accuracy peaked at 0.992 at the V3 stage and fell to 0.899 at R2, a drop of 8.4 percent from the V5 value, as severe leaf overlap, insufficient local topological cues, and false stem cores created by vertically dense foliage took their toll. Planting density mattered systematically: at the tightest 0.4-meter spacing, accuracy fell from 0.989 at V3 to 0.884 at V12, a relative decrease of 10.6 percent, while the 0.8-meter spacing held 0.948 at V12, 7.2 percent higher. Variety also mattered. The compact Zhengdan 958 achieved the highest accuracy, 0.944 at V12 and 0.917 at R2, while Sanbao Black Rose, with large leaf base angles and long leaves that invade neighboring plants’ growing space, dropped to 0.862 at R2, the lowest of the three. Against benchmarks, SPGrowing led throughout: at the R2 stage it maintained an AP50 of 0.924 while the Corn Seg and Nonrandom RAIN methods fell to 0.788 and 0.753 respectively, and across all stages it posted an mIoU of 0.898, about 3.9 percent above the second-best method, GD Quickshift++.

The segmentation translated into usable trait measurements. Plant height validated against manually annotated point clouds achieved a mean absolute error of 0.022 meters and a coefficient of determination of 0.985; against 300 manual field measurements taken with a telescopic ruler, the error rose to 0.068 meters, a gap the authors attribute to UAV reconstruction limits and systematic errors in manual measurement. Canopy width, evaluated only against the annotated point clouds, reached a mean absolute error of 0.059 meters with a coefficient of determination of 0.812, with underestimation of large canopies becoming more pronounced in later stages when the aggregation merged adjacent regions aggressively under dense canopy conditions. Population-level analysis revealed growth patterns with breeding relevance: plant height followed a sigmoidal trajectory, canopy width declined in the reproductive stage as nutrients shifted from leaves to grains, and canopy volume kept expanding through R1 and R2 even as horizontal spread stabilized, indicating ongoing three-dimensional architectural reorganization. At the R1 stage, Zhengdan 958 showed the highest height-to-width ratio near 2.0 and the lowest vertical structural weighted center around 0.6, consistent with evenly distributed foliage, while the two semi-compact varieties clustered foliage higher, near 0.7.

The authors are explicit about the limits. The experiments took place at a single site with a single drone platform under moderate planting densities, so performance in ultra-high-density commercial systems and transferability across locations, platforms, and more heterogeneous environments remain untested. The algorithm’s reliance on stem-centered growth may also limit generalization to crops with prostrate growth or multiple tillering, and future work points toward adaptive growth priors and weakly supervised strategies. Within those bounds, the study demonstrates that a consumer drone, oblique photogrammetry, and a growth-aware unsupervised algorithm can resolve individual maize plants in dense field canopies, delivering plant-level trait data at a granularity that canopy-averaged approaches cannot match, with direct value for breeding selection and field management.

Subject of Research: Unsupervised segmentation of individual maize plants from UAV-derived 3D point clouds for in-field phenotyping

Article Title: SPGrowing: an unsupervised individual plant segmentation pipeline from UAV-derived point clouds for high-throughput in-field maize phenotyping

Article References: Niu, Z., Huang, T., Yang, C., Sun, X., He, Y., & Qiu, Z. (2026). SPGrowing: an unsupervised individual plant segmentation pipeline from UAV-derived point clouds for high-throughput in-field maize phenotyping. Artificial Intelligence in Agriculture. https://doi.org/10.1016/j.aiia.2026.10.007

Image Credits: AI Generated

DOI: 10.1016/j.aiia.2026.10.007

Keywords: maize phenotyping, UAV photogrammetry, 3D point clouds, unsupervised segmentation, SPGrowing, plant height, canopy volume, high-throughput phenotyping, precision agriculture, breeding selection, superpoint graph, stem core extraction