Smartphone Neural Rendering Resolves Entangled Crop and Weed Architectures in 3D

Agricultural scientists evaluating multi-species crop systems have demonstrated that consumer-grade smartphone video processed through neural scene reconstruction can accurately quantify three-dimensional plant structures in heavily entangled, mixed-species canopies. In a controlled greenhouse study conducted at Texas A&M University, researchers used Neural Radiance Fields (NeRF) to extract detailed phytomorphological metrics from overlapping cotton plants and competitive weeds. The digital reconstructions yielded close statistical agreement with destructive physical measurements, achieving centimeter-level accuracy for individual leaf dimensions and whole-plant height. By capturing volumetric shoot occupancy and the surface area of washed root systems without specialized laser hardware, the workflow provides a practical framework to investigate competitive plant interactions that have historically resisted non-destructive measurement.

Understanding morphological dynamics in diverse plant communities is essential for optimizing sustainable intercropping systems and developing variable-rate weed management strategies that respond directly to weed size and architectural form. In mixed stands, aggressive weeds such as Palmer amaranth (Amaranthus palmeri) and barnyardgrass (Echinochloa crus-galli) compete vigorously against crops like upland cotton (Gossypium hirsutum) for critical above-ground light and subterranean resources. However, tracking these competitive interactions over time has remained a formidable challenge in agronomic research. Traditional manual phenotyping requires harvesting and physical dissection, making continuous temporal tracking impossible, while conventional two-dimensional imaging pipelines systematically fail when upper leaves obscure understory vegetation and overlapping species become physically entangled.

While plant phenomics has increasingly adopted three-dimensional tools to overcome the occlusion inherent to flat imaging, existing methods present operational trade-offs in dense canopies. High-resolution Light Detection and Ranging (LiDAR) systems require expensive instrumentation, lack inherent color data, and suffer from shadowing when outer foliage blocks laser pulses. Structure from Motion (SfM) photogrammetry provides a lower-cost alternative but relies on matching distinct geometric pixels across views, frequently producing noisy, incomplete point clouds when attempting to resolve thin, repetitive structures such as narrow grass tillers or overlapping broadleaf foliage. To bypass these limitations, the research team applied neural rendering frameworks—principally NeRF and 3D Gaussian Splatting (3DGS)—which synthesize continuous volumetric scene representations from multi-view photographs by optimizing deep representations rather than relying strictly on feature matching.

The experimental trials were conducted between late summer and fall 2024 at the Borlaug Center for Southern Crop Improvement Greenhouse Facility in College Station, Texas. Using a target-neighbor layout arranged in a completely randomized design, the researchers cultivated plants in 11.3-liter containers across 18 distinct competition treatments, varying species combinations and planting densities. Multi-view datasets were captured across three vegetative and reproductive growth stages—at 47, 87, and 129 days after transplanting—using a handheld Google Pixel 6a smartphone. Smooth freehand camera sweeps around grouped pots yielded approximately 350 frames per scene, which were downsampled into multi-scale image pyramids to accelerate ray-marching convergence and preserve fine morphological margins before camera poses were calculated with structure-from-motion software.

Across 80 collected video sequences of canopies and washed root systems, 72 were successfully reconstructed into three-dimensional models. Comparative benchmarking between NeRF (nerfacto) and 3D Gaussian Splatting (splatfacto) showed that 3DGS achieved superior image-space rendering metrics, delivering a mean peak signal-to-noise ratio of 28.61 compared to 22.55 for NeRF, a higher structural similarity index of 0.877 versus 0.627, and faster rendering speeds of 182 frames per second against 2.19. However, when evaluating the exported point clouds for downstream biological trait extraction, qualitative inspection revealed that NeRF generated more contiguous representations along leaf blades, stems, and grass tillers, with fewer floating primitive artifacts. Consequently, the researchers selected NeRF as the primary modeling engine, combining it with an automated green-reference scaling algorithm to resolve metric scale ambiguity before filtering noise with statistical outlier removal.

Validation against physical ground-truth measurements obtained through destructive sampling confirmed the precision of the digitally extracted leaf parameters. Across 128 individually segmented leaves evaluated at the early reproductive stage, digital measurements matched readings from a portable laser leaf area meter with a coefficient of determination (R²) of 0.908 and a root mean square error (RMSE) of 0.0014 square meters. Similar precision was recorded for leaf length (R² = 0.938, RMSE = 0.0208 meters) and leaf width (R² = 0.950, RMSE = 0.0112 meters). An analysis of variance confirmed that the digital models successfully distinguished species-level foliar differences, showing that cotton exhibited significantly larger mean leaf areas than either Palmer amaranth or barnyardgrass, whose individual leaf dimensions did not differ significantly from one another.

The neural reconstruction framework demonstrated substantial advantages over traditional flat imaging when quantifying vertical plant height. While standard two-dimensional planar measurements exhibited severe error at the late reproductive stage due to stem lodging, leaf occlusion, and perspective distortion (R² = 0.131, RMSE = 0.454 meters), heights measured directly on segmented three-dimensional point clouds agreed closely with manual yardstick recordings across all stages (R² = 0.988, RMSE = 0.022 meters). Using these precise height metrics, the researchers resolved clear vertical stratification within the entangled canopies: Palmer amaranth emerged as the significantly tallest competitor, followed by barnyardgrass, with cotton remaining the shortest component of the canopy mixture.

At the whole-plant level, three-dimensional spatial traits served as effective non-destructive indicators of above-ground dry biomass. Vertical plant height showed the strongest linear correlation with total dry mass (Pearson r = 0.785), while a uniform-subsampling occupancy index (Vocc) designed to capture spatial extent yielded strong positive correlations at both late reproductive (r = 0.683) and early vegetative (r = 0.641) stages, outperforming two-dimensional canopy footprints. In subterranean evaluations of washed root networks, rasterized two-and-a-half-dimensional root surface area and volumetric occupancy indices were strongly associated with root dry weight (r = 0.803 and r = 0.755, respectively). These integrated profiles revealed that Palmer amaranth combined the highest absolute height with the greatest subterranean root surface area, illustrating how multi-dimensional structural traits track the competitive suppression of cotton.

The researchers also investigated non-destructive physiological assessment by evaluating point-cloud color distributions against leaf SPAD chlorophyll measurements. During early vegetative growth, digital green-channel mode values correlated strongly with SPAD readings (r = −0.834, R² = 0.695), successfully separating lighter-green barnyardgrass foliage from the darker broadleaf species without destructive sampling. However, predictive accuracy declined markedly by the reproductive stage (R² = 0.180), as dense canopy development caused complex internal shadowing and uncalibrated exposure variations degraded color fidelity. The authors noted that future iterations must incorporate in-scene radiometric calibration charts to maintain consistent physiological tracking under varying lighting conditions.

Although neural scene reconstruction substantially democratizes high-resolution plant phenomics by relying on standard consumer mobile hardware, several operational bottlenecks remain before field deployment is practical. Manual species and organ segmentation within point-cloud software currently represents the primary throughput constraint, requiring up to 15 minutes of operator effort per multi-plant scene. Furthermore, camera registration failures in approximately ten percent of captures highlight the ongoing challenge of feature matching across repetitive, weakly textured foliage. Translating the workflow from wind-free greenhouses to open agricultural fields will ultimately require robust dynamic motion modeling, automated semantic segmentation, and on-device training architectures to enable real-time spatial monitoring of multi-species crops.

To execute the computationally intensive reconstruction pipeline, the researchers deployed a standardized desktop workstation configured with an eight-core AMD Ryzen 7 9700X processor, 64 gigabytes of system memory, and a 16-gigabyte NVIDIA GeForce RTX 5070 Ti graphics processing unit. Accommodating this Blackwell architecture required constructing an isolated software environment under Ubuntu 24.04, deploying a nightly build of PyTorch configured with CUDA 12.8, and compiling specialized libraries such as tiny-cuda-nn and gsplat directly from source. Across each two-pot scene comprising roughly 350 base frames, the automated processing stages ran efficiently without manual intervention. Multi-scale image pyramid generation required 21 to 35 seconds, camera pose estimation through COLMAP finished in under five minutes, and model convergence over 30,000 training iterations completed in under 15 minutes. Scripted point-cloud export, metric scale calibration, and statistical outlier filtering subsequently required under two minutes.

The point-cloud extraction protocol generated 10 million coordinate points per scene positioned at expected ray-termination depths along training camera rays, capturing exterior plant surfaces rather than solid internal volumes. To resolve scale ambiguity, the team segmented an in-scene 0.91-meter green tape using color-space thresholds and principal component analysis to compute a global scaling factor, followed by a double-pass statistical outlier removal filter assessing 20 nearest neighbors. Morphological trait extraction then distinguished two volumetric descriptors: a rasterized height-field approach that projected points onto a one-millimeter planar grid to calculate outer surface area and total spatial displacement beneath the canopy surface, and a three-dimensional spatial occupancy index derived from uniform minimum-distance subsampling. Because the height-field method necessarily enclosed empty intra-canopy voids, the subsampled occupancy index provided a closer relative approximation of reconstructed biological surfaces without counting internal empty space.

Downstream morphological parsing also revealed distinct architectural patterns across the species. Manual segmentation divided cotton and Palmer amaranth point clouds into separate leaf, stem, and seed components, while barnyardgrass was partitioned into foliage and seeds. Additionally, statistical comparisons of washed roots demonstrated that Palmer amaranth produced significantly greater surface area than both cotton and barnyardgrass, which did not differ from each other.

Subject of Research: Three-dimensional neural scene reconstruction for plant phenotyping in mixed-species crop systems

Article Title: Neural scene reconstruction and 3D spatial analysis for high-throughput phytomorphology of shoot and root structures in complex mixed-species systems

Article References: Johnson, J., Gyawali, P., & Bagavathiannan, M. (2026). Neural scene reconstruction and 3D spatial analysis for high-throughput phytomorphology of shoot and root structures in complex mixed-species systems. Smart Agricultural Technology, 15, Article 102669. https://doi.org/10.1016/j.atech.2026.102669

Image Credits: AI Generated

DOI: 10.1016/j.atech.2026.102669

Keywords: Neural scene reconstruction, Neural Radiance Fields, 3D Gaussian Splatting, High-throughput phenotyping, Plant phytomorphology, Crop-weed competition, Cotton, Palmer amaranth, Barnyardgrass, Point cloud segmentation, Root phenotyping, Volumetric occupancy, Scale calibration, Precision agriculture, Non-destructive phenotyping, Digital colorimetry, Sustainable agriculture