Geometry-First Reconstruction and Mask Lifting Improve 3D Maize Kernel Counting Accuracy

Automating the measurement of kernel numbers across intact ears of maize has long posed a stubborn obstacle in digital plant phenotyping, where close physical contact between adjacent grains routinely confounds standard computer vision algorithms. Evaluated across a development cohort of nine harvested maize ears, their complete geometry-first pipeline achieved a mean absolute error of 3.00 kernels and a coefficient of determination of 0.999 across ground-truth counts ranging between 314 and 729 kernels per ear. This performance marked a 71.0 percent reduction in counting error compared to their preceding InvNeRF-Seg pipeline, which had recorded an average discrepancy of 10.33 kernels on the identical testbed. Rather than treating kernel counting as an unconstrained surface reconstruction task, the authors addressed the systematic tendency of neighboring visual boundaries to collapse into monolithic three-dimensional structures during multi-view neural synthesis.

Despite its fundamental importance, determining kernel counts at whole-ear scale has historically required laborious manual tallying or destructive grain detachment. Two-dimensional imaging configurations, including single-camera snapshots and panoramic unwrapping systems, inevitably suffer from severe perspective distortion near the ear tips and cannot directly register kernels occluded along the rear circumference of the cylindrical cob. Multi-view three-dimensional imaging theoretically overcomes these line-of-sight constraints by registering an entire intact ear across diverse camera viewpoints.

To establish the empirical benchmark, the researchers utilized nine mature Zea mays ears harvested at physiological maturity and stripped clean of protective outer husks. The nine specimens naturally exhibited considerable morphological diversity in total kernel population and packing arrangement. Multi-view image capture was performed indoors under uniform illumination using a handheld iPhone 13 Pro smartphone recording high-definition video at 1920 by 1080 pixels. The extracted video frames, sampled at four frames per second to yield between 90 and 190 sequential RGB exposures per ear, provided dense multi-angle visual coverage. Camera orientations and intrinsic optical parameters were subsequently solved through bundle adjustment within the open-source COLMAP structure-from-motion framework without the aid of external calibration markers or specialized active depth sensors.

A critical conceptual insight established in the study involves the nature of two-dimensional supervisory annotations and their propensity to induce downstream geometric fusion. Because only the outward-facing apex of each kernel is visible to an exterior camera, annotators cannot delineate hidden basal or lateral margins. The authors consequently annotated visible outward-facing surface patches across four multi-angle RGB frames selected from distinct ears using the AnyLabeling polygon tool. A U-Net segmentation architecture built upon an ImageNet-initialized ResNet-34 encoder was initialized from an earlier kernel checkpoint and trained on 60 synthetic augmentations per source image using combined binary cross-entropy and Dice losses. Although these predicted masks effectively captured exposed surface textures across remaining views, their broad planar footprints frequently touched or came within a few pixels of adjacent kernels. Projecting such broad, unconstrained annotations into three-dimensional space systematically established persistent volumetric bridges across adjacent instances, demonstrating that maximizing two-dimensional mask coverage actively harms three-dimensional instance separation.

To counteract this structural bridging effect, the researchers introduced an explicit morphological shrinking mechanism termed proportional mask erosion, transforming mask geometry into an active control parameter. Operating on the initial binary U-Net predictions, the algorithm calculated an L2 Euclidean distance transform smoothed by a three-by-three Gaussian filter. Local peaks separated by at least seven pixels and an absolute threshold of three pixels were identified as kernel centers, while spurious noise artifacts were pruned using image-space density-based spatial clustering of applications with noise, configured with an epsilon of 80 pixels and a minimum sample size of three. For each retained peak, a circular footprint was constructed using a radius defined by the product of a scaling factor and the local distance transform value, enforcing a hard minimum radius of two pixels. Across systematic sensitivity sweeps testing scaling factors of 0.2, 0.4, 0.6, 0.8, and 1.0, counting accuracy followed a pronounced non-monotonic trajectory: extreme reduction at 0.2 starved the pipeline of sufficient surface points (mean absolute error of 356.4 kernels), whereas an unpruned scaling factor of 1.0 reintroduced excessive inter-kernel contact (mean absolute error of 116.0 kernels), establishing 0.6 as the optimal operational setting.

The study demonstrated that the utility of these reduced proportional dot masks depends strictly upon the structural architecture chosen for semantic integration. Previous neural radiance field frameworks, including FruitNeRF and the authors’ prior InvNeRF-Seg implementation, incorporated semantic segmentation directly into volumetric optimization by replacing RGB reconstruction targets with mask values or appending semantic loss heads during scene training.

In the final geometry-first pipeline, scene geometry was reconstructed entirely from multi-view RGB imagery using SVRaster independently of kernel segmentation data. Trained for 20,000 iterations per individual ear within a normalized scene bounding scale of 0.85, the model rendered depth and volumetric transmittance maps that were integrated into a truncated signed distance field spanning voxel resolution levels from 8 to 10. Explicit surface meshes were extracted within a bounding box scale of 0.8 at distance thresholds below 0.5. Once three-dimensional ear geometry was definitively anchored, two-dimensional proportional masks were lifted onto the fixed surface points by projecting each vertex into available camera views, filtering through an alpha visibility threshold of 0.9, and selecting the mask value associated with the viewpoint presenting the minimum discrepancy between rendered depth and projected point distance. Because geometry optimization remained wholly uncoupled from semantic painting, alterations to mask morphology could not distort the reconstructed physical ear surface.

Following mask painting, the pipeline isolated kernel-specific coordinates by filtering points where every RGB color channel exceeded an intensity of 220, followed by a global cleaning clustering step with an epsilon of 0.08 and a minimum sample threshold of 10 to discard stray floating artifacts. To accommodate morphological variations along the ear axis, the retained point cloud was aligned via principal component analysis, trimmed at extreme axial percentiles, and partitioned into distinct head, center, and tail regions by identifying candidate cut planes that minimized axial point density. Thin inter-kernel connection bridges were severed by computing local point density from four-nearest-neighbor distances and discarding points falling below the tenth percentile in the center region and the first percentile at the tapered ends. Initial regional clustering was then executed using localized density thresholds, filtering residual dust components against dynamic size factors tied to regional median cluster sizes.

To resolve persistent clusters that resisted morphological erosion, the architecture incorporated a target-adaptive secondary splitting mechanism. Lifted components exceeding 2.40 times the regional median size in the center or 2.35 times at the extremities were flagged as oversized multi-kernel aggregates. Each oversized component was subjected to a secondary density-based split using tight radii of 0.0065 in the center or 0.0085 at the ends. Disabling this topological refinement stage caused the proportional-mask SVRaster error to jump from 3.00 to 14.67 kernels, confirming that pre-lifting mask erosion and post-lifting topological splitting act as indispensable, complementary mechanisms for severing complex volumetric bridges.

The experimental ablations systematically highlighted the failure modes associated with unconstrained semantic transfer. Direct projection of unscaled U-Net surface masks onto SVRaster geometry without topological split refinement produced a catastrophic mean absolute error of 80.22 kernels across the nine development ears. This aggregate discrepancy was heavily distorted by ear 4, which registered an extreme overcount of 649 predicted instances against 314 ground-truth kernels, while the remaining eight ears exhibited a collective mean absolute error of 48.4 kernels across errors that overall included three over-counts and six under-counts. In contrast, incorporating proportional mask reduction dropped the no-split error to 14.67 kernels, and introducing adaptive split refinement achieved final ear-level predictions within zero to six kernels of physical manual counts across all nine specimens, including a near-perfect prediction of 317 kernels for ear 4.

Notwithstanding these robust counting results, the authors emphasized several operational and statistical boundaries inherent to their experimental setup. Because global hyperparameters—such as the proportional scaling factor of 0.6 and fixed regional density percentiles—were selected empirically to minimize aggregate error across the nine development ears, the reported accuracy reflects a method-development benchmark rather than an unbiased generalization assessment on unseen biological cultivars. Furthermore, the SVRaster pipeline constitutes a scene-specific reconstruction framework requiring independent camera-pose estimation and dedicated neural optimization for every evaluated specimen; an optimized ear model cannot be directly transferred to classify new samples. The authors also noted that ear-level counting totals do not establish one-to-one spatial correspondence with individual physical kernels, leaving open the possibility of localized offsetting errors between over-fragmentation and residual merges. The public software repository releases full prediction tables and execution scripts, establishing a reproducible foundation for extending geometry-first instance separation across diverse agricultural phenotyping environments.

Subject of Research: Computer vision and 3D reconstruction for automated maize kernel counting

Article Title: Geometry-first mask lifting and topology-aware instance separation for 3D maize kernel counting

Article References: Zhao, J., & Cui, X. (2026). Geometry-first mask lifting and topology-aware instance separation for 3D maize kernel counting. Smart Agricultural Technology, 15, Article 102720. https://doi.org/10.1016/j.atech.2026.102720

Image Credits: AI Generated

DOI: 10.1016/j.atech.2026.102720

Keywords: maize kernel counting, 3D reconstruction, SVRaster, neural radiance fields, mask erosion, instance segmentation, U-Net, topology-aware clustering, agricultural phenotyping, InvNeRF-Seg, FruitNeRF, SA3D, DBSCAN, crop yield analysis, Zea mays, TSDF fusion, post-hoc mask lifting