Virtual Orchards: Synthetic Data Closes the Gap for AI Fruit Detection

Training an artificial intelligence to spot apples in an orchard sounds simple enough: point a camera at a tree, label the fruit, and let a neural network learn. In practice, it is one of the most stubborn bottlenecks in agricultural robotics. Collecting thousands of orchard photographs is laborious enough, but annotating every visible apple with a bounding box is worse, and the resulting datasets are often riddled with inconsistencies that quietly sabotage the models trained on them. A new study published in Smart Agricultural Technology argues that the solution may not lie in collecting more real images, but in manufacturing better ones inside a computer.

The research, conducted by Jhonny Hueller, Massimo Vecchio, and Fabio Antonelli, takes an unusually rigorous approach to a question that has mostly been answered piecemeal. Rather than demonstrating yet another synthetic-data pipeline for a single task, the team systematically benchmarked all ten publicly available apple-detection datasets, measured how well models trained on each one generalised to a new real-world test set, and then used those findings to iteratively refine a family of fully procedural synthetic datasets. The central question was not whether synthetic images can be generated at scale, but under what conditions they actually teach a detector something transferable to the real world.

The benchmark itself produced a striking result. When the researchers trained the same YOLOv26 Medium detector, with identical hyperparameters and a five-fold cross-validation scheme, on each public dataset and evaluated it on their own newly collected OpenIoT test set of 104 smartphone photographs containing 6,573 annotated apples, three datasets stood far above the rest: MinneApple, MetaFruit, and APPLE MOTS, with peak F1 scores around 0.58 to 0.64. Seven other collections languished far below, some with peak F1 scores under 0.35. Crucially, performance did not correlate with dataset size. Some of the largest collections performed worst, while a manual audit revealed that the failures traced back to low resolution, poor lighting, incomplete labels, duplicated imagery, and, perhaps most importantly, inconsistent annotation geometry, such as bounding boxes that were oversized or drawn from a limited set of predefined shapes.

That last finding shaped the entire synthetic-data strategy. The team built their virtual orchards in Blender, using an open-source procedural tree generator to create apple trees from random seeds and parameter ranges, and then rendered scenes in Unreal Engine 5, chosen for its photorealistic real-time rendering. Because every object in a virtual scene has known geometry, ground-truth annotations come for free and are pixel-perfect by construction. They produced three synthetic collections of 3,840 images each. Synthetic A captured broad scene diversity. Synthetic B was redesigned after analysing the best public datasets, aligning camera angles, optics, and fruit-generation parameters so that bounding-box sizes matched the statistical profile of MinneApple and MetaFruit. Synthetic C added finer geometric refinements, including aspect-ratio alignment and camera-pose regularisation.

The results of this iterative refinement were decisive. Models trained purely on Synthetic A already beat all seven lower-quality public datasets, but, tellingly, adding more synthetic images made performance worse rather than better, with peak F1 scores actually rising as the dataset was downsampled from 3,840 to 240 images. Synthetic B, with its aligned annotation geometry, delivered a consistent jump of roughly five to eight percentage points in F1 score and mean average precision across all size tiers. Synthetic C added a further two to three points, reaching a peak F1 of about 0.52. The team quantified the geometric alignment using the Wasserstein distance between bounding-box-area distributions and found a moderate negative correlation with detection performance, suggesting that how tightly annotations hug the fruit matters as much as how many images you render.

Even so, purely synthetic training never quite matched the best real data. The gap to MinneApple remained statistically significant in bootstrap comparisons over 10,000 resamples. The authors attribute the residual difference to the inherent limits of simulation: a finite library of tree and fruit models, simplified botanical structure, procedurally placed fruit that ignores real branch correlations, subtle rendering artefacts invisible to humans, and missing confounds such as haze, lens aberration, and sensor noise. Deep networks, it turns out, notice things our eyes do not.

The most practically important finding came when the researchers mixed real images into the synthetic training sets. With a conservative blend of one real image for every four synthetic ones, performance leapt dramatically. The best hybrid, built on Synthetic C, reached a peak F1 of 0.652, statistically indistinguishable from the model trained entirely on MinneApple. A systematic sweep of real-data fractions showed that the benefit follows a curve of diminishing returns: the first five to ten percent of real images accounts for most of the improvement, and below roughly five percent the loss relative to a fully mixed training set becomes statistically significant. In other words, a training corpus that is overwhelmingly synthetic, salted with a small handful of real photographs, can perform within a few F1 points of a fully real-data pipeline while eliminating the vast majority of annotation labour.

The study also probed a question that often goes unexamined: does dataset redundancy matter? Using perceptual hashing and feature embeddings from a frozen DINOv2 vision transformer to detect near-duplicate images, the team found that redundancy varied wildly across datasets, from essentially zero to nearly complete duplication in APPLE MOTS, where 97 percent of frames were near-identical at the structural level. Yet redundancy showed little correlation with final performance. High-quality and low-quality collections alike appeared at both extremes. Within the scope of fruit detection, at least, annotation geometry appears to matter far more than whether images repeat.

The authors are careful about the boundaries of their claims. The absolute performance figures come from a single test set of 104 images photographed in one orchard on one day with one smartphone, and every model tested was a YOLOv26 Medium detector. The real-data fraction at which the sim-to-real gap closes may be a property of this particular pipeline rather than a universal constant, and whether the ordering of training sets survives with transformer-based detectors or different cultivars remains untested. They suggest two complementary paths forward: embracing domain randomisation, which deliberately randomises textures, colours, and lighting rather than chasing photorealism, or pushing realism further with modern 3D reconstruction techniques such as Gaussian splatting and neural radiance fields.

What the study establishes, with unusual methodological care, is an ordering: carefully engineered synthetic datasets beat poorly curated real ones, refined annotation geometry narrows the gap to the best real data, and a small dose of reality finishes the job. For a field where every labelled apple costs human time and every harvesting robot must cope with changing light, weather, and growth stages, that is a genuinely useful recipe. The vision of machines that learn to farm inside a simulator before ever touching soil moved a measurable step closer.

Subject of Research: Synthetic dataset design for apple fruit detection with computer vision in agriculture

Article Title: Designing effective synthetic datasets for fruit detection in agriculture

Article References: Hueller, J., Vecchio, M., & Antonelli, F. (2026). Designing effective synthetic datasets for fruit detection in agriculture. Smart Agricultural Technology, 15, Article 102605. https://doi.org/10.1016/j.atech.2026.102605

Image Credits: AI Generated

DOI: 10.1016/j.atech.2026.102605

Keywords: synthetic data, apple detection, agricultural computer vision, YOLO, sim-to-real transfer, Unreal Engine 5, Blender, object detection, annotation geometry, dataset benchmarking, precision agriculture, domain randomisation