AI Scans Beat Human Modelers on Speed, but Not Yet on Quality, in Metaverse 3D Tests

Building a believable virtual world is one of the most stubborn bottlenecks on the road to the metaverse. Every realistic 3D scene—whether a living room, an office, or a conference hall—must be painstakingly assembled from meshes, the nets of vertices, edges, and faces that give digital objects their shape, and then wrapped in textures that mimic real materials. The more detailed a model is, the more convincing it looks, but the harder a computer has to work to render it in real time. That trade-off between fidelity and performance has long forced developers to choose between beauty and speed. Now, a team of researchers from the University of Salerno, the Universitat Politècnica de Catalunya, and Aalborg University Copenhagen has put two of the most hyped AI-powered shortcuts to the test, and the results offer a refreshingly sober picture of what automation can and cannot yet do.

The study, published in the journal Cluster Computing, set out to answer a deceptively simple question: can ordinary people with a smartphone and a free app produce 3D scenes good enough for the metaverse, without any of the specialized artistic and technical skills that professional modeling demands? To find out, the researchers reconstructed the same three real-world environments three times over—once by hand in the open-source modeling software Blender, and twice using mobile photogrammetry applications. The first app, Polycam, relies on a classical computer-vision technique called Multi-View Stereo, or MVS. The second, LumaAI, is built on Neural Radiance Fields, or NeRF, a deep-learning approach that has electrified the graphics community in recent years.

The two photogrammetry techniques could hardly be more different under the hood. MVS works like digital triangulation on steroids: it detects visual features in overlapping photographs, matches them across views to estimate where the camera stood for each shot, and then triangulates the matched points into a dense cloud from which a solid polygonal mesh is extracted. Textures are generated by projecting the original photos back onto that mesh. NeRF takes an entirely different route. Instead of building geometry directly, it trains a neural network to encode the scene itself, learning to map every point in three-dimensional space, viewed from any direction, to a color and a density. The result is a continuous volumetric model of how light behaves in the scene, which can produce startlingly photorealistic reconstructions of complex surfaces and lighting effects.

For the experiment, the team selected three environments chosen to reflect the kinds of spaces metaverse developers actually want to build: a domestic living room, a working office, and a social meeting hall. Each was filled with objects varying in size, shape, and surface properties, from coffee tables and plants to desks, documents, and stationery. A single experimenter with professional 3D modeling expertise carried out all reconstructions, first photographing each room for reference and then either modeling everything from scratch in Blender or scanning it with a smartphone. After a pilot run in a kitchen to fine-tune the procedure, the reconstructed assets were assembled into coherent scenes and exported to Unity3D, a widely used game engine, where each scene was explored in real time with a first-person camera for five minutes per trial.

The headline finding concerns time, and it is dramatic. On average, manually modeled scenes took roughly 7 hours and 25 minutes to complete, while the photogrammetry pipelines—fed by the same captured videos—needed only about 2 hours and 12 minutes. Automated reconstruction cut the modeling workload by more than two-thirds. Intriguingly, the time required for the automated approach barely varied with the complexity of the scene, because filming an object takes roughly the same effort whether it is ornate or plain, whereas manual modeling time scaled directly with the number of objects and their level of detail.

Speed, however, came at a cost measured in polygons. Manual modeling consistently produced the leanest meshes, ranging from just 8,991 triangles in the meeting hall to 126,307 in the office, reflecting the clean, deliberate topology a human modeler builds by manipulating vertices and faces from primitive shapes. LumaAI, true to its NeRF origins, generated extraordinarily dense meshes exceeding 2.8 million polygons in every scene and peaking above 4 million in the living room. Polycam landed in between, producing between roughly 74,000 and 534,000 polygons thanks to an internal optimization mechanism that trims mesh density while preserving essential detail. That middle ground matters enormously: polygon count is a direct proxy for the computational burden a scene places on a graphics card, and the metaverse lives or dies on real-time rendering.

The performance tests made that burden vivid. On an NVIDIA RTX 2080, manual and Polycam scenes held acceptable frame rates of around 55 to 58 frames per second, while LumaAI scenes collapsed to roughly 30 to 32 FPS with GPU usage climbing above 80 percent. The mid-tier RTX 3060 handled manual and Polycam content comfortably at 60 FPS with GPU loads between 32 and 39 percent, but strained under LumaAI models, which pushed GPU usage toward 80 percent and dropped frame rates to the mid-thirties. Only the flagship RTX 4090 rendered everything effortlessly, sustaining 115 to 120 FPS on the lighter scenes and still above 90 FPS even for the dense NeRF meshes. The lesson is stark: without serious post-processing, NeRF-derived assets remain effectively out of reach for VR headsets and mobile platforms, precisely the devices the metaverse is meant to serve.

Visual fidelity told a more nuanced story. The researchers compared rendered images of each virtual scene against photographs of the real rooms using the Structural Similarity Index, a metric that scores similarity on a scale from minus one to one. No single technique dominated. In the cluttered living room, Polycam achieved the highest average similarity at roughly 0.58, while LumaAI performed worst despite its geometric richness. In the office, Polycam and manual modeling were statistically comparable, and in the simpler meeting hall, the hand-built scene actually won, likely because its plain object shapes made manual modeling and texture mapping especially effective. The finding that LumaAI’s hyper-detailed meshes did not translate into superior visual similarity suggests weaknesses in texture generation or lighting reconstruction rather than geometry. A complementary measure, the Hausdorff distance, quantified how far the two automated tools diverged from each other object by object: differences were large for complex, occluded items like lamps, sofas, and curtains, but fell below 0.3—and sometimes below 0.1—for simple, regular shapes such as chairs and cabinets.

Perhaps the most practically valuable insights concern which objects suit which technique. Photogrammetry excelled at small, intricate items—plants, telephones, keyboards—whose compact size allowed full-angle coverage and whose rich textures gave the algorithms abundant visual cues. It stumbled on large objects that demand more physical space to scan, on reflective surfaces whose changing highlights confuse frame-to-frame matching, and, counterintuitively, on flat, featureless furniture like tables and wardrobes, where the lack of shading variation starves the reconstruction algorithms of information. A human modeler, by contrast, can mentally reconstruct occluded geometry, ignore misleading reflections, and assemble simple shapes in minutes from basic primitives. The choice between the two apps followed similar logic: LumaAI delivered more complete, higher-fidelity meshes and handled mixed-surface objects better, while Polycam proved more robust with glossy, reflective materials and produced lighter models friendlier to constrained hardware.

So can AI-powered scanning replace the 3D artist? The researchers’ answer is an unambiguous no, at least for now. Photogrammetry is a genuine accelerant and a democratizing force, opening scene creation to people without modeling expertise and slashing prototyping time, but it surrenders control over mesh topology, produces dense and sometimes flawed geometry, and struggles with entire categories of materials. Rather than a replacement, the authors argue, these tools should be treated as complements to manual craft—ideal for rapid prototyping and complex organic objects, but still dependent on human optimization for performance-critical, professional-grade assets. Looking ahead, the team points to lightweight NeRF variants and emerging techniques like Gaussian Splatting, automated post-processing pipelines for mesh simplification and artifact correction, and human-centered studies measuring perceived realism and immersion as the next frontiers. For a metaverse that promises millions of simultaneous users in lifelike worlds, the message of this study is both cautionary and constructive: the machines are getting fast, but the artists are not obsolete yet.

Subject of Research: Comparative evaluation of manual 3D modeling and AI-based photogrammetry techniques for metaverse scene generation

Article Title: Towards AI-powered automatic 3D scene generation for the metaverse: a comparative analysis of manual and photogrammetry techniques

Article References: Pentangelo, V., Di Dario, D., De Martino, V., Buono, M. D., & Lambiase, S. (2026). Towards AI-powered automatic 3D scene generation for the metaverse: a comparative analysis of manual and photogrammetry techniques. Cluster Computing, 29(12), Article 715. https://doi.org/10.1007/s10586-026-06530-x

Image Credits: AI Generated

DOI: 10.1007/s10586-026-06530-x

Keywords: metaverse, photogrammetry, 3D scene generation, NeRF, Multi-View Stereo, Polycam, LumaAI, real-time rendering, virtual environments, 3D modeling, GPU performance, artificial intelligence