What makes a medical simulation feel real, and more importantly, what makes it actually train a surgeon well? A new study from a team of German researchers suggests that the answer depends on which of those two questions you are asking, and that confusing the two may lead training programs to invest in the wrong kind of technology. In research published in the International Journal of Computer Assisted Radiology and Surgery, Florian Heinrich, Marvin Kohpeiß, Christian Hansen, and Danny Schott report that the visual polish of a mixed reality environment and the physicality of the tools used within it contribute in distinctly different ways to how medical trainees perform and how they experience a precision-critical needle insertion task. The findings arrive at a moment when hospitals and medical schools worldwide are rapidly adopting virtual and mixed reality training platforms, often under the assumption that the more realistic the simulation looks, the better it will prepare clinicians for the operating room.
The research team, based at the University of Magdeburg, set out to disentangle two dimensions of simulation design that are frequently bundled together under the vague umbrella term of “realism.” The first is visual fidelity, or VF, which encompasses the geometric detail, textures, lighting, and overall richness of what the user sees. The second is interaction fidelity, or IF, which concerns how closely the way a user acts within the simulation corresponds to the way they would act in the real procedure, including the physical instruments they hold and the tactile feedback those instruments provide. Drawing on Bonfert and colleagues’ Interaction Fidelity Model, the authors treat fidelity as a multidimensional design property rather than a single slider of realism, distinguishing what users perceive visually from how users can act within the simulation. This conceptual separation allowed them to design a controlled experiment in which each dimension could be switched independently between high and low settings, something rarely done in prior work that has tended to focus almost exclusively on visual factors.
The task at the heart of the study was a guided needle insertion, a procedure representative of image-guided, minimally invasive interventions such as biopsies and percutaneous therapies. Rather than reproducing an entire clinical workflow, the researchers distilled the scenario into three sequential sub-tasks of escalating precision: moving an instrument table into the interventional workspace, disinfecting a laser-marked puncture site on a patient model, and finally inserting a needle toward a predefined anatomical target at a specified angle and depth. Only the third sub-task, the needle insertion itself, was used for performance analysis, since the first two serve primarily as low-precision warm-up activities that familiarize participants with the environment and the tools. A laser-based navigation aid, inspired by commercial systems such as ATLAS, indicated the puncture site and insertion angle, while a stopper on the needle shaft enforced the target depth across all conditions.
The experimental environment was a hybrid one, blending virtual content rendered through a head-mounted display with real, physically tracked objects. Participants wore a Valve Index head-mounted display fitted with an Ultraleap Leap Motion Controller 2 for markerless hand tracking, and tangible instruments, including the needle and the instrument table, were tracked in six degrees of freedom using HTC Vive Trackers 3.0, spatially registered to their virtual counterparts. The physical centerpiece of the setup was an MRI scanner mock-up that anchored the virtual interventional room. In the tangible interaction conditions, a candle gel phantom was aligned with the virtual patient anatomy so that participants could feel the puncture surface as they worked. In the virtual interaction conditions, by contrast, everything was done mid-air: tissue resistance was approximated with a simplified physics-based model, depth was detected via raycasting, and beyond four centimeters of insertion, progressive constraints gently pulled the needle tip back toward its original trajectory, a so-called rubber band effect that mimicked the growing resistance of deeper tissue layers. The simulation was built in Unity using the High Definition Render Pipeline, running on a workstation equipped with an AMD Ryzen 9 5950X processor and an NVIDIA GeForce RTX 3090 graphics card.
The high visual fidelity condition featured high-polygon models, realistic textures, dynamic lighting, and a fully furnished environment, while the low fidelity condition reduced everything to simplified geometry, monochrome surfaces, minimal shading, and an absence of props, deliberately evoking the look of a low-effort prototype. A fifth condition, a non-immersive baseline in which participants performed the same tasks on the physical setup without a head-mounted display, served as a real-world reference point. Thirty-one medical students, aged 19 to 33, completed the study, recruited exclusively from the human medicine program to guarantee basic familiarity with clinical environments. The sample size yielded a statistical power of 0.91 for detecting medium effects. Each participant experienced every condition in a counterbalanced order determined by a double Latin square design, completing four repetitions of the task per condition, with the first serving as training and the remaining three feeding into the analysis.
Performance was measured with clinical precision. Task completion time ran from the moment the needle was picked up to an acoustic stop signal, at which point the final needle pose was recorded. Accuracy was decomposed into positioning error, alignment error, and depth error, capturing distinct stages of the insertion process. Subjective experience was assessed with standardized instruments: the raw NASA Task Load Index for workload, the short User Experience Questionnaire for overall experience, and the igroup Presence Questionnaire for presence and its sub-scales, including spatial presence, involvement, and perceived realism. The study received ethical approval from the university ethics board and adhered to the Declaration of Helsinki.
The results tell a clear story with a twist. On the performance side, interaction fidelity was the star. Tangible interaction produced significantly shorter task completion times than virtual interaction, with a robust statistical effect, and dramatically lower needle alignment errors, one of the strongest effects in the entire study. Depth control also improved with tangible tools. Visual fidelity, by contrast, showed no significant effect on any of these performance metrics, and Bayesian analyses provided moderate evidence in favor of the null hypothesis, suggesting that the lack of visual effects was not merely a failure to detect a small signal. Intriguingly, the researchers found small but significant interaction effects between the two fidelity dimensions on both task completion time and alignment error: the performance advantage of tangible interaction was larger when the visual environment was sparse. In other words, realistic physical tools can partially compensate for a visually barren simulation, particularly when it comes to how quickly and how accurately trainees work.
On the subjective side, the picture reversed. High visual fidelity did exactly what one would expect: it significantly increased spatial presence, involvement, and perceived realism, the psychological ingredients of the sensation of being there. Tangible interaction also boosted perceived realism, but its most consistent subjective benefits were elsewhere, reducing perceived workload and enhancing user experience. Participants overwhelmingly preferred the combination of high visual fidelity and tangible tools, and their intuitions aligned with the data in one respect: 90 percent reported that the interaction modality had a noticeable impact on task performance, compared to only 68 percent who said the same about visual fidelity. The non-immersive baseline, analyzed descriptively, reminded readers that real physical practice remains a reference point that immersive systems have yet to fully surpass.
The authors are careful to frame these findings not as a rejection of visual realism but as a call for purposeful design. If the goal of a simulation is to sharpen procedural accuracy and efficiency in a precision-critical task, their data suggest that investment in tangible, tracked instruments and genuine haptic surfaces will pay off more than investment in photorealistic rendering. If, however, the goal is to create a convincing, engaging experience that increases immersion and involvement, as might be desirable for orientation training or patient education, then visual fidelity is the lever that matters. The interaction effects add a further nuance: tangible tools appear to offer their greatest marginal benefit precisely when visual resources are limited, hinting that resource-constrained training programs, perhaps in settings where high-end graphics hardware is unavailable, can still deliver effective procedural practice by prioritizing physical interaction over visual polish.
The study also addresses a persistent methodological problem in the field. Fidelity-related design choices in medical MR research are often poorly documented, making studies difficult to compare or replicate. By explicitly operationalizing visual and interaction fidelity, reporting their implementation details, and publishing the work open access, the team has provided a template for how future fidelity research might be conducted. The limitations are acknowledged as well: participants were medical students rather than practicing clinicians, the needle path was predefined and guided rather than freely planned, and a single registration drift issue forced the exclusion of positioning error from the analysis. Whether the findings generalize to unguided insertions, to experienced interventionalists, or to entirely different procedural domains remains an open question.
Still, the central message is likely to resonate far beyond this single experiment. As virtual and mixed reality flood into surgical education, procurement decisions are often driven by demo-day impressions of visual spectacle. This study provides some of the strongest controlled evidence yet that the hand, not just the eye, is the critical pathway to procedural skill, and that the sense of realism a trainee feels and the skills a trainee acquires are governed by separable design choices. For a field that has spent years chasing photorealism, the most realistic thing a simulation can offer may simply be something real to hold.
Subject of Research: People
Subject of Research: Medicine
Article Title: Mixed Reality Simulation of Guided Needle Insertion: Tangible Tools Beat Visual Polish for Precision Training
Article References: Heinrich, F., Kohpeiß, M., Hansen, C., & Schott, D. (2026). Mixed reality simulation of guided needle insertion: effects of interaction and visual fidelity. International Journal of Computer Assisted Radiology and Surgery. https://doi.org/10.1007/s11548-026-03756-3
Image Credits: AI Generated
DOI: 10.1007/s11548-026-03756-3
Keywords: Mixed reality, Medical training, Visual fidelity, Interaction fidelity, Needle insertion, Surgical simulation, User experience, Presence, Tangible interaction, Workload
Cite Scienmag News
APA MLA Chicago
Ophelia Keating. (September 9, 2026). Mixed reality needle insertion simulation: how interaction and visual fidelity matter. Scienmag. https://scienmag.com/mixed-reality-needle-insertion-simulation-how-interaction-and-visual-fidelity-matter/
Ophelia Keating. “Mixed reality needle insertion simulation: how interaction and visual fidelity matter.” Scienmag, 9 September 2026, https://scienmag.com/mixed-reality-needle-insertion-simulation-how-interaction-and-visual-fidelity-matter/. Accessed 9 September 2026.
Ophelia Keating. “Mixed reality needle insertion simulation: how interaction and visual fidelity matter.” Scienmag. September 9, 2026. https://scienmag.com/mixed-reality-needle-insertion-simulation-how-interaction-and-visual-fidelity-matter/
Copy citation Download RIS
Tags: design of effective surgical simulation environmentsenhancing precision in medical proceduresimmersive surgical trainingimpact of visual polish on medical trainingimpact of visual quality on surgical skill developmentmedical simulation realismmedical training simulation effectivenessMixed reality medical simulationmixed reality technology for surgeonsmixed reality training for surgeonsneedle insertion trainingneedle insertion training simulationphysicality of simulation toolsphysicality of surgical toolsrealism versus effectiveness in medical simulationsrealism versus interaction in medical trainingsimulation design for medical skill acquisitionsurgeon training technologyvirtual and augmented reality in surgeryvirtual and mixed reality in medical educationvirtual reality fidelity and performancevirtual reality in healthcare educationvisual fidelity in medical VRvisual fidelity in virtual reality
