Scoping review finds markerless surgical tool tracking in open surgery still at proof-of-concept stage

Tracking a surgeon’s instruments without attaching any sensors or markers to them has long been a solved-looking problem in minimally invasive surgery, where the endoscope hands computer vision systems a stable, well-lit view of the tools. Open surgery is a different story. A new scoping review published in Machine Learning with Applications has, for the first time, systematically mapped the research on vision-based markerless tool tracking in open surgical procedures, and its conclusion is sobering: after nearly a decade of work, only 14 studies meeting strict evaluation criteria could be found, and the field remains dominated by simulated environments rather than real operating rooms.

The review, conducted according to the PRISMA-ScR guidelines, began with 2,626 records identified through five databases, including PubMed, Web of Science, IEEE Xplore, Scopus and the ACM Digital Library. After duplicate removal and screening, 407 articles underwent full-text assessment, and just 14 studies published between 2016 and 2026 satisfied the eligibility criteria. To qualify, studies had to report markerless tracking or pose estimation of surgical instruments during open procedures, including open microsurgery, and had to include quantitative tracking outcomes such as accuracy, precision or localization error. Studies limited to minimally invasive surgery, hand-only tracking, marker-based or sensor-based systems, or purely virtual reality environments were excluded.

The reason open surgery poses such a challenge for computer vision comes down to physics and anatomy. In minimally invasive procedures, the endoscopic camera provides a constrained scene where instruments remain consistently visible. In open surgery, instruments are frequently occluded by the surgeon’s hands, assistants, tissue or surrounding equipment, and the surgeon enjoys a substantially higher degree of freedom in instrument manipulation, producing complex motion patterns and rapidly changing viewpoints. The scarcity of publicly available annotated datasets for open surgical procedures compounds the problem, which the review’s authors identify as a central bottleneck for the field.

Despite these obstacles, the 14 included studies spanned a surprising range of specialties: cardiothoracic surgery, neurosurgery, spinal surgery, ophthalmology, orthopedic surgery and otolaryngology. Reported applications included vein harvesting during coronary artery bypass grafting, microvascular anastomosis, spinal surgery, cataract surgery, bone drilling and mastoidectomy. Surgical drills were the most frequently tracked instruments, appearing in six studies, followed by tweezers, needle holders, ophthalmic needles and screwdrivers. Most studies focused either on training and skill assessment or on infrastructure development and benchmarking, with only two studies aiming at intraoperative surgical navigation.

The technical landscape is heavily tilted toward deep learning. Twelve of the 14 studies used deep learning approaches, one used classical computer vision, and one combined the two. The deep learning pipelines split into two broad families: eight studies produced 3D pose estimation, recovering the full six-degree-of-freedom position and orientation of an instrument, while six generated 2D motion tracking, linking instrument tip or centroid locations across consecutive video frames to produce image-space trajectories. Correspondence-based networks that predict keypoints or dense surface correspondences and then solve for pose geometrically proved popular, as did direct regression models and architectures combining instance segmentation with pose regression.

Performance varied enormously depending on the imaging setup. Multi-view pipelines reported the lowest 3D pose errors: in one cadaveric navigation study using calibrated RGB-D cameras, positional errors of about 1 to 2.8 millimeters and orientation errors of 0.89 to 3.33 degrees were achieved for a drill and screwdriver in the best five-camera configuration. Another multi-view RGB pipeline with recurrent temporal modeling reported sub-millimeter positional accuracy and angular errors well below one degree in controlled settings. By contrast, monocular 3D pose pipelines without explicit depth cues showed far more variable results, with one study reporting positional errors of 52.5 millimeters on simulated data and 106.2 millimeters on real data, alongside detection rates of 99.3 percent and 86 percent respectively.

On the 2D side, deep learning detectors applied to microscope video achieved mean average precision values near 0.99 at an intersection-over-union threshold of 0.5 for primary instruments in one mastoidectomy study, while a YOLOv2-based tip detector in a microvascular anastomosis study reached only moderate detection performance with mean average precision around 0.63 to 0.65, though instrument regions achieved a mean Dice coefficient of about 0.87. A recurring pattern emerged across the 2D pipelines: even when overall segmentation quality appeared high, precise localization of small, fine-grained landmarks such as tool tips remained substantially more difficult, and performance dropped consistently for instruments that were small, thin, or occupying only a few pixels in the frame.

Perhaps the most striking finding concerns data. Simulated datasets, including phantom-based environments, cadaveric procedures, rendered virtual scenes and synthetic imagery, were used in ten of the 14 studies, reflecting the practical and ethical challenges of collecting and annotating large-scale open surgical video. Real clinical datasets were limited primarily to microscope-guided procedures such as cataract surgery and mastoidectomy, where the surgical microscope conveniently provides a stable, magnified video feed. Only one study explicitly employed a hybrid strategy combining synthetic and real data to bridge the domain gap. The review’s authors note that obtaining accurate 3D ground truth during routine surgery is so difficult that many pipelines rely on synthetic renders, structured phantoms, or simplified tasks, where ground truth can be generated automatically from a rendering engine.

The review also uncovered a striking algorithmic gap: tool tracking is inherently a temporal problem, yet ten of the 14 pipelines estimate tool state independently in each frame, with no learned temporal modeling at all. Only one study incorporated recurrent units to propagate pose-related features across multiple views and time steps, and it was also the only one to quantify frame-to-frame temporal stability directly, showing that its temporal component improved robustness under partial occlusion. In the remaining 13 pipelines, long-range temporal dependencies, recovery after occlusion, and uncertainty accumulation are handled heuristically or not at all, rather than being modeled directly.

Comparing the studies proved nearly impossible, and the review is candid about why. Evaluation metrics were heterogeneous, with each research group selecting its own combination of 3D pose errors, 2D detection scores, segmentation metrics, or simple distance measures, often without clear justification for how these choices relate to clinically meaningful tolerances. Test sets ranged from individual sequences to thousands of images, thresholds such as IoU of 0.4 or 0.5 and pixel-distance cutoffs of 10 or 20 pixels were used inconsistently, and no common benchmark dataset, metric suite, or reporting standard exists for open surgery, in contrast to minimally invasive surgery where shared public datasets have supported common evaluation protocols. The authors conclude that current technologies should be viewed as demonstrations of proof of concept rather than robust solutions ready for adoption in operating rooms, and they call for shared benchmark datasets, task-derived accuracy targets defined jointly with clinical end users, and standardized reporting practices that include cross-dataset validation, explicit failure-mode analysis, and temporal robustness evaluation alongside average frame-wise accuracy.

Subject of Research: Vision-based markerless surgical tool tracking in open surgery

Article Title: Vision-based markerless surgical tool tracking in open surgery: A scoping review

Article References: Hamza, H., Naim, R., Shabir, D., Shaban, K., Abinahed, J., Deng, Z., & Navkar, N. V. (2026). Vision-based markerless surgical tool tracking in open surgery: A scoping review. Machine Learning with Applications, 26, Article 101050. https://doi.org/10.1016/j.mlwa.2026.101050

Image Credits: AI Generated

DOI: 10.1016/j.mlwa.2026.101050

Keywords: markerless tracking, open surgery, surgical tool tracking, pose estimation, computer vision, deep learning, scoping review, surgical navigation, skill assessment, 6DoF pose, surgical datasets, PRISMA-ScR