Touchless gesture control in hospitals still falls short of the lab, review finds

Surgeons and clinicians may one day control computers without touching anything, but a new systematic review suggests that the technology is not ready for routine hospital use. The review, published in Artificial Intelligence Review, examined vision-based touchless gesture recognition systems intended for clinical human-computer interaction, where sterility and uninterrupted workflow are critical.

Following PRISMA guidelines, the authors analysed literature published between 2018 and 2026, focusing on systems that have been deployed in clinical settings or are close to deployment, rather than purely algorithmic benchmarks. The settings covered include operating rooms, intensive care units and rehabilitation environments.

The technologies assessed span three main sensing approaches: standard RGB cameras, RGB-D depth cameras, and skeletal-vision systems that track body joints. Recent deep learning methods applied to these inputs include convolutional neural networks, long short-term memory networks, transformers, and multimodal fusion models.

In laboratory conditions, these methods have been shown to achieve recognition accuracy of more than 95 percent. However, the review reports that real-world performance degrades by 1 to 23 percentage points once systems leave the lab.

The identified causes of this degradation are specific. Occlusion of hands or bodies, high-intensity surgical lighting in the range of 500 to 2000 lux, complex backgrounds, and hardware limitations all reduce accuracy in clinical environments compared with controlled test conditions.

Beyond raw accuracy, the review compares clinical scenarios and identifies major trade-offs between accuracy and latency, along with characteristic failure modes of operation.

To navigate these trade-offs, the authors propose a three-tier deployment model. Lightweight techniques would be considered for safety-critical commands only after task-specific validation. Hybrid CNN-LSTM models would handle regular interactions. Computationally costly models would be reserved for offline or experimental applications.

The review also identifies structural problems that go beyond computer vision. Dataset bias, ambiguity in how gestures are defined, and the lack of interoperability with proprietary electronic health record systems are described as the main issues standing in the way of practical deployment.

For future research, the authors point to domain adaptation, federated learning, and edge-optimised multimodal systems as promising directions.

The overall picture is that touchless gesture interfaces remain a promising sterile, non-contact option for clinical settings, but their practical deployment remains substantially unrealised, and the gap between laboratory benchmarks and real-world performance is the central challenge the field must close.

Subject of Research: Vision-based touchless gesture recognition for human-computer interaction in clinical settings

Article Title: Vision-based touchless gesture recognition for clinical HCI: deployment challenges, trade-offs, and future directions

Article References: Niloy, M. B. A., Islam, F., Zadid, T. H., Zihad, M. T., Likhon, M. R. I., Biswas, T., Jisha, S. S., Miah, M. S. U., Lamagna, C. Z., & Liew, T. H. (2026). Vision-based touchless gesture recognition for clinical HCI: deployment challenges, trade-offs, and future directions. Artificial Intelligence Review. https://doi.org/10.1007/s10462-026-11739-8

Image Credits: AI Generated

DOI: 10.1007/s10462-026-11739-8

Keywords: touchless gesture recognition, clinical HCI, systematic review, computer vision, deep learning, operating rooms, intensive care units, rehabilitation, accuracy-latency trade-off, edge computing, federated learning, healthcare technology