Surgical AI Learns to Adapt When Smoke and Darkness Blur the Robot’s View

Robot-assisted surgery depends on a camera that never blinks, but the images it returns are anything but stable. During a minimally invasive procedure, smoke from energy devices drifts across the field, blood pools over tissue, illumination dims, and defocus blur softens instrument edges. A deep learning model trained to segment surgical instruments can perform beautifully on clean validation frames and then fall apart mid-procedure, because the incoming image distribution has quietly drifted away from everything it was trained on. A research team at Chengdu University of Information Technology has now mapped exactly when online adaptation helps such a model, when it actively harms it, and which mechanism actually does the heavy lifting, in a study published in Frontiers in Robotics and AI.

The team, led by Shiqi Jiang, built a framework called Lightweight Spatiotemporal Test-Time Adaptation, or LiteST-TTA. The core idea is test-time adaptation: instead of freezing a trained segmentation network during inference, the model adjusts itself on the fly using unlabeled surgical video, without ever seeing ground-truth labels or source-domain training images. The source network combines a MobileNetV3 encoder with an LR-ASPP segmentation head, a deliberately lightweight architecture suited to real-time endoscopic processing. For each incoming frame, the model first produces an instrument foreground probability map, then quantifies its own uncertainty pixel by pixel using normalized binary predictive entropy, which rises as a predicted probability approaches the ambiguous value of 0.5.

Crucially, LiteST-TTA does not update on every frame. Because surgical instruments occupy only a fraction of the image, a simple average of pixel entropies would drown out localized uncertainty around instrument boundaries and degraded regions with a sea of confidently predicted background. The framework instead aggregates the 20 percent highest-entropy pixels into a frame-level uncertainty score and triggers adaptation only when this score exceeds a threshold of 0.08. When a frame does trigger an update, the system exploits the temporal structure of video: Farneback dense optical flow is estimated between the current and previous frames, and the stored historical prediction is warped into the current coordinate system so that normal instrument and camera motion is not mistaken for a segmentation error.

Even a motion-aligned historical prediction can carry errors forward, so LiteST-TTA constructs supervision only where both the current prediction and the warped historical prediction are confident, filtering by entropy thresholds and a confidence margin around the 0.5 decision boundary. A photometric reliability term further downweights pixels with large RGB residuals between the aligned frames. Within this reliable region, the historical probability serves as a soft temporal target, and the final adaptation objective blends current-frame entropy minimization with cross-frame temporal consistency, weighted at 0.7 and 0.5 respectively. Gradient updates touch only the affine scale and bias parameters of Batch Normalization layers, roughly 0.77 percent of the network’s 3.2 million parameters, while convolutional weights remain frozen.

The evaluation protocol is a controlled stress test of unusual rigor. Using the EndoVis 2017 robotic instrument segmentation dataset, the team generated twelve degradation conditions from the same 901 validation frames: blood interference, imaging blur, low illumination, and smoke, each at three severity levels with explicitly reported digital generation parameters and a deterministic per-frame seeding scheme so that all five compared methods faced identical degraded files. The methods spanned the fixed Source model, a Batch Normalization statistics baseline, the continual entropy-minimization method TENT, a selective entropy variant with the same uncertainty gate, and the full LiteST-TTA pipeline.

The headline numbers are strikingly close. TENT achieved the highest macro-average Dice similarity coefficient of 0.8881 across the twelve conditions, with LiteST-TTA at 0.8879, while the fixed Source model trailed at 0.8533. But the condition-wise breakdown reveals a far more interesting story. Under severe low illumination, the Source model collapsed to a DSC of 0.369, essentially failing to segment the instruments, while LiteST-TTA recovered to 0.872, an absolute improvement of more than half a Dice point. Under strong blur, a smaller but real benefit emerged, with adaptation overtaking the Source model only at the highest severity. Under blood interference and smoke, however, the pattern reversed: the Source model remained superior at every severity level, and the gap widened as degradation intensified, with adaptation differences under smoke growing increasingly negative from minus 0.017 to minus 0.027.

Sequence-level repeated-measures analysis confirmed that this is not visual coincidence. When the ten video sequences were treated as repeated-measure subjects, the degradation-type-by-severity interaction was statistically significant for absolute DSC and overwhelmingly strong for the adaptation gain relative to Source, with corrected p values below 0.001 and partial eta squared near 0.89. In plain terms, the benefit of online adaptation is jointly conditioned by what kind of visual change is occurring and how severe it is, not by degradation presence alone. A model that still handles smoke or blood well can be damaged by unnecessary parameter updates, because unsupervised entropy minimization can push predictions toward greater confidence without guaranteeing they move toward the truth.

The team then ran mechanism controls that substantially qualify their own results. In the original configurations, LiteST-TTA beat Selective Entropy in all twelve cells, but the two methods evolved wildly different update rates, 81 percent versus 23 percent, so the difference could not be attributed to the temporal term. When both methods were forced to update exactly the same frames on a fixed periodic schedule, their macro-average DSC values became virtually identical: 0.886967 versus 0.886966, with a sequence-level test giving p equals 0.496 and an effect size effectively zero. The apparent advantage of the temporal machinery was an artifact of the update trajectory, not a standalone causal gain.

A Batch Normalization ablation proved even more revealing. Simply placing the model in training mode during testing, so that running statistics update from the incoming test stream without any gradient optimization, raised the macro-average DSC to 0.8864, nearly matching the full pipeline. Optimizing only the affine parameters with frozen source statistics fell to 0.8473, collapsing under the strongest low-light shift. The joint configuration reached 0.8879, a modest increment of 0.0014 over statistics-only adaptation. Test-time normalization statistics, not entropy minimization or temporal constraints, account for most of the aggregate recovery. Meanwhile, the cost is real: LiteST-TTA’s optical-flow estimation, temporal supervision construction, and second forward pass pushed latency to 70.70 milliseconds per frame, roughly six times the 11.07 milliseconds of the statistics baseline, for that 0.145 percentage-point gain.

The authors also confronted a subtle evaluation problem head-on. In the original protocol, the source checkpoint was selected by maximizing DSC on the same clean validation content later used to generate the degraded conditions, a model-selection overlap that could favor the Source baseline. They therefore retrained once, locked a sequence-disjoint split before any scoring, selected a checkpoint using only five complete video sequences, and evaluated all methods on five entirely held-out sequences. The dominant low-light pattern persisted: under severe low illumination, LiteST-TTA reached 0.8083 against 0.3251 for Source, while Source remained superior in ten of the twelve held-out conditions. The central conclusion survives the stricter design, though the authors caution that a single fixed split is a sensitivity check, not external validation.

The broader lesson reaches beyond one segmentation benchmark. Predictive uncertainty, the study shows, indicates how confident a model is about its current output but cannot determine whether an update will improve it; under smoke, uncertainty and update frequency stayed high as severity rose while performance continued to decline. The authors argue that future surgical perception systems should separate the question of whether a prediction is unstable from whether adaptation is likely to help, using adaptation-effectiveness estimates, historical update outcomes, and short-term temporal stability. They envision an adapt-or-abstain architecture for robotic deployment: when perception remains reliable, assistance continues; when adaptation fails or the model state drifts from a safe range, the system freezes parameters, restores a stable state, reduces automation, or requests human confirmation. For intelligent surgical robotics, the ability to detect one’s own adaptation failure may matter more than chasing incremental gains in a single metric.

Subject of Research: Test-time adaptation for surgical instrument segmentation under controlled visual degradations

Article Title: Test-time adaptation for surgical instrument segmentation under visual degradations: selective updating and temporal constraints

Article References: Jiang, S., Peng, H., Gu, S., Zhang, D., Zhu, Y., Zhong, E., & Wu, F. (2026). Test-time adaptation for surgical instrument segmentation under visual degradations: selective updating and temporal constraints. Frontiers in Robotics and AI, 13, 1991989. https://doi.org/10.3389/frobt.2026.1991989

Image Credits: AI Generated

DOI: 10.3389/frobt.2026.1991989

Keywords: surgical robotics, test-time adaptation, instrument segmentation, endoscopic vision, uncertainty estimation, batch normalization, low illumination, optical flow, domain shift, EndoVis 2017, predictive entropy, robot-assisted surgery