Clinicians in the Cardiothoracic ICU Say AI Must Earn Its Place Before They Will Trust It

Before any algorithm can help save lives in a cardiothoracic intensive care unit, it has to convince the people already working there. A field study conducted at a large academic medical center found that nurses, advanced practice providers, and physicians in a CT-ICU were cautiously optimistic about predictive artificial intelligence but attached strict conditions to its adoption. The system would need to add genuine clinical value, earn trust through transparency and consistent accuracy, and fit into a workflow already crowded with alarms, documentation, and devices. The study, published in Frontiers in Digital Health, involved fourteen participants and is presented as an early step in a longer project to develop, deploy, and evaluate a predictive AI system in this demanding environment.

The research team observed and interviewed fourteen members of the CT-ICU team over three consecutive days in November 2023, totaling twenty-five hours of interviewing and observation. The group included seven nurses, three advanced practice providers such as nurse practitioners and physician assistants, two cardiovascular critical care attending physicians, and two surgeons. Participants ranged in age from twenty-six to fifty-three, with critical care experience spanning from one month to twenty-two years. Interviews were semi-structured, and some were conducted observationally while participants worked in the unit. Audio recording was permitted only in a private interview space because patient privacy rules barred recording inside the ICU itself, and five of the fourteen interviews relied on written notes instead.

The proposed AI system at the center of the study would assign patient-specific risk scores for multiple adverse outcomes, including acute kidney injury, prolonged ventilation, and stroke. It would draw on data already flowing through the electronic health record, such as demographics, risk factors, laboratory results, vital signs, clinical observations, and medications. Static predictions are batch or point-in-time calculations based on collected data, while real-time predictions make continuous or serial updates as patient data changes. The researchers emphasized that this field study was the first step in a long-term multidisciplinary project and that no prototype or sample interface was shown to participants, a limitation the authors themselves acknowledged.

Attitudes toward AI were described as cautiously optimistic with thoughtful skepticism. Participants expressed excitement about the potential of predictive tools and said they would welcome involvement in shaping them. One participant noted that a system could be helpful and asked to be involved in more feedback. At the same time, concerns surfaced about potential negative impacts on workflow, accuracy, and whether the tool would offer anything beyond what clinicians already know. A system that is perfectly accurate at telling the care team what they already know, participants pointed out, simply adds unnecessary effort. One physician remarked that the tool had the potential to create more noise without helping.

Added clinical value emerged as the single most important concept. Participants consistently said any AI system introduced into the CT-ICU must be highly accurate and provide knowledge or insights beyond what is already known to the clinical team. One participant said they would have to see it be right a couple of times and out-think them before trusting it. Another explained that if critical alerts arrived once a week and proved reliable, that would be of huge value. The confidence threshold participants considered useful ranged from seventy-five percent to one hundred percent, with no agreement on an exact number. One participant said the confidence would have to be nearly perfect or they would ignore it entirely. Conveying uncertainty in predictions was also flagged as important, with requests for sensitivity, specificity, and confidence intervals. Timing mattered too. Participants described a sepsis prediction tool available in the electronic medical record that was rarely used in the CT-ICU, partly because notifications arrived hours after the relevant changes in vital signs or labs, leaving data that was old or no longer pertinent.

Trust in the AI system hinged on several factors the participants identified. Accurate predictions with minimal false positives or false alarms would increase trust, as would evidence of accuracy through clinical testing or clinical trials. Opportunities for clinical team feedback during development, transparency about how and why the system makes a prediction, and the ability to see the top reasons behind any prediction were all cited as trust builders. One participant illustrated this by describing how a patient who had not received adequate nutrition for a week, was not moving much, and had certain lab levels might be flagged for increased pressure injury risk, allowing the team to evaluate whether the source data was valid. Data validity was a critical concern at both the individual patient level and the training data level. Participants worried about technological artifacts, such as a rhythm monitor alarming for ventricular tachycardia when a patient was simply brushing their teeth, and about human delays in charting, since nurses may fall behind on documentation during busy periods and vitals may only be recorded every few hours. A lag in patient data could leave an AI system without the most up-to-date information, leading to inaccurate or delayed predictions. Participants also raised concerns that the algorithm could be skewed by the unique conditions of the CT-ICU population, noting that sepsis can look very similar to cardiogenic shock, which is commonly seen in these patients.

Alarm fatigue was by far the most prominent theme across all fourteen participants and was identified as a potentially significant barrier to AI adoption. One participant estimated that about eighty-five percent of alarms are false but said clinicians are obligated by ethics to check every one. Another described the experience of multiple alarms going off as making it impossible to think. Providers were consistently concerned that AI algorithms could contribute to alarm fatigue by generating yet another stream of alerts. They suggested that an alarm generated by the AI would only be useful in the event of a high-risk event, and that careful threshold setting would be needed. The unit operates with an unofficial hierarchy of alarms coded by color: red for critical events, yellow for moderate ones, and blue for low-risk situations. Each level has distinct auditory and visual signatures, and team members learn to recognize them by sound. Participants suggested that AI-driven alarms may need to fit within this hierarchy, and that for a predictive tool intended to warn hours in advance rather than signal an imminent life-threatening event, visual communication may be more appropriate than audible cues.

Documentation burden and click fatigue added further layers of concern. Participants valued automatic integration into the electronic medical record, citing examples such as glucometer data that populates the patient chart directly. They worried that AI predictions could be buried deep within medical systems and not readily available, and that they might need to manually document prediction outputs in the chart. Automated alerts received through the electronic record often require a click to acknowledge, and participants reported that a high volume of these alerts were unhelpful and ignored. One participant said ninety-nine percent or more of the alerts coming through the system were unhelpful. The study also documented the sheer volume of technology already in use. During one observation in a patient room, the team noted six IV pumps, multiple monitors, a continuous venovenous hemofiltration machine, a ventilator, a Bair Hugger system, a computer, a bed alarm, and sequential compression devices. Participants described technostress as an additional consideration, noting that lower comfort with technology may make learning a new system harder, and that a rigorous onboarding plan would be critical.

Personal wellness and the ability to disconnect from work devices also emerged as themes. Several participants described feeling too tethered to technology, with one physician noting they were in touch with the hospital twenty-four hours a day, seven days a week, whether on call or not. Others spoke about the difficulty of taking a mental break when alarms continue to arrive even during a short lunch. The authors suggest that user interfaces for AI tools could include customization at the unit and individual levels, with silence features, sign-out options, or the ability to turn notifications off when a clinician is not the responsible provider. Participants also suggested that more experienced team members may be less likely to embrace AI, while newer staff might benefit from it sooner, and that comfort with technology appeared to influence hesitancy.

From these findings, the researchers synthesized a preliminary framework called Trusted AI for Clinical Teams, or TACT. It lays out five recommended stages: ideation, development, clinical validation, deployment, and feedback, with eight steps and seventeen key insights. The framework emphasizes a human-in-the-loop model in which the clinician remains the decision-maker, engagement of the clinical team in iterative co-design, tailoring training datasets to local or comparable populations, and displaying temporal trends and top AI features as detail-on-demand to avoid cluttering the primary view. The authors stress that TACT is preliminary and emerged from a single-site study with a small sample, and that future work will be needed to validate it across a variety of healthcare settings. They also note that clinical guidelines or nursing protocols accompanying machine-learning predictions would benefit from clinical trials demonstrating improved outcomes over the current standard of care, and that medico-legal implications, while beyond the scope of this paper, must be considered when implementing an AI system in clinical practice.

Subject of Research: A qualitative field study of clinician needs and barriers to implementing real-time AI predictive analytics in a cardiothoracic intensive care unit.

Article Title: Needs and barriers of healthcare teams to implement real-time AI & predictive analytics in cardiothoracic ICU: a field study

Article References: Barry, C. L., Purdue, S. K., Nourani, M., Forlines, C., Goodwin, M. S., & Tory, M. (2026). Needs and barriers of healthcare teams to implement real-time AI & predictive analytics in cardiothoracic ICU: a field study. Frontiers in Digital Health, 8, Article 1943663. https://doi.org/10.3389/fdgth.2026.1943663

Image Credits: AI Generated

DOI: 10.3389/fdgth.2026.1943663

Keywords: artificial intelligence, predictive analytics, cardiothoracic ICU, clinical decision support, alarm fatigue, workflow burden, trust in AI, human factors, electronic health record, technostress, qualitative research, health informatics