Deliberately Breaking Medical Devices: A Three-Method Test Could Make Hospital Technology Safer

Medical devices fail in the real world in ways that laboratory testing rarely anticipates. Nurses work under time pressure, interruptions, fatigue, and cognitive overload, and they occasionally press the wrong button, skip a step, or misread a screen. Yet most usability evaluations of hospital equipment still rely on a single method, typically observing users perform tasks correctly under ideal conditions. A new study published in BMC Health Services Research argues that this one-method approach leaves dangerous blind spots, and it proposes a solution: triangulate three complementary assessment techniques so that each catches what the others miss.

The research, led by Calvin Kalun Or of The University of Hong Kong and the Hospital Authority’s New Territories East Cluster, together with colleagues from Harbin Institute of Technology, Nanjing Forestry University, and Tianjin University, introduces what the authors call an assessment triangulation paradigm. The three methods at its core are deliberate misuse testing, heuristic evaluation, and simulated-use testing. Rather than treating these as competing alternatives, the paradigm deploys them in sequence so that their methodological strengths reinforce one another and their individual weaknesses are offset.

The first method, deliberate misuse testing, is perhaps the most provocative. Instead of asking users to operate a device correctly, researchers intentionally introduce abnormal operations and errors designed to replicate the misuse and error conditions that pose genuine safety risks. The goal is to examine how the device responds to these disruptions. Standard testing, by contrast, feeds devices only expected and correct inputs, which means it systematically overlooks what happens when a tired clinician enters an out-of-range value, presses keys in the wrong sequence, or attempts an operation the designer never anticipated. By deliberately stressing the device with plausible mistakes, this method surfaces operation problems that conventional evaluation would never reveal.

The second method, heuristic evaluation, brings in experts with domain knowledge who systematically inspect the device interface against a curated set of general and device-specific usability and safety heuristics. These principles, drawn from human factors engineering and clinical experience, act as a checklist of known design pitfalls: inconsistent labeling, ambiguous feedback, alarm fatigue risks, confusing menu hierarchies, and similar deficiencies. Because heuristic evaluation does not require end users to be present, it can be conducted relatively quickly and can identify interface design flaws early, before they ever reach a clinical ward. Its limitation is that it depends on the insight of the evaluators, which is precisely why the paradigm pairs it with empirical methods.

The third method, simulated-use testing, restores the human element in a controlled but realistic setting. Nurse representatives of the actual end users perform tasks relevant to real clinical use of the device, allowing researchers to objectively evaluate user-device-task interaction performance. This method captures how performance unfolds in practice: where users hesitate, where they err, which steps take too long, and where the design fights against the workflow. What it cannot easily do, on its own, is probe rare error conditions or expose latent design defects that users unconsciously work around. That is exactly the gap the other two methods fill.

To demonstrate that the paradigm works in practice, the team validated it step by step using smart infusion pumps as contextual clinical testbeds. Smart infusion pumps are an ideal proving ground. These devices deliver medications and fluids intravenously at precisely programmed rates, and they are among the most ubiquitous and most safety-critical pieces of equipment in modern hospitals. A programming error on an infusion pump can deliver a tenfold overdose or interrupt a life-sustaining medication, which is why these pumps have long been a focus of patient safety concern and why they are equipped with dose error reduction software, drug libraries, and alert systems.

The validation confirmed that each method performed its intended complementary function. Deliberate misuse testing allowed the discovery of device operation problems that would not emerge from standard testing in which only correct inputs are used. Heuristic evaluation detected interface design deficiencies that structured expert inspection is uniquely positioned to catch. Simulated-use testing provided a realistic environment for objectively measuring how nurses actually interacted with the pumps. Together, the three methods generated complementary information across different dimensions of the device, giving evaluators a more comprehensive and deeper understanding of its usability and safety issues than any single method could provide.

The significance of this work extends beyond infusion pumps. The authors argue that single-method assessments present significant gaps that put patient safety at risk, because no one technique can detect the full breadth of design and use-related problems that devices exhibit in clinical service. A device might pass simulated-use testing with flying colors while harboring a catastrophic failure mode that only appears under misuse. An expert heuristic review might flag a confusing screen, but only observed testing can reveal how that confusion plays out under real workflow pressure. Triangulation, in the methodological sense of combining multiple perspectives on the same object of study, is the mechanism by which these blind spots are closed.

The implications for regulation and practice could be substantial. The authors suggest that the paradigm may reshape how relevant entities in the medical device safety and patient safety fields, including the U.S. Food and Drug Administration, the International Organization for Standardization, and leading practitioners, approach their evaluations. Regulators currently rely on standards and guidance that emphasize summative usability testing with correct task performance; a triangulation requirement would push manufacturers to demonstrate not only that their devices work when used properly, but that they fail safely when used improperly. The study also offers step-by-step instructions for implementing the paradigm, lowering the barrier for hospitals, manufacturers, and assessment teams that want to adopt it without designing a methodology from scratch.

The research was conducted with ethical approval from the Joint Chinese University of Hong Kong and Hospital Authority New Territories East Cluster Clinical Research Ethics Committee, and written informed consent was obtained from all participants. Published open access on 9 October 2026, the study arrives at a moment when hospitals are deploying increasingly complex connected devices, from smart pumps to clinical decision support systems, into environments where human error is inevitable. The paradigm’s central message is disarmingly simple: if you want to know how a device will behave in the messy reality of clinical care, you must test it the way it will actually be used, inspect it the way an expert would, and break it the way a stressed human might. Only the combination of all three, the authors conclude, gives safety engineers and clinicians the full picture they need before a device ever touches a patient.

Subject of Research: A triangulated usability and safety assessment paradigm for medical devices validated with smart infusion pumps

Article Title: Rethinking the usability and safety assessment of medical devices – an assessment paradigm involving triangulation of deliberate misuse testing, heuristic evaluation, and simulated-use testing and practical validation through a smart infusion pump case study

Article References: Rethinking the usability and safety assessment of medical devices – an assessment paradigm involving triangulation of deliberate misuse testing, heuristic evaluation, and simulated-use testing and practical validation through a smart infusion pump case study. (n.d.). https://doi.org/10.1186/s12913-026-15697-3

Image Credits: AI Generated

DOI: 10.1186/s12913-026-15697-3

Keywords: medical devices, usability testing, patient safety, deliberate misuse testing, heuristic evaluation, simulated-use testing, smart infusion pump, assessment triangulation, human factors engineering, health services research, device safety, clinical workflow