Artificial intelligence systems that make life-changing decisions in hospitals, courtrooms, cockpits and trading floors face a stubborn dilemma: the models that predict best are often the ones humans understand least. A systematic review published in the journal AI Perspectives & Advances examines this tension in detail, arguing that explainability is not a cosmetic add-on for high-stakes autonomous AI but a foundational requirement for trust, accountability and regulatory compliance. The study, led by Uday Nedunuri with Abhijit Das Gupta and Debashis Guha, was published on 13 February 2026 as an open-access review.
The review followed the PRISMA framework, the standard protocol for systematic literature reviews. The authors searched the ScienceDirect database with three query strings covering high-stakes autonomous AI systems and performance, explainability and performance, and AI governance frameworks. Those searches returned 3,953 records: 1,854 from the first string, 1,344 from the second and 755 from the third. After removing 91 duplicates, applying a publication window of 2018 to 2025, which excluded 376 articles, and screening out book chapters, conference proceedings and abstract-only records, the pool narrowed to 1,590 articles. A final relevance assessment left 45 peer-reviewed studies for in-depth analysis.
The central finding is what the authors call the explainability-performance paradox. Complex models, especially deep learning architectures and ensemble methods, achieve superior predictive accuracy, but that accuracy comes bundled with opacity. The reviewed literature shows that performance gains in medical and industrial domains often make it harder to audit a model’s rationale, because the deeper and more heavily parameterized a network becomes, the more its internal reasoning resists human inspection. Conversely, techniques designed for intrinsic interpretability tend to sacrifice some predictive power, since simplified model logic and reduced representational depth limit what the model can learn.
Importantly, the review argues that this is not a simple binary trade-off. The authors frame it as a socio-technical equilibrium problem shaped by algorithmic complexity, reliability requirements, user cognition, governance expectations and domain-specific risk. They build their analysis on four interdependent constructs. Reliability, the ability to produce consistent outcomes across variable conditions, underpins user confidence and regulatory approval in fields like aviation and medical diagnostics, but tightening it can restrict a system’s adaptability to new environments. Performance quality drives optimization and efficiency. Explainability acts as the epistemic bridge between algorithmic reasoning and human understanding. Trustworthiness integrates all three within broader expectations of fairness, accountability and governance.
The synthesis is organized into four thematic clusters. The first confirms that explainability-performance tensions recur across healthcare, aviation and autonomous decision systems. Post-hoc interpretability tools such as SHAP, LIME and Grad-CAM improve transparency, but the review finds they often produce only partial causal insight. Citing work by Lipton, the authors note that many so-called explanations merely approximate a model’s reasoning rather than reveal it, and that approximate explanations may fall short of the ethical, evidentiary or compliance standards required in legally consequential settings. The tools can also be unreliable under out-of-distribution or adversarial conditions.
The second theme addresses reliability under uncertainty. Studies spanning aviation, defense and medical imaging show that even highly accurate models degrade when confronted with distributional shifts, adversarial noise or environments they were never modeled to handle. Researchers in the reviewed literature advocate uncertainty quantification, calibration mechanisms and robustness testing as prerequisites for trustworthy AI, while acknowledging that reliable systems can lose agility and real-time adaptability, which complicates optimization further.
The third theme is perhaps the most socially consequential: trust and acceptance depend as much on social and organizational factors as on raw technical performance. Evidence from cognitive research indicates that when explanations are missing or too complex, people’s willingness to rely on AI and collaborate with it declines. The review highlights Shin’s argument that explainability alone does not guarantee trust; what matters is causability, the extent to which an explanation enables users to build a clear, coherent causal understanding of how the system operates. On this view, explainability carries cognitive and communicative dimensions, and trustworthiness emerges as a relational construct built on transparency, stakeholder engagement and sensitivity to context rather than technical design alone.
The fourth theme surveys emerging hybrid approaches that attempt to dissolve the paradox rather than merely manage it. The authors describe a progression from AutoML to AutoAI and toward what Cao terms AutoAI+, systems that pair automated optimization with more cognitively aligned capabilities such as intention inference and contextual adaptability. Industrial examples suggest interpretability and performance need not be mutually exclusive: Cavalcanti and colleagues combined genetic algorithms, simulation-driven machine learning models and data-envelopment analysis to improve thermodynamic plant operations while retaining interpretable control logic. Concept-whitening approaches align a network’s latent representations with human-understandable concepts, offering greater transparency with limited loss in accuracy. Hybrid symbolic-neural systems, prototype-based reasoning and blockchain-AI integration, which proposes decentralized infrastructures with transparent mechanisms and verifiable data provenance, all aim to embed interpretability closer to the model itself. The review cautions, however, that many autonomous systems still rest on rigid assumptions that limit their effectiveness in dynamic, uncertain environments, and that open questions remain about the scalability, cross-domain generalizability and resilience of these hybrid designs.
On the regulatory side, the review finds momentum accelerating. Analyses of the EU AI Act underscore that human oversight, traceability and explanation requirements form core safeguards for high-risk applications, and the authors point to U.S. executive guidelines on AI oversight and FDA regulatory pathways for medical AI as increasingly mandating transparency, documentation and meaningful human oversight. These shifts, they argue, position explainability not merely as a desirable design feature but as a formal compliance requirement, reinforcing that technical progress alone cannot resolve the tension without alignment with institutional and legal expectations.
The authors are candid about the review’s limits. The search drew only on ScienceDirect, so relevant work in IEEE Xplore, the ACM Digital Library, PubMed or arXiv may have been missed, and the 45-study sample may reflect regional or disciplinary skews toward Western academic and policy contexts. The review is conceptual in scope and does not empirically test the trade-offs it identifies; the real-world impact of hybrid models and explainability tools on trust, clinical accuracy, operational safety or compliance remains largely unexamined. Terminology across the field is inconsistent, with no shared definition of explainability itself. Two of the study’s five research questions, how end-users such as doctors, pilots and judges actually weigh explainability against performance, and under what conditions reduced accuracy is acceptable in exchange for interpretability, remain insufficiently answered by the existing literature. The authors call for shared benchmarks, domain-specific explainability criteria, risk-tiered frameworks linking transparency requirements to specific legal obligations, and empirical evaluations involving clinicians, judges, pilots and auditors. Their overarching conclusion is that strong performance and meaningful human understanding should be treated not as opposing aims but as mutually reinforcing foundations for safe, transparent and trustworthy AI deployment.
Subject of Research: The trade-off between explainability and performance in high-stakes autonomous AI systems
Article Title: The paradox of explainability vs. performance in high-stakes autonomous AI systems: a systematic review of trade-offs, regulatory gaps, and emerging solutions
Article References: Nedunuri, U., Das Gupta, A., & Guha, D. (2026). The paradox of explainability vs. performance in high-stakes autonomous AI systems: a systematic review of trade-offs, regulatory gaps, and emerging solutions. AI Perspectives & Advances, 8(1), Article 1. https://doi.org/10.1186/s42467-026-00018-5
Image Credits: AI Generated
DOI: 10.1186/s42467-026-00018-5
Keywords: explainability, autonomous AI, systematic review, deep learning, EU AI Act, trustworthy AI, interpretability, high-stakes domains, hybrid AI, PRISMA, AI governance, post-hoc explanation
News Source: Denise Maddox. (October 28, 2026). Why the smartest AI systems are the hardest to trust, review finds. Scienmag.
