When AI Agents Get Tools, Safety Becomes an Architecture Problem

Large language models have quietly outgrown their original role as text generators. The most capable systems now operate inside software harnesses that retrieve documents, plan subtasks, call external tools, edit code, run tests, and adapt their behavior through memory and feedback. That shift changes the nature of the safety problem in a fundamental way. When a model merely produces text, a failure misleads a reader. When an agent can write files, execute commands, or call services, a failure can change the state of the world. A new Perspective published in Frontiers in Computer Science argues that the field has been asking the wrong question, and that agentic AI safety should be treated not only as a problem of model trust but as a problem of software architecture.

…

The distinction the authors draw between misleading a reader and altering external state deserves closer examination, because it reframes what counts as a safety failure. A hallucinated answer in a chat window is recoverable; a hallucinated answer that becomes a deleted database table, a merged pull request, or an executed shell command is not. The authors ground this observation in the trajectory of recent research: retrieval-augmented generation brought external documents into model context, reasoning-action loops connected inference to environment-directed behavior, and tool-use research wired models to external application programming interfaces. Each of these steps increased capability, and each also increased the blast radius of a single bad decision. The Perspective’s response is to relocate part of the safety burden from the model itself to the arrangement of components around it.

That relocation has a long pedigree in computer security, even if it is relatively new to machine learning discourse. The authors invoke the reference-monitor concept, which requires that every security-relevant access be mediated by a component that is always invoked, tamper-proof, and simple enough to analyze. They also draw on Saltzer and Schroeder’s classic protection principles, including least privilege, separation of privilege, complete mediation, and fail-safe defaults. What is notable is how directly these decades-old ideas map onto contemporary agent failures. An agent that holds both the authority to decide what to do and the credentials to do it violates separation of privilege in exactly the way a monolithic application with a single superuser account does. The novelty is not the principle but the target: instead of protecting files from users, the topology protects external state from autonomous components.

The five-plane reference topology makes this concrete. The intent plane captures what the user actually wants and preserves a stable reference to it, so that goal drift during long task execution can be detected rather than silently absorbed. The orchestration plane plans and decomposes work but, crucially, holds no effectful tools of its own. The execution plane does the acting—modifying files, calling services, running commands—but only under scoped, time-limited, task-specific authority granted at the tool boundary. The oversight plane can reject, request revision, quarantine, or halt, but it deliberately does not wield the same tools whose outputs it judges, preventing the evaluator from becoming an unaccountable actor. The adaptation plane handles memory and self-improvement, with the constraint that reflection may change how a task is approached but never what the system is authorized to do.

The core invariant binding these planes together is stated with unusual economy: components that decide should not directly act, and components that act should not act without supervision. This is an architectural statement rather than a behavioral one. It does not depend on the model being well-aligned, honest, or robust; it depends only on the wiring being correct. A misaligned planner in this topology can produce a bad plan, but it cannot execute it. A compromised executor can act, but only within narrowly scoped authority and only on paths that a separate oversight component has gated. The design philosophy resembles defense in depth, but applied to the internal structure of an AI system rather than to a network perimeter.

Complete mediation is where this approach becomes most demanding. The authors insist that every consequential message path be declared, every worker result be gated before acceptance, every uncertain verdict fail closed, and every high-consequence action require authorization at the tool boundary. Failing closed is a particularly important commitment. In many current agent frameworks, when a validator or checker is uncertain, the system proceeds anyway, treating ambiguity as permission. The topology proposed here inverts that default: if oversight cannot confidently approve an action, the action does not happen. This converts safety from a probabilistic property of model behavior into a checkable property of system structure.

The Polos specification, which instantiates the topology, encodes these commitments in machine-checkable form. Role contracts define what each component may do. Typed message schemas constrain what can flow between components. Declared flow graphs make the actual control paths explicit and auditable, so that an undocumented shortcut from planner to executor is not merely bad practice but a detectable violation. Bounded loops impose step, time, cost, and scope ceilings on the iterative plan-act-observe-revise cycles that characterize agentic behavior, addressing the well-documented risk that unattended loops drift, exhaust resources, or accept false completion signals. Validator-checked invariants mean the topology itself can be verified, not just documented.

The treatment of self-improvement is especially worth noting. Reflective memory and learning from feedback are among the most powerful capabilities emerging in agentic systems, and among the most dangerous. A system that can rewrite its own instructions, expand its own tool access, or reinterpret its own objectives is a system whose safety properties at deployment time may not hold a week later. The Perspective’s answer is to channel adaptation through the governed adaptation plane, where memory can refine strategy but authority boundaries remain fixed and externally enforced. This keeps self-improvement useful while denying it the ability to dissolve the very constraints that make it safe.

The authors are careful to position this work as complementary rather than adversarial to model-centered safety. Reinforcement learning from human feedback and related alignment methods have produced genuine improvements in instruction following and the reduction of undesirable outputs. Red teaming, guardrails, trajectory monitoring, and diagnostic frameworks such as those proposed in recent agent-safety literature all retain their value. The argument is about sufficiency, not replacement. Alignment improves the behavior of individual components; topology limits what the failure of any component can accomplish. A well-aligned model inside a well-governed topology is safer than either alone, because the two layers fail differently and independently.

The prompt-injection literature supplies the sharpest motivation for this claim. Indirect prompt injection research has shown that malicious instructions hidden in retrieved documents can manipulate language-model-integrated applications, including those with tool access. More recent work demonstrates attacks that target tool selection itself and operate through AI-assisted development environments, tool poisoning, hidden parameters, cross-tool interactions, and unauthorized tool invocation. Against these threats, prompt-level defenses are inherently fragile, because the attack surface is the input stream itself. A topology that separates decision from action and gates every consequential path at the tool boundary does not need to detect the injection; it needs only to ensure that whatever the model decides, the action still passes through scoped, supervised, authorized channels. Provenance-oriented research that tracks how untrusted context propagates into agent decisions fits naturally into this picture, providing the oversight plane with evidence about whether proposed actions are justified by trustworthy sources.

There are practical implications for how organizations deploy agents. Under a model-trust paradigm, procurement reduces to evaluating benchmarks and hoping behavior generalizes. Under a topology paradigm, deployment becomes an engineering discipline: define role contracts, declare flow graphs, scope credentials at tool boundaries, impose loop ceilings, and verify invariants continuously. Auditing changes character too. Instead of trying to reconstruct why a model behaved as it did from opaque internal states, an auditor can inspect the declared topology, compare it to runtime message traces, and identify exactly where authority was granted, exercised, or exceeded. This inspectability is what the authors mean when they describe the topology as a substrate for evaluating, constraining, and governing tool-using agents.

The epistemic dimension the authors raise also merits attention. They argue, drawing on recent work, that agentic safety is partly an epistemic property: self-improving systems must remain teachable and correctable over time. A system that drifts beyond correction—whose memory has overwritten the operator’s intent, or whose adaptation has reinterpreted its own mandate—cannot be governed even with perfect oversight machinery. Preserving a stable user-intent reference across the intent plane, and confining adaptation to strategy rather than authority, are the topological mechanisms that keep the system correctable. This connects agent safety to questions of institutional control that extend well beyond any single model checkpoint.

None of this eliminates the hard problems. Determining the right granularity of scoped authority, designing oversight components that can genuinely evaluate complex agent outputs, and handling the tension between bounded autonomy and useful persistence all remain open engineering challenges. The topology also imposes overhead: declared paths, gated results, and validator checks add latency and complexity that ad hoc agent frameworks avoid. The Perspective’s contribution is not to solve these problems but to relocate them—to a layer where they can be specified, checked, and enforced with the ordinary tools of software engineering. As agents take on repository-level software tasks that require issue interpretation, codebase navigation, file editing, execution, and test-based validation, and as research increasingly identifies execution environments, tool interfaces, and governance as independent determinants of reliability, the question shifts from whether a model can be trusted to whether the system around it is built so that no single component, however capable or however compromised, can convert error into unsupervised action.

Subject of Research: Structural software-architecture governance for safe tool-using agentic AI systems

Article Title: From model trust to software topology: a Perspective on structurally governed agentic AI systems

Article References: Morin, M., & Sufi, F. (2026). From model trust to software topology: a Perspective on structurally governed agentic AI systems. Frontiers in Computer Science, 8, Article 1919950. https://doi.org/10.3389/fcomp.2026.1919950

Image Credits: AI Generated

DOI: 10.3389/fcomp.2026.1919950

Keywords: agentic AI, AI safety, software topology, large language models, prompt injection, Polos specification, agent governance, tool use, oversight gates, bounded autonomy, AI control, software architecture