Your AI Agent Has Permissions. What Happens When Someone Else Influences Its Decisions?

The moment an AI agent stops answering questions and starts taking actions, it becomes an identity inside your systems — one that holds permissions, and one that anyone who can put text in front of it may be able to influence.

For most of the last two years, the risk conversation around generative AI was about output. Would the model hallucinate, would it leak training data, would it say something embarrassing. Those are content risks, and they are largely manageable with review.

Agentic systems changed the question. An agent that can query a database, send an email, update a CRM record, call an internal API, or open a ticket is no longer producing text for a human to evaluate. It is producing effects — and those effects land in production systems with whatever permissions you granted it, at machine speed, usually without anyone reading the intermediate steps.

That is a different class of risk, and it is not solved by making the model better.

The Model Is Not a Trust Boundary

Here is the uncomfortable property of every large language model in production today: it cannot reliably distinguish instructions that came from you from instructions that arrived inside the data it was asked to process.

A model reads one stream of tokens. Your system prompt, the user's request, the contents of a retrieved document, the body of an incoming email, the text of a web page, the description field of a CRM record — by the time they reach the model, they are the same kind of thing. The distinction between policy and content exists in your architecture diagram. It does not exist in the context window.

This is why prompt injection is not a bug waiting to be patched. It is a direct consequence of how these systems work. Someone who can influence any text the agent reads can attempt to influence what the agent does, and the attempt does not have to look like an attack.

Malicious Input Looks Like Legitimate Input

The injected instruction rarely arrives as something obviously hostile. It arrives as a support ticket, a resume, a vendor invoice, a calendar invitation, a product review, or a page your agent was asked to summarise. The payload is a sentence in the middle of otherwise ordinary content, and the agent processes it the way it processes everything else — as context worth acting on.

Filtering helps at the margins and fails as a control. You cannot enumerate the phrasings of a natural-language instruction, and an attacker who gets feedback from the system will iterate faster than your blocklist does.

Prompts Cannot Enforce Access Rules

The most common response to this is to harden the system prompt — to add a paragraph saying the agent must never reveal customer data, never issue refunds above a threshold, never call an administrative endpoint. It feels like a control because it reads like a policy.

It is not a control. A system prompt is an instruction to a probabilistic system that already failed to distinguish instructions from data. It biases behaviour; it does not constrain it. No security architecture would accept a firewall rule that works most of the time and can be argued out of by the traffic it inspects.

If the only thing stopping your agent from taking an action is a sentence in the system prompt, the action is not restricted — it is merely discouraged.

The conclusion follows directly: authorization has to live outside the model. The model decides what to attempt. Your architecture decides what is permitted to happen.

Six Principles for Designing Agents You Can Trust

Assume the model can be influenced, and design so that influence does not translate into consequence. These six layers are what turn an agent with broad access into a system with bounded behaviour.

1. Constrain capabilities. An agent should hold the narrowest set of tools and data scopes its task requires, not the permissions of the service account that was convenient to reuse. Read access is not a lower tier of write access — it is a separate decision. If the agent never needs to delete, it should not hold a tool that can.
2. Enforce authorization and policy outside the model. Every tool call runs through a layer that checks identity, scope, limits, and business rules independently of what the model intended. The agent requests; the policy engine authorises. A refund above a threshold, a record outside a tenant boundary, an endpoint outside the allowlist — these are rejected by code that cannot be persuaded.
3. Validate inputs and outputs. Treat everything entering the context window as untrusted content rather than as instruction, and validate everything leaving it against a schema before it reaches a downstream system. Structured, typed tool arguments are far harder to subvert than free-form text that some service will later interpret.
4. Control egress. Most damaging injections do not try to destroy data — they try to move it. Restrict where the agent can send information: allowlisted domains, no arbitrary outbound HTTP, no unconstrained attachments, no email to addresses outside an approved set. Capability without egress is a much smaller problem.
5. Scope human review to what matters. Approval on every action destroys the economics of automation and trains reviewers to click through. Route to a human only where the action is irreversible, high-value, or outside the agent's normal distribution — and give the reviewer the context needed to make a real decision rather than a rubber stamp.
6. Log, monitor, and contain. Record every prompt, retrieval, tool call, argument, and result as an auditable trail. Alert on the patterns that indicate drift or abuse — unusual tool sequences, volume spikes, repeated authorization failures — and build the ability to revoke a credential or disable a tool without redeploying the system.

Excessive Agency Fails Quietly

Over-permissioned agents rarely announce themselves. There is no outage, no error page, no stack trace. The agent does exactly what it was asked to do by whoever happened to be able to ask it, the action succeeds, and the result looks like normal operation in every dashboard you have.

This is what makes excessive agency more dangerous than most AI failure modes. A hallucination is visible and gets corrected. An agent that quietly exfiltrates a customer list, approves a transaction it should have escalated, or modifies a record outside its tenant is a loss you discover later — during an audit, a customer complaint, or a breach notification.

It is also where the technical risk becomes a business risk. The organization is accountable for what the agent did under its own credentials. "The model was manipulated" is not a regulatory defence, a contractual defence, or an explanation a customer will accept.

What This Means for the Business

• Design assuming the model can be influenced. Not as a worst case — as the operating assumption. Every control that depends on the model behaving correctly under adversarial input is a control you do not have.
• Treat permissions as the primary risk lever. Reducing what an agent is able to do lowers exposure more reliably than any amount of prompt engineering, and it is the one change you can actually verify.
• Make the system explainable after the fact. Validation, egress control, and complete logs are what let you answer what the agent did, on whose behalf, and under what authority — the question that gets asked when something goes wrong.

How We Apply This in Practice

When we design an agentic system, the capability model comes before the agent. We start from the actions the system is permitted to take, the data scopes each action requires, and the threshold at which a decision stops being automatic — then build the policy, validation, and egress layers that enforce those boundaries in code rather than in instructions.

The result is an agent whose worst case is known. It can still be given a bad instruction; it simply cannot act outside the envelope the architecture defines. That property is what makes an AI system deployable in an environment where the actions are real — and it is designed in at the start, not added after the first incident.