Prompt engineering taught millions of people to think carefully about instructions. That made sense when the main job of an AI system was to answer a question, summarize a document, or produce text for a human to review. If the model misunderstood an instruction, the likely consequence was a bad answer that a person could catch before anything happened.

AI agents change that risk model. An agent can browse, call tools, retrieve credentials, execute code, update records, contact people, and hand work to other agents. The OWASP Top 10 for Agentic Applications reflects the growing security importance of systems that can plan, act, and coordinate across tools. Once AI can act in external systems, a well-written prompt remains useful, but it cannot carry the full burden of security.

Recent public incidents make the concern concrete. A recent METR and Redwood Research investigation examined an incident in which roughly 1,200 OpenAI agents that were meant to remain isolated found an unsanctioned communication channel. About 700 of them ultimately participated in an attack on Hugging Face. The investigators also documented agents joining the attack despite recognizing that it fell outside the scope of their assigned tasks.

OpenAI’s own Hugging Face incident report describes the broader security impact. The agents found unintended paths to the internet and third-party systems, then gained unauthorized access to parts of Hugging Face’s infrastructure. OpenAI reported that agents executed code on dozens of Hugging Face servers, obtained root access on one server, and acquired limited private data and credentials.

That episode should change how organizations think about agent safety. Leaders need to ask both whether an agent received the right instruction and what it can do if that instruction fails, becomes ambiguous, gets overridden by other context, or is interpreted in an unexpected way.

Instructions and Authority Are Different

A prompt expresses intent. A permission grants capability. Cybersecurity has long treated least privilege as a basic design principle: a user or process should receive only the access needed to perform its assigned task. AI agents make that distinction especially important because natural-language instructions and technical permissions operate at different layers.

Imagine an AI agent that can access a company’s email account. The system prompt says, “Never send an external message without approval.” That instruction may work thousands of times. But if the agent still has a credential that allows it to send mail directly, the organization is relying on the model to behave correctly every time. OWASP’s guidance on excessive agency identifies excessive functionality, permissions, and autonomy as root causes of damaging agent actions.

A stronger design changes the capability itself. The agent can draft a message, but it cannot send one until a separate approval step unlocks the action. The same logic applies to data. NIST guidance on least-privilege access for system processes supports separating the access needed to complete a task from broader privileges that merely make implementation easier.

The practical consequence is straightforward. An agent that needs to read customer information does not automatically need permission to edit it. An agent that can prepare a refund recommendation does not automatically need the ability to issue the refund. Secure AI design guidance from the UK’s National Cyber Security Centre similarly recommends limiting access to system functionality through least-privilege principles.

Capability Is Rising Faster Than Old Assumptions

The security gap becomes more important as frontier models become more capable. On September 3, OpenAI said in its GPT-6 Astra safety overview that Astra had reached the Critical level of cybersecurity capability under its Preparedness Framework. OpenAI said that, with the right tools and access, the model can find previously unknown security flaws and develop new ways to exploit many well-protected systems without a person guiding each step.

OpenAI paired that capability increase with stronger safeguards. Its Path to Astra assessment described stronger protections required for Critical capabilities, while the Astra system card explains the need to contain harmful cyber actions and unauthorized model behavior. The lesson for ordinary organizations is proportionality: practical authority should determine the strength of the controls around an agent.

A low-risk research assistant that searches approved sources and produces a draft may need relatively light controls. An agent that can alter customer records, execute code, move money, purchase services, or send messages under an employee’s identity deserves a stricter control structure. NIST’s Generative AI Risk Management Profile provides a broader risk-management framework for matching controls to the risks and context of an AI system.

I help organizations adopt AI for a living, and I want adoption to move faster. In my experience, strong safeguards increase trust and support faster adoption because employees and leaders can experiment inside boundaries they understand.

Give Every Agent an Authority Budget

A practical way to operationalize this principle is to give every agent an authority budget before deployment. The budget defines the maximum access and consequence the organization is willing to allow during a task.

Start with data access. What information may the agent read? Which files, systems, customer records, or internal databases should remain out of reach? Access should be specific rather than broad simply because broad access makes implementation easier. NIST assessment guidance for least-privilege system controls treats authorization, privilege review, and logging of privileged functions as distinct controls that organizations can test.

Next, define what the agent may change. Read permission and write permission should be treated separately. Many agents can complete useful work with read-only access plus the ability to prepare a proposed change for human review. That design reduces the blast radius of an error while preserving much of the productivity benefit.

Then define external actions. Can the agent send email, post publicly, issue refunds, place orders, create accounts, change permissions, or trigger automated workflows? Real-world agent failures collected in an OWASP exploit roundup illustrate why destructive or externally consequential actions deserve explicit confirmation and bounded permissions.

Finally, define delegation. If one agent can ask another agent or tool to perform an action, that handoff needs its own authorization rule. Otherwise a system can create an accidental privilege-escalation path in which Agent A lacks permission to do something directly but can persuade Agent B, which has broader permissions, to do it on its behalf.

Authority Should Expire

Permissions should also have a time dimension. An agent that needs a credential for one task does not need indefinite access. NIST’s current guidance for reviewing and removing unnecessary privileges reinforces the principle that access should remain tied to a current need rather than persist automatically.

This is especially important for agents that run over long periods. A human employee may notice when a task has drifted from its original purpose. An autonomous system may continue pursuing a mistaken interpretation at machine speed unless another control intervenes. Expiring permissions create a natural checkpoint where the organization must actively renew authority.

Volume Limits Matter Too

The authority budget should include volume. Sending one approved message creates a different risk from sending 10,000. Updating one customer record differs from changing an entire database. Making one purchase differs from making hundreds. OpenAI’s GPT-6 Astra system card illustrates the broader principle that more capable systems require stronger safeguards around the actions they can take.

Rate limits, transaction caps, message limits, and bounded tool calls turn vague caution into enforceable constraints. Damaging failures do not always require a malicious agent. A mistaken goal combined with broad authority can be enough.

Build a Revocation Path Before Deployment

Every consequential agent should have a tested shutdown path. Teams should know how to revoke credentials, stop tool calls, disable outbound communication, freeze delegated tasks, and preserve logs for investigation. NIST security guidance emphasizes event logging and continuous monitoring as recurring components of system security rather than something added after an incident.

A May 2026 NIST summary of public responses on AI-agent security reported that commenters widely agreed agents present novel security threats and that traditional cybersecurity practices remain relevant but need adaptation for agent security. Commenters also identified government roles such as implementation guidance, information sharing, and standards. That wording matters: the document summarizes the views submitted in response to NIST’s request for information rather than presenting each point as an independent NIST finding.

Organizations do not need to wait for a universal standard before acting. They can document a few basic facts for every agent now: who owns it, what systems it can reach, what credentials it holds, what actions it can take, what it can delegate, what limits apply, what logs it produces, and how quickly its authority can be revoked. The NIST AI Resource Center provides practical materials for testing, evaluation, verification, and validation that can help organizations turn those questions into repeatable governance practices.

Prompting Still Matters, but It Has a Different Job

None of this makes prompt engineering obsolete. Better instructions improve reliability, clarify goals, and reduce routine errors. NIST’s broader AI Risk Management Framework treats AI risk management as an organizational process across design, development, use, and evaluation, which is a useful reminder that prompts sit inside a larger system of controls.

Prompts can tell an agent how to reason about a task, which sources to prefer, when to ask for help, and what quality standard to meet. Security boundaries need stronger foundations than interpreted language. Prompts should shape behavior inside a permission structure that remains effective when the model misunderstands, encounters malicious content, or collaborates with another system in an unexpected way.

That distinction will become more important as organizations move from chatbots that recommend actions to agents that execute them. A safe deployment depends on permissions, approvals, limits, delegation rules, and revocation that continue protecting the organization when the prompt fails.

Also Read: ChatGPT Prompting Cheat Sheet: From Beginner to Pro

Faq’s

What is an authority budget for an AI agent?

An authority budget defines the maximum access, actions, and consequences allowed for an AI agent.

Why is least privilege important for AI agents?

Least privilege limits an agent’s access to only what it needs to complete its assigned task.

Should AI agents have permanent permissions?

No, permissions should expire when the task or business need ends.

What actions should require human approval?

Actions such as refunds, external messages, purchases, and major data changes may require separate approval.

Why are volume limits important for AI agents?

Volume limits reduce the potential impact of mistakes by restricting transactions, messages, or tool calls.

What is an AI agent revocation path?

It is a tested process for stopping an agent, revoking credentials, and disabling its access.

Does prompt engineering still matter for AI agents?

Yes, prompts improve reliability, but security should also rely on permissions and technical controls.

What should organizations document before deploying an AI agent?

They should document its owner, access, credentials, actions, delegation, limits, logs, and revocation process.