An AI agent control plan defines what an agent may see, what it may propose, what it may change and how people can reconstruct every consequential run. It should exist before production credentials are issued, not after the first incident. The plan is both an architecture artifact and an operating agreement among product, security, legal, data, engineering and workflow owners. This guide is for teams searching for a practical way to govern tool-using agents: classify actions, enforce permissions outside the model, design meaningful approvals, preserve evidence, test failure paths and increase autonomy only when measured performance justifies it.
What the control plan must answer
Start with the workflow, not a generic list of AI principles. Name the business owner, intended users, affected people, systems of record, permitted outcomes and prohibited outcomes. Record the model and service dependencies, but do not make model behavior the primary control. NIST's Generative AI Profile treats risk management as contextual across design, deployment, operation and decommissioning. The control plan should therefore cover the complete lifecycle and remain reviewable as tools, models, policies and business use change.
- Purpose: Which measurable business outcome is the agent allowed to pursue?
- Authority: On whose behalf does it act, and how is delegated authority verified?
- Data: Which sources and fields can it read, retain, infer or disclose?
- Actions: Which tools exist, and which arguments, rates and destinations are allowed?
- Oversight: Which conditions pause execution, who may approve and what must they see?
- Evidence: Which events, versions, inputs, outputs and decisions must be retained?
- Recovery: How is work cancelled, retried, compensated, corrected or escalated?
- Change: Who approves new models, prompts, tools, permissions and autonomy levels?
Classify actions by impact and reversibility
A single label such as safe or unsafe is too vague. Classify each tool operation using the sensitivity of data, scope of effect, financial or rights impact, reversibility, time to detect harm and ease of recovery. The same verb can occupy different tiers: drafting a refund note is not issuing a refund; updating an internal tag is not changing a customer's access. Classify the concrete operation, not the tool's marketing name.
| Tier | Typical operation | Default execution rule | Required safeguards |
|---|---|---|---|
| 0: Observe | Read an authorized record or calculate a value | May execute automatically | Field-level access, purpose limits, logging and rate limits |
| 1: Draft | Prepare a reply, recommendation or change set | No external effect until a person submits it | Evidence display, provenance and clear draft status |
| 2: Reversible action | Create an internal task or update a low-impact status | May execute within narrow policy after validation | Idempotency, change limits, notification and rollback |
| 3: Consequential action | Send externally, change money or alter access | Deterministic approval before execution | Authorized approver, preview, reason, strong authentication and audit event |
| 4: Prohibited | Action outside purpose or beyond acceptable risk | Tool is absent or policy always denies | Denial logging, alerting and periodic attempted-use review |
Build a control plane around the agent
| Control component | Decision it owns | Evidence it emits |
|---|---|---|
| Identity broker | Which human, service and agent identity is involved | Principal, delegated scope, authentication method and session |
| Tool registry | Which operations exist and their schemas and risk tiers | Tool version, operation, validated arguments and destination |
| Policy decision point | Whether this principal may perform this operation now | Allow or deny result, matched rule and policy version |
| Approval service | Whether an accountable person authorizes a paused action | Approver, displayed preview, decision, reason and timestamp |
| Execution gateway | How the validated action reaches the target system | Request ID, idempotency key, response and resulting state |
| Trace and incident service | How events are correlated, retained and escalated | Run timeline, alerts, exception owner and response actions |
This separation makes the agent replaceable. The model may change, but identity, policy and execution remain stable contracts. It also applies zero-trust reasoning: every action is evaluated using current identity, resource and context rather than trusted because it originated inside an agent loop. The broader application pattern is covered in zero trust for business applications.
Enforce least privilege and least action
OWASP describes excessive agency as damage enabled by too much functionality, permission or autonomy. Address all three. Remove operations the workflow does not need; issue an agent-specific identity rather than borrowing a broad user token; restrict records, fields, destinations and transaction size; and require approval where autonomy is not justified. Microsoft recommends starting with no permitted actions and enabling capabilities incrementally, while AWS guidance applies permission boundaries to agentic workflows. These controls belong in IAM, gateways and target systems, not only in prompt instructions.
- Separate read, draft and execute operations even when a vendor API combines them.
- Issue short-lived credentials for one agent, environment and purpose; never share administrator credentials.
- Constrain resource scope, fields, destinations, amounts, frequency and operating windows.
- Validate tool arguments against schemas and business rules before authorization.
- Use idempotency keys for writes so retries do not duplicate effects.
- Require target systems to enforce policy; an orchestration-layer check alone is insufficient.
- Review unused permissions and denied calls, then retire access that the workflow does not need.
Design approvals that support real judgment
A human-in-the-loop control is effective only if it is deterministic and informative. Microsoft explicitly recommends enforcing review for high-risk or irreversible actions through orchestrator logic rather than model reasoning. The reviewer should see the intended goal, affected records, proposed change, source evidence, policy trigger, alternatives, known uncertainty and whether the action can be reversed. Approval must bind to an immutable action digest so the system cannot execute altered arguments after consent. Expire approvals when data, policy or the change set changes.
Route decisions to roles with competence and authority, not merely whoever is available. Provide reject, edit, request-more-information and escalate outcomes. Track approval quality as well as speed: habitual approval without inspection is automation bias, while long queue times may push users to bypass the system. Applicable regulation may impose specific obligations. For example, the EU AI Act's requirements for high-risk systems include risk management, logging and effective human oversight. Organizations should obtain jurisdiction- and use-case-specific legal advice rather than assume every agent has the same classification.
Capture an audit trail that can reconstruct the run
| Event group | Minimum fields | Why it matters |
|---|---|---|
| Request | Run ID, purpose, initiating principal, time and workflow record | Connects the agent run to accountable business work |
| Configuration | Model, prompt, tool catalog, policy and retrieval versions | Makes behavior reproducible enough to investigate change |
| Evidence access | Source identifiers, authorization result, freshness and retrieved segments | Shows what information influenced the proposal |
| Tool proposal | Operation, validated arguments, risk tier and model response reference | Separates proposed intent from executed effect |
| Approval | Preview digest, approver, decision, reason and time | Demonstrates meaningful authorization |
| Execution | Target request, idempotency key, result and before-or-after reference | Supports reconciliation and recovery |
| Outcome | Success criteria, user correction, exception and final owner | Links technical activity to business quality |
Do not indiscriminately log private prompts, hidden credentials or sensitive source content. Define field-level redaction, access, retention, integrity protection and deletion rules. The objective is sufficient evidence, not maximum data. Logs should be tamper-evident, time-correlated and accessible to authorized operations and incident responders without exposing them to every developer.
Threats, failures and control responses
| Scenario | Preventive control | Detective or recovery control |
|---|---|---|
| Indirect prompt injection in an email or document | Untrusted-content boundary, narrow tools and policy enforcement | Injection telemetry, denied-call alert and affected-run review |
| Agent attempts a broader resource scope | Agent identity, field and resource restrictions | Authorization denial trend and permission review |
| Duplicate action after timeout | Idempotency key and transactional outbox | Reconciliation job and duplicate-effect alert |
| Approval for one change reused for another | Bind approval to action digest and expiry | Gateway rejects mismatch and records security event |
| Model or prompt update changes behavior | Versioned configuration and pre-production regression suite | Canary comparison, rollback and incident review |
| Agent loops or consumes resources without progress | Turn, time, tool and spend budgets | Loop detection, cancellation and visible exception queue |
Practical example: a supplier bank-detail change
A finance inbox agent may read a supplier request, find the vendor record and draft a change case. It should not update bank details. The request itself is untrusted content and could contain injected instructions; the proposed change affects money and is difficult to detect after payment. The agent can extract claimed fields, compare them with the supplier master, flag inconsistencies and request independent verification. A finance approver then sees the original evidence, verification status and exact before-and-after values. A separate execution identity performs the update only after approval, and the system notifies a second channel and records the change.
This example shows why control plans are operation-specific. A broad updateVendor tool creates unnecessary exposure. A createBankChangeCase tool can collect structured evidence without possessing the authority to alter payment data. The final update operation can be isolated behind strong authentication, separation of duties and a policy that no model can bypass.
Test controls as adversarial system behavior
- Attempt every tool with an unauthorized agent, user, record, field and destination.
- Place conflicting and malicious instructions in retrieved documents, emails and tool outputs.
- Change action arguments after approval and confirm the execution gateway denies them.
- Inject timeouts before and after writes to verify idempotency and reconciliation.
- Remove or stale a required source and verify the agent asks, stops or escalates instead of guessing.
- Exhaust turn and spend budgets and confirm the workflow enters an owned exception state.
- Run regression tests for every model, prompt, policy, schema and tool version change.
- Exercise incident response, credential revocation, rollback and evidence export with operations staff.
Roll out the control plan in five gates
- Gate 1, inventory: approve purpose, owner, data map, action catalog, prohibited outcomes and success measures.
- Gate 2, pre-production: pass functional, security, privacy and trajectory evaluations with synthetic or controlled data.
- Gate 3, shadow: compare proposals with human decisions while production writes remain disabled.
- Gate 4, assisted: allow drafts and approval-required actions for a trained cohort with daily exception review.
- Gate 5, bounded autonomy: enable one reversible operation, then review outcomes, denied actions, incidents, cost and user bypasses before expansion.
Use NIST's Govern, Map, Measure and Manage functions as a review rhythm rather than a one-time checklist. Governance sets ownership and tolerance; mapping documents context and affected parties; measurement tests risks and controls; management prioritizes treatment, monitoring and response. Maintain an agent registry containing purpose, owner, environments, identities, tools, data sources, risk tier, evaluation status and retirement date.
Key takeaways
- Classify concrete operations by impact and reversibility before granting credentials.
- Enforce identity, policy, schemas and approval in deterministic services outside the model.
- Reduce functionality, permissions and autonomy together to limit excessive agency.
- Bind approvals to an exact change and give reviewers enough evidence to decide.
- Capture reconstructable, privacy-aware traces across request, evidence, decision and effect.
- Test hostile content, partial failure, retries and change regressions before increasing autonomy.
FAQ: Can a strong system prompt control an AI agent?
No. Instructions influence model behavior but cannot replace authorization, data controls, schema validation, rate limits or approval gates. Treat model output as an untrusted proposal that must pass deterministic controls before execution.
FAQ: What belongs in an AI agent audit log?
Record the accountable request, configuration versions, evidence references, proposed operation, policy decision, approval, execution result and business outcome. Redact secrets and minimize sensitive content. Retention and access should follow the workflow's legal, security and operational requirements.
FAQ: When can an agent act without approval?
Automatic execution is most defensible for narrow, low-impact, reversible actions with reliable detection, tested recovery and stable quality. Approval may still be required by policy or law. Autonomy is a per-operation decision, not a permanent label applied to the entire agent.
FAQ: Who owns the agent control plan?
The business workflow owner should be accountable for purpose and outcomes, with engineering, security, data, privacy, legal or compliance and operations contributing controls in their domains. Name one decision owner and document approval responsibilities so gaps do not hide between committees.
Conclusion
A trustworthy agent is not one that promises to behave; it is one whose authority is technically bounded, whose consequential actions are meaningfully reviewed and whose effects can be reconstructed and managed. Build the control plane before broad tool access, test it under hostile and failing conditions, and let evidence justify each autonomy increase. Readers who need the underlying agent architecture can continue with how AI agents work in business workflows. Edilec's AI automation services can translate a control plan into tool contracts, approval states, evaluation cases and production telemetry.