Building Human Approval into High-Risk AI Agent Actions Safely

In this blog post Building Human Approval into High-Risk AI Agent Actions Safely we will explain how to let AI agents complete routine work while requiring a person to approve actions that could create serious cost, security, privacy or operational consequences.

At a high level, an AI agent is different from a chatbot. A chatbot usually answers a question, while an agent can use business systems and tools to perform tasks such as sending emails, changing accounts, processing invoices or updating customer records.

That ability creates the problem. An agent can make a reasonable-looking mistake and carry it out before anyone notices. If the action involves money, sensitive data or access to critical systems, one incorrect decision can quickly become a costly incident.

Human approval should be a control, not a bottleneck

Human approval means the agent prepares an action but cannot execute it until an authorised person approves it. The workflow pauses, presents the relevant information and continues only after receiving a valid decision.

The aim is not to put a person in front of every task. That would remove most of the productivity benefit. Approval should be reserved for actions where the potential impact is greater than the cost of waiting.

This builds on the practical process covered in adding human approval to Microsoft Foundry agent workflows. Here, the focus is deciding which actions need approval and designing that control so it works under real business pressure.

Start by defining what high risk means

Many organisations begin with a vague rule such as “ask for approval when the agent is uncertain.” That is not enough. AI confidence can be inconsistent, and a confident answer can still be wrong.

A better approach is to classify risk according to business impact. Approval should normally be required when an action is:

• Financially significant, such as paying an invoice, issuing a refund or committing to a supplier contract.
• Difficult to reverse, such as deleting records, cancelling a service or publishing information externally.
• Security-sensitive, such as creating an administrator account or changing access permissions.
• Privacy-sensitive, such as sharing employee, customer, health or financial information.
• Externally visible, such as sending a legal notice, customer communication or public statement.
• Outside normal patterns, such as a payment to a new bank account or an unusually large data export.

Before production deployment, these rules should be documented as part of an AI agent risk assessment. This gives executives, system owners and security teams a shared definition of acceptable automation.

Show the approver what they actually need

An approval request saying “Approve this agent action?” is almost useless. Busy managers will approve it without understanding the consequences, creating the appearance of oversight rather than meaningful control.

A useful approval request should show:

• What the agent wants to do.
• Why it believes the action is required.
• Which customer, employee or system will be affected.
• The financial value or security impact.
• The source information used to reach the decision.
• Whether the action can be reversed.
• What will happen if the request is rejected or expires.

Approvers should be able to approve, reject or request changes. If they modify the action, the revised details should be recorded rather than silently replacing the original request.

Separate the AI recommendation from execution

The safest design treats recommendation and execution as two different steps. The agent can prepare a payment, draft an email or propose an access change, but a controlled service performs the final action only after approval.

A simplified policy might look like this:

risk = assess_business_risk(proposed_action)

if risk == "high":
 pause_workflow()
 request_approval(
 action=proposed_action,
 approver=authorised_owner,
 expiry="24 hours"
 )
else:
 execute_with_limits(proposed_action)

The technology behind this is straightforward. The agent proposes a tool call, which is a structured request to use another system. An approval layer intercepts the request, saves the workflow state and sends the details to an authorised reviewer through an application such as Microsoft Teams or Power Automate.

Once the reviewer responds, the system checks their identity and authority before allowing the workflow to continue. The agent itself should never be able to approve its own action or bypass the approval service.

Use identity and permissions as the real enforcement point

A prompt telling an agent to “always ask permission” is not a security control. Prompts can be misunderstood, manipulated or overridden by untrusted content.

This risk is particularly important when agents read emails, documents, websites or code. Malicious instructions can be hidden in that content, as discussed in our article on security risks affecting Claude Code users.

The approval requirement must therefore be enforced outside the AI model. The agent’s technical identity should have only the minimum access needed, and high-risk tools should reject any request that does not include a current, verifiable approval.

The approver must also be suitable for the action. A service desk employee might approve a standard software request, but should not approve their own administrator access. Large payments may require two approvers from different teams.

These controls support the intent of Essential 8, the Australian government’s cybersecurity framework that many organisations use to reduce common attack paths. Human approval does not replace controls such as multi-factor authentication, restricted administrator access, patching and application control. It adds another layer around AI-driven actions.

Make sure paused workflows resume safely

An approval might take five minutes or two days. During that time, systems can restart, information can change and the same request can accidentally be submitted twice.

A production workflow should preserve its state, set an expiry time and confirm that the underlying facts remain valid before execution. A bank account change, for example, should be checked again after approval rather than relying on old data.

The workflow also needs protection against duplicate execution. If an approver clicks twice or a system retries after a network problem, the payment or account change should still happen only once. Our guide to resumable AI agent workflows explains this pause-and-resume pattern in more detail.

Consider a practical accounts payable scenario

Imagine a 200-person company using an AI agent to process supplier invoices. The agent reads invoices, checks purchase orders and prepares payments, reducing hours of manual data entry each week.

Matched invoices below $1,000 from established suppliers could proceed automatically. An invoice above $10,000, a changed bank account or a supplier without a valid purchase order would pause for finance approval.

The finance manager would see the invoice, purchase order, supplier history, bank account status and the reason for escalation. If approved, the payment service would execute the transaction and record the result.

This model preserves the productivity gain while placing people around the small number of decisions capable of causing major financial loss. It also produces clear evidence for audits and incident investigations.

Keep a reliable approval record

For every high-risk action, record what was proposed, which policy triggered approval, who reviewed it, what they decided and what was ultimately executed. Logs should also capture the agent and policy version so changes can be traced.

Do not store unnecessary personal or confidential information in approval messages and logs. Australian privacy obligations still apply when AI systems handle personal information, and approvals should reveal only what the reviewer needs to make the decision.

Monitoring matters too. Repeated rejections may indicate poor instructions, unreliable data or an attempted attack. Approval rates and waiting times can also show where rules should be adjusted without weakening safeguards.

A practical rollout plan

1. List every action the agent can perform, not just the answers it can generate.
2. Rate each action by financial, security, privacy and operational impact.
3. Define clear approval triggers and authorised approvers.
4. Enforce approval outside the model using identity and access controls.
5. Design expiry, rejection, escalation and recovery paths.
6. Test manipulation attempts, duplicate requests and unavailable approvers.
7. Review approval records regularly and refine the thresholds.

CloudProInc applies these principles across Microsoft Foundry, Azure, Microsoft 365, OpenAI and Anthropic Claude environments. Our experience as a Microsoft Partner and Wiz Security Integrator helps connect AI workflow design with the identity, device and cloud security controls surrounding it.

The goal is not to make AI agents harmless by preventing them from doing useful work. It is to let them act quickly inside clear boundaries while keeping people responsible for decisions that can materially affect the business.

If you are unsure which AI actions your organisation should automate and which should require approval, CloudProInc can review the workflow and its security controls. We are a practical Melbourne-based consultancy serving organisations across Australia and internationally, and we are happy to take an initial look with no strings attached.