Taking AI Agents to Production with Closed-Loop Evaluations That Work

In this blog post Taking AI Agents to Production with Closed-Loop Evaluations That Work we will explain how CIOs can keep AI agents accurate, safe and commercially useful after they move beyond the pilot stage.

The uncomfortable reality is that an AI agent can perform well during a controlled demonstration and still fail in daily business use. Employees ask unexpected questions, company information changes, connected systems behave differently and seemingly minor updates can create new risks.

Closed-loop evaluation addresses this problem by creating a continuous cycle. The organisation measures what the agent is doing, identifies failures, improves the agent, tests the change and then monitors the updated version in production.

What is a closed-loop evaluation?

An evaluation is a structured test of whether an AI agent produced the right outcome. It goes beyond asking whether the response sounded professional.

For example, did the agent complete the requested task? Did it use reliable company information? Did it follow the approved process? Did it avoid exposing sensitive data or taking an action outside its authority?

The “closed-loop” part means those results are fed back into the way the agent is managed. Production activity becomes evidence for the next round of testing and improvement rather than disappearing into a technical log that nobody reviews.

Microsoft Foundry provides the platform for running and managing these evaluations. It can capture agent activity, score selected interactions and help teams investigate the steps an agent took before producing an answer or completing an action.

How the technology works in plain English

A Microsoft Foundry agent normally combines an AI model with instructions, approved business information and tools. Those tools might search SharePoint, check a customer record, create a service ticket or start an approval process.

Foundry can record a trace, which is effectively a step-by-step history of what happened. It can show the request, information retrieved, tools called, response generated, time taken and resources consumed.

Evaluators then score that activity against defined criteria. Built-in evaluators can measure areas such as task completion, relevance, groundedness and safe behaviour. Groundedness simply means whether the answer is supported by the approved information available to the agent rather than being invented.

Organisations can also create custom scoring rules. A finance agent, for example, might be tested on whether it confirmed the supplier, matched the purchase order and requested human approval before processing an exception.

This builds on the hosting, access control and monitoring foundations covered in our guide to moving an AI agent from prototype to production. Closed-loop evaluation is what helps maintain that service once real employees and customers begin using it.

The five parts of an effective evaluation loop

1. Define success as a business outcome

Many AI projects begin with vague goals such as “improve response quality”. That is difficult to measure and even harder to govern.

A better target would be: resolve 80% of standard IT requests without escalation, while maintaining approved security checks and reducing the average handling cost by 25%.

Each evaluation should connect to an outcome the business understands, such as cases resolved, minutes saved, errors avoided, customer satisfaction or cost per transaction.

2. Test realistic situations before release

Pre-production testing should include normal requests, incomplete information, unusual wording, conflicting instructions and attempts to make the agent break policy. Testing only the happy path creates false confidence.

Microsoft Foundry can assess individual answers or complete conversations. The second option matters because an agent may give several reasonable responses but still fail to complete the customer’s overall task.

Representative test cases should come from real business processes, with personal or confidential details removed. Synthetic cases can fill gaps, but they should not completely replace examples reviewed by employees who understand the process.

3. Set clear release gates

An agent should not be released simply because the project team believes it has improved. Agree on minimum standards before approving a change.

An illustrative release policy might look like this:

Task completion rate 90% or higher
Grounded answers 95% or higher
Critical safety failures Zero
Average cost per case Below $1.20
High-risk actions Human approval required

These are business controls, not just technical measures. They give the CIO, risk owner and process owner a common basis for deciding whether a release is acceptable.

4. Evaluate a controlled sample of production activity

Testing does not stop when the agent goes live. A selected sample of production conversations can be evaluated continuously or on a schedule, depending on risk, volume and cost.

The monitoring view can bring together quality scores, response times, failures and token consumption, which is one of the main drivers of AI usage costs. Alerts can then flag falling completion rates, unusual spending or unsafe outputs before the problem becomes widespread.

Not every interaction needs the same level of review. A low-risk internal knowledge assistant may only require sampling, while an agent handling financial approvals or personal information needs much tighter oversight.

5. Turn failures into controlled improvements

When an evaluation fails, the team needs to identify the cause. The problem may be outdated source material, unclear instructions, excessive permissions, poor tool selection or an unrealistic user request.

The fix is tested against the existing evaluation set and previous failures. This regression testing checks that solving one problem has not reintroduced an old issue or damaged another part of the service.

The change then passes through the governance boundaries described in our article on governing Microsoft Foundry AI agents in production. An evaluation result should inform a release decision, not allow an agent to rewrite and redeploy itself without supervision.

A practical business scenario

Consider a 180-person professional services firm using an agent to answer internal policy questions and prepare draft client communications. During the pilot, employees reported that it was fast and helpful.

Production evaluations later showed that responses based on older SharePoint documents were creating inconsistent advice. The agent also produced unnecessarily long answers, increasing usage costs and slowing employees down.

The company introduced three controls: only approved policy libraries could be searched, answers had to identify the policy date and responses were assessed against a concise company writing standard.

After the next release, the business could measure fewer escalations, shorter responses and more consistent policy guidance. The value did not come from simply deploying a more powerful model. It came from finding failure patterns and closing the improvement loop.

What CIOs should ask before approving production use

• Who defines what a successful result looks like?
• Which failures would create financial, privacy or reputational harm?
• Are complete tasks being assessed, or only individual answers?
• How often is production activity sampled and reviewed?
• Who investigates failures and approves changes?
• Can the business compare quality and cost between versions?
• Are sensitive details removed or protected in evaluation records?
• Are any preview features supporting controls that require a production guarantee?

For Australian organisations, evaluation data should be managed like other potentially sensitive business records. Access should be restricted, retention periods should be defined and personal information should not be collected merely because the technology allows it.

Closed-loop evaluations also support the intent of the Essential Eight, the Australian Government’s cybersecurity framework, by encouraging controlled changes, limited access and reliable monitoring. However, evaluations are not a substitute for Essential Eight controls, privacy assessments or human accountability.

Make AI performance visible to the business

The goal is not to produce another technical dashboard. It is to give decision-makers evidence that an AI service remains useful, controlled and worth its operating cost.

CloudProInc combines more than 20 years of enterprise IT experience with hands-on Microsoft Foundry, Azure, Microsoft 365, OpenAI, Claude, Defender and Wiz expertise. As a Melbourne-based Microsoft Partner and Wiz Security Integrator, we help organisations connect AI evaluation to practical security, governance and financial controls.

If you are unsure whether your AI agent is genuinely ready for production—or whether anyone would notice when its performance starts to decline—we are happy to review your evaluation approach with you, no strings attached.