In this blog post Building Audit Ready AI Agents with Azure Cosmos DB Securely we will explain how to design AI agents that do useful work without becoming a compliance headache.
Many organisations are moving from AI experiments to AI agents that can search internal knowledge, summarise customer records, create tickets, update workflows, and help staff make decisions. That is exciting, but it also creates a simple leadership problem: if the agent gives a wrong answer, accesses the wrong data, or changes something it should not have changed, can you prove what happened?
That is where audit-ready design matters. It is not enough for an AI agent to be fast or impressive in a demo. It needs a trustworthy record of what it saw, what it did, who asked it, what data it used, what tools it called, and whether a human approved the action.
Azure Cosmos DB is useful here because it can act as the operational memory and evidence store for AI agents. In plain English, Azure Cosmos DB is Microsoft’s cloud database for applications that need to store flexible data, search it quickly, scale as usage grows, and support modern AI patterns such as vector search, which means finding information by meaning rather than only by exact keywords.
CloudProInc has been helping organisations make Microsoft cloud systems secure and practical for more than 20 years. As a Microsoft Partner and Wiz Security Integrator based in Melbourne, we often see the same pattern: the AI idea is promising, but the governance trail is an afterthought. That is fixable if you design the data layer properly from day one.
Why AI agents need an audit trail
For a normal chatbot, a bad answer is annoying. For an AI agent connected to business systems, a bad action can create financial, privacy, or operational risk.
Imagine an internal finance agent that reviews supplier invoices and recommends which ones should be paid. A manager asks why a $42,000 payment was flagged as low risk. If the only answer is “the AI said so”, the business has a problem.
An audit-ready agent should be able to show the source documents it checked, the policy version it used, the confidence score, any exceptions, and whether a person approved the final action. That does not remove all risk, but it gives leaders evidence they can review, improve, and explain.
This is especially important for Australian organisations working toward Essential 8, the Australian government’s cybersecurity framework that many organisations now use as a practical baseline. Essential 8 does not magically make AI safe, but it pushes the right behaviours: access control, patching, application control, logging, backups, and reducing the chance that one compromised account can do serious damage.
What Azure Cosmos DB does in an AI agent architecture
Think of Azure Cosmos DB as the agent’s working memory and filing cabinet.
It can store structured business records, such as customer IDs, case numbers, workflow status, and approval history. It can also store unstructured information, such as document snippets, conversation history, extracted facts, and AI-generated summaries.
Most importantly for modern AI, Azure Cosmos DB can store embeddings. An embedding is a numerical fingerprint that represents the meaning of a piece of text. This lets an AI agent find relevant information even when the user does not type the exact words used in the document.
For example, a staff member might ask, “Can we give this client extended payment terms?” The agent may need to search credit policy, customer history, contract notes, and recent account activity. Keyword search alone may miss important context. Vector search helps find related material by meaning, while traditional search still helps match exact terms such as customer names, invoice numbers, and policy codes.
Azure Cosmos DB now supports this type of combined search pattern, which is useful because it reduces the need for a separate vector database. Fewer systems usually means lower integration cost, simpler security, and fewer places for sensitive data to leak.
If you want the broader infrastructure view, we covered that in Designing Secure AI Agent Infrastructure on Azure. This article goes one layer deeper into the database and audit design.
The business outcome is evidence, not just automation
The biggest mistake we see is treating the AI agent’s database as just a place to store chat history. That is too narrow.
For an audit-ready agent, the database should capture four types of information:
- Business context: the customer, employee, supplier, case, project, or asset involved.
- Agent memory: what the agent has learned or been allowed to remember for future interactions.
- Retrieval evidence: which documents, records, or knowledge items were used to produce the answer.
- Action history: what the agent recommended, what it changed, which system it called, and who approved it.
This gives CIOs and CTOs something very practical: a way to answer the uncomfortable questions that always come later.
Who accessed that data? Why did the agent recommend that action? Was sensitive information included in the prompt? Did the agent use an outdated policy? Did a human approve the final step? Can we delete or retain records according to policy?
A practical Cosmos DB design for audit-ready agents
A simple design might use separate Cosmos DB containers, which are like organised collections of related records. Keeping these separate helps with security, cost control, retention, and reporting.
- AgentSessions: one record per user interaction or workflow run.
- AgentMessages: user prompts, agent responses, system instructions, and safety checks.
- KnowledgeItems: approved document chunks, policies, procedures, and their embeddings.
- ToolCalls: every API, database, ticketing, finance, or HR system call made by the agent.
- Approvals: human review steps, decisions, exceptions, and sign-off records.
- AuditEvents: security and operational events written in a consistent format.
The key is to design these records around the questions the business will need to answer, not just the needs of the developer building the agent.
Example audit event
Here is a simplified example of what an audit event could look like. The goal is not to impress anyone with code. The goal is to make sure every important action leaves a clear record.
{
"id": "audit-20260714-00192",
"sessionId": "session-88421",
"agentName": "supplier-risk-agent",
"userId": "manager@company.com.au",
"businessObject": {
"type": "supplierInvoice",
"id": "INV-10488"
},
"eventType": "recommendation_created",
"dataUsed": [
"supplier-policy-v4",
"invoice-INV-10488",
"supplier-history-ACME"
],
"decision": "manual_review_required",
"reason": "Invoice amount exceeds normal approval threshold",
"humanApprovalRequired": true,
"timestampUtc": "2026-07-14T02:18:44Z"
}This type of record helps your team prove that the agent did not simply “make a decision”. It checked defined sources, applied a rule, created a recommendation, and required a person to review it.
Use Cosmos DB change feed for near real-time oversight
Azure Cosmos DB includes a feature called change feed. In plain English, change feed is a running list of changes made to a database container, in the order they happened.
That matters because audit data should not sit quietly until someone remembers to check it. Change feed can send important events to monitoring, reporting, security tools, or a data platform for compliance review.
For example, if an agent suddenly starts making far more tool calls than normal, accessing records outside its usual business area, or repeatedly triggering failed permission checks, the security team should know quickly. This is where Microsoft Defender, Azure Monitor, Microsoft Sentinel, and Wiz can become part of the operating model rather than an afterthought.
As a Wiz Security Integrator, CloudProInc often looks at this from both sides: how the application is designed, and whether the cloud environment around it exposes unnecessary risk.
Design access so the agent only sees what it needs
One of the most important security principles for AI agents is least privilege. That means the agent should only have access to the data and actions it genuinely needs to do its job.
In Azure, this usually means using Microsoft Entra ID, which is Microsoft’s identity platform, and managed identities, which let Azure services authenticate without storing passwords or secrets in code. Instead of giving an agent a broad database key, you assign it a specific identity and tightly controlled permissions.
For leaders, the business benefit is simple. If the agent is compromised, misconfigured, or tricked by a malicious prompt, the blast radius is smaller.
This also supports Essential 8 thinking. Strong identity, controlled administrative access, logging, and recoverability all matter when AI agents are allowed anywhere near operational systems.
Separate memory from evidence
Agent memory is useful, but it can also become risky if it stores too much for too long.
For example, an HR agent may need to remember that a manager prefers short summaries. It probably should not remember sensitive employee medical details unless there is a clear lawful reason, retention policy, and access model.
A good design separates personalisation memory from audit evidence. Personalisation memory can often be short-lived and minimal. Audit evidence may need longer retention, stronger controls, and restricted access.
Azure Cosmos DB supports time to live, often called TTL. This automatically deletes records after a defined period. In plain English, it helps you avoid keeping data forever just because nobody built a cleanup process.
Control cost before the pilot becomes production
AI agent costs can surprise businesses because usage grows in small invisible ways. More prompts, more retrieved documents, more stored messages, more search queries, and more logs all add up.
Azure Cosmos DB uses a capacity model based on request units. A request unit is a way of measuring the database work needed to read, write, or query data. Leaders do not need to understand every detail, but they should ask whether the design has cost guardrails.
Good guardrails include limiting how much conversation history is stored, archiving old audit records, using sensible partition keys, and testing search patterns before production. A partition key is the field Cosmos DB uses to spread data efficiently. Choosing the wrong one can make the system slower and more expensive as it grows.
This is where practical architecture matters. The cheapest AI pilot is not always the cheapest production system.
A real-world scenario
Consider a 180-person professional services firm with teams in Melbourne, Sydney, and Singapore. They want an AI agent to help staff find project documents, summarise client history, and draft internal task updates.
The first prototype works well, but it stores everything in one database collection with broad access. There is no clear record of which documents informed each answer. Sensitive client notes appear in general conversation history. Nobody has defined how long agent memory should be retained.
A more audit-ready design uses Cosmos DB for approved knowledge, session records, tool calls, and audit events. The agent can search by meaning, but only across documents the user is allowed to access. Every answer records the source material used. Higher-risk actions, such as updating a client record, require human approval.
The business outcome is not just a smarter assistant. It is faster document retrieval, fewer repeated admin tasks, lower privacy risk, and a cleaner story for internal governance or client assurance reviews.
Five practical steps before you build
- Define what must be explainable. List the decisions, recommendations, and actions that need an evidence trail.
- Classify the data. Separate public, internal, confidential, personal, and regulated data before connecting it to an agent.
- Design the audit schema early. Decide what every session, tool call, approval, and exception must record.
- Use identity-based access. Avoid broad keys where possible and give each agent only the permissions it needs.
- Test retrieval quality and security together. A fast answer is not useful if it uses the wrong data or ignores access rules.
If you are already exploring Microsoft Agent Framework, .NET, OpenAI, or Claude-based agents, this Cosmos DB audit layer fits naturally with those patterns. We touched on production build considerations in Build Production AI Agents with Microsoft Agent Framework and .NET and safe automation patterns in How Azure AI Agents Automate Repetitive Business Processes Safely.
Final thought
AI agents will become part of normal business systems, not just side experiments. The organisations that get the most value will be the ones that can trust, monitor, and explain what their agents are doing.
Azure Cosmos DB gives you a strong foundation for agent memory, retrieval, audit events, and operational evidence. But the technology only works well when the design reflects business risk, compliance needs, and real user behaviour.
If you are not sure whether your AI agent design is audit-ready, or whether your current Azure setup will stand up to security and compliance scrutiny, CloudProInc is happy to take a look. No pressure, no scare tactics, just practical advice from a Melbourne-based team that builds and secures these systems every day.
Discover more from CPI Consulting
Subscribe to get the latest posts sent to your email.