{"id":59774,"date":"2026-11-13T07:01:53","date_gmt":"2026-11-12T21:01:53","guid":{"rendered":"https:\/\/www.cloudproinc.com.au\/index.php\/2026\/11\/13\/running-ai-workloads-on-aks-with-distributed-storage-caching\/"},"modified":"2026-11-13T07:03:34","modified_gmt":"2026-11-12T21:03:34","slug":"running-ai-workloads-on-aks-with-distributed-storage-caching","status":"publish","type":"post","link":"https:\/\/www.cloudproinc.com.au\/index.php\/2026\/11\/13\/running-ai-workloads-on-aks-with-distributed-storage-caching\/","title":{"rendered":"Running AI Workloads on AKS with Distributed Storage Caching"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">In this blog post Running AI Workloads on AKS with Distributed Storage Caching we will explain why expensive AI infrastructure can sit idle while model files are still loading, and how a distributed cache can bring frequently used data closer to your Azure Kubernetes Service workloads.<\/p>\n\n\n\n<!--more-->\n\n\n\n<p class=\"wp-block-paragraph\">At a high level, Azure Kubernetes Service, or AKS, runs AI applications across groups of cloud servers. Those applications often need to retrieve very large model files or datasets from Azure Blob Storage before they can start useful work. If the same files are downloaded after every restart, update or scaling event, the business pays for powerful graphics processing units, or GPUs, that are waiting rather than processing requests.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Distributed storage caching addresses this problem by temporarily keeping frequently requested data on fast disks inside the AKS cluster. Azure Blob Storage remains the permanent source of truth, while the cache provides a shorter path to data that applications repeatedly need.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Why storage becomes an AI cost problem<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Traditional business applications might load a few hundred megabytes when they start. A production AI model can require tens or hundreds of gigabytes of model weights, which are the files containing what the model learned during training.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">When one AI application starts, downloading those files may be manageable. The problem appears when several application instances start together, failed workloads restart or the business adds more GPU capacity to handle demand.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Every new instance may request the same files from remote storage. This can lead to slower deployments, repeated storage traffic and unpredictable recovery times during an incident.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The commercial issue is simple. GPU servers are among the most expensive resources in an AI platform. A ten-minute storage delay across multiple GPUs is not merely an IT inconvenience; it is paid computing capacity producing no business value.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How distributed storage caching works<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Microsoft\u2019s Distributed Accelerator for Cloud Storage, or DACS, is a read-through cache designed for data-heavy Kubernetes workloads. \u201cRead-through\u201d means the application asks the cache for data first. If the data is not available locally, it is retrieved from Azure Blob Storage and placed in the cache for later requests.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>AI application on AKS\n |\n v\nDistributed cache checks for the requested model data\n |\n +-- Cache hit --&gt; Read from fast local storage\n |\n +-- Cache miss --&gt; Read from Azure Blob Storage\n and retain a temporary cached copy<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The cache is distributed across several cache servers rather than relying on one machine. Each server uses local NVMe storage, which is a fast disk attached directly to the server. The client identifies which cache server should hold a particular piece of data and sends the request there.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This design spreads capacity and traffic across the cache layer. It also means cache capacity can grow as additional cache servers are introduced, subject to careful network and storage planning.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Importantly, the cached copy is temporary. If a cache node disappears, the original model or dataset still exists in Azure Blob Storage. DACS is not a backup platform, a permanent file system or a place to store training results and checkpoints.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What this changes for the business<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">1. Faster AI application startup<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The first request for a model may still need to retrieve data from Blob Storage. Later application instances can reuse the cached data, reducing the time between allocating GPU capacity and serving real requests.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This matters when an AI service must scale quickly, recover after a node failure or complete a rolling update without creating a long period of reduced capacity.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">2. Less paid GPU waiting time<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Consider an illustrative company running eight GPUs for an internal AI service. If each GPU waits ten minutes for model data during two deployment or restart events per day, the company loses roughly 2.7 GPU-hours daily.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Caching will not remove every delay, but recovering even part of that idle time can improve the economics of the platform. The saving becomes more significant as the number and cost of GPU servers increase.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">3. More predictable scaling<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">AI demand is rarely constant. A customer-facing assistant may become busy during business hours, while document processing may peak after large file uploads.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">When scaling depends on repeatedly downloading large models from remote storage, adding GPU capacity does not guarantee an immediate increase in usable capacity. A warm cache, meaning the required data is already present, can make scaling behaviour more consistent.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">4. Reduced repeated storage traffic<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Repeatedly transferring identical model files creates unnecessary activity between storage and compute. A cache can reduce those repeated reads when applications share the same models or training jobs reuse portions of the same dataset.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The result may be lower storage-related traffic and less pressure on the remote storage path. However, the cache has its own server, disk and management costs, so the correct goal is lower total cost rather than simply lower Blob Storage usage.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Where distributed caching will not help<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Caching delivers value when data is reused. If every workload reads different information once and never requests it again, the cache may add cost and complexity without producing enough cache hits.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It is also important not to confuse storage caching with other AI caches. A model-data cache accelerates access to model files and datasets. It does not replace a container image cache, and it is not a large language model KV cache, which temporarily stores information created while generating a response.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Capacity must be based on the active working set, meaning the models and dataset portions used regularly, rather than every file the organisation owns. Leaders should also allow space for overlapping model versions during upgrades and for data to be removed when the cache becomes full.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Security and preview limitations need attention<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">At the time of writing, Microsoft lists DACS as a preview capability without a production service-level agreement. That makes it suitable for controlled evaluation, but organisations should be cautious about placing it directly under a critical production service without fallback and recovery planning.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The current security model also treats the AKS cluster as one trusted environment. Workloads that can reach the cache may be able to access cached information, so it should not be treated as a security boundary between unrelated customers, departments or tenants.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For Australian organisations, this deserves particular attention when model data contains personal, financial, health or commercially sensitive information. Access should be limited using Microsoft Entra identities, network controls and tightly governed AKS permissions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Caching also does not replace the Essential Eight, the Australian Government\u2019s cybersecurity framework for reducing common attack risks. Patch management, restricted administration, application controls, monitoring and reliable backups remain necessary.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">A practical assessment before you build<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Before adding distributed caching, ask your platform team or provider to measure the current problem. Useful questions include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>How long do AI workloads wait for model data before becoming ready?<\/li>\n<li>How often are the same models downloaded after restarts or scaling events?<\/li>\n<li>How many paid GPU-hours are lost to storage-related delays?<\/li>\n<li>What percentage of model and dataset reads are repeated?<\/li>\n<li>How much fast cache storage would the active working set require?<\/li>\n<li>What happens when the cache is empty, unavailable or being upgraded?<\/li>\n<li>Can every workload in the cluster be treated as equally trusted?<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Start with a representative model and a non-production AKS environment. Compare cold-cache performance, where nothing has been stored locally, with warm-cache performance. Then test restarts, scale-outs, cache-node loss and fallback to Blob Storage.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If you want a closer look at the Azure-specific implementation, our related article on how DACS reduces AI startup delays and GPU waste on Azure AKS explores its use cases and preview limitations in more detail.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The goal is better GPU use, not more infrastructure<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Distributed storage caching can make AI workloads on AKS faster and more economical when large models or datasets are read repeatedly. The strongest business case usually comes from shorter startup times, more predictable scaling and fewer expensive GPU minutes spent waiting.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It is not automatically the right answer for every AI platform. The likely cache-hit rate, security model, operational overhead and preview status should all be assessed before making it part of a production design.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">CloudProInc is a Melbourne-based Microsoft Partner and Wiz Security Integrator with more than 20 years of enterprise IT experience across Azure, AKS, OpenAI, Claude and cloud security. If you are unsure whether storage delays are increasing the cost of your AI platform, we are happy to review the setup and identify where the real bottleneck sits\u2014no strings attached.<\/p>\n\n\n","protected":false},"excerpt":{"rendered":"<p>Distributed storage caching can reduce AI startup delays and wasted GPU time on AKS. Here is how it works, where it helps and what leaders should assess before investing.<\/p>\n","protected":false},"author":1,"featured_media":59776,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":true,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_hide_featured_image":false,"_jetpack_featured_image_duplicate_dismissed":0,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[24,139,13,124],"tags":[],"class_list":["post-59774","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","category-azure-kubernetes-service","category-blog","category-cloud-cost-finops"],"jetpack-related-posts":[{"id":59758,"url":"https:\/\/www.cloudproinc.com.au\/index.php\/2026\/11\/12\/how-dacs-reduces-ai-startup-delays-and-gpu-waste-on-azure-aks\/","url_meta":{"origin":59774,"position":0},"title":"How DACS Reduces AI Startup Delays and GPU Waste on Azure AKS","author":"CPI Staff","date":"November 12, 2026","format":false,"excerpt":"DACS keeps frequently used AI model data close to Azure GPUs. Learn where it could reduce startup delays, storage traffic and wasted compute\u2014and where its preview limitations matter.","rel":"","context":"In &quot;AI&quot;","block_context":{"text":"AI","link":"https:\/\/www.cloudproinc.com.au\/index.php\/category\/ai\/"},"img":{"alt_text":"","src":"\/wp-content\/uploads\/2026\/11\/how-dacs-reduces-ai-startup-delays-and-gpu-waste-on-azure-aks.png","width":350,"height":200,"srcset":"\/wp-content\/uploads\/2026\/11\/how-dacs-reduces-ai-startup-delays-and-gpu-waste-on-azure-aks.png 1x, \/wp-content\/uploads\/2026\/11\/how-dacs-reduces-ai-startup-delays-and-gpu-waste-on-azure-aks.png 1.5x, \/wp-content\/uploads\/2026\/11\/how-dacs-reduces-ai-startup-delays-and-gpu-waste-on-azure-aks.png 2x, \/wp-content\/uploads\/2026\/11\/how-dacs-reduces-ai-startup-delays-and-gpu-waste-on-azure-aks.png 3x, \/wp-content\/uploads\/2026\/11\/how-dacs-reduces-ai-startup-delays-and-gpu-waste-on-azure-aks.png 4x"},"classes":[]},{"id":59520,"url":"https:\/\/www.cloudproinc.com.au\/index.php\/2026\/11\/03\/how-aks-flex-nodes-extend-azure-clusters-to-your-own-compute\/","url_meta":{"origin":59774,"position":1},"title":"How AKS Flex Nodes Extend Azure Clusters to Your Own Compute","author":"CPI Staff","date":"November 3, 2026","format":false,"excerpt":"AKS Flex Nodes connect your own servers and virtual machines to an Azure Kubernetes cluster. Learn where the preview could reduce costs and where it adds risk.","rel":"","context":"In &quot;Azure&quot;","block_context":{"text":"Azure","link":"https:\/\/www.cloudproinc.com.au\/index.php\/category\/microsoft-azure\/"},"img":{"alt_text":"","src":"\/wp-content\/uploads\/2026\/11\/how-aks-flex-nodes-extend-azure-clusters-to-your-own-compute.png","width":350,"height":200,"srcset":"\/wp-content\/uploads\/2026\/11\/how-aks-flex-nodes-extend-azure-clusters-to-your-own-compute.png 1x, \/wp-content\/uploads\/2026\/11\/how-aks-flex-nodes-extend-azure-clusters-to-your-own-compute.png 1.5x, \/wp-content\/uploads\/2026\/11\/how-aks-flex-nodes-extend-azure-clusters-to-your-own-compute.png 2x, \/wp-content\/uploads\/2026\/11\/how-aks-flex-nodes-extend-azure-clusters-to-your-own-compute.png 3x, \/wp-content\/uploads\/2026\/11\/how-aks-flex-nodes-extend-azure-clusters-to-your-own-compute.png 4x"},"classes":[]},{"id":59381,"url":"https:\/\/www.cloudproinc.com.au\/index.php\/2026\/10\/24\/securing-aks-ingress-why-the-alb-controller-1-12-4-update-matters\/","url_meta":{"origin":59774,"position":2},"title":"Securing AKS Ingress Why the ALB Controller 1.12.4 Update Matters","author":"CPI Staff","date":"October 24, 2026","format":false,"excerpt":"ALB Controller 1.12.4 includes security patches, but upgrading safely requires compatibility checks. Here is what AKS leaders need to review before approving the change.","rel":"","context":"In &quot;Azure&quot;","block_context":{"text":"Azure","link":"https:\/\/www.cloudproinc.com.au\/index.php\/category\/microsoft-azure\/"},"img":{"alt_text":"","src":"\/wp-content\/uploads\/2026\/10\/securing-aks-ingress-why-the-alb-controller-1-12-4-update-matters.png","width":350,"height":200,"srcset":"\/wp-content\/uploads\/2026\/10\/securing-aks-ingress-why-the-alb-controller-1-12-4-update-matters.png 1x, \/wp-content\/uploads\/2026\/10\/securing-aks-ingress-why-the-alb-controller-1-12-4-update-matters.png 1.5x, \/wp-content\/uploads\/2026\/10\/securing-aks-ingress-why-the-alb-controller-1-12-4-update-matters.png 2x, \/wp-content\/uploads\/2026\/10\/securing-aks-ingress-why-the-alb-controller-1-12-4-update-matters.png 3x, \/wp-content\/uploads\/2026\/10\/securing-aks-ingress-why-the-alb-controller-1-12-4-update-matters.png 4x"},"classes":[]},{"id":59598,"url":"https:\/\/www.cloudproinc.com.au\/index.php\/2026\/11\/06\/how-to-secure-mcp-servers-on-aks-with-a-governed-ai-agent-gateway\/","url_meta":{"origin":59774,"position":3},"title":"How to Secure MCP Servers on AKS with a Governed AI Agent Gateway","author":"CPI Staff","date":"November 6, 2026","format":false,"excerpt":"Learn how Application Gateway for Containers can give AKS-hosted MCP servers one governed entry point, reducing exposure and improving control over production AI agents.","rel":"","context":"In &quot;Azure Kubernetes Service&quot;","block_context":{"text":"Azure Kubernetes Service","link":"https:\/\/www.cloudproinc.com.au\/index.php\/category\/azure-kubernetes-service\/"},"img":{"alt_text":"","src":"\/wp-content\/uploads\/2026\/11\/how-to-secure-mcp-servers-on-aks-with-a-governed-ai-agent-gateway.png","width":350,"height":200,"srcset":"\/wp-content\/uploads\/2026\/11\/how-to-secure-mcp-servers-on-aks-with-a-governed-ai-agent-gateway.png 1x, \/wp-content\/uploads\/2026\/11\/how-to-secure-mcp-servers-on-aks-with-a-governed-ai-agent-gateway.png 1.5x, \/wp-content\/uploads\/2026\/11\/how-to-secure-mcp-servers-on-aks-with-a-governed-ai-agent-gateway.png 2x, \/wp-content\/uploads\/2026\/11\/how-to-secure-mcp-servers-on-aks-with-a-governed-ai-agent-gateway.png 3x, \/wp-content\/uploads\/2026\/11\/how-to-secure-mcp-servers-on-aks-with-a-governed-ai-agent-gateway.png 4x"},"classes":[]},{"id":435,"url":"https:\/\/www.cloudproinc.com.au\/index.php\/2026\/10\/12\/how-to-deploy-a-wiz-outpost-to-microsoft-azure\/","url_meta":{"origin":59774,"position":4},"title":"How to Deploy a Wiz Outpost to Microsoft Azure","author":"CPI Staff","date":"October 12, 2026","format":false,"excerpt":"This Wiz Security article will explain how to deploy a Wiz outpost to Microsoft Azure cloud infrastructure. Listen to this article Before we discuss how to deploy an outpost, it is important to understand why many companies do so. A Wiz outpost is the deployment of the Wiz security scanner\u2026","rel":"","context":"In &quot;Azure&quot;","block_context":{"text":"Azure","link":"https:\/\/www.cloudproinc.com.au\/index.php\/category\/microsoft-azure\/"},"img":{"alt_text":"","src":"\/wp-content\/uploads\/2024\/07\/How-to-Deploy-a-Wiz-Outpost-to-Microsoft-Azure.webp","width":350,"height":200,"srcset":"\/wp-content\/uploads\/2024\/07\/How-to-Deploy-a-Wiz-Outpost-to-Microsoft-Azure.webp 1x, \/wp-content\/uploads\/2024\/07\/How-to-Deploy-a-Wiz-Outpost-to-Microsoft-Azure.webp 1.5x, \/wp-content\/uploads\/2024\/07\/How-to-Deploy-a-Wiz-Outpost-to-Microsoft-Azure.webp 2x, \/wp-content\/uploads\/2024\/07\/How-to-Deploy-a-Wiz-Outpost-to-Microsoft-Azure.webp 3x, \/wp-content\/uploads\/2024\/07\/How-to-Deploy-a-Wiz-Outpost-to-Microsoft-Azure.webp 4x"},"classes":[]},{"id":59558,"url":"https:\/\/www.cloudproinc.com.au\/index.php\/2026\/11\/05\/what-the-aks-mcp-gateway-means-for-production-ai-agent-security\/","url_meta":{"origin":59774,"position":5},"title":"What the AKS MCP Gateway Means for Production AI Agent Security","author":"CPI Staff","date":"November 5, 2026","format":false,"excerpt":"Microsoft\u2019s MCP gateway for AKS creates one governed entry point for AI tools. Here is what it improves, what it does not, and how to assess production readiness.","rel":"","context":"In &quot;AI Agents&quot;","block_context":{"text":"AI Agents","link":"https:\/\/www.cloudproinc.com.au\/index.php\/category\/ai-agents\/"},"img":{"alt_text":"","src":"\/wp-content\/uploads\/2026\/11\/what-the-aks-mcp-gateway-means-for-production-ai-agent-security.png","width":350,"height":200,"srcset":"\/wp-content\/uploads\/2026\/11\/what-the-aks-mcp-gateway-means-for-production-ai-agent-security.png 1x, \/wp-content\/uploads\/2026\/11\/what-the-aks-mcp-gateway-means-for-production-ai-agent-security.png 1.5x, \/wp-content\/uploads\/2026\/11\/what-the-aks-mcp-gateway-means-for-production-ai-agent-security.png 2x, \/wp-content\/uploads\/2026\/11\/what-the-aks-mcp-gateway-means-for-production-ai-agent-security.png 3x, \/wp-content\/uploads\/2026\/11\/what-the-aks-mcp-gateway-means-for-production-ai-agent-security.png 4x"},"classes":[]}],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"\/wp-content\/uploads\/2026\/11\/running-ai-workloads-on-aks-with-distributed-storage-caching.png","_links":{"self":[{"href":"https:\/\/www.cloudproinc.com.au\/index.php\/wp-json\/wp\/v2\/posts\/59774","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.cloudproinc.com.au\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.cloudproinc.com.au\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.cloudproinc.com.au\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.cloudproinc.com.au\/index.php\/wp-json\/wp\/v2\/comments?post=59774"}],"version-history":[{"count":2,"href":"https:\/\/www.cloudproinc.com.au\/index.php\/wp-json\/wp\/v2\/posts\/59774\/revisions"}],"predecessor-version":[{"id":59777,"href":"https:\/\/www.cloudproinc.com.au\/index.php\/wp-json\/wp\/v2\/posts\/59774\/revisions\/59777"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.cloudproinc.com.au\/index.php\/wp-json\/wp\/v2\/media\/59776"}],"wp:attachment":[{"href":"https:\/\/www.cloudproinc.com.au\/index.php\/wp-json\/wp\/v2\/media?parent=59774"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.cloudproinc.com.au\/index.php\/wp-json\/wp\/v2\/categories?post=59774"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.cloudproinc.com.au\/index.php\/wp-json\/wp\/v2\/tags?post=59774"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}