As AI Workloads Shift From Training to Inference, VAST and AMD Bet on the Data Layer

AI infrastructure is moving from one-time training to continuous, around-the-clock inference. On July 24, 2026, VAST Data and AMD published a reference architecture aimed at the path between chip and storage, where the new cost and contention live.

For two years, the loudest numbers in AI were about training: how many GPUs, how many days, how large the model. Those runs still happen, but the everyday spend is now serving the trained model to users, and that work is bottlenecked by data movement. Every prompt has to fetch the model's working memory, called a KV cache, from somewhere. Every response has to write new memory back. A single user session can hold megabytes of cache; a million concurrent sessions turn that into a production cost line, not a footnote.

VAST and AMD's expanded tie-up is the most explicit attempt yet to handle that path as one design problem.

At the silicon end, VAST has selected AMD's 6th-generation EPYC processors, the chip formerly known by the codename "Venice," to power the 6th-generation CBox and 3rd-generation EBox appliances that anchor the VAST AI Operating System. The CPU is no longer the sidekick. It is part of the inference budget.

The bigger move is the joint reference architecture. VAST, AMD, and DriveNets are co-developing a stack that puts AMD's Helios rack-scale AI infrastructure alongside the VAST software layer and DriveNets' networking, sized for training, inference, reinforcement learning, and the KV cache workloads that surround them. The packet distributed on GlobeNewswire and mirrored on Yahoo Finance describes optimisations for inference and KV cache offload using AMD Instinct GPUs, AMD's Infinity Context software, and AMD's ROCm stack, with AMD Pensando Pollara 400 AI network cards moving data between Instinct GPUs and VAST storage clusters.

BlocksAndFiles calls the resulting footprint "widespread AMD CPU/GPU collaboration." It is, more precisely, an attempt to lock down the network card, the storage cluster, and the cache policy as one decision rather than three.

VAST's own early testing on an AMD Instinct MI355X GPU recorded a ninefold improvement in time-to-first-token and a 9.7-fold increase in token throughput when VAST handled KV cache offloading under high-concurrency agentic workloads, where the AI takes multi-step actions across tools and sessions rather than answering one prompt at a time. The figures are vendor-reported, not independently verified by AMD, and the source itself flags that performance depends on hardware and workload. Anyone planning a deployment should treat them as a starting point for their own testing.

The same VAST data lifecycle policies that govern ordinary file expiry can also automatically delete cached KV cache data. A transformer model's working memory can hold the user's last several messages, a slice of a document, or fragments of personal context; if a regulated workflow caches all of that and never expires it, the regulator reads that as a retention problem. VAST is positioning the expiry rule as a feature, and the joint reference architecture inherits the position. Whether enterprise customers wire that expiry through to production is a separate question, and one the release does not answer.

What an enterprise architect takes home from this is a mental model: chip → NIC → KV cache → storage, with cache lifecycle sitting alongside model performance as a design constraint. The standard coverage will frame the announcement as a chip partnership. The substance is a stack decision, and a privacy question to put to every inference vendor in the stack: what governs the cache, and when does it expire?

The reference architecture is a design, not a shipping product. Pricing, ship dates, and named customers are absent from the joint release. VAST's 6th-generation CBox datasheets describe the platform, not the AMD partnership. The 9x and 9.7x numbers will be tested by someone independent, or they will not be tested at all.