AINEWS
The model quality race is over. The next chip fight is over who can run the answers cheapest, fastest, and longest, and the alliance map is already shifting.
GPT-3 answered 43.9% of the questions on a tough knowledge-and-reasoning exam in 2020. Four years later, GPT-4o hit 88.7% on the same test, roughly matching human experts. That gap closed because the models got better, but the smarter-model race is essentially over. What is now deciding who profits, who controls the AI stack, and what you pay to use it is the cost of running a model every time you ask it a question. That cost, in industry shorthand, is inference: the moment a trained model actually does something, whether answering a question, writing code, or generating an image. It is the doing, not the years-long study phase that built it (the part called training).
The doing has gotten expensive in a way the study phase never was. Two changes are doing the work. First, the newest reasoning models re-ask themselves many times before they answer, the way a careful student might work a problem on scratch paper before turning it in. With reasoning effort set high, those models can produce up to 20 times as much text per query as a model that just answers straight (The AI Inference Revolution Is Here). Twenty times the output means twenty times the silicon time, twenty times the memory, and twenty times the electricity. Second, agentic AI now keeps working after you close the tab. Instead of answering one question and stopping, the model runs continuously toward a goal, calling tools, checking its own work, and waiting for hours or days.
Put those two together and the workload changes shape. Inference is no longer a quick reply to a question. It is a continuous, memory-hungry, around-the-clock process, and the chips that handled training are not always the chips that handle it well. At Nvidia's GTC 2026 conference, Jensen Huang told the audience that 2026 is the "inflection point of inference" (The AI Inference Revolution Is Here). Matt Kimball of Moor Insights & Strategy put it more bluntly: training is "yesterday's news" relative to what chief information officers now care about (The AI Inference Revolution Is Here).
The chip map is reorganizing around that shift in ways that look strange until you see the workload behind them. OpenAI and Amazon have both put production inference on Cerebras wafer-scale chips, silicon roughly the size of a dinner plate built for memory-heavy work rather than the small, dense GPUs that dominate training (Cerebras press release on Llama 405B inference; Cerebras CS-4 product page). Amazon is the unusual case: it owns Trainium, an in-house AI chip, yet routes parts of its inference to Cerebras anyway. The split is deliberate. Trainium runs the computationally heavy part of a query; Cerebras handles the memory-intensive part where the wafer-scale design pays off (The AI Inference Revolution Is Here). When Amazon, which makes its own chips, still needs a rival's silicon, the underlying workload has changed shape.
Nvidia, which sells most of the GPUs used to train large models, has moved to absorb talent and intellectual property from inference startup Groq in a deal industry sources call controversial (The AI Inference Revolution Is Here). The pattern is consistent: when a market leader is buying its way into a smaller rival's specialty, the buyer is hedging that the specialty becomes the new center of gravity.
It would be wrong to call training over. The largest training clusters are still being built and funded, and the labs that run them are still training larger models (The AI Inference Revolution Is Here). The conversation has shifted, not the budget. But the budget is starting to follow, and the chip map is shifting with it. A research paper on GPU memory bottlenecks in large-batch inference, "Mind the Memory Gap," lays out the technical reason: when a reasoning model produces long outputs and an agent runs for hours, the bottleneck moves from raw compute to memory bandwidth, and most GPUs were not designed for that (Mind the Memory Gap, arXiv 2503.08311). Independent GPU references for inference workloads show the same shape: the chips that win for training do not always win for serving a model in production (Inference Engineering GPU reference).
The next time an AI product feels slow, expensive, or always on, the reason is here. The smarter-model race ended, and the throughput race started, and the chips behind your screen are being rearranged to match.