An H100 GPU draws 700 watts under full compute load. Eight of them in a server node consume 10 to 12 kilowatts — the equivalent of 120 standard gaming PCs running simultaneously. A single rack of these nodes pulls 80 to 140 kilowatts. A 10,000-GPU training cluster draws 10 to 15 megawatts. Enough to power a small town. And that is before the next generation of AI features ships.GPUnex
The math is not abstract. The International Energy Agency projects global data center electricity consumption — currently 415 terawatt-hours — will nearly double to 945 TWh by 2030, roughly equivalent to Japan is entire national power demand.SemiEngineering In the United States, data centers consumed 4.4 percent of total electricity in 2023. The Department of Energy, in a report from Lawrence Berkeley National Laboratory, projects that figure will reach 6.7 to 12 percent by 2028, depending on how aggressively AI infrastructure builds out.DOE The US grid absorbed 176 TWh from data centers in 2023. By 2028, it could be absorbing between 325 and 580 TWh.
Every reasoning chain that AI labs are racing to ship — every agentic loop, every multi-step planning step — consumes additional kilowatts. A single generative AI query already uses up to ten times the power of a traditional search. More capable models do more work. More work means more heat. The physics is not a metaphor.SemiEngineering
The industry is already hitting the consequences. Northern Virginia, the world is largest data center market, has utility connection wait times of three to five years for new large-scale deployments.GPUnex Occupancy rates across major US markets are expected to exceed 95 percent by the end of 2026 — not because servers are full, but because electrical capacity is committed. Over 100 gigawatts of new data center capacity is in various stages of planning and development in the US alone. The interconnection queue for new generation and storage projects in the US stands at over 2,500 gigawatts.
Air cooling is dead for AI workloads. Any deployment involving eight H100s or Blackwell-class hardware requires liquid cooling — direct-to-chip or full immersion.GPUnex This immediately disqualifies most existing colocation facilities built for 10 to 20 kilowatts per rack. Liquid cooling infrastructure adds 500,000 dollars to 2 million dollars per megawatt of capacity in capital costs. For a 10-megawatt GPU cluster, cooling infrastructure alone runs 5 million dollars to 20 million dollars.
The industry is answer to the power wall is also creating new engineering problems. Traditional monolithic chip scaling — the path that delivered better performance per generation for decades — has hit physical and economic limits. The solution is chiplets, 3D stacking, and heterogeneous integration: combining multiple specialized dies in a single package to keep the performance curve alive. The advanced packaging market, valued at 46 billion dollars in 2024, is projected to reach 79.4 billion dollars by 2030 at a 9.5 percent compound annual growth rate, driven by AI accelerators, GPUs, and chiplet architectures.Yole Technologies including TSMC is CoWoS and SoIC, Intel is EMIB, and Samsung is I-Cube are the new frontier of compute density.
But stacking chiplets vertically introduces its own thermal constraints. Die warpage and localized hotspots threaten reliability and yield. Pulling heat away from vertically integrated dies — each generating hundreds of watts in a volume measured in cubic centimeters — is, in the words of engineers working on the problem, one of the most pressing engineering challenges of the current era. Solving it requires breakthroughs in hybrid bonding processes, advanced thermal interface materials, and liquid cooling integration. And it cannot be done by any single company in isolation.SemiEngineering
The SEMI Advanced Packaging and Heterogeneous Integration Technology Coalition — the APHI group — was formed specifically because the bottlenecks are pre-competitive. Foundries, outsourced assembly and test providers, material suppliers, and equipment makers all need to solve the same foundational problems before competitive differentiation can occur. TSMC has acquired additional capacity specifically to scale CoWoS.Yole Intel, Samsung, SK Hynix, and Sony lead the advanced packaging landscape, with TSMC, ASE, and Amkor expanding US capacity in line with CHIPS Act incentives to serve customers including NVIDIA and Apple.
The thermal wall is not a future problem. It is the reason AI labs are building in places with power surplus, paying premiums for guaranteed electricity allocations, and racing to secure cooling infrastructure years before new capacity comes online. It is why reasoning chains and agentic features carry a physics cost that product managers are only beginning to price in. The semiconductor industry has run into the same wall every industrial revolution eventually hits. The question now is not whether the physics applies — it does. The question is who solves it first.
