The money flowing into artificial intelligence is increasingly tied to physical infrastructure.
Large technology companies are expected to spend on the order of $700 billion in a single year on data centers and the chips that fill them, while Goldman Sachs projects that U.S. data center power demand could double within two years.
But beneath the spending is a less visible challenge: the systems built to run AI are being assembled faster than many reliability practices can adapt.
AI workloads can be unpredictable, latency-sensitive and demanding on data integrity. The International Energy Agency has warned that AI data centers already show rapid and large swings in demand, creating new pressure on the infrastructure that supports them.
Semyon Slepov is a site reliability engineer and IEEE Young Professionals member who works on large-scale distributed systems. His view is that AI is exposing weaknesses in reliability architectures built for more predictable web traffic.
The workload that refuses to behave
For years, capacity planning rested on a familiar assumption: traffic could be forecast with some confidence.
Usage rose and fell in daily, weekly or seasonal patterns. Engineers could provision around those patterns and adjust as demand changed.
AI complicates that model. Training and inference workloads can create sharp, uneven bursts of demand. The IEA has also projected that power use from AI-focused facilities could triple by 2030.
When demand no longer behaves predictably, provisioning for average usage becomes risky. The peak, not the average, is where systems often fail.
Slepov has written about this problem in the context of round-robin load balancing fails under heterogeneous traffic. The point is straightforward: when machines and workloads are not behaving identically, treating them as interchangeable can create instability.
For AI infrastructure, that means systems need to understand real-time resource consumption rather than simply spreading work evenly by default.
When one failure can stop the job
AI infrastructure can also be more brittle than traditional web systems.
A failed server in a conventional web service may be routed around. But large AI training jobs can be tightly synchronized, meaning a single failed component can interrupt the broader workload. One documented training run saw unexpected hardware failures once every three hours across a large GPU cluster.
That makes failure containment more important.
Distributed systems need boundaries so that one bad release, overloaded cluster or failing component does not spread across the entire service. For AI teams, that can mean isolating workloads across clusters, regions or service boundaries so problems remain contained.
The principle is not new. But AI raises the cost of ignoring it.
Latency still matters
Speed is another reliability issue.
A widely cited Amazon finding linked every 100 milliseconds of latency to reduced sales by roughly one percent, while later research found that even small load-time improvements could lift conversions by more than eight percent.
AI services introduce new latency challenges because inference can be expensive and uneven. Users may not care whether the delay comes from retrieval, model selection, tool use or validation. They experience one product.
That means teams have to engineer for stable latency, not just average latency. A system that performs well most of the time but regularly produces slow outliers can still feel unreliable to users.
Reliability has to become a shared discipline
The tools needed to close the gap already exist in distributed-systems engineering: workload-aware resource management, failure isolation, data-integrity checks and disciplined latency design.
The challenge is making those practices standard across teams building production AI.
That matters because reliability work is already consuming more engineering time. A 2026 survey found many reliability and operations professionals spending 40 percent or more of their time managing incidents.
Slepov has argued for spreading reliability knowledge beyond small specialist teams, including in his writing on teaching product engineers to own their own incidents.
That approach fits the moment. As AI systems become more central to products, reliability cannot be treated as a cleanup function that arrives only after something breaks.
The road ahead
AI infrastructure will continue to grow, but capacity alone will not make it dependable.
The systems underneath AI need to handle strange workloads, isolate failures, protect data integrity and keep latency under control. Those are not glamorous problems, but they determine whether AI products can survive real use.
The next phase of AI engineering will not be defined only by larger models or more powerful chips. It will also be defined by whether teams can build infrastructure that keeps its promises when the workload gets unpredictable.
©2026 Cox Media Group
