AI datacenters are approaching a point where adding more processors no longer guarantees faster results. As machine-learning models expand across hundreds or thousands of accelerators, the central challenge is increasingly not raw computing power but the time required for those devices to communicate. A new review in Nature Reviews Electrical Engineering presents a vision for overcoming this problem: turning a distributed datacenter into something that behaves, from the perspective of an AI workload, like one enormous computer chip.
Modern AI systems divide computation among many graphics processing units, tensor processing units and other specialized accelerators. These devices must constantly exchange model parameters, intermediate results and gradients during training and inference. In a small system, communication delays may be manageable. At large scale, however, even tiny differences in the time taken by individual messages can spread across the entire machine. The slowest communication path can force thousands of otherwise idle processors to wait, reducing the efficiency of the whole system.
This problem is known as latency variability, or latency spread. It is different from average network speed. A network may offer high bandwidth and a low average delay while still producing unpredictable execution if some messages encounter congestion, software overhead or longer routes. AI workloads are particularly sensitive because many operations are tightly synchronized. If one accelerator finishes late, the next stage of a calculation may not begin until every participating device has caught up. The result is a datacenter whose performance is governed by its most delayed moments.
The review by Minseok Jung, Hyunwoo Woo, Joonwon Kim and colleagues argues that conventional software-managed distributed systems are reaching structural limits under these conditions. In traditional architectures, separate devices communicate through layers of operating systems, network protocols and software coordination. These layers provide flexibility, but they can also introduce variation into the timing of memory access and data movement. For large AI models, the authors propose extending the execution principles of a single chip across physically separated machines and devices.
At the center of this approach is Compute Express Link, or CXL, a high-speed interconnect technology designed to connect processors with memory and accelerators. Unlike a conventional network connection, CXL can support more direct, coherent communication between components. It allows devices to access shared or pooled memory through a fabric, potentially making memory resources available wherever they are needed rather than tying each accelerator to a fixed local pool.
Memory pooling could transform how AI servers are assembled. Large models frequently require more memory than a single accelerator can provide, while other devices may have memory capacity sitting unused. A CXL-based fabric can make these resources appear as part of a larger, addressable memory system. In principle, an accelerator could retrieve data from a remote memory device with a level of integration closer to accessing memory inside a computer system than communicating with a separate server over a conventional network.
Yet the review emphasizes that CXL standards alone cannot guarantee chip-like behavior. The technology defines important functions, but predictable execution depends on how the hardware and the overall infrastructure are designed. Every traversal path through a CXL fabric can introduce delays caused by switches, queues, memory controllers, protocol processing and competing traffic. If these sources of variation are not controlled, a system may offer shared memory without delivering the stable timing that tightly coupled AI computation requires.
The proposed solution involves hardware mechanisms that identify and regulate the causes of latency variation. Rather than treating the fabric as a general-purpose network, designers would need to manage routes, traffic, buffering and resource access with execution predictability in mind. The review also points to hierarchical deployment, in which resources are arranged in layers that resemble the organization of a chip. Closely connected components could form local scale-up domains, while larger fabrics link those domains over longer physical distances.
This hierarchy could allow datacenter architects to balance speed, capacity and physical reach. Shorter links would serve the most latency-sensitive interactions, while higher-level connections would provide access to additional accelerators and memory. Such an arrangement may reduce unnecessary traffic and make the timing behavior of the system easier to model. The goal is not simply to connect more hardware, but to make thousands of separate components operate according to shared execution rules.
The next stage of this evolution may involve optical communication. Electrical links face limits in power consumption, signal integrity and reach as systems grow. Optical interconnects can carry data over longer distances with potentially higher bandwidth and lower energy costs for large-scale communication. CXL-over-optics could extend the one-chip-like model beyond tightly packed racks and into broader portions of a datacenter. If engineers can combine optical links with latency-control mechanisms and hierarchical fabrics, future AI infrastructures may behave less like collections of networked servers and more like giant, spatially distributed processors.
The authors’ perspective reframes the race to scale AI. The decisive innovation may not be a faster accelerator alone, but an infrastructure capable of making distant memory and computation feel predictable and immediate. CXL provides a foundation for that shift, while new hardware controls, architectural hierarchies and optical technologies could determine whether the vision becomes practical. As AI models continue to grow, the datacenter itself may increasingly be designed as a single machine—one whose “chip” spans rooms, racks and eventually entire facilities.
Subject of Research: One-chip-like datacenter infrastructures for large-scale AI workloads using CXL-based scale-up fabrics
Article Title: One-chip-like datacenter design enabled by CXL-based scale-up fabrics
Article References: Jung, M., Woo, H., Kim, J. et al. “One-chip-like datacenter design enabled by CXL-based scale-up fabrics.” Nature Reviews Electrical Engineering (2026). https://doi.org/10.1038/s44287-026-00315-5
Image Credits: AI Generated
DOI: 10.1038/s44287-026-00315-5
Keywords: Artificial intelligence, AI datacenters, Compute Express Link, CXL, scale-up fabrics, memory pooling, distributed computing, latency variability, optical interconnects, CXL-over-optics
Tags: AI datacenter communication bottlenecksCXL scale-up fabrics for data centersCXL technology for AI acceleratorsdistributed AI system efficiencyhigh-performance interconnects for AI workloadslarge-scale accelerator data exchangelatency variability in AI trainingone-chip-like AI hardware architectureoptimizing communication in AI model trainingovercoming communication delays in AI systemsreducing latency spread in data centersscalable data center design for machine learning
