The economics of artificial intelligence are increasingly being shaped by a simple constraint. Every megawatt consumed by supporting infrastructure is a megawatt unavailable for computing, yet reducing that infrastructure too aggressively risks leaving expensive processors waiting for data.
This tension is becoming particularly important as developers plan AI factories with power requirements of 50MW or more. The challenge is no longer simply installing sufficient computing capacity, but ensuring that storage, networking and memory systems can sustain performance without consuming a disproportionate share of the available electricity.
World Wide Technology (WWT) and DDN have developed a storage reference design intended to address this problem, using NVIDIA’s Vera Rubin NVL72 architecture as the basis for large-scale AI deployments.
The design targets read throughput exceeding 2.5GB per second per GPU. With 72 GPUs in each Vera Rubin rack, the required aggregate throughput reaches 180GB per second. DDN says a single 2U AI400X3M storage appliance can deliver 190GB per second, providing sufficient rated capacity to support an entire rack.
The significance extends beyond storage performance. By reducing the space and power required to supply data, the companies aim to allow operators to allocate more of their fixed electricity budget to GPUs while limiting the time those processors spend idle.
Another challenge is emerging as AI shifts towards increasingly complex inference workloads, particularly applications involving extended reasoning, long contexts and autonomous agents.
These workloads generate key-value caches, which retain information produced while processing prompts. When that information exceeds available GPU memory, it may need to be discarded and subsequently recalculated, consuming computing resources and delaying responses.
The proposed architecture introduces an additional memory tier using NVIDIA CMX and BlueField-4 STX infrastructure, with DDN Infinia enabling inference nodes to share cached information rather than repeatedly generating it.
DDN reports that its approach can load cached information up to 55 times faster. In one test involving 112,000 tokens, retrieving cached data took 2.1 seconds compared with 57 seconds to recompute it. These figures are workload-dependent and should not be interpreted as universal performance improvements.
The wider design combines DDN EXAScaler for training data and checkpoints, Infinia for cache management, NVIDIA Dynamo for cache placement and Spectrum-X Ethernet networking. WWT is responsible for integrating and validating the components through its AI Proving Ground.
For data centre developers, the approach highlights an increasingly important investment decision. GPU performance alone does not determine the productive capacity of an AI factory. Storage efficiency, memory management and network architecture also influence how much useful computing can be delivered within a fixed power allocation.
As AI facilities grow larger, infrastructure efficiency may become as important as processor performance in determining their operational economics. The question is shifting from how many GPUs a data centre can accommodate to how effectively it can keep them working.
