New data center routing design cuts AWS networking energy costs by 40%, Amazon claims

Amazon has started deploying a completely new routing architecture in AWS data centers which it says will deliver higher throughput from fewer physical switches while slashing electricity consumption. The company claims the architecture, dubbed Resilient Network Graphs (RNG) by the AWS Networking Lab researchers who developed it, offers a more efficient alternative to the traditional ‘fat tree’ topology that dominates data centers today. According to Amazon’s overview , RNG has been the default routing architecture for most new AWS data centers since April, spurred by the architecture’s ability to deliver 33% better throughput from 69% fewer routers. Importantly, in an industry where operating costs are always a focus, using fewer switch-routers has led to a projected reduction in network infrastructure electricity consumption of 40%. “For customers, it means more resilient infrastructure behind every API call, database query, and machine learning training job, without changing a single line of code,” said Amazon’s researchers. Random graph theory Tech is overwhelmed with big claims, especially regarding energy efficiency, which has turned out to be a fundamental limit in an era where power consumption is a major constraint. Does this one stand up? The answer to this question begins with the limitations of today’s fat tree routing. First used in 1990’s supercomputing, fat tree routing was widely adopted in the 2000s because it scaled well to handle the huge data center bandwidth demands. Fat tree is hierarchical: Switch-router infrastructure is layered, and packets move up and down these layers with the structure dictating how the packets find the shortest path. The drawback is that, as data center networks get bigger, the architecture requires ever more switch and cabling infrastructure to maintain throughput. In practice, this means that data center designers are forced to cut corners for cost reasons, leading to higher congestion. A theoretical alternative that’s been discussed for years is to use a non-hierarchical ‘random graph’ topology, for example, the one proposed by Jellyfish , a university project from 2012. In principle, this is more efficient; switches connect to each other randomly in a flat mesh that avoids the need for multiple switch-routing layers. It is also more fault tolerant, Amazon’s researchers explained: “No single router is more important than any other. The loss of 1% of routers results in a roughly 1% capacity loss.” Unfortunately, the random graph topology has downsides, principally the need for impossibly complex cabling between switches across varying distances inside a data center. It also requires each node to hold a huge routing table that sets out every possible data path in its memory. Quasi-random Amazon’s researchers say they solved this by developing a new routing algorithm called ‘Spraypoint’ which combines the basic idea of a random graph topology with some of the hierarchy of fat tree to effect a “quasi-random” compromise. Traffic is randomly ‘sprayed’ to neighbors, giving it a wide selection of possible paths to its destination. But as packets get near to their destination, they are routed via ‘waypoint’ switches using a conventional shortest path algorithm. However, the biggest innovation is a new type of data center device called a ‘ShuffleBox’. This concentrates the complex wiring normally required in random graph topologies into a single box, allowing random interconnection between switches without long cable runs. Although the efficiencies claimed for RNG have not been independently verified, the fact that Amazon plans to make the architecture its default for most new data centers offers some validation. “The first quasi-random network went live near Dublin, Ireland, at the end of 2024, carrying real production traffic. We validated performance against the mathematical predictions, identified operational refinements, and applied them in two additional deployments,” said Amazon. Not for everyone Ryan Ries , chief AI and data scientist at AWS consultancy and MSP Mission Cloud, was positive about the development. “Across the industry, there is growing pushback on data center expansion, tied to energy demand, water use, and local community impact, so power and water performance have become two of the most important issues facing cloud providers today,” said Ries. “The efficiency claims are credible because AWS is saying RNG is already in production, and it’s now the default architecture for most new builds globally.” An obvious upside with RNG, added Amruth Laxman of cloud VoIP provider 4Voice, is that it proves that random graph features can be built into data center networks after all. However, its proprietary nature means that its direct influence is likely to be limited for now. “AWS is known to design most of their networking equipment. The big question at this point is how flexible they have made their design. Most hyperscale customers are not able to absorb the costs, while AWS has the resources to absorb the entire redesign costs,” he pointed out. Re-equipping existing data centers with any radically new technology would incur significant expense, which is why Amazon only plans to use RNG in new data centers, he noted, so, in the immediate future, “don’t expect other companies to copy this design.”