Amazon Rearchitects the Data Centre

Introduces new Arm-based Graviton2 chip

As data traffic soars and workloads become increasingly sophisticated and high-value, Arm has long targeted Intel’s server heartland with the promise of performance at lower cost and with significantly greater energy efficiency. Several manufacturers of Arm server chips have tried to push the industry forward, but their efforts came up short. AMD, Broadcom (with its Vulcan line of Arm server chips), Qualcomm (with its Centriq processor design) and others have put massive amounts of money into the Arm server market, only to later cease operations or find themselves acquired by other companies.

More recently, companies including Hewlett Packard Enterprise with its Moonshot server systems, Amazon Web Services (AWS) with its Graviton server chip, Huawei with its HiSilicon subsidiary and Marvell with its ThunderX have been expanding the Arm server market. Their efforts are helping Arm gain a foothold in hyperscale, cloud and enterprise data-centre computing environments, but Intel and its x86 server products still outpace deployments of Arm server chips by a huge margin.

However, there are signs of change and an acceleration in Arm’s momentum in servers with Amazon’s efforts to diversify the solutions it uses for its AWS cloud services.

At its recent AWS re:Invent event in Las Vegas, Amazon announced that its AWS cloud computing unit has built a second-generation data-centre chip called Graviton2. The 7 nm chip delivers seven times the CPU performance, four times the number of computing cores, double the cache memory and a five-fold increase in memory speed compared with the first generation of Graviton processors.

Based on Arm’s 64-bit Neoverse intellectual property rather than the company’s older Cortex-A72 technology, the Graviton2 is designed to significantly push the price-to-performance ratio for AWS’ Elastic Compute Cloud (EC2) customers. Although the first generation of Graviton processors didn’t have a dramatic impact on the market, the priority was to deploy, learn and begin to establish the ecosystem.

The vision for Graviton2 is much broader. AWS presented a host of benchmarks showing the processor against Intel running EC2 instances. The result was a clear lead in performance for Amazon’s chip. Despite the caveat that benchmarks were specific to AWS workloads, this is the market Graviton is designed for.

Central to the chip is AWS’ Nitro System, a hypervisor that abstracts the underlying hardware. Simply put, it allows AWS to more easily move workloads between architectures and remove the ties to the legacy hardware and instruction sets, which have served as an important barrier to change in the data-centre market and Arm, in particular. This is the enabling software for the hardware vision.

The server opportunity is vast and Graviton doesn’t have to compete broadly against products from Intel and AMD to be successful. AWS will use Graviton for its AWS instances, enabling optimization for a narrower set of workloads. The company needn’t worry about being pitted against Intel in the enterprise space, for example.

This is where Graviton gets wildly misreported. The chip isn’t about ending AWS’ dependence on Intel and AMD; it’s about providing choice. Intel and AMD will continue to be important partners, as will Nvidia for artificial intelligence training and increasingly inference-based workloads. As workloads become evermore diversified, choice becomes critical and software the key determinant that enables that choice. Intel’s One API initiative highlights a similar shift within its own range of processor types.

In the long term, there’s little doubt that Graviton2 is a threat to Intel and AMD for cloud workloads. Amazon is investing in chips for a reason. Its new processor increases flexibility, choice and performance (depending on the workload), but also stands to improve AWS margin if it scales to account for a sizeable proportion of AWS’ business. Margin pressure on its cloud business is likely to increase over the medium term as competition grows. Amazon is investing to improve cloud profitability by reducing the premium for merchant silicon. It’s a capital-intensive way to do it, but a no-brainer for a business that drives the majority of Amazon’s total company profit.