Amazon Unveils Graviton4: A 96-Core ARM CPU with 536.7 GBps Memory Bandwidth
by Anton Shilov on November 29, 2023 4:30 PM EST- Posted in
- CPUs
- Arm
- Datacenter
- Amazon
- AWS
- Neoverse V2
- Graviton4
- Graviton
Nowadays many cloud service providers design their own silicon, but Amazon Web Services (AWS) started to do this ahead of its rivals and by now its Annapurna Labs subsidiary develop processors that can well compete with those from AMD and Intel. This week AWS introduced its Graviton4 SoC, a 96-core ARM-based chip that promises to challenge renowned CPU designers and offer unprecedented performance to AWS clients.
"By focusing our chip designs on real workloads that matter to customers, we are able to deliver the most advanced cloud infrastructure to them," said David Brown, vice president of Compute and Networking at AWS. "Graviton4 marks the fourth generation we have delivered in just five years, and is the most powerful and energy efficient chip we have ever built for a broad range of workloads."
The AWS Graviton4 processor packs 96 cores that offer on average 30% higher compute performance compared to Graviton3 and is 40% faster in database applications as well as 45% faster in Java applications, according to Amazon. Given that Amazon did not reveal many details about its Graviton4, it is hard to attribute performance increases to any particular characteristics of the CPU.
Yet, NextPlatform believes that the processor uses Arm Neoverse V2 cores, which are more capable than V1 cores used in previous-generation AWS processors when it comes to instruction per clock (IPC). Furthermore, the new CPU is expected to be fabricated using one of TSMC's N4 process technologies (4nm-class), which offers a higher clock-speed potential than TSMC's N5 nodes.
"AWS Graviton4 instances are the fastest EC2 instances we have ever tested, and they are delivering outstanding performance across our most competitive and latency sensitive workloads," said Roman Visintine, lead cloud engineer at Epic. "We look forward to using Graviton4 to improve player experience and expand what is possible within Fortnite.”
In addition, the new processor features a revamped memory subsystem with a 536.7 GB/s peak bandwidth, which is 75% higher compared to the previous-generation AWS CPU. Higher memory bandwidth improves performance of CPUs in memory intensive applications, such as databases.
Meanwhile, such a major memory bandwidth improvement indicates that the new processor employs a memory subsystem with a higher number of channels compared to Graviton3, though AWS has not formally confirmed this.
Graviton4 will be featured in memory-optimized Amazon EC2 R8g instances, which is particularly useful to boost performance in high-end databases and analytics. Furthermore, these R8g instances provide up to three times more vCPUs and memory than Graviton 3-based R7g instances, enabling higher throughput for data processing, better scalability, faster results, and reduced costs. To ensure security of AWS EC2 instances, Amazon equipped all high-speed physical hardware interfaces of Graviton4 CPUs.
Graviton4 R8g is currently in preview, these instances will be available widely in the coming months.
Sources: AWS, NextPlatform
Facebook
Twitter
RSS
35 Comments
View All Comments
SarahKerrigan - Wednesday, November 29, 2023 - link
"Meanwhile, such a major memory bandwidth improvement indicates that the new processor employs a memory subsystem with a higher number of channels compared to Graviton3, though AWS has not formally confirmed this.""Given that Amazon did not reveal many details about its Graviton4, it is hard to attribute performance increases to any particular characteristics of the CPU.
Yet, NextPlatform believes that the processor uses Arm Neoverse V2 cores, which are more capable than V1 cores used in previous-generation AWS processors when it comes to instruction per clock (IPC). Furthermore, the new CPU is expected to be fabricated using one of TSMC's N4 process technologies (4nm-class), which offers a higher clock-speed potential than TSMC's N5 nodes."
https://aws.amazon.com/blogs/aws/join-the-preview-...
I don't know why the writer is acting like the uarch and memory are some kind of mystery worthy of speculation. Amazon says outright that it's Neoverse V2 and has 12-channel DDR5-5600.
bwj - Wednesday, November 29, 2023 - link
It's great, abstractly, if these are power efficient, but since you can't buy these it only matters to Jeff Bezos. For the rest of us, we only care about the sticker price.trevor23 - Wednesday, November 29, 2023 - link
People and companies who care about sustainability care about power consumption.Threska - Wednesday, November 29, 2023 - link
That's why statements like this:"Nowadays many cloud service providers design their own silicon, but Amazon Web Services (AWS) started to do this ahead of its rivals and by now its Annapurna Labs develops processors that can well compete with those from AMD and Intel."
Are kind of a huh.
mode_13h - Thursday, November 30, 2023 - link
> It's great, abstractly, if these are power efficient,It should translate into lower pricing, because it costs money both to power the CPUs and then remove the waste heat.
mode_13h - Thursday, November 30, 2023 - link
> I don't know why the writer is acting like the uarch and memory are some kind of mysteryBecause the link cited by the article is different than yours. It doesn't say what kind of core they have:
https://press.aboutamazon.com/2023/11/aws-unveils-...
Also, the 30% number seems to refer to the entire CPU. For a CPU with 50% more cores, then switching to the N2 seems in line with having only a 30% speedup. If they meant per-core performance improved by 30%, that's more than I'd expect for the gen-on-gen difference between Neoverse-V cores.
mode_13h - Thursday, November 30, 2023 - link
I guess what makes 30% plausible as a single-thread speedup number would be if they increased the clock speed significantly. Give that the V2 allegedly delivers 13% better IPC than V1, that leaves 15% to be accounted for (1.13 * 1.15 = 1.3).If this is achieved by increasing the clock speed from Graviton 3's 2.6 GHz to 3.0 GHz, then it works out perfectly. However, that's a little at odds with their efficiency goals. NextPlatform instead worked backwards from the claimed power budget to arrive at a clock speed of only 2.7 GHz. If true, there's still about 11% that's unaccounted for. NextPlatform suggests this might've been accomplished via additional cache.
SarahKerrigan - Thursday, November 30, 2023 - link
Right, but I'm not making this stuff up. The link I provided is first-party, from Amazon. It has V2. That's not an area of ambiguity.mode_13h - Thursday, November 30, 2023 - link
> Right, but I'm not making this stuff up.Yes, I saw your link. Thanks for sharing it. The point of my comment was just to explain why Anandtech, NextPlatform, and others tried to speculate - they were working off less information than you found.
eastcoast_pete - Sunday, December 3, 2023 - link
One aspect the AWS blog entry doesn't comment on that the Nextplatform one speculates on is that Graviton 4 might be SMT enabled. On a somewhat related note, Ampere is currently dealing with/trying to deal with a limitation for ARM cores in the Linux kernel, that right now can only handle 256 physical cores. While Ampere's new CPU has "only" 192 cores, their new CPU is also capable of dual-socket operation. Except that, right now, doing so would exceed the kernel limit of the very OS that most ARM-based servers use. If Graviton 4 does indeed have SMT, it would sidestep that issue and still allow two socket operation while offering more than 256 threads. Of course, among the first ARM-based server CPUs to use SMT was/is actually the one from Huawei/HiSilicon. They now also use that SMT-capable core in their Smartphone SoC for the Chinese market. AFAIK, right now the only smartphone SoC to have SMT.