Test Bed and Setup - Compiler Options

For the rest of our performance testing, we’re disclosing the details of the various test setups:

Ampere "Mount Jade" - Dual Altra Q80-33

Obviously, for the Ampere Altra system we’re using the provided Mount Jade server as configured by Ampere.

The system features 2 Altra Q80-33 processors within the Mount Jade DVT motherboard from Ampere.

In terms of memory, we’re using the bundled 16 DIMMs of 32GB of Samsung DDR4-3200 for a total of 512GB, 256GB per socket.

CPU ​2x Ampere Altra Q80-33 (3.3 GHz, 80c, 32 MB L3, 250W)
RAM 512 GB (16x32 GB) Samsung DDR4-3200
Internal Disks Samsung MZ-QLB960NE 960GB
Samsung MZ-1LB960NE 960GB
Motherboard Mount Jade DVT Reference Motherboard
PSU 2000W (94%)

The system came preinstalled with CentOS 8 and we continued usage of that OS. It’s to be noted that the server is naturally Arm SBSA compatible and thus you can run any kind of Linux distribution on it.

Ampere makes special note of Oracle’s active support of their variant of Oracle Linux for Altra, which makes sense given that Oracle a few months ago announced adoption of Altra systems for their own cloud-based offerings.

The only other note to make of the system is that the OS is running with 64KB pages rather than the usual 4KB pages – this either can be seen as a testing discrepancy or an advantage on the part of the Arm system given that the next page size step for x86 systems is 2MB – which isn’t feasible for general use-case testing and something deployments would have to decide to explicitly enable.

The system has all relevant security mitigations activated, including SSBS (Speculative Store Bypass Safe) against Spectre variants.

AMD - Dual EPYC 7742

For our AMD system, unfortunately we had hit some issues with our Daytona reference server motherboard, and moved over to a test-bench setup on a SuperMicro H11DSI0.

We’re also equipping the system with 256GB per socket of 8-channel/DIMM DDR4-3200 memory, matching the Altra system.

CPU ​2x AMD EPYC 7742 (2.25-3.4 GHz, 64c, 256 MB L3, 225W)
RAM 512 GB (16x32 GB) Micron DDR4-3200
Internal Disks OCZ Vector 512GB
Motherboard SuperMicro H11DSI0
PSU EVGA 1600 T2 (1600W)

As an operating system we’re using Ubuntu 20.10 with no further optimisations. In terms of BIOS settings we’re using complete defaults, including retaining the default 225W TDP of the EPYC 7742’s, as well as leaving further CPU configurables to auto, except of NPS settings where it’s we explicitly state the configuration in the results.

The system has all relevant security mitigations activated against speculative store bypass and Spectre variants.

Intel - Dual Xeon Platinum 8280

For the Intel system we’re also using a test-bench setup with the same SSD and OS image actually – we didn’t have enough RAM to run both systems concurrently.

Because the Xeons only have 6-channel memory, their maximum capacity is limited to 384GB of the same Micron memory, running at a default 2933MHz to remain in-spec with the processor’s capabilities.

CPU 2x Intel Xeon Platinum 8280  (2.7-4.0 GHz, 28c, 38.5MB L3, 205W)
RAM 384 GB (12x32 GB) Micron DDR4-3200 (Running at 2933MHz)
Internal Disks OCZ Vector 512GB
Motherboard ASRock EP2C621D12 WS
PSU EVGA 1600 T2 (1600W)

The Xeon system was similarly run on BIOS defaults on an ASRock EP2C621D12 WS with the latest firmware available.

The system has all relevant security mitigations activated against the various vulnerabilities.

Compiler Setup

For compiled tests, we’re using the release version of GCC 10.2. The toolchain was compiled from scratch on both the x86 systems as well as the Altra system. We’re using shared binaries with the system’s libc libraries.

Topology, Memory Subsystem & Latency SPEC - Single-Threaded Performance
Comments Locked

148 Comments

View All Comments

  • tygrus - Saturday, December 19, 2020 - link

    Next step would be to see how ARM performed with 256MB (or more) of cache. The early models didn't suit many workloads compared to the general purpose kings (x86-64). Each generation of ARM based server chips have seen an increase to the number of suitable workloads thus more general purpose. Adding more specialised instructions to x86 has diminishing returns that increase the complexity of decoding & execution it's always good to be challenged "can we make it simpler & faster?".
  • mode_13h - Sunday, December 20, 2020 - link

    ARM has been doing some of the same, though. Look at the evolution of ARMv8-A as it has aged, and you'll see several bolt-ons to target additional markets:
    * new atomics
    * signed, saturating multiplies
    * CRC instructions
    * half-precision floating point
    * SVE (their answer to AVX/AVX-512)
    * complex number support
    * an instruction specifically for floating-point conversion in Javascript
    * integer dot products
    * random number generation
    * matrix multiply & manipulation
    * BFloat16 support
    * a smattering of other virtualization and security-oriented additions

    That's not a small list, and definitely not a less-is-more approach.
  • Leeea - Sunday, December 20, 2020 - link

    Very interesting.

    Great article.
  • Sivar - Sunday, December 20, 2020 - link

    Typo report, conclusion: "The se*ver landscape is changing very quickly. "
  • Sivar - Sunday, December 20, 2020 - link

    Imagine the response if someone made a prediction of this statement just five years ago:

    "Intel’s current Xeon offering simply isn’t competitive in any way or form at this moment in time. Cascade Lake is twice as slow and half as efficient – so unless Intel is giving away the chips at a fraction of a price, they really make no sense."
  • mode_13h - Sunday, December 20, 2020 - link

    Heh, good call!
  • Makste - Monday, December 21, 2020 - link

    Truly exciting times. Thanks for the review.
    There's going to be a massive restructuring in "computeverse".
    I hope they'll be a merger at one point.
  • hyc - Monday, December 21, 2020 - link

    It would've been more appropriate to compare Q64-33 to AMD to assess the merits of each architectural design. Could you repeat some of these tests, limiting the Altra to only 64 cores/threads?
  • Wilco1 - Tuesday, December 22, 2020 - link

    It might be an interesting comparison with SMT disabled since the extra 25% of cores are the alternative to adding SMT. However should we also limit EPYC to 32MB L3 to make things more equal?
  • hyc - Friday, December 25, 2020 - link

    Are you saying that turning off 16 cores in the Q80-33 is not the equivalent of running a Q64-33?

    Turning off SMT may have some merits, depending on workload.

Log in

Don't have an account? Sign up now