AMD Launches NVMe RAID Support For Ryzen Threadripper Platform

A week later than originally planned, AMD is launching NVMe RAID support for their Ryzen Threadripper platform. Grab the Windows 10 drivers and release notes here. This updated RAID capability should work with any X399/Threadripper motherboard and any NVMe SSDs, enabling RAID modes 0, 1 and 10 for up to 10 NVMe SSD connected to the CPU's PCIe lanes, but a motherboard firmware update may be required first.

AMD's Ryzen Threadripper launched with support for RAID modes 0, 1 and 10 for arrays of SATA drives connected through the X399 chipset's eight SATA ports. The X399 chipset doesn't offer RAID capability for NVMe SSDs, but given its limited PCIe lane counts, this isn't much of an inconvenience. The Ryzen Threadripper processors themselves have plenty of PCIe lanes to go around. In a typical configuration, the motherboard will have up to three M.2 slots providing PCIe x4 connections to the Threadripper CPU (leaving 48 PCIe lanes for GPUs and four lanes for the connection to the X399 chipset). These PCIe x4 ports can also operate as SATA ports to support M.2 SATA SSDs. However, since these ports are provided by the CPU and not the chipset, they're not covered by the RAID capabilities of the chipset.

To allow for RAID arrays of NVMe SSDs, the Ryzen Threadripper platform needed another RAID implementation. Nobody has manufactured a true NVMe hardware RAID controller—one that completely abstracts the individual drives in the array and presents a single NVMe device to the host. Like all previous NVMe RAID implementations and almost all motherboard RAID implementations for SATA drives, AMD's NVMe RAID for Ryzen Threadripper is at heart a software RAID system. That means it needs new drivers for the OS, and matching support in the motherboard's UEFI to enable booting from a RAID array.

Chipset RAID implementations for SATA typically rely on switching the SATA controller out of AHCI mode and into a proprietary RAID mode. This ensures that the operating system's standard storage drivers won't be able to access the drives, so there's no danger of the array being accidentally corrupted by an OS that isn't aware the drive is part of an array. When Intel added NVMe RAID to their consumer platform with the Sunrise Point 100-series chipsets accompanying Skylake processors, they included a similar mechanism that hides NVMe devices that are connected through the chipset and instead makes them accessible through non-standard interfaces on the SATA controller.

For Intel's X299 HEDT platform for Skylake-X, the same chipset RAID capabilities are available, but the more interesting feature is Intel's Virtual RAID on CPU (VROC) for NVMe SSDs attached to the CPU's PCIe lanes. VROC is a software RAID system that builds atop the Intel Volume Management Device (VMD) feature of Skylake-X and Skylake-SP processors. VMD allows some of the CPU's PCIe ports to be partitioned off into a separate PCI domain behind a virtual PCI bridge. VMD has largely been described by Intel as a feature intended to simplify support for hotplug and enclosure management (status LEDs) of NVMe SSDs, but it apparently also helps build a robust software RAID stack. Intel has not shared much technical information about how VROC works, but they have announced that its features will be segmented by requiring the purchase of a hardware key to unlock RAID capability, and a premium key to unlock RAID-5.

AMD has not described any features of their Zen architecture that sound particularly similar to Intel's VMD or the NVMe remapping feature of Intel's consumer platform chipsets, so it's not clear what hardware features (if any) are helping to enable their NVMe RAID implementation. From a user perspective, setting up NVMe RAID is very similar to setting up chipset SATA RAID: configure the array through the motherboard firmware and load an extra driver when installing Windows. Unlike Intel's plans for VROC, AMD isn't trying for any extra product segmentation and isn't charging extra for any of their NVMe RAID functionality.

One important warning: the new RAID system is incompatible with the previous SATA RAID system for X399. Users with an existing SATA RAID array will not be able to access it using the new RAID drivers. The only migration path AMD currently offers is to re-create the array using the new drivers and firmware, and restore the data from a back-up. For users booting off a RAID array, this means re-installing Windows.