NVMe Recap

Before diving into our results, I want to spend a bit of time talking about NVMe (Non-Volatile Memory Express) as a command set for PCIe-based storage first. NVMe has been in the OEM and enterprise space for over a year, but it's still very much a new thing in the end-user system builder space due to the fact that NVMe drives were not available through regular retail channels until now. So let's start things by spending a moment to recap what NVMe is, how it works, and why it is such an important improvement over AHCI for SSDs.

Traditional SATA drives, such as mechanical hard drives or SSDs, are connected to the system through a controller sometimes referred to as a Host Bus Adapter (HBA) or Host Controller. This is powered through the south bridge of the system, which hosts the input/output and talks to the main processor. On the "upstream" side of the HBA to the main processor is a PCI Express-like connection, and on the downstream side from the chipset to the drives are SATA links. Almost all SATA controllers, including the ones built in to the chipsets on motherboards, adhere to the Advanced Host Controller Interface (AHCI) standard, which allows them to all work with the same class driver. SATA drives connected through such controllers are accessed using the ATA command set, which maintains a standard language for all drives to use.

Early PCIe SSDs on the other hand either implemented proprietary interfaces requiring custom drivers, or they implemented older AHCI and ATA command set and as such appeared to the OS to be a SATA drive in every way (except for the 6Gb/s speed limit). As a transition mechanism to help smooth out the rollout of PCIe SSDs, using AHCI over PCIe was a reasonable short-term solution, however over time AHCI itself became a bottleneck to what the PCIe interface and newer SSDs were capable of.

  NVMe AHCI
Latency 2.8 µs 6.0 µs
Maximum Queue Depth Up to 64K queues with
64K commands each
Up to 1 queue with
32 commands each
Multicore Support Yes Limited
4KB Efficiency One 64B fetch Two serialized host
DRAM fetches required

To allow SSDs to better take advantage of the performance possible through PCI Express, a new host controller interface and command set called NVMe was developed and standardized. NVMe's chief advantages are lower latency communication between the SSD and the CPU, and lower CPU usage when communicating with the SSD (though the latter usually only matters in enterprise scenarios). The big downside is that it's not backwards-compatible: in order to use an NVMe SSD you need an NVMe driver, and in order to boot from an NVMe drive your motherboard's firmware needs NVMe support. The NVMe standard has now been around long enough that virtually every consumer device with an M.2 slot providing PCIe lanes should have NVMe support or a firmware update available to add it, so booting off the 950 Pro poses no particular trouble (as odd as it sounds, some older enterprise/workstation systems may not have an NVMe update, and users should check with their system manufacturer).

Meanwhile on the software side of matters, starting with Windows 8.1 and Windows 10, Windows has a built-in NVMe driver that implements all the basic functionality necessary of NVMe for everyday use. But basic is it; the default NVMe class driver is missing some features necessary for things like updating drive firmware and accessing some diagnostic information. For that reason, and to accommodate users who can't update to a version of Windows that has an NVMe driver available from Microsoft, most vendors are providing a custom NVMe driver. Samsung provided a beta version of their driver, as well as a beta of their SSD Magician software that now supports the 950 Pro (but not their previous M.2 drives for OEMs). Almost all of the features of SSD Magician require Samsung's NVMe driver for use with the 950 Pro.

Finally, some of our benchmarking tools are affected by the switch to NVMe. Between most of our tests, we wipe the drive back to a clean state. For SATA drives and PCIe drives using AHCI, this is accomplished using the ATA security features. The NVMe command set has a format command that can be specified to perform a secure erase, producing the same result but requiring a different tool to issue the necessary commands. Likewise, our usual tools for recording drive performance during the AnandTech Storage Bench tests don't work with NVMe, so we're using a different tool to capture that data, but the same tools to process and analyze it.

Measuring PCIe SSD Power Consumption

Our SSD testbed is now equipped to measure power consumption of PCIe cards, and we're using an adapter to extend that capability to PCIe M.2 drives. M.2 drives run on 3.3V where most SATA devices use 5V, but we account for that difference by using reporting power draw in Watts or milliWatts. This review is our first look at the power draw behavior of a PCIe drive and our first opportunity to explore the power management capabilities of PCIe drives. To offer some points of comparison, we're re-testing our samples of the Samsung SM951 and XP941, earlier PCIe M.2 drives that were sold to OEMs but not offered in the retail channel.

This analysis has turned up some surprises. For starters, the 950 Pro's power consumption noticeably increases as it heats up - indicating that the heat/leakage effect is enough of a factor to be measurable here - and I've seen its idle power climb by as much as 4.5% from power on to equilibrium. Pointing a fan at the drive quickly brings the power back down. For this review, I made no special effort to cool the 950 Pro. Samsung ships it without a heatsink and assures us that it has built-in thermal management capabilities, so I tested it as-is in our standard case scenario with the side panel removed.

It appears that Samsung's NVMe drives have much higher idle power consumption than the AHCI drives, even when using the same UBX controller. It's clear that our system configuration is not putting the 950 Pro or the SM951 with NVMe into a low power state when idle, but the cause for what that happens is not clear. The power levels reported in the graph below are all attainable even before the operating system has loaded and they don't improve any once Samsung's NVMe driver loads, which points to an issue with either the drive firmware or the motherboard firmware.

M.2 PCIe Drives Idle Power

Furthermore, we have a clear indication of at least one motherboard bug. PCI Express Active State Power Management (ASPM) is a feature that allows a PCIe link to be slowed down to save power, something that is quite useful for a SSD that experiences long idle periods. ASPM can be activated in just the downstream direction (CPU to device) or in both directions. The latter is what offers significant power savings for a SSD. Our testbed motherboard offers options to configure ASPM, but when enabling the more aggressive bidirectional ASPM level, it locks up very frequently. I tried to test ASPM on my personal Haswell-based machine with a different motherboard from a different vendor, but it didn't offer any option to enable ASPM.

Using a slightly older Ivy Bridge machine with an Intel motherboard, I was able to confirm that the 950 Pro doesn't have any issues with ASPM, and that it does offer significant power savings. However, I wasn't able to dig for further power savings on that system, and all of the power measurements reported with the performance benchmarks in this review were performed on our usual testbed with ASPM off, as it has been for all previous reviews.

Motherboard power management bugs are tragically common in the desktop space, and devices that incorrectly implement ASPM are common enough that it is seldom enabled by default. As PCIe peripherals of all kinds become more common, the industry is going to have to shape up in this department, but for now consumers should not assume that ASPM will work correctly out of the box.

Introduction Performance Consistency
POST A COMMENT

142 Comments

View All Comments

  • SunnyNW - Saturday, October 24, 2015 - link

    Actually shame on you for telling others what their priorities should be. Boot times are Very important to me and was one of the main reasons I upgraded from a hdd to a ssd in the first place. I dont want to have to wait more than the 15 seconds it takes my system to boot right now. People have to boot/restart their machine for various reasons and variable amounts of times, I dont want to have to wait more than 20 sec or so Every time I update software/drivers that require a reboot (windows update, gpu driver update, etc.). Almost every time I leave my house for more than 8 hrs or so I shut down my machine, I just dont have good luck with sleep on it, not sure if it's because of the radeon or what but ehh. Reply
  • Rajinder Gill - Saturday, October 24, 2015 - link

    If you leave the house for 8 hours, your time cannot be worth enough money to be worried about 20 seconds of BOOT time :) Reply
  • SunnyNW - Sunday, October 25, 2015 - link

    LOL!!! You are one Hilarious (my most polite way of saying pathetic) character. If you had any clue as to how much I Actually earn for my time, from all my various ventures especially my business, your ugly little smiles at the end of all your comments would more accurately be portrayed with a symbol that expressed your jaw wide open and hitting the floor. : 0 And how many times are you going to mention S3, lol, let me guess it's something you just recently learned about so you spout off about it Every single chance you get. Please don't reply to my comments in the future. I come here for intelligent interactions, not some nonsense from someone that has No Clue. Thank You in advance, Oh and just for you ;) Reply
  • Rajinder Gill - Sunday, October 25, 2015 - link

    S3 resume, USE IT! One more time just for you :) Reply
  • xype - Monday, October 26, 2015 - link

    SunnyNW, I never met one single person who made a shitload of money "with their time" who spent that time arguing online on tech sites, and arguing trivialities like SSD boot times—congrats, you’re the first!

    No person I know that makes $150+/hour gives a shit about boot times, and the people who do (working in IT, specifically high availability infrastructure ops) are in a whole different market from the one that AnandTech usually covers.
    Reply
  • SunnyNW - Monday, October 26, 2015 - link

    You are entitled to your opinion. But I agree, this was the first time I made a reference to my personal life finances thru comments and there is no need for that here. I just become Very annoyed when one judges others' priorities and I was already upset from other bs. Anyway not one of my finer moments. Reply
  • SunnyNW - Monday, October 26, 2015 - link

    Lastly Ill add that 'making wrong assumptions' (especially when insulting) and 'questioning others priorities' are my pet peeves. On any other site I would not have replied in the first place. I dont know why but I feel differently when it comes to anandtechs community, I feel there is a higher standard here and many of the comments are usually very informative and worthwhile discussions. I work extremely hard and do not need someone telling me how much my time is worth. I hate when people sit behind their computer making assumptions about someone that they know absolutely nothing of, I do not expect that from the community here. I highly respect most of the commenters on anandtech and would like that in return. I feel anandtech is full of mostly intellectual persons. There are not many commenters on anandtech like mr. gill and I just let him get to me way more than I should have. Again I shouldnt have to explain myself to you but iunno here I am doing just that...I just turned 22 recently so gimme a beak, I'm learning. Reply
  • Rajinder Gill - Saturday, October 24, 2015 - link

    If the sleep issues are due to the Radeon and BOOT time is that important, then perhaps a S3 resume complaint VGA would get you back to the desktop quicker. ;) Reply
  • SunnyNW - Sunday, October 25, 2015 - link

    What is a "S3 resume complaint?" I have no complaints with my GPU and am very content with the 15 seconds it takes my machines to reach the desktop. Reply
  • ewitte - Thursday, November 12, 2015 - link

    Working in IT the most annoying thing is boot times on OTHERS machines. I still run into spinning drives and insufficient memory which could be upwards of 10 minutes before performance stabilizes after a reboot. Reply

Log in

Don't have an account? Sign up now