Performance Consistency

We've been looking at performance consistency since the Intel SSD DC S3700 review in late 2012 and it has become one of the cornerstones of our SSD reviews. Back in the days many SSD vendors were only focusing on high peak performance, which unfortunately came at the cost of sustained performance. In other words, the drives would push high IOPS in certain synthetic scenarios to provide nice marketing numbers, but as soon as you pushed the drive for more than a few minutes you could easily run into hiccups caused by poor performance consistency. 

Once we started exploring IO consistency, nearly all SSD manufacturers made a move to improve consistency and for the 2015 suite, I haven't made any significant changes to the methodology we use to test IO consistency. The biggest change is the move from VDBench to Iometer 1.1.0 as the benchmarking software and I've also extended the test from 2000 seconds to a full hour to ensure that all drives hit steady-state during the test.

For better readability, I now provide bar graphs with the first one being an average IOPS of the last 400 seconds and the second graph displaying the IOPS divided by standard deviation during the same period. Average IOPS provides a quick look into overall performance, but it can easily hide bad consistency, so looking at standard deviation is necessary for a complete look into consistency.

I'm still providing the same scatter graphs too, of course. However, I decided to dump the logarithmic graphs and go linear-only since logarithmic graphs aren't as accurate and can be hard to interpret for those who aren't familiar with them. I provide two graphs: one that includes the whole duration of the test and another that focuses on the last 400 seconds of the test to get a better scope into steady-state performance. These results are for all drives 240GB and up.

Steady-State 4KB Random Write Performance

Steady-state performance is fairly good, although given that the SX930 employes 12% over-provisioning it's expected to perform better than drives with only 7%. 

Steady-State 4KB Random Write Consistency

Consistency, on the other hand, isn't that good. The good news is that the JMF670H is more consistent than the SM2246EN in the BX100, but there's a long way to achieve Samsung-level consistency. 

ADATA XPG SX930
Default
25% Over-Provisioning

As the SX930 isn't very consistent, we saw performance dropping below 1,000 IOPS. It seems that consistency is one of the areas that truly separate Samsung and Marvell drives from the rest because JMicron, Silicon Motion and Phison based drives all have trouble sustaining steady performance. Basically, the baseline performance is about 1,000 IOPS for all controllers with frequent peaks of +5,000 IOPS, whereas Samsung and Marvell drives have very little variation in performance. I suspect the controller design itself has something to do with this because Samsung and Marvell controllers are all multi-core, but JMicron and Silicon Motion rely on single-core designs for higher cost efficiency. 

ADATA XPG SX930
Default
25% Over-Provisioning
Introduction, The Drives & The Test AnandTech Storage Bench - The Destroyer
POST A COMMENT

67 Comments

View All Comments

  • Oxford Guy - Friday, July 17, 2015 - link

    We'll just pretend that the 840 EVO had not been heavily hyped by commentators like you all over the net for quite a long time as well as by review sites. We will also just pretend that it did not have very heavy sales figures. Instead because Samsung put out with a new drive that you want to push we will just pretend that anyone who bought the previous generation drives were idiots. Reply
  • Oxford Guy - Friday, July 17, 2015 - link

    Plus, it was Samsung who decided to stick with the EVO name. Reply
  • sonny73n - Saturday, July 18, 2015 - link

    @Oxford Guy,

    I couldn't agree more with you. Thank you for explaining things clearer than I ever could have.
    Reply
  • sonny73n - Friday, July 17, 2015 - link

    @Stochastic

    840 EVO sequential read at ~100MB/s. My 5900RPM Seagate 2TB HDD is at ~135MB/s. Lol
    Reply
  • leexgx - Friday, July 17, 2015 - link

    what about random data speeds (as that's more the problem with HDDS) Reply
  • sonny73n - Saturday, July 18, 2015 - link

    What about a product that doesn't meet advertised specs? "Fool me once, shame on you..." I'll make sure the second line of that saying won't apply to me. Reply
  • leexgx - Saturday, July 18, 2015 - link

    if you're using a 840 EVO you can update the firmware to restore the drive to more so normal performance (unless your on linux or OSX then best not) even so without the firmware update compared to a HDD the 840 evo SSD is overall faster Reply
  • sonny73n - Saturday, July 18, 2015 - link

    By the way, I'm still rocking my i5-2500K system OCed @4.5GHz. I always had an SSD for boot drive (C:) and 2 HDD drives for storage since the beginning. Why should there be problems with the HDDs as they're just storage drives? Damn why did I feel the urge to explain to a 5th grader? Reply
  • bill.rookard - Thursday, July 16, 2015 - link

    I was never crazy about the TLC premise either. I prefer the 830's for boot drives since they're very reliable and have a more proven track record. I think the 850's with the 3D NAND should be pretty durable as well. In my servers I have the Crucial MX100's (a pair of 256Gs - one for data, one for sync, but -not- raided) and they're solid drives and have not had a single spot of trouble with those. Reply
  • fokka - Thursday, July 16, 2015 - link

    after the 840 evo fiasko i'm reluctant to put my trust in samsung, especially when there are perfectly capable, performant and affordable SSDs like the bx100 available. 3d nand is nice and all, but for me crucial just seems like the safer bet right now. Reply

Log in

Don't have an account? Sign up now