Performance Consistency

We've been looking at performance consistency since the Intel SSD DC S3700 review in late 2012 and it has become one of the cornerstones of our SSD reviews. Back in the days many SSD vendors were only focusing on high peak performance, which unfortunately came at the cost of sustained performance. In other words, the drives would push high IOPS in certain synthetic scenarios to provide nice marketing numbers, but as soon as you pushed the drive for more than a few minutes you could easily run into hiccups caused by poor performance consistency. 

Once we started exploring IO consistency, nearly all SSD manufacturers made a move to improve consistency and for the 2015 suite, I haven't made any significant changes to the methodology we use to test IO consistency. The biggest change is the move from VDBench to Iometer 1.1.0 as the benchmarking software and I've also extended the test from 2000 seconds to a full hour to ensure that all drives hit steady-state during the test.

For better readability, I now provide bar graphs with the first one being an average IOPS of the last 400 seconds and the second graph displaying the standard deviation during the same period. Average IOPS provides a quick look into overall performance, but it can easily hide bad consistency, so looking at standard deviation is necessary for a complete look into consistency.

I'm still providing the same scatter graphs too, of course. However, I decided to dump the logarithmic graphs and go linear-only since logarithmic graphs aren't as accurate and can be hard to interpret for those who aren't familiar with them. I provide two graphs: one that includes the whole duration of the test and another that focuses on the last 400 seconds of the test to get a better scope into steady-state performance.

Steady-State 4KB Random Write Performance

Barefoot 3 has always done well in steady-state performance and the Vector 180 is no exception. It provides the highest average IOPS by far and the advantage is rather significant at ~2x compared to other drives.

Steady-State 4KB Random Write Consistency

But on the down side, the Vector 180 also has the highest variation in performance. While the 850 Pro, MX100 and Extreme Pro are all slower in terms of average IOPS, they are a lot more consistent and what's notable about the Vector 180 is how the consistency decreases as the capacity goes up. 

OCZ Vector 180 240GB
Default
25% Over-Provisioning

Looking at the scatter graph reveals the source of poor consistency: the IOPS reduce to zero or near zero even before we hit any type of steady state. This is known behavior of the Barefoot 3 platform, but what's alarming is how the 480GB and 960GB drives frequently drop to zero IOPS. I don't find that acceptable for a modern high-end SSD, no matter how good the average IOPS is. Increasing the over-provisioning helps a bit by shifting the dots up, but it's still clear that 240GB is the optimal capacity for Barefoot 3 because after that the platform starts to run into issues with consistency due to metadata handling.

OCZ Vector 180 240GB
Default
25% Over-Provisioning
SSD Guru: The New OCZ Toolbox AnandTech Storage Bench - The Destroyer
Comments Locked

89 Comments

View All Comments

  • LB-ID - Thursday, March 26, 2015 - link

    Looks like an overpriced product that doesn't stack up against competition in the same price range. Given OCZ's extensive track record of failures and attempts to use their user base as unpaid beta testers I can understand their desire to include a feature to give more peace of mind, but in reality it's just not that useful and certainly insufficient to overcome years of accumulated ill will.
  • serndipity - Thursday, March 26, 2015 - link

    The specifications say it all.

    On a $, performance, reliability, endurance, warranty basis, the OCZ 180 falls significantly short of even the consumer grade competition (e.g. Crucial or Samsung).

    Unfortunately, the OCZ story when from deceiving investors, to customers, ending in bankruptcy.

    Sadly, the same management etc., remains in place

    Be smart and avoid OCZ
  • ocztosh - Monday, March 30, 2015 - link

    Hi LB-ID, thank you for your feedback. We have shipped countless Barefoot 3 based drives across multiple series products and never once had to rev silicon. We have made updates to firmware to continually improve our products. The addition of the PFM+ feature was a decision to support customers that are really on the edge of enthusiast and workstation. This is a required feature for many enterprise drives and we felt there was a value to including this feature into our Vector 180 Series. We agree that not every customer necessarily needs this feature, and for those customers across the client spectrum we have other BF3 based drives that do not include the additional circuitry, but for those customers that really would like that extra layer of protection we were able to integrate this feature without a price premium over our previous Vector 150 Series. We have already received a very positive response from some users but will continue to offer a range of drives to meet the needs of different user applications.
  • voicequal - Friday, March 27, 2015 - link

    Wow, great review and a ton of new tests and data. The bathtub curve on the mixed sequential read/write performance will be very interesting to compare to forthcoming NVMe drives. I was surprised by how much the IOPS were reduced by commingling of sequential reads/writes. Seems to be room for improvement in this area.
  • Ethos Evoss - Saturday, March 28, 2015 - link

    The worst company of SSD's some OCZ or OWC or PVC .. jeez trash.. it's been sold to toshiba anyway ..
  • ocztosh - Monday, March 30, 2015 - link

    Hello Ethos Evoss. We are sorry that you feel this way and/or if you had a negative experience in the past. We are indeed now part of Toshiba and OCZ Storage Solutions is a very different organization as a subsidiary of Toshiba. Toshiba has provided the ability to vastly enhance resources for product development as well as given us complete access to their premium NAND. We certainly will continue to strive to meet the high expectations of you, our valued customers. Thank you again for your input.
  • daerron - Tuesday, May 26, 2015 - link

    Nice in depth analysis Anandtech! Read the article as I'm receiving a new Vector 180 240GB drive this week as my original Vector bricked itself earlier this month after around 2 years of service. I have to say that I'm really impressed with OCZ's customer service as I couriered the drive last week and will be receiving a brand new and latest SSD in return this week. Fortunately I had my data backed up as it really just died without any warning. Based on the analysis I was also thinking this would be a killer laptop drive, till I saw the lack of slumber support which is a bit of a disappointment and opportunity missed. Still the PFM+ and low power usage is also most welcome in my workstation PC. We have regular power cuts in my country so its great to have that extra peace of mind.
  • bogdan_kr - Thursday, December 17, 2015 - link

    Hello guys. I would like to ask how exactly this Storage Bench - Light is designed. The description says it is designed to be an accurate illustration of basic usage with Total IO Operations 832,339, Total GB Read 17.97 and Total GB Written 23.25.
    With its 23 GB of writes it should account for roughly two or three days of average usage (i.e. 7-11GB per day).

    How long (typically) does it take to finish this bench?
    Is it designed to last for few hours (which would account for a typical daily use) or is it designed to finish as fast as possible, in few minutes perhaps?
  • bogdan_kr - Saturday, January 23, 2016 - link

    I am aware that most likely it is a rhetorical question but can anyone from AnandTech answer it, please? I just wanted to be sure about that benchmark.

Log in

Don't have an account? Sign up now