Benchmark Overview

For our testing, depending on the product, we attempt to tailor the presentation of our global benchmark suite down into what users who would buy this hardware might actually want to run. For CPUs, our full test suite is typically used to gather data and all the results are placed into Bench, our benchmark database for users that want to look at non-typical benchmarks or legacy data. For motherboards we run our short form CPU tests, the gaming tests with half the GPUs of our processor suite, and our system benchmark tests which focus on non-typical and non-obvious performance metrics that are the focal point for specific groups of users.

The benchmarks fall into several areas:

Short Form CPU

Our short form testing script uses a straight run through of a mixture of known apps or workloads, and requires about four hours. These are typically the CPU tests we run in our motherboard suite, to identify any performance anomalies.

CPU Short Form Benchmarks
Three Dimensional Particle Movement (3DPM) v1 3DPM is a self-penned benchmark, derived from my academic research years looking at particle movement parallelism. The coding for this tool was rough, but emulates the real world in being non-CompSci trained code for a scientific endeavor. The code is unoptimized, but the test uses OpenMP to move particles around a field using one of six 3D movement algorithms in turn, each of which is found in the academic literature. This test is performed in single thread and multithreaded workloads, and uses purely floating point numbers. The code was written in Visual Studio 2008 in Release mode with all optimizations (including fast math and -Ox) enabled. We take the average of six runs in each instance.
v2 The second version of this benchmark is similar to the first, however it has been re-written in VS2012 with one major difference: the code has been written to address the issue of false sharing. If data required by multiple threads, say four, is in the same cache line, the software cannot read the cache line once and split the data to each thread - instead it will read four times in a serial fashion. The new software splits the data to new cache lines so reads can be parallelized and stalls minimized.

As v2 is fairly new, we are still gathering data and results are currently limited.
WinRAR 5.01 WinRAR is a compression based software to reduce file size at the expense of CPU cycles. We use the version that has been a stable part of our benchmark database through 2015, and run the default settings on a 1.52GB directory containing over 2800 files representing a small website with around thirty half-minute videos. We take the average of several runs in this instance.
POV-Ray 3.7 beta POV-Ray is a common ray-tracing tool used to generate realistic looking scenes. We've used POV-Ray in its various guises over the years as a good benchmark for performance, as well as a tool on the march to ray-tracing limited immersive environments. We use the built-in multithreaded benchmark.
HandBrake  HandBrake is a freeware video conversion tool. We use the tool in to process two different videos - first a 'low quality' two hour video at 640x388 resolution to x264, then a 'high quality' ten minute video at 4320x3840. The low quality video scales at lower performance hardware, whereas the buffers required for high-quality can stretch even the biggest processors. At current, this is a CPU only test.
7-Zip 7-Zip is a freeware compression/decompression tool that is widely deployed across the world. We run the included benchmark tool using a 50MB library and take the average of a set of fixed-time results.

System Benchmarks

Our system benchmarks are designed to probe motherboard controller performance, particularly any additional USB controllers or the audio controller. As general platform tests we have DPC Latency measurements and system boot time, which can be difficult to optimize for on the board design and manufacturing level.

System Benchmarks
Power Consumption One of the primary differences between different motherboads is power consumption. Aside from the base defaults that every motherboard needs, things like power delivery, controller choice, routing and firmware can all contribute to how much power a system can draw. This increases for features such as PLX chips and multi-gigabit ethernet.
Non-UEFI POST Time The POST sequence of the motherboard becomes before loading the OS, and involves pre-testing of onboard controllers, the CPU, the DRAM and everything else to ensure base stability. The number of controllers, as well as firmware optimizations, affect the POST time a lot. We test the BIOS defaults as well as attempt a stripped POST.
Rightmark Audio Analyzer 6.2.5 Testing onboard audio is difficult, especially with the numerous amount of post-processing packages now being bundled with hardware. Nonetheless, manufacturers put time and effort into offering a 'cleaner' sound that is loud and of a high quality. RMAA, with version 6.2.5 (newer versions have issues), under the right settings can be used to test the signal-to-noise ratio, signal crossover, and harmonic distortion with noise.
USB Backup USB ports can come from a variety of sources: chipsets, controllers or hubs. More often than not, the design of the traces can lead to direct impacts on USB performance as well as firmware level choices relating to signal integrity on the motherboard.
DPC Latency Another element is deferred procedure call latency, or the ability to handle interrupt servicing. Depending on the motherboard firmware and controller selection, some motherboards handle these interrupts quicker than others. A poor result could lead to delays in performance, or for example with audio, a delayed request can manifest in distinct audible pauses, pops or clicks.

Gaming

Our Gaming test suite is still our 2015 implementation, which remains fairly solid over gaming title updates. We are still working on a 2017 suite update, with a move to Windows 10. This will allow most of the titles to be replaced with DirectX 12, indie and eSports games.

Gaming Benchmarks
Alien: Isolation If first person survival mixed with horror is your sort of thing, then Alien: Isolation, based off of the Alien franchise, should be an interesting title. Developed by The Creative Assembly and released in October 2014, Alien: Isolation has won numerous awards from Game Of The Year to several top 10s/25s and Best Horror titles, ratcheting up over a million sales by February 2015. Alien: Isolation uses a custom built engine which includes dynamic sound effects and should be fully multi-core enabled.
Total War: Attila The Total War franchise moves on to Attila, another The Creative Assembly development, and is a stand-alone strategy title set in 395AD where the main story line lets the gamer take control of the leader of the Huns in order to conquer parts of the world. Graphically the game can render hundreds/thousands of units on screen at once, all with their individual actions and can put some of the big cards to task.
Grand Theft Auto V The highly anticipated iteration of the Grand Theft Auto franchise finally hit the shelves on April 14th 2015, with both AMD and NVIDIA in tow to help optimize the title. GTA doesn’t provide graphical presets, but opens up the options to users and extends the boundaries by pushing even the hardest systems to the limit using Rockstar’s Advanced Game Engine. Whether the user is flying high in the mountains with long draw distances or dealing with assorted trash in the city, when cranked up to maximum it creates stunning visuals but hard work for both the CPU and the GPU.
GRID: Autosport No graphics tests are complete without some input from Codemasters and the EGO engine, which means for this round of testing we point towards GRID: Autosport, the next iteration in the GRID and racing genre. As with our previous racing testing, each update to the engine aims to add in effects, reflections, detail and realism, with Codemasters making ‘authenticity’ a main focal point for this version.
Middle-Earth: Shadow of Mordor The final title in our testing is another battle of system performance with the open world action-adventure title, Shadow of Mordor. Produced by Monolith using the LithTech Jupiter EX engine and numerous detail add-ons, SoM goes for detail and complexity to a large extent, despite having to be cut down from the original plans. The main story itself was written by the same writer as Red Dead Redemption, and it received Zero Punctuation’s Game of The Year in 2014.
Test Bed and Setup BIOS
Comments Locked

37 Comments

View All Comments

  • ddriver - Monday, February 6, 2017 - link

    Great more rainbows, cuz nothing spells "pro" like rainbows do. IMO people already know that RGB LEDs implies they can change color, so rainbows aren't really necessary, unless trying to appeal to a specific and fairly narrow market niche.
  • tarqsharq - Monday, February 6, 2017 - link

    Are you guys looking to update your review GPU?

    A GTX980 is seriously falling behind the times and will bottleneck a CPU in many titles even at 1080p.
  • TEAMSWITCHER - Monday, February 6, 2017 - link

    If a GTX 980 is a bottleneck for 1080p gaming, then I'm officially done with the PC Gaming Industry ... forever. Have fun playing games that look pretty and run crappy.
  • TheinsanegamerN - Wednesday, February 8, 2017 - link

    Oh no! a 2014 GPU is a bottleneck in the newest games in 2017?!? WHO WOUDDA THUNK?!?

    The 980 is a good gaming chip, but for benchmarks like this, it's been long known that there are better parts for the job. 980ti/1070 should be the minimum.
  • close - Monday, February 6, 2017 - link

    Since the gaming PC is the only area where PC sales aren't slumping that niche might not be so small. You'd be amazed how many people dream of having a gaming system with the typical gaudy computer case, mandatory transparent side-panel, as many LEDs splashed around in every little component, the RGB-er the merrier, hopefully any kind of custom watercooling loop to show fluorescent tubing on top of everything else.

    It even says "Gaming Pro", not "Pro".

    Ian, are you sure about the $0.002 per HDMI connector thingy?
  • ddriver - Monday, February 6, 2017 - link

    I meant leprechauns doh :D
  • 80-wattHamster - Tuesday, February 7, 2017 - link

    It's probably not far off. When you get to quantities in the tens of thousands and up, stuff like that gets CHEAP. It's not $0.002 in final cost; production time and resources aren't free. Having an HDMI connector vs. not probably ends up being a difference of $0.10 (+/- $0.05). Folks that know electronics manufacturing better than I, please correct.
  • Murloc - Thursday, February 9, 2017 - link

    ddriver wasn't complaining about the RGB LEDs, but rather that to advertise them they put them in a rainbow configuration, he thinks that it should be obvious and so they shouldn't do it, unless they were trying to attract people who like rainbows.
    I don't agree with him, rainbow advertising material makes it clear that the LEDs are INDEPENDENTLY controlled.
  • Flunk - Monday, February 6, 2017 - link

    I don't think I agree with you. I'd say the best feature of customizable RGB lighting is that you can turn it off or just pick a suitable colour scheme. That way it doesn't matter what you want, the board will still appeal. I'm not big for the gaudy crap (I once bought an Alienware m14x R2 because of the massive discount and ran it with only the keyboard lights on an set to white the whole time), but it doesn't really bother me if the LEDs are there as long as I can turn it off. The reason you're seeing these everywhere is that they don't add very much cost to the bill of materials.
  • ddriver - Monday, February 6, 2017 - link

    I don't think you got my point. I don't have a problem with customizable RGB LEDs, I was just pointing out that the rainbow showcasing really ruins the "pro" looks of the product.

Log in

Don't have an account? Sign up now