Benchmarking Performance: CPU System Tests

Our first set of tests is our general system tests. These set of tests are meant to emulate more about what people usually do on a system, like opening large files or processing small stacks of data. This is a bit different to our office testing, which uses more industry standard benchmarks, and a few of the benchmarks here are relatively new and different.

All of our benchmark results can also be found in our benchmark engine, Bench.

FCAT Processing: link

One of the more interesting workloads that has crossed our desks in recent quarters is FCAT - the tool we use to measure stuttering in gaming due to dropped or runt frames. The FCAT process requires enabling a color-based overlay onto a game, recording the gameplay, and then parsing the video file through the analysis software. The software is mostly single-threaded, however because the video is basically in a raw format, the file size is large and requires moving a lot of data around. For our test, we take a 90-second clip of the Rise of the Tomb Raider benchmark running on a GTX 980 Ti at 1440p, which comes in around 21 GB, and measure the time it takes to process through the visual analysis tool.

System: FCAT Processing ROTR 1440p GTX980Ti Data

FCAT likes single threaded performance, whcih shows the high frequency parts with faster memory near the top.

Dolphin Benchmark: link

Many emulators are often bound by single thread CPU performance, and general reports tended to suggest that Haswell provided a significant boost to emulator performance. This benchmark runs a Wii program that ray traces a complex 3D scene inside the Dolphin Wii emulator. Performance on this benchmark is a good proxy of the speed of Dolphin CPU emulation, which is an intensive single core task using most aspects of a CPU. Results are given in minutes, where the Wii itself scores 17.53 minutes.

System: Dolphin 5.0 Render Test

Dolphin is also pure ST frequency driven, however a surprise twist in that our Xeon W-2155 beats the Core i7-8086K in this test, although with a margin of error. 

3D Movement Algorithm Test v2.1: link

This is the latest version of the self-penned 3DPM benchmark. The goal of 3DPM is to simulate semi-optimized scientific algorithms taken directly from my doctorate thesis. Version 2.1 improves over 2.0 by passing the main particle structs by reference rather than by value, and decreasing the amount of double->float->double recasts the compiler was adding in. It affords a ~25% speed-up over v2.0, which means new data.

System: 3D Particle Movement v2.1

3DPM likes fast cache and frequency, and the W-2195 is almost fighting with the Core i9-7980XE here, and is let down slightly by its slow memory. The 1950X is still top dog.

DigiCortex v1.20: link

Despite being a couple of years old, the DigiCortex software is a pet project for the visualization of neuron and synapse activity in the brain. The software comes with a variety of benchmark modes, and we take the small benchmark which runs a 32k neuron/1.8B synapse simulation. The results on the output are given as a fraction of whether the system can simulate in real-time, so anything above a value of one is suitable for real-time work. The benchmark offers a 'no firing synapse' mode, which in essence detects DRAM and bus speed, however we take the firing mode which adds CPU work with every firing.

System: DigiCortex 1.20 (32k Neuron, 1.8B Synapse)

DigiCortex is a memory focused benchmark, but can also take advantage of AVX2 and sometimes AVX512, hence why the W-2195 is sat at the top. That being said, it is above the i9-7980XE, despite the latter having dual AVX512 ports.

Agisoft Photoscan 1.3.3: link

Photoscan stays in our benchmark suite from the previous version, however now we are running on Windows 10 so features such as Speed Shift on the latest processors come into play. The concept of Photoscan is translating many 2D images into a 3D model - so the more detailed the images, and the more you have, the better the model. The algorithm has four stages, some single threaded and some multi-threaded, along with some cache/memory dependency in there as well. For some of the more variable threaded workload, features such as Speed Shift and XFR will be able to take advantage of CPU stalls or downtime, giving sizeable speedups on newer microarchitectures.

System: Agisoft Photoscan 1.3.3 (Large) Total Time

Agisoft is a mixture of workloads, although the big multithreaded bit in the middle tends to dominate. Both the W-2195 and W-2155 score the same time, with a cluster of results around it. The Core i9-7960X sits on top though, with a seemingly better mix of cores and threads.

Benchmarking Performance: CPU Office Tests Benchmarking Performance: CPU Rendering Tests
Comments Locked

74 Comments

View All Comments

  • 0ldman79 - Tuesday, July 31, 2018 - link

    He was an Intel fanboy undercover as an AMD shill who's in Intel's pocket.

    He's a triple agent.

    The only way to keep is cover is to not show bias, be objective and report the facts as they stand, otherwise his employers will figure out he's turned (and turned and turned) and they'll cut him loose.

    True story.
  • BambiBoom - Monday, July 30, 2018 - link

    Anandateers,

    As a Xeon / workstation user for many years, this is an interesting review, but seems eccentric to typical workstation buyers' tendencies.

    There are three kinds of Xeon users based on the emphasis in applications: one, single thread performance such as 3D modeling, animation, and simulation, two, multi-threaded which includes CPU rendering, some well-threaded simulation, some aspects of video and photo-editing, and database, computational application, and three, a very common one in which the workstation has to be reasonably good at both single-threaded and multi-threaded. Solidworks is very useful to evaluate workstations as it's well-written and also demanding of both very high single and multi-thread performance. If you look at worsktation listings, the modeling units are i7-8800K A lot of Solidworks modelers are running i7-8800K systems- with the required Quadro. ECC is not as universal as it was five years ago. But, if the system is also used for the very well-threaded Solidworks rendering, they will use 10 or 12-core i7 or i9 that have high Turbo clock speeds.

    My solution to reasonable performance in both single and multi-threaded work is a Xeon E5-1680 v2 which I run at 4.3GHz on all 8 cores. Yes, you can overclock E5-1680 v2, E5-1660 v2, and E5-1650 v2. A friend of mine using Solidworks and Maya runs an E5-1680 v2 at 4.7GHz using a large custom-designed external cooling tower. Because a fast 8-core is the sweet spot in single and multi- thread balance, I'm sorry the review did not include tests of the Xeon W2145 (8C@3.7/4.5), also the W2135 (6C@3.7/4.5), and especially, the i7-7820X (8C@3.6/4.3) overclockable- there are a lot of 4.6-4.8GHz on Passmark and it's under $600 as compared to $1100 for the W-2145. The i7-7820X could be the perfect workstation CPU except for the limited PCIe lanes. The W-2145 has an excellent average single threaded mark of 2537 on Passmark, but that has a locked multiplier. The i7-7820X however at 4.8GHz has a calculated single thread up to 3100 and the top performer on Passmark at about 3185. This is even more interesting in comparison to the recent AMD Ryzen 2700X which has greatly improved Ryzen single thread performance. However, the highest CPU rating in Passmark calculates to about 2600- still really excellent: the average i7-7700K is 2583. The 2700X is tempting from a cost/performance standpoint, but those needing the highest possible single-threaded performance will stay with Intel.

    Overall I'm very glad to see a review of Xeon W's, but in my view, the inclusion of the low end OEM 4-core models, the inclusion of the limited issue i7-8086K and the obsolete i7-7700k instead of the i7-8800K, plus the exclusion of the 8-core Xeon W-2145 and it's i7 counterpart i7-7820X makes it less useful to the typical workstation buyer. Some comparative tasks within workstation applications would be more informative too than synthetic benchmarks.

    BambiBoom

    P.S. In my view, Intel is making a huge mistake with the upcoming i9-9700K. It's 8-core and has high clock speeds- 3.6/4.9GHz and will be overclockable, but minus hyperthreading is going to send many, many workstation buyers to whatever the Ryzen 2800X turns out to be. The lack of hyperthreading will make it a gaming-only CPU and what games will make much if any use of all 8 cores?
  • mode_13h - Monday, July 30, 2018 - link

    You mean i7-8700K, but yeah.

    A good workstation has more cores (and PCIe lanes) than desktop, but still good single-thread perf. It's mostly servers where you really care about aggregate performance more than single-thread.
  • Ian Cutress - Monday, July 30, 2018 - link

    >Overall I'm very glad to see a review of Xeon W's, but in my view, the inclusion of the low end OEM 4-core models, the inclusion of the limited issue i7-8086K and the obsolete i7-7700k instead of the i7-8800K, plus the exclusion of the 8-core Xeon W-2145 and it's i7 counterpart i7-7820X makes it less useful to the typical workstation buyer. Some comparative tasks within workstation applications would be more informative too than synthetic benchmarks.

    1) Unfortunately these were the only 5 SKUs we were able to get ahold of. Please pester Intel if you want to see more reviewed.
    2) We have many other CPUs tested in our database, www.anandtech.com/Bench
    3) We can't test every overclockable CPU at every frequency. Overclocking is in itself a niche (as much as people talk about it online, and we only pull out OC data unless it's universal: Running our suite CPU X at Frequencies YXZ just multiplies our testing time.
    4) Benchmarks: I've repeatedly asked in reviews over the years for users to get in contact with their preferred professional benchmarks. Some ideas were good (Agisoft, DigiCortex), some were not (licensable software doesn't scale over 5 systems testing simultaneously). But please keep emailing suggestions.

    Please bear in mind, not 100% of benchmarks have to be specifically for you. I get so many complaints about 'why include benchmark X?' because it doesn't pertain to that user. There are other users who prefer other benchmarks. Even if 50% or 20% of the benchmarks are relevant to you, that's the take-home data for you, not the others. The others are for other people. Don't expect 100% of all the data points to be relevant for you.
  • diehardmacfan - Tuesday, July 31, 2018 - link

    Have you looked into using SPECviewperf 13 for testing? It's probably the best all-in-one suite for workstation performance.
  • BambiBoom - Tuesday, July 31, 2018 - link

    Ian Cutress,

    I don't expect any review to have any particular content specific to me and don't understand your creating that inference.

    My point was that the review was less relevant to buyers considering Xeon W's by omitting an important category of processors, the fast 6-core and moreso the W-2145 8-core. As those processors were unavailable, we'll look forward to seeing something about them perhaps another time.

    It seems that those buying HP, Dell, Lenovo, Boxx, and Puget systems are going to buy quite a few W-2123, W-2125, W-2135, W-2145 plus the i7-and i9 alternatives including i7-8700K and i7-7820X and I would have liked to have seen those reviewed. Those with $2,000 can have good performance in both in 10, 12 and greater core count CPU's, but the 8-core is important in having a good balance at a comfortable cost. Tests have shown- e.g. the Puget Systems articles that demonstrate that multi-threaded applications often (e.g. Adobe) have peak core utiltization at 5-6 cores and so a fast 8- core is a good solution. As the Xeon W-2145 and i7-7820X are both 8-cores @ 3.7/4.5, it seemed that would be a good "center" for a Xeon W- review. I also don't expect a review to try every overclocked CPU, but in the example of comparing the W-2145 and i7-7820X, it seems interesting to note that the i7-7820X may be run at 4.8GHz for $500 less (-45%) than the 4.5GHz W-1245.

    You seem to have stored up a good supply of impatience and anger towards readers making requests or suggestions. Keep in mind we're not as expert, have the range of technical or market knowledge, nor access to so many components as do you. Do you enjoy this work?

    BambiBoom
  • tmediaphotography - Tuesday, July 31, 2018 - link

    "You seem to have stored up a good supply of impatience and anger towards readers making requests or suggestions." You have to empathize with Ian, and the other reviewers. With every article / review they publish here there seems to be a small cadre of users that will criticize everything they write, deride them, belittle them. After a time it gets old, and starts to weigh heavily. All while working under intense pressure from deadlines.

    Constructive criticism is a thing most people should read into. In my job as a photographer / trainer, the basic idea is "Okay, this is what is good about the image, this is what is wrong and here is how to improve upon it in the future." While your comment had much of the workings for a proper CC, I also don't see Ian's anger or impatience. I only see him reiterating a point he's had to make dozens of times.
  • 0ldman79 - Tuesday, July 31, 2018 - link

    Agreed.

    I've seen where Ian was getting irritated, this didn't appear to be one of those moments.
  • BambiBoom - Wednesday, August 1, 2018 - link

    imediaphotography,

    As Anandtech is owned by Purch Group which has a marketing function; "to better reflect its growing portfolio of brands and products focused on purchasing decisions", one would think that mentioning median level products not reviewed but that were, in my view, probably more likely to be purchased by workstation buyers would have had a better reception. Instead of addressing the specifics of the content and intention of my comments, Ian's reply was almost entirely a general rant as to readers' unfair comments and those applied applied to me.

    Reviewers/ critics that take every comment personally seem unhappy in their work. Yes, reviewing is difficult and under pressure of deadlines, but your idea of focusing on constructive criticism is the proper mode of response. In a professional media situation, applying general irritation based on past experience to an individual has no excuse. Ian could have simply said that the processors missing from the review would be reviewed when available and not use most of his comments repeating that I insist every review revolve around me.

    I've been a reviewer / critic, having had a radio program in Los Angeles for six years. During that run I became used to criticism of my choices for review and comments, and once in a while this was vehement. I once had a fellow make semi-obscene comments and implied threats concerning something I said- about Mozart. Still, my perplexity and pique was never applied to any other caller.

    BambiBoom
  • JlHADJOE - Tuesday, July 31, 2018 - link

    To me it looks like forcibly introducing an i9 SKU and it's need for differentiation with an i7 messed up the whole lineup.

    If they had stuck with the traditional lineup, then we could have had 8/16 i7, 6/12 i5, and 4/8 i3 and the entire line would have been much better across the board.

Log in

Don't have an account? Sign up now