AMD Socket-F Opteron vs. Intel Woodcrest
by Jason Clark & Ross Whitehead on December 18, 2006 12:05 AM EST- Posted in
- IT Computing
Benchmark Methodology
For AnandTech Database Benchmarks, we have always focused on "real world" Benchmarks. To achieve this, we have used real applications with loads such that CPU utilization was 80-90%. Recently we discussed how most Enterprise Database Servers do not average 80-90% CPU utilization, but rather something closer to the 30-60% range. We thought it would make more sense to show performance where it is most likely going to be used, as well as the saturation numbers for the situations where the CPU is maxed.
We feel this is consistent with how GPUs are reviewed, and how you might test drive a car. With GPUs, the cards are tested with varying resolutions, and anti-aliasing levels. With a car, you don't just hit the highway and see what the top end is.
We settled on six load points 20%, 40%, 60%, 80%, 100%, and 120+% for testing the varying ranges of load. These load points are consistent across all platforms and are throttled from the client, independent of the platform being measured. We chose these load points as they split the load range into 6, roughly, equal parts and allow us to extrapolate data between the points. The 120+% load point was included to verify that our 100% load point really was 100%.
The 100% load point was determined by starting an execution of the client and adding threads until the CPU utilization was between 95% and 100%. The other load points were determined by altering the number of threads from the client, thus adjusting the rate of client request per second, until the appropriate ratio of Orders/Minute for Dell and Transactions/Minute for Forums was obtained relative to the 100% load point. These thread counts were recorded and maintained consistent across all platforms.
For any given load point, there is a defined number of threads. Each test is 20 minutes in duration, which includes an 8 minute warm up period followed by a 12 minute measured period. For a given load point, the client submits requests to the DB server as fast as the DB server will respond. The rate which the client is able to submit requests is measured during the final 12 minutes of the test and averaged to determine the Orders/Minute for Dell, and Transactions/Minute for Forums.
After much blood, sweat, and almost tears we were able to produce repeatable loads with an average deviation of 1.6%.
For each platform we ran the test 5 times for each load point and than averaged the 5 results, this was repeated for all loads, all tests, on all platforms... that is 300 test executions!.
Dell & Forum SQL Trace Analysis
The Dell and Forum benchmarks are quite different workloads, which you will see in the benchmark results. Dell executes approximately 10 times more queries during the test, and the durations are approximately 4 times less than that of the Forum benchmark durations. To summarize, Dell is a workload with a high transaction volume, and each query executes in a very short amount of time. The Forum workload has a medium transaction volume, and the queries execute in a reasonable amount of time but are much more read intensive (larger datasets are returned).
Dell DVD Store Information
For AnandTech Database Benchmarks, we have always focused on "real world" Benchmarks. To achieve this, we have used real applications with loads such that CPU utilization was 80-90%. Recently we discussed how most Enterprise Database Servers do not average 80-90% CPU utilization, but rather something closer to the 30-60% range. We thought it would make more sense to show performance where it is most likely going to be used, as well as the saturation numbers for the situations where the CPU is maxed.
We feel this is consistent with how GPUs are reviewed, and how you might test drive a car. With GPUs, the cards are tested with varying resolutions, and anti-aliasing levels. With a car, you don't just hit the highway and see what the top end is.
We settled on six load points 20%, 40%, 60%, 80%, 100%, and 120+% for testing the varying ranges of load. These load points are consistent across all platforms and are throttled from the client, independent of the platform being measured. We chose these load points as they split the load range into 6, roughly, equal parts and allow us to extrapolate data between the points. The 120+% load point was included to verify that our 100% load point really was 100%.
The 100% load point was determined by starting an execution of the client and adding threads until the CPU utilization was between 95% and 100%. The other load points were determined by altering the number of threads from the client, thus adjusting the rate of client request per second, until the appropriate ratio of Orders/Minute for Dell and Transactions/Minute for Forums was obtained relative to the 100% load point. These thread counts were recorded and maintained consistent across all platforms.
For any given load point, there is a defined number of threads. Each test is 20 minutes in duration, which includes an 8 minute warm up period followed by a 12 minute measured period. For a given load point, the client submits requests to the DB server as fast as the DB server will respond. The rate which the client is able to submit requests is measured during the final 12 minutes of the test and averaged to determine the Orders/Minute for Dell, and Transactions/Minute for Forums.
After much blood, sweat, and almost tears we were able to produce repeatable loads with an average deviation of 1.6%.
For each platform we ran the test 5 times for each load point and than averaged the 5 results, this was repeated for all loads, all tests, on all platforms... that is 300 test executions!.
Dell & Forum SQL Trace Analysis
The Dell and Forum benchmarks are quite different workloads, which you will see in the benchmark results. Dell executes approximately 10 times more queries during the test, and the durations are approximately 4 times less than that of the Forum benchmark durations. To summarize, Dell is a workload with a high transaction volume, and each query executes in a very short amount of time. The Forum workload has a medium transaction volume, and the queries execute in a reasonable amount of time but are much more read intensive (larger datasets are returned).
Dell DVD Store Information
Facebook
Twitter
RSS
38 Comments
View All Comments
minidad - Thursday, December 21, 2006 - link
Hi,Anandtech has done a lot of hard work here, and should be commended for this, but the methodology appears flawed. The metric of comparison between the different systems is the % cpu utilization at 6 different load points. However, if you examine the Dell DVD Store cpu utilization graphs, the CPU utilization for each load point is different for different cpus except for the two heaviest load points. They should be the same at each load point for correct comparison. In other words, when the opteron 2218 is running at 65% cpu load in load point 3, the woodcrest is running at 50%. Since the load points for the different cpus are not comparable, the conclusions of the article are unfortunately not usable.
Mantruch - Thursday, December 21, 2006 - link
Woodcrests are faster? well, thats all i need to knowNighteye2 - Monday, December 18, 2006 - link
Does that version of Windows server support NUMA? It could make a significant impact on results...BikeDude - Thursday, December 21, 2006 - link
NUMA is supported onWindows Server 2003
WinXP SP2
and newer
See reqs at:
http://msdn.microsoft.com/library/en-us/dllproc/ba...">http://msdn.microsoft.com/library/en-us/dllproc/ba...
gouyou - Monday, December 18, 2006 - link
I think it would be nice to have a test using a linux plateforme. I'm wondering if there is any performance gain for AMD using scheduling and memory management algorithms made with a NUMA set-up in mind. I guess that in some scenari we might see the opteron performance closer to the Intel one.JarredWalton - Monday, December 18, 2006 - link
The Dell test runs on Linux, while our forums benchmark runs on Windows Server 2003 x64. We will be providing additional benchmarks in the near future comparing Opteron and Xeon in other ways, so stay tuned.Spacecomber - Monday, December 18, 2006 - link
Sorry if I overlooked where this was mentioned in the article, but are these comparable systems comparably priced?Nehemoth - Monday, December 18, 2006 - link
Why you don't include information test for Terminal services, for example in out company with have plans to migrate from an old version of Citrix Metaframe to the Windows 2003 server terminal services.And don't care much about the power consumption (in out country the electricity bill is always high not matters what) but i do care much about the upgrade path, for example :
(And taking in mind the HP solutions , DL365 opteron VS DL380G5)
1-If i choose Opteron over Woodcrest will be easy or more cheap to buy more memory next year end?
2-What about Quad Core, i know that i can buy woodcrest QC now but it will become conductible this upgrade concerning the bus of intel or should i see beyond to opteron QC (anyway for an upgrade for a system bought it in january 2007 shall be are less january 2008).
These are the things that matters to me right now and i hope that AT answer those question sooner than later.
Nehemoth - Monday, December 18, 2006 - link
HP has curious quad core upgrade pathhttp://www.theinquirer.net/default.aspx?article=36...">http://www.theinquirer.net/default.aspx?article=36...
mino - Monday, December 18, 2006 - link
As for upgrage path, go AMD.While Woodcrest is usually a bit better than AMD, K8L will be better in allmost every aspect to Clovertown.
Also I doubt 45nm Penryn-derived 4C Xeons will be compatible with current platforms.
As of now I would go for some serious 16DIMM board with cheaper DC like 2214. And plan upgrade in Q407 or Q108 to K8L.