SQL Stress Tool Benchmark

Our first benchmark was custom-written in .NET, using ADO.NET to connect to the database. The AnandTech Forums database, which is over 14GB in size at the time of the benchmark, was used as the source database. We'll dub this benchmark tool "SQL Stress Tool" for the purposes of discussing what it does. We have done some updates to the tool since we first used it; it now supports Oracle, and MySQL. We also adjusted the test time for this test and future tests to 20 minutes. The reason for this was to ensure that we used as much memory as possible for future planned 64 bit tests.


Click to enlarge.

SQL Stress allows us to specify the following: an XML based workload file for the test, how long the test should run, and how many threads it should use in which to load the database. The XML workload file contains queries that we want executed against the database, and some random ID generator queries that populate a memory resident array with ID's to be used in conjunction with our workload queries. The purpose of using random ID's is to keep the test as real-world as possible by selecting random data. This test should give us a lot of room for growth, as the workload can be whatever we want in future tests.

Example workload:


    Select1
    select count(iuserid) as usercount from ftdb_forumusers where iforumid = 1


    Select2
    select count(u.iuserid) as currusercount from ftdb_users u,ftdb_forumusers fu where fu.iforumid = 1 and u.iuserid = fu.iuserid and dtlastvisiteddate > '[q]qGetLastVisitDate[/q]'

Example Random ID Generator:


    qGetLastVisitDate
    select dtlastvisiteddate,newid() as ldate from ftdb_users where dtlastvisiteddate is not null order by ldate


The workload used for the test was based on every day use of the Forums, which are running FuseTalk. We took the most popular queries and put them in the workload. Functions, such as reading threads and messages, getting user information, inserting threads and messages, and reading private messages, were in the spotlight. Each reiteration of the test was run for 20 minutes, with the first being from a cold boot. SQL was restarted in-between each test that was run consecutively.

The importance of this test is that it is as real world as you can get; for us, the performance in this test directly influences what upgrade decisions we make for our own IT infrastructure.


Test hardware configuration SQL Stress Results
Comments Locked

97 Comments

View All Comments

  • Jason Clark - Monday, February 14, 2005 - link

    An article we are contemplating is desktop parts in a SQL test, and web. Lots of folks in smaller orgranizations and even medium to some extent build their own boxes.

    Interesting?
  • Regs - Monday, February 14, 2005 - link

    Thanks for Clarifying me #24. For some odd reason I'm thinking about the differences between the branch predicator of a A64 and Intel and I got in over my head.

    But you are right about the cache, spatial and temporal locality.
  • rivieracadman - Monday, February 14, 2005 - link

    I would suspect that the raw speed of the Xeon coupled with the larger cash to reduce latency would make the Xeon perform well in any benchmark that was both threaded and delt with small data sets, such as reads, queries, and searches. On the other hand, the Opteron due to its lower memory access overhead, and shear bandwidth, would do better in areas with large data sets such as data transfers, data recovery, and large complex calculations. If this is correct, which you have pretty much confirmed, then I would suspect that the Opteron would do better in the web server tests as long as the pages served were larger then say 15K. Not that this is any magical number, but the Xeon would have to pull from memory more at this point.

    As for the HT bus. I wouldn't think you would use the entire 1Ghz bus on a database benchmark. You really need to perform some workstation benchmarks to fill the bus.

    Since everyone else here is adding to the wish list. I would like to see a real world combined query, read, change, write benchmark. I think the Xeon does better when searching and reading because of its shear speed, but the Opteron would do better when a record is altered and resubmitted to a database. This is more of a real world example in my opinion, and since both are architectually diffrent, it would allow for both CPUs to show their true colors in what would be considered every tasks.
  • blckgrffn - Monday, February 14, 2005 - link

    Having repetitive data is what having cache is all about, the long pipelined architecture of the P4 needs the large local cache to minimize time-expensive ram lookups to compensate for the time-expensive deep pipe operations that get tossed when mis-predicted. So, the 2meg cache could help the prescott in many places and is not limited only to SQL. I think that we can probably look at the the EE P4's and get a feeling for what the new prescotts will bring to the table, but we can hope that all of those additions that were made to the Prescott core are allowed to shine with more cache present.
  • fitten - Monday, February 14, 2005 - link

    #9, this was a server benchmark test. Servers are about stability and such. Anyone who overclocks a critical server (database, etc.) should be fired on the spot.

    They may do overclocking tests in the workstation review that was mentioned.
  • Ross Whitehead - Monday, February 14, 2005 - link

    #20 - I agree the AMDs instructions/clock count is high, but we were surprised that the 25% increase in HT did not provide any measurable difference.
  • Regs - Monday, February 14, 2005 - link

    *Their IPC counts are higher*

    Need more coffee
  • Regs - Monday, February 14, 2005 - link

    #11 - I doubt there will be a performance gain for games with just added cache. The problem with the prescott is it's low IPC core and leakage. Anyways, Apps on the desk top use a lot of repeatedly used data arrays with similar instruction sets. So why would the CPU core benefit a larger L2 cache for games when it's just going to be the similar type of code it just processed?

    #16 - AMD's are not bandwidth starved. Their high instructions per clock count are higher. So the pipeline is a lot shorter which means it does not run a risk of pipeline stalls if it was not fed enough data from the bus unlike the Intel.
  • Jason Clark - Monday, February 14, 2005 - link

    mickyb,
    Quad 3.6 Xeon systems don't exist as far as I know, correct me if I'm wrong. Quad Xeon systems are still the 400MHz FSB Xeons that are clocked at most 3GHz.
  • mickyb - Monday, February 14, 2005 - link

    The Intel XEON has always been competitive. You guys are thinking about gaming. I would like to see 4 way perforamnce and see a graph on benchmarks compared to number of CPUs. AMD has previously done well in this area.

Log in

Don't have an account? Sign up now