Infrastructure as a Service: Benchmarking Cloud Computing
by Johan De Gelas on June 2, 2011 8:50 PM EST- Posted in
- IT Computing
- IT Computing general
- Cloud Computing
Benchmarking the Terremark Cloud
We wanted to compare the virtual IaaS servers of the Terremark Enterprise Cloud with a virtualized physical server, because that is the decision you will have to make: will you deploy your application on a server in your own local data center, or will you deploy to a virtual server in an IaaS environment?
The "In House" Reference Machine
Nowadays most applications find a home inside a virtual machine on top of a hypervisor. Since Terremark servers have Intel's Xeon 7500s inside, we decided to use a reference machine based on the same platform. We used the QSSC-4R machine, equipped with four Xeon X7560 CPUs running at 2.26GHz. We ran vSphere 4.1 Update 1, basedpon the 64-bit ESX 4.1.0 b348481 hypervisor on top of this server.
| CPU | 4x Xeon X7560 at 2.26GHz |
| RAM | 16x4GB Samsung Registered DDR3-1333 at 1066MHz |
| Motherboard | QCI QSSC-S4R 31S4RMB00B0 |
| Chipset | Intel 7500 |
| BIOS version | QSSC-S4R.QCI.01.00.S012,031420111618 |
| PSU | 4x Delta DPS-850FB A S3F E62433-004 850W |
Typically, a group of virtual machines share the CPU, memory and storage resources that have been allocated to their "resource pool", so we tested the "in house" machine in two ways. In the first benchmark run, virtual machines were only limited by the amount of virtual CPUs they were given. The one OLAP virtual machine got to eight virtual CPUs, and our three web servers each got two virtual CPUs. With sixteen total CPU cores, that means the OLAP machine is able to use up to eight physical CPUs and each web server is able to use two physical CPUs.
In the second benchmark setup, we limited the virtual machines (14 virtual CPUs in total) to a resource pool of 10GHz of CPU power. This is similar to the Terremark setup (as well as other "cloud" setups), which also use resource pools to make optimal use of the underlying hardware. After all, it is costly to reserve hardware resources if they are not being used.
The Terremark Virtual Server Infrastructure
We reserved 5GHz (10GHz limit) of CPU power, 10GB of RAM, and 215GB of storage space in the Terremark Enterprise Cloud. We tested this IaaS cloud in two ways. First, we disabled the burst function, which means that we are limited to a maximum of 10GHz of CPU power. Second, we enabled the burst function. In that case, the Terremark Infrastructure will offer extra CPU power, but the amount of processing power that will be made available to your server depends on how heavy the Terremark cluster is currently loaded. Terremark guarantees that in all circumstances 20% extra resources are available, and during our tests we saw up to 24GHz was made available to us.
Facebook
Twitter
RSS
29 Comments
View All Comments
TRodgers - Thursday, June 2, 2011 - link
I like the way you have broken this subject it to small succinct snipets of value information. I work in a place where many of our physical resources are being converted into virtual ones, and it is so often difficult to break down the process, the reasoning, and benefit trees etc to the many different audiences we have.johnsom - Friday, June 3, 2011 - link
You said:Renting 5GB of RAM is pretty straightforward: it means that our applications should be able to use up to 5GB of RAM space.
However this is not always the case with IaaS. vSphere allows memory over committing which allows you to allocate more memory across the virtual machines than the physical hardware makes available. If physical RAM is exhausted your VM gets swap file space tanking your VM memory performance. Likely killing performance when you need it most, peak memory usage.
GullLars - Friday, June 3, 2011 - link
If the pools are well dimentioned, this should almost never happen.If the pagefile is on something like an ioDrive, performance wouldn't tank but be a noticable bit slower. If the pagefile is on spinning disks, the performance would be horrible if your task is memory intensive.
duploxxx - Sunday, June 5, 2011 - link
THat is designing resource pools, if a service company is that idiot they will run out of business.Although swapping on SSD (certainly on next gen vsphere) is another way to avoid the slow performance as much as possible it is still slower and provides Hypervisor overhead.
Ram is cheap, well chosen servers have enough memory allocation.
ckryan - Friday, June 3, 2011 - link
I'm quite pleased with the easy, informative way the article has been presented; I for one would like to see more, and I'm sure future articles on the way. Keep it up, I think it's facinating.JohanAnandtech - Sunday, June 5, 2011 - link
Thank you for taking the time to let us know that you liked the article. Such readers have kept me going for the past 13 years (started in 1998 at Ace's ) :-).HMTK - Monday, June 6, 2011 - link
Yes, you're old :p The main reason I read Anand's these days is exactly for your articles. I liked them at Ace's, like them even more now. Nevertheless, nostalgia sometimes strikes when I think of Aces's and the hight quality of the articles and forums there.bobbozzo - Friday, June 3, 2011 - link
Hi, please include costs of the systems benchmarked... in the case of the Cloud, in $/hour or $/month, and in the case of the server, a purchase price and a lease price would be ideal.Thanks for all the articles!
bobbozzo - Friday, June 3, 2011 - link
Oh, and include electric consumption for the server.krazyderek - Friday, June 3, 2011 - link
i agree, showing a simple cost comparison would have really rounded out this article, it was mentioned several time "you pay for bursting" but how much? put it in perspective for us, relate it to over purchasing hardware for your own data center.