A Second Shot: Windows Vista SP1

by Ryan Smith on February 27, 2008 12:00 AM EST

Vista vs. Vista SP1

We’ll start with our Futuremark benchmark applications, 3DMark 2006 and PCMark Vantage. 3DMark in spite of being a graphics benchmark is sensitive enough to pick up on any changes in CPU or GPU performance (in this case any optimizations that reduce overhead), while PCMark Vantage is a full suite of benchmarks to measure overall system performance. It’s also one of the few benchmarks that we’re using that has a 64-bit mode.


Starting with 3DMark, the change in performance is effective imperceptible at 0.2%, well within experimental variance. This just goes to show there haven’t been any changes in overhead. PCMark however is far more interesting; here we get a full 7% score increase, indicating that SP1’s effects are felt outside of our earlier benchmarks. Drilling down ino PCMark’s subscores, we found that the higher score is a result of improvements in data compression scores, data encryption scores, and searching in Windows Mail, all of which can be attributed to improvements in file I/O performance. All other subscores are virtually unchanged.

Moving on to our application specific benchmarks, we have our DivX encoding test, our iTunes/LAME MP3 encoding test, and the Retouch Artists speed test for Photoshop. DivX stresses I/O somewhat, while the rest of the tests are largely memory and CPU-bound.


Here we see no notable changes in performance moving to SP1. All of the application tests come back with virtually identical scores.

Finally we have our gaming tests. Games tend to be good a great way to stress all the components in a system, so this should give us a better idea of how far improvements in Vista’s file I/O system stretch. For our games we have the RTSes World in Conflict and Company of Heroes, and the FPSes Crysis and Unreal Tournament 3.


In spite of the more rounded nature of gaming tests compared to our application tests, the results are the same with no perceivable improvement in performance. At this point it’s clear that what performance improvements Vista does offer are limited to a handful of situations where we are specifically file or network I/O bound. What this means for any given application is that it is unlikely to see a performance improvement due to SP1.

We also ran some quick testing with startup and shutdown times to see if Vista improved performance there at all; there were a couple of hotfixes in SP1 that dealt with these matters.

Vista Startup/Shutdown Performance

Pre-SP1
Post-SP1
Startup Time
33 Seconds
28 Seconds
Shtudown Time
32 Seconds
31 Seconds
.

While shutdown time doesn’t see any real performance improvements, we are surprised to see an improvement in startup time by several seconds. Vista is not notably slow to start up in the first place, so we weren’t expecting much improvement if there was to be any at all. Shaving off 5 seconds for a 14% improvement in startup time (getting startup below 30 seconds altogether) is a pleasant surprise.

The Test Observations & Closing Thoughts
Comments Locked

62 Comments

View All Comments

  • blppt - Wednesday, February 27, 2008 - link

    I could be just imagining things, but I could swear that Aero is noticably more responsive after the SP1 update.

    System has been up for 14 days since I installed the RTM SP1 on x64 Home Premium. I'd say so far it is quite stable.
  • Griswold - Wednesday, February 27, 2008 - link

    It does feel snappier, especially switching to non-aero mode, for example when maximizing/minimizing mediacenter.

    Fruthermore, the UAC secure desktop mode doesnt feel as sluggish anymore when the system is under heavy load.
  • Etern205 - Wednesday, February 27, 2008 - link

    Can any tell me what buid number is this sp1 and is this the same version as it was released to RTM?

    From what I've read some where it says the build number 1806 or is it 1084 has a major bug and there is another version with the build number 20xx.

  • Tristesse27 - Wednesday, February 27, 2008 - link

    I'm always amazed at the way some people refer to Vista ("wretch" was used in this thread I believe). Vista Hate has taken on a life of its own, with I think many people claiming to hate Vista despite never having used it.

    The benchmarks that review sites keep coming out with seem rather silly. Vista vs SP1 vs XP SP2 vs Vista x64 SP1 vs XP SP1 vs .... As a user I'm more concerned with stability, and Vista has been very good to me throughout all my gaming. One horrible BSOD loop occurred from what I can remember, and that's because I was trying to use Alcohol 120's XP version. Plus I have to admit I'm a sucker for prettiness, and Vista is pretty damn slick.
  • WaltC - Thursday, February 28, 2008 - link

    Every time Microsoft introduces a new OS the nutballs come crawling out of the woodwork to post their "expert" hypercritical opinions...;) It's happened every single time, without exception. The "Vista sucks" bandwagon is par for the course at the moment, as basically you're hearing from people who don't like change and are really, really upset that Microsoft is forcing them back to school once again, or forcing them to buy new hardware, or both. I think it's fairly sad. But not to worry--a year from now these same hypercritical "experts" will be using nothing but Vista and they'll be swearing up and down that they never, ever said what they said about Vista...;) It is as inevitable as the sunrise.

    I was going to comment about what a poorly written article this was, but it's so full of mistakes and propaganda that I just don't have the time or inclination to write the lengthy rebuttal to this article that ought to be written, point by point. So a general critique will have to suffice.

    I've been using Vista on several networked machines at home for a year and have been running a lot of 3d-games and application software without experiencing any of the problems alluded to in this article. I did not "wait" on SP1 and have absolutely no regrets. The only thing about Vista SP1 that mildly interests me is that like all service packs it rolls months of updates into a single installation package.

    What's really ironic to me is that when I installed XP in November of 2001, I had a lot more issues--mainly driver issues--with XP than I have ever had with Vista. None of these so-called "experts" seems to have experienced that for themselves with XP (I think they really did but don't choose to remember it because it might get in the way of their Vista bashing).

    Indeed, in the lopsided and dishonest way that these "experts" write about XP, someone new to computers in general today would think that the current state of XP in terms of drivers and updates is the same as the original state of XP when it shipped. The truth is that on both counts the XP experience today is nothing like the XP experience just one year after XP initially shipped.

    Furthermore, prior to Vista shipping, these same people couldn't criticize XP enough; but once Vista shipped then they adopted XP as the Holy Grail of OSes...;) I don't have time to speculate on the reasons why these people are doing this, but I will say that I think that their fear that Vista is too good of an OS is actually what drives them. They think that by criticizing Vista and instead recommending yesterday's XP (which prior to Vista they could never say anything good about) they will be hurting Microsoft in the process, and that I think is their real goal as unfortunate as it may be.

    The really pathetic part of all of these Vista-bashing festivities is that the people who write these kinds of silly articles focus on gnats while they swallow camels whole. I mean, take for instance the teeny tiny spectrum of complaints this author pontificates on to the point of tedium in this article.

    You would think by reading it that this handful of complaints constitutes all the difference there is between XP and Vista. The author would have the reader focus on the tree and remain blind to the forest behind it. From new driver models to greatly enhanced security, just for starters, Vista is leaps and bounds ahead of XP. But you'd never, ever guess that to be true from reading this article, would you?

    Here are just *a few* problems I had with this article:

    *"We’re still waiting for someone besides Apple to start shipping consumer machines (or motherboards) with UEFI support, so this will be an issue we’ll pick up another day. (Ed: We did see a few demonstrations of UEFI boards at CES, though they're not yet publicly available.)"

    As I understand it, EFI is merely an Intel technology that Apple ships in Macs alongside a lot of other Intel technology, and I see the primary benefit of that to Apple being that it makes it that much harder to install and run OSX on non-Apple x86 hardware platforms which are essentially all BIOS driven--which is exactly the way Apple wants to keep it for as long as possible.

    It would have been nice if the author had bothered to tell us why he thinks EFI is better than a BIOS for x86 32-bit machine environments. It would also be nice and lend additional credibility to authors hypercritical of anything Microsoft does if they'd refrain from alluding to "Apple" every time they want to criticize Microsoft. That sort of allusion is a dead giveaway of their intent, imo.

    *"Developers may simply skip Direct3D 10.1 and go for Direct3D 11 when it is finally released, otherwise sticking with 10.0 for the time being. (Ed: We've heard from Microsoft and several game developers that DX10.1 is not a major update and that they will do exactly that.)"

    Ahem...developers did not "skip" DX9 to go to DX10. Not even close. Of the developers currently supporting DX10, *all of them* also support DX9 with those games. Indeed, Vista itself did not skip DX9 in order to support DX10, but Vista also supports DX9 completely and fully--as DX9 is but a subset of DX10, and thus it has always been with D3d. All of my DX9-only games run great under Vista, without exception.

    The implication that developers may "skip" 10.1 to "wait" for DX11 is therefore ludicrous and silly as , just like DX9.0c, developers could support it or not as they chose without the necessity of having to "wait" on DX10. DX9.0c was also billed by Microsoft as a minor update--which did not keep any software developer who wished to do so from supporting it. Indeed, now, most of them support DX9.0c, don't they?

    The salient point is that minor update or not, DX10.1 is real and some developers will choose to support it even though they will also continue to support DX9 and DX10.0 at the same time and in the same games. There is no need to "wait" on DX11 at all, nor is there a reason to do so.

    *"With the addition of this API, developers will be able to control how DEP functions, so that if their code isn’t completely DEP-safe, they may disable certain parts of DEP for their specific application, allowing some protection from DEP without the need to rewrite the offending code or require that DEP be disabled for that program entirely. This is effectively a precursor towards Windows being globally DEP enabled at some later point."

    While the last sentence above is certainly true, I fear the author has once again allowed his obvious prejudices and personal opinions to influence his judgment. As I understand it, the purpose of the DEP API is *not* to provide developers with a way to get around DEP, but is rather to teach them how to program their software so that it will not break when DEP is enforced. Think about it a moment...the purpose of DEP is purely security--if Microsoft was to provide an API to developers to show them how to get around DEP then Microsoft might as well simply remove DEP from the OS altogether. Obviously, the purpose of the DEP API is *not* to allow developers to "get around" DEP.

    *"Among the 24 pages(!) of hotfixes that have been rolled into Vista SP1 are favorites such as the virtual address space fix and a fix for a conflict with NVIDIA’s USB controller and >2GB of RAM. Other additions include fixes for ejecting iPods, a fix for HybridSLI/HybridCrossfire (which is why the launch of these technologies is tied to SP1), and a fix for AMD Barcelona processors causing system reboots during Windows installations. While we could rattle off the entire 24 page list of hotfixes, the important thing to note here is that there are a number of small issues that have been “fixed” prior to SP1 but are only now being widely corrected."

    Talk about bias--note the exclamation point rendered after the phrase "24 pages(!)"...;) The intent here, of course, is to greatly exaggerate the "bugginess" of pre-SP1 Vista. There is no other intent possible here, because the "24 pages (!)" the author alludes to without explaining what he means by saying "24 pages" is not just a list of the updates and patches--it is the number of pages listing not only the updates and patches themselves, but also *all of the accompanying documentation* on each and every one of those updates and/or patches.

    I know this must be true because when I go out to Vista Update and display my history of patches applied, the entire list takes up less than a single on-screen page display. This kind of tactic is just too sad for words--as the author also fails to mention that people like me, who haven't yet installed Vista SP1, but who have already updated their installation via Vista Update *already have* better than 90% of SP1 installed and running at the present time.

    Here again we see some mention of Apple--this time for the iPod eject mechanism. It would have been nice if the author had bothered to remember that Apple had updated its own iPod-related software to run properly under Vista some months ago--which was not a Microsoft fix but purely an Apple fix. But obviously he can't be tasked with criticizing anyone except Microsoft.

    Last, of course, here I am puttering along with pre-SP1 Vista, which the author more or less characterizes as being non-usable because of the "24 pages (!)" of bugs--and, gee, I'm not having any problems!!!! Wonder how that's possible? Could it be because out of the *tens of millions* of lines of code in Vista that the "24 pages(!)" of bug fixes in Vista SP1 are very, very small potatoes? I certainly think so.

    *"This won’t change the public perception of UAC (or Apple jokes on the subject), but any reduction is welcome and perhaps will stem the tide of Vista users who are completely turning off this critical system feature."

    Apple jokes?...;) Wow, what an informative bit of drivel that is...;) Apple people routinely crack jokes about subjects they have no understanding of, so what else is new? It would have been nice if the author would have bothered to tell us what UAC is about and why it exists--but he cannot be bothered with doing anything aside from talking about his own impressions of "public perceptions" it seems. Such a pity.

    My own perception of UAC is that I have become so used to it that it simply doesn't bother me at all anymore, and the thought of "turning it off" seems really ridiculous--as why would I want to drop back to the lesser security levels of XP??? So sad that in his haste to write negative commentary the author cannot manage to find the time or the words to explain what UAC is and why it's not a good idea for people to turn it off.

    The change that SP1 brings to UAC file operations strikes me as entirely positive--where's the beef? Are we to encourage the ignorance of general computer users by letting them think that turning off UAC is a good thing simply because "Apple users" might ignorantly crack jokes about it? How does that sort of attitude help AT's readership?

    *"The advantage of this is that it will reduce the number of computer owners thinking something is wrong because Windows doesn’t “see” all of their RAM; on the other hand this is clearly disadvantageous because they will no longer be informed that Windows in fact isn’t using all of their RAM, nor will there be an easy way any longer to tell how much RAM it is capable of using."

    The whole point of this change is that users will correctly see that the amount of ram they have installed is present and accounted for. It would be nice if the author would tell us what *his* solution would be for this situation. But, alas, he doesn't--he just, once again, criticizes Microsoft whether they do or they don't.

    *"For SP1 we were hoping for a complete overhaul of the MMCSS so that it ceased adversely affecting network performance, unfortunately what we’re getting is something about mid-way towards that."

    I don't have much to say here except that the "problem" the author spends a great deal of time and wordage expounding upon is one I have not experienced. Possibly that could well be because I'm too busy using and enjoying Vista to run benchmark after benchmark in hopes of uncovering something wrong with it which only those benchmarks will indicate...;) I have better things to do with my time.

    *"When it comes to performance, anything is better than Explorer here. The silver lining here is that SP1 has improved WinRAR’s already fast performance by a further and unexpected 28%, making the argument to use anything but Explorer a very easy one."

    If there's something wrong with the idea that Vista is not intended to allow the end user to avoid purchasing or using third-party software, I'd like to know what it is...;) I mean, criticizing "MineSweeper" because it isn't "The Witcher" just strikes me as ludicrous and a waste of verbiage. Who cares? These ancillary programs in all Windows versions are meant to provide basic functionality and that's all. To complain that they aren't the best available just makes no sense. But I guess when your basic goal is to dump on Windows everything is in bounds, isn't it?

    *"We’re not convinced Microsoft is going to see much use of exFAT outside of Windows Mobile 6 devices given the high degree of overlap with NTFS; if the time comes for mobile devices where FAT32 is too little, they may very well switch to NTFS due to the much wider base of support."

    So? This is a "problem" for whom, exactly? Seems to me the more choice people have in matters of this type, the better off they are. Yawn...another mountain out of a molehill.

    *"Vista vs. Vista SP1"

    Gee, after presumably reading through the "24 pages (!)" written about SP1, you'd think the author might have concluded that SP1 doesn't contain anything relative to the three little software benchmarks he wastes an entire page talking about, including a display of colorful frame-rate bar charts.

    I mean, what is the point here, relative to SP1? When the author says "Vista versus Vista SP1, is he talking about Vista with *no updates* versus Vista with SP1? Or is he contrasting Vista + all updates currently obtainable through Vista Update versus Vista with SP1? We don't know and he doesn't say. What is his point here? Is it to suggest that there's no reason to go to SP1 because he sees no performance improvement in these three benchmarks? He doesn't tell us.

    *"One thing that is unfortunate for Microsoft with SP1 is that there is a good chance that system performance immediately following the patching process will be lower than it was prior to patching."

    Oh, gee--installing SP1 clears the caches and the author seeks to imply that SP1 is going to "run slower" than non-SP1 for up to "a couple of days." Wow, what a criticism that is--I need to chuck Vista out of my window right now!...;) Pity he doesn't bother to tell us whether or not after SP1 rebuilds the caches if our performance for the next *few years* will be a bit better. I guess it just didn't occur to him to think about it that way...Oh, well...

    *"Finally, coming into SP1 we heard some concerns about application and driver compatibility."

    Really? I didn't. What I heard was that there was a problem with some driver *install* routines that didn't mesh well with SP1 precisely because the driver developers had not adhered to Microsoft's guidelines when they wrote their driver install software. That's what *I* heard--I heard zero about "driver compatibility."

    You'd think the author would have researched this as opposed to relying on "what I heard"...;) Wouldn't you? Indeed, the only software I have ever seen to "break" under a service pack is software that was improperly coded by its developers in the first place. Microsoft doesn't publish its programming rules and guides for the heck of it. Microsoft has a purpose in doing so, and much of that purpose revolves around future compatibility. Most developers follow the guidelines and that's why the great bulk of software and driver install routines *don't break* under a service pack.

    I've said all I want to say here and I'll close with the admonition that AT much better vet it's articles in the future so that they provide relevant and informative information as opposed to providing little more than anti-Microsoft, or as in this case, anti-Vista, marketing propaganda. I still use XP on my principle machine at work because my employer is too cheap to update his hardware and software, and there's not a day goes by but when I think to myself how crude and rude XP is by comparison to Vista. That's my opinion on the matter.


  • chizow - Wednesday, February 27, 2008 - link

    Good job covering the main changes and fixes in SP1 Ryan. Very readable yet maintains a lot of technical info without getting bogged down. Linked for future reference once I get SP1 RTM installed as I'm still running RC1.

    As others stated though, it would've been nice to have compared to original RTM or even XP. I know that's exceedingly difficult to do with Windows since its very hard to run snapshots and your system may not run at all without Windows updates, but once you start updating you can't stop the flood of updates. Still, I think a lot of "performance" and "compatibility" improvements will be lost when comparing a fully updated/hotfixed pre-SP1 install to SP1 as many of the SP1 hotfixes were released and available prior to SP1.

    At first I thought you overlooked file and network transfers but on closer examination saw it covered on Page 3 extensively. That was a big concern about Vista and while initial speeds with RTM were very slow, there were a few patches that gradually brought my copy speeds closer to those of XP. Also, wouldn't testing copy speeds with a single HDD skew your copy testing a bit? When I copy Raptor to Raptor my copy speeds are very close to the synthetic scores I see with HDtach/HDtune etc at ~70 MB/s.

    In any case, nice job again with this and I enjoy reading your articles on Vista/Windows. You seem to be extremely familiar with Vista and the various stack changes compared to XP as demonstrated in your Messy Transition articles and some keen insights here (like the multimedia stack/transfer caps). I believe Derek (any relation?) mentioned AT would be doing a final XP to Vista 64 comparison for gaming/performance prior to switching to Vista 64 for all test suites and I'd personally think you would be best suited for the task. Now that SP1 has RTM'd this would be the perfect time to document the differences as well.
  • jamawass - Wednesday, February 27, 2008 - link

    I'm interested in the comments on recommending Vista 64 as one's main system. Aren't there still driver issues? Also I use citrix clients for remote access does anyone know if it's compatible with x64? I haven't had any problem with it on vista 32 even though MS pops up a message stating that it isn't compatible.
  • 7Enigma - Thursday, February 28, 2008 - link

    Interested here as well. I'll be building a new system in the next 2-4 months and will need it to do everything from general OS to gaming. I'm leaning towards Vista64 with 4gigs of ram (2X2) allowing me to upgrade to 8 if needed in the future. I just don't know how the driver support is currently.
  • takumsawsherman - Wednesday, February 27, 2008 - link

    One of the more interesting experiences I've had is moving a file to the desktop. For example, an HP LaserJet PCL6 driver, (12-15MB?) being moved to the desktop under Vista took something like 20 seconds on a customer's system.

    Now, the clever among you will ask "why move a laserjet driver to the desktop?". Good point. It doesn't matter what file. Pick any file of that size. What's the hold up? Also, UAC wants to crab about it unless disabled. Has this been improved? XP takes just a couple of seconds to move it (really, isn't it just updating the record on the filesystem anyway?). It took longer to move than to download, on Vista.

    I can't say I've tested Vista too extensively, as I have mostly business customers, and I have been having them stick to XP. However, a few people went out and bought laptops. They call me, miserable, because of the lousy performance and confusing nature of this wretch called Vista.
  • TA152H - Wednesday, February 27, 2008 - link

    They use the term x64.

    The term x86 came from the fact there were processors like 186, 286, 386, 486. x86-64 is OK, or something like that, or 64-bit x86, but x64 is derived from what? There were 164 chips, made by Intel? It sounds more like something of Alpha origin.

    Stop being lazy and learn how to type.

Log in

Don't have an account? Sign up now