Speed tests: Google curbed Chrome 3 speed prior to stable release
By Scott M. Fulton, III | Published September 17, 2009, 11:40 AM
![]()
Two days ago, Google signed off on a stable version of series 3 of its Chrome Web browsers; and since that time, users everywhere are noticing two not-so-subtle changes: First, the New Tab panel has a different (and, we feel, better) layout. Second, it's noticeably faster.
Google promised speed increases of about 30% (often quoted as "one-third") for users who'll find themselves bumped up to Chrome 3 (Google's browser diligently updates itself). Last month, Betanews tested that claim, and projected speed increases of more like 24.5% -- still in Google's ballpark, just along the edge. But since that time, we noticed the company had made dramatic strides, with both beta and dev channel (Chrome 4) builds posting record speed numbers in our tests, for gains that could possibly break the 40% barrier.
But was Google able to deliver those numbers to stable, non-testing customers? We've been thoroughly testing our own tests in recent days, in response to an overwhelming number of mostly positive comments from readers that diligently point out that even "real-world" benchmarks tend to be error prone. So we have a project that's "under construction" at the moment, with regard to nailing down numbers that folks can trust, and that represents the actual speed differences that users experience as opposed to the artificial speed differences imposed by many tests under unrealistic circumstances.
While we've been hard at work on this, we have some early numbers for Google's speed increases. And the answer to that big question is actually a surprising no: On our physical test platform, version 3.0.195.21 -- the first officially stable release of Chrome 3 -- ended up 6.5% slower than the record numbers for Chrome 3's last beta version.
On our Windows XP SP3-based physical test platform (it's XP that Google appears to be targeting for its Chrome OS netbook-oriented project), the first stable Chrome 3 posted an index score of 18.68, which is near the results we were seeing when Google first made its 30% speed boost claim. We estimate that the new Chrome 3 will be 24.1% faster overall on Windows XP than the final stable Chrome 2. By comparison, the current stable Firefox 3.5.3 posted an index score of 9.97 on XP SP3, and the latest nightly developers' distribution of Firefox 3.6 Alpha 2 posted a record score for Mozilla of 11.53.
For those of you not familiar with our index scoring system, it's based on a scale where 1.0 is the assessed performance of the slowest browser in current use on the slowest updated platform it can be used on: Microsoft Internet Explorer 7 (not the most recent IE8, which is faster) on Windows Vista SP2, on the same physical hardware. We chose a slow browser on purpose as something against which all our faster browsers may be triangulated.
Where we had been seeing speed gains for Chrome were in three key departments: 1) page load time, where Google was already a major contender (but where Opera still has advantages); 2) JavaScript control flow, which involves the handling of heavily nested functions; 3) handling of arrays in memory. But now, not only have all those gains been scaled back to pre-August levels, but our early tests suggest that the latest Chrome 4 developers' builds have seen the same deceleration. On XP SP3, Chrome 4 has slowed down from a record 20.09 on our index on August 25, to 18.39 now -- actually a few points slower than the stable Chrome 3.
What do a few tenths of a point matter? Google's own JavaScript speed test suite, built around its V8 kernel, would suggest it matters a heck of a lot -- it generates heavy workloads based on the assessed capabilities of the browser on its test platform. But then it tends to magnify the differences as workloads get larger, which is one reason the V8 suite isn't part of ours. However, as features such as dynamic graphics rendering and complex DHTML become more and more a feature of real-world Web usage (and as browsers like Firefox and Chrome get more capable, this will happen sooner rather than later), those elements which make a few tenths of a point of difference on our current scale will actually become more magnified in the real world, as the V8 suite suggests. You'll be playing a 3D game through a pair of browsers, and you'll notice one renders more crisply and cleanly than the other, and this will be why.
Right now, Chrome's nearest competitor in the speed department is Apple's Safari 4, which posted a 16.57 on our index in Windows XP SP3. A recent look at its developers' work on the underlying WebKit rendering engine suggests a future version of Safari could very well blast Chrome off the center of the podium, assuming Apple's team sufficiently blends the rendering speed gains we're seeing with other important features such as control flow and memory conservation.
Yes, yes, we're working on new charts now.
This is worthless. My OS has nothing to do with my surfing speeds at this point. I hit 30mps too often (maxed out my connection from my ISP). If I were to switch, I would not see any increase at all. The only problem I ever have is with FireFox pulling up certain sites (very slow). That has more to do with those sites and less to do with FireFox. It's because I have to filter out all the advertisements. So, you guys can throw out all your benchmarks and so on. It would be a waste of time for me to switch to Chrome for speed. The ONLY thing that I can do is wait until I can get DOCSIS 3.0 from my ISP.
Score: -1
|Is it really just me, or does anyone else think that Chrome is REALLY REALLY SLOW loading images?
On any webpage with a lot of images, you can see them gradually loading, where on the same pages with IE and FF the whole page (including images) just: BLAM - sits there, loaded many seconds faster! (and yes, of course I cleared the cache before testing).
I run Vista 64 and I see this issue on all my systems.
I'm pretty sure the first versions of Chrome (1.x) wasn't like this, and they were really quick, but since v2 Chrome is pretty much the SLOWEST of all browsers if you take this into account. And I personally think that loading images quickly/slowly is more noticable than running a javascript a few microseconds faster...
Score: 0
|I get the opposite effect. There were issues of Chrome loading animated gifs very slowly, could that be it?
Score: -1
|No, not animated gifs. Just a bunch of jpgs - say 10 of them on the same (simple) webpage. You can SEE them loading and rendering almost one by one. But in IE7, IE8, Firefox, Opera they load and render instantly. It's a huge difference! I've seen it on several systems.
Score: 0
|You might want to reconsider the baseline for your benchmark. IE7 is exiting use as a standard browser by both corps and users.
Score: -1
|I don't know about you, but this kind of articles are making me sick.
One browser 100 nano seconds faster then other...
OS providers should work more like this. IMHO i think the browsers are already fast enough, the only thing that slows browsers is the network bandwidth, since it has been more and more used by spamming emails, worms, ..., and crap like that.
Just my 2 cents of the month.
Regards
Score: -3
|tontito "I don't know about you, but this kind of articles are making me sick."
Your whine is having the same affect on me.
Score: 0
|So is your post.
Score: -1
|Gosh you know how to hurt a chap.
Score: -1
|Does it involve a louisville slugger? If it does, I know how too!
Score: -1
|Google translated, we did laugh so.
Score: -1
|I've switched to this browser for everyday use for the time being. Really liking the render speed and KISS interface compared to Firefox/Opera. I tried Privoxy but it was fairly buggy, ended up going with a hosts file instead.
Score: 0
|Me too.
Score: -1
|how stable can it be when there is the flash player crashes and no way to install one?
Score: -2
|Uninstall it using the latest flash uninstaller, ( http://kb2.adobe.com/cps/141/tn_14157.html ), reboot, then reinstall it using the plug-ins option on Adobe's site:
http://get.adobe.com/flashplayer/
Score: 0
|This build seems to have solved the flash issue for me.
Yay!
Score: -1
|unfortunately, the re-installation of the flash player and the required reboot still didn't overcome the gold bar message and frowny face produced by chrome citing: the flash player crashed.
i'm testing and receiving the error as i browse through the target.com weekly ad which requires flash player.
Score: -1
|will be a graph image ?
Score: 0
|Your articles about the Browsers Performance (including the bars graph) is very very good.
Score: 0
|Been about 20 minutes now since I updated...no shockwave crashes so far. I've been horribly plagued with those in the previous version. Every 5 minutes or so.
(Note: I only use chrome for Jango right now, so it's open pretty much all day...)
Score: 2
|Ever tried Pandora? It seems to be very similiar to Jango. Never heard of Jango till now...
Score: -1
|Used pandora before I found Jango. Pandora was giving me *way* too many oddball suggestions.
(My Metal station was frequently playing country for some reason)...
Although Jango is now getting *much* more annoying with their pop-ups and crap.
It's kinda starting to piss me off. None of them seem to be able to stop with the BS and just, oh....I don't know...play freaking music?
Score: -1
|Also check out grooveshark.
Pandora is a dead service now, they limit how much you can listen in a month now.
Score: -1
|Nice.
Started it today...not much in the way of a tutorial, but once I found the "autoplay" button, things got interesting.
Thanks for the suggestion!
Score: -1
|Will do. Tried Jango and after my first song, received up a pop-up to listen to some add!
Score: -1
|Yeah. Between those pop-ups and the "New Artist!" pop-ups I was about to go postal on my desktop...
Score: -2
|