Speed tests: Google curbed Chrome 3 speed prior to stable release

By Scott M. Fulton, III | Published September 17, 2009, 11:40 AM

Banner: Test Results

Two days ago, Google signed off on a stable version of series 3 of its Chrome Web browsers; and since that time, users everywhere are noticing two not-so-subtle changes: First, the New Tab panel has a different (and, we feel, better) layout. Second, it's noticeably faster.

Google promised speed increases of about 30% (often quoted as "one-third") for users who'll find themselves bumped up to Chrome 3 (Google's browser diligently updates itself). Last month, Betanews tested that claim, and projected speed increases of more like 24.5% -- still in Google's ballpark, just along the edge. But since that time, we noticed the company had made dramatic strides, with both beta and dev channel (Chrome 4) builds posting record speed numbers in our tests, for gains that could possibly break the 40% barrier.

But was Google able to deliver those numbers to stable, non-testing customers? We've been thoroughly testing our own tests in recent days, in response to an overwhelming number of mostly positive comments from readers that diligently point out that even "real-world" benchmarks tend to be error prone. So we have a project that's "under construction" at the moment, with regard to nailing down numbers that folks can trust, and that represents the actual speed differences that users experience as opposed to the artificial speed differences imposed by many tests under unrealistic circumstances.

While we've been hard at work on this, we have some early numbers for Google's speed increases. And the answer to that big question is actually a surprising no: On our physical test platform, version 3.0.195.21 -- the first officially stable release of Chrome 3 -- ended up 6.5% slower than the record numbers for Chrome 3's last beta version.

On our Windows XP SP3-based physical test platform (it's XP that Google appears to be targeting for its Chrome OS netbook-oriented project), the first stable Chrome 3 posted an index score of 18.68, which is near the results we were seeing when Google first made its 30% speed boost claim. We estimate that the new Chrome 3 will be 24.1% faster overall on Windows XP than the final stable Chrome 2. By comparison, the current stable Firefox 3.5.3 posted an index score of 9.97 on XP SP3, and the latest nightly developers' distribution of Firefox 3.6 Alpha 2 posted a record score for Mozilla of 11.53.

For those of you not familiar with our index scoring system, it's based on a scale where 1.0 is the assessed performance of the slowest browser in current use on the slowest updated platform it can be used on: Microsoft Internet Explorer 7 (not the most recent IE8, which is faster) on Windows Vista SP2, on the same physical hardware. We chose a slow browser on purpose as something against which all our faster browsers may be triangulated.

A word about Betanews' Windows Web browser test suite

Where we had been seeing speed gains for Chrome were in three key departments: 1) page load time, where Google was already a major contender (but where Opera still has advantages); 2) JavaScript control flow, which involves the handling of heavily nested functions; 3) handling of arrays in memory. But now, not only have all those gains been scaled back to pre-August levels, but our early tests suggest that the latest Chrome 4 developers' builds have seen the same deceleration. On XP SP3, Chrome 4 has slowed down from a record 20.09 on our index on August 25, to 18.39 now -- actually a few points slower than the stable Chrome 3.

What do a few tenths of a point matter? Google's own JavaScript speed test suite, built around its V8 kernel, would suggest it matters a heck of a lot -- it generates heavy workloads based on the assessed capabilities of the browser on its test platform. But then it tends to magnify the differences as workloads get larger, which is one reason the V8 suite isn't part of ours. However, as features such as dynamic graphics rendering and complex DHTML become more and more a feature of real-world Web usage (and as browsers like Firefox and Chrome get more capable, this will happen sooner rather than later), those elements which make a few tenths of a point of difference on our current scale will actually become more magnified in the real world, as the V8 suite suggests. You'll be playing a 3D game through a pair of browsers, and you'll notice one renders more crisply and cleanly than the other, and this will be why.

Right now, Chrome's nearest competitor in the speed department is Apple's Safari 4, which posted a 16.57 on our index in Windows XP SP3. A recent look at its developers' work on the underlying WebKit rendering engine suggests a future version of Safari could very well blast Chrome off the center of the podium, assuming Apple's team sufficiently blends the rendering speed gains we're seeing with other important features such as control flow and memory conservation.

Yes, yes, we're working on new charts now.

Comments

View comments by with a score of at least

This is worthless. My OS has nothing to do with my surfing speeds at this point. I hit 30mps too often (maxed out my connection from my ISP). If I were to switch, I would not see any increase at all. The only problem I ever have is with FireFox pulling up certain sites (very slow). That has more to do with those sites and less to do with FireFox. It's because I have to filter out all the advertisements. So, you guys can throw out all your benchmarks and so on. It would be a waste of time for me to switch to Chrome for speed. The ONLY thing that I can do is wait until I can get DOCSIS 3.0 from my ISP.

Score: -1

|

Is it really just me, or does anyone else think that Chrome is REALLY REALLY SLOW loading images?
On any webpage with a lot of images, you can see them gradually loading, where on the same pages with IE and FF the whole page (including images) just: BLAM - sits there, loaded many seconds faster! (and yes, of course I cleared the cache before testing).

I run Vista 64 and I see this issue on all my systems.
I'm pretty sure the first versions of Chrome (1.x) wasn't like this, and they were really quick, but since v2 Chrome is pretty much the SLOWEST of all browsers if you take this into account. And I personally think that loading images quickly/slowly is more noticable than running a javascript a few microseconds faster...

Score: 0

|

I get the opposite effect. There were issues of Chrome loading animated gifs very slowly, could that be it?

Score: -1

|

No, not animated gifs. Just a bunch of jpgs - say 10 of them on the same (simple) webpage. You can SEE them loading and rendering almost one by one. But in IE7, IE8, Firefox, Opera they load and render instantly. It's a huge difference! I've seen it on several systems.

Score: 0

|

You might want to reconsider the baseline for your benchmark. IE7 is exiting use as a standard browser by both corps and users.

Score: -1

|

I don't know about you, but this kind of articles are making me sick.
One browser 100 nano seconds faster then other...

OS providers should work more like this. IMHO i think the browsers are already fast enough, the only thing that slows browsers is the network bandwidth, since it has been more and more used by spamming emails, worms, ..., and crap like that.

Just my 2 cents of the month.

Regards

Score: -3

|

tontito "I don't know about you, but this kind of articles are making me sick."
Your whine is having the same affect on me.

Score: 0

|

So is your post.

Score: -1

|

Gosh you know how to hurt a chap.

Score: -1

|

Does it involve a louisville slugger? If it does, I know how too!

Score: -1

|

Google translated, we did laugh so.

Score: -1

|

I've switched to this browser for everyday use for the time being. Really liking the render speed and KISS interface compared to Firefox/Opera. I tried Privoxy but it was fairly buggy, ended up going with a hosts file instead.

Score: 0

|

Me too.

Score: -1

|

how stable can it be when there is the flash player crashes and no way to install one?

Score: -2

|

Uninstall it using the latest flash uninstaller, ( http://kb2.adobe.com/cps/141/tn_14157.html ), reboot, then reinstall it using the plug-ins option on Adobe's site:
http://get.adobe.com/flashplayer/

Score: 0

|

This build seems to have solved the flash issue for me.

Yay!

Score: -1

|

unfortunately, the re-installation of the flash player and the required reboot still didn't overcome the gold bar message and frowny face produced by chrome citing: the flash player crashed.

i'm testing and receiving the error as i browse through the target.com weekly ad which requires flash player.

Score: -1

|

will be a graph image ?

Score: 0

|

Your articles about the Browsers Performance (including the bars graph) is very very good.

Score: 0

|

Been about 20 minutes now since I updated...no shockwave crashes so far. I've been horribly plagued with those in the previous version. Every 5 minutes or so.

(Note: I only use chrome for Jango right now, so it's open pretty much all day...)

Score: 2

|

Ever tried Pandora? It seems to be very similiar to Jango. Never heard of Jango till now...

Score: -1

|

Used pandora before I found Jango. Pandora was giving me *way* too many oddball suggestions.

(My Metal station was frequently playing country for some reason)...

Although Jango is now getting *much* more annoying with their pop-ups and crap.

It's kinda starting to piss me off. None of them seem to be able to stop with the BS and just, oh....I don't know...play freaking music?

Score: -1

|

Also check out grooveshark.

Pandora is a dead service now, they limit how much you can listen in a month now.

Score: -1

|

Nice.

Started it today...not much in the way of a tutorial, but once I found the "autoplay" button, things got interesting.

Thanks for the suggestion!

Score: -1

|

Will do. Tried Jango and after my first song, received up a pop-up to listen to some add!

Score: -1

|

Yeah. Between those pop-ups and the "New Artist!" pop-ups I was about to go postal on my desktop...

Score: -2

|

PDC 2009: What have we learned this week?

There was the freebie that no one will forget, the heebie-jeebies courtesy of Scott Guthrie, and a teensy bit clearer picture of how this cloud thingie should work.

Live report: Will Google Chrome OS change Linux?

The mysteries of just what Chrome OS is, and how much of an operating system it truly is, may be resolved today.

PDC 2009: Microsoft cares about Web browser performance

The effort to give users of the world's dominant Web browser the impression of quality, is a personal one for the man who leads that battle.

Nokia re-affirms its commitment to Symbian, sort of

Maemo won't necessarily be replacing Symbian in the Nokia N-Series, but that's definitely a place where it will be found.

E-book readers will be in short supply this holiday season

E-readers are hot this year, and a lot of compelling new products have been released, but are there enough electrophoretic displays to go around?

Sony looks to finally open a single storefront for downloads

Sony has had many different download portals for movies, music, e-books, and games, and now it's looking to make a single shop for all of it.

Tuning out the tablet: Time to give the endless speculation a rest

Wide Angle Zoom: Wishing and hoping and thinking and praying....won't put an iTablet on the market.

Five improvements for IT managers in 2010

If businesses are to improve their efficiency for next year, they need to stop and reassess the basic tenets of their job.

AOL's spinoff from Time Warner to shed 2,500 jobs

As AOL moves toward become an independent company again, it will cut nearly a third of its workforce.

Gartner: SMS-based money transfer will be bigger than mobile browsing, search

Gartner issues its predictions for the 10 things our phones will be doing in 2012.

Don't forget to upgrade to Firefox 3.6 beta 3 today

Mozilla has released the latest beta its Firefox 3.6 browser software, just over one week after beta 2.