RSSFirefox 3.5 Beta 4 beats IE8 in Microsoft's own load times test
By Scott M. Fulton, III | Published April 29, 2009, 1:45 PM
Last month, Microsoft published a white paper suggesting that, as first adopters installed the final build of Internet Explorer 8, they judge the real performance of the new browser based on what they feel when they use it. Specifically, the authors of the paper (PDF available here) advised users to load their favorite Web sites and feel how much less time it takes to see results. But since many Web sites take only microseconds to load anyway -- perhaps beyond the range of everyday human feeling -- they went on to suggest that folks take videos of their Web sites loading, and use the time index cues to tell the story of whether they should feel pages loading faster or not.
Betanews took Microsoft up on its suggestion...kind of. No, we didn't set up a video camera. But we did set up a clean virtual machine with an adaptation of WebMonkey's Browser Load Time Stopwatch, to see just how many microseconds we should be feeling if our feelers felt that sensitively. At the time, our tests concluded that the latest edition of Mozilla's production release of Firefox available at that time, version 3.0.7, could load 25 selected pages from comScore's Top 50 Web sites (plus Betanews thrown in for good measure) 13% faster on average than the final IE8, and Firefox 3.1 Beta 3 loaded those pages 31% faster than IE8.
In a fresh round of Betanews tests this morning, however, we discovered the gap had widened considerably, at least with respect to Mozilla's latest public beta. Firefox 3.5 Beta 4, released yesterday (actually the next public beta after 3.1 Beta 3), blew away IE8 by loading pages 81% faster on average [an adjusted figure following a math correction by some of our readers, much obliged].
We used 23 of the 25 Web sites chosen by Microsoft for its own test -- which, incidentally, concluded IE8 was generally on a par with other Web browsers. Two sites -- Blogger.com and MySpace -- do not test accurately with the timer because of how they use JavaScript, so we replaced those with two sites in comScore's list of hot properties, TV.com and Weather.com. Then we added Betanews for good measure.
The Web sites that loaded faster in IE8 than in Firefox 3.5 Beta 4 were Google (certainly among the most used), YouTube by a small margin, eBay, and if you can believe it, Betanews. These results held up over multiple tries, and we gave IE8 the fastest time whenever we tried more than once. To ensure that we weren't hitting on the memory cache, we cleaned the cache completely for both browsers prior to loading each time trial.
In tests of JavaScript and CSS rendering performance yesterday, we found Firefox 3.5 Beta 4 to have 477% the speed of IE8 when performing calculations. Add to this the fact that the new beta renders fresh pages in about three-fourths the time, and we begin to wonder just what it is we're supposed to be feeling.
firefox is junk... opera is junk... chrome is junk! IE8 is by far the most secure, fastest browser.
Score: 0
|well, duh..
firefox is faster than ie because it has no real security enhancements.
it might as well be called, naked browser.
"no different than not wearing underwear. it is faster to get naked when the race begins for the pants to come off"
Score: 0
|Bottom line is, Firefox rulez and Internet Exploder sucks......the Big one!! I haven't used IE since I was Forced to, when I worked for Microslop and that was over 5 years ago!! Haven't even looked at it since. MS should get out of the Browser business, as they don't have a clue what they're doing. Hell, they should get out of the O/S business too, while they're at it!! :)
Score: -1
|Ok, I know I am strange. But, they should include Opera 9.x in these stats.
Score: -4
|No! They should not include Opera.... get over it.
Score: 0
|Blah, blah.
90% of all statistics can be made to say whatever you want 50% of the time.
Do we really need a statistical analysis of every release candidate of every browser every single time? Don't people have anything better to do?
Score: -1
|Where are those days gone when Microsoft used to lead the browser war, intimidate and crush small-time companies. Have I missed something here?
Score: 0
|Percentage-wise they are still the most dominant browser in the market more than 2 to 1. While their market share is eroding, their features are meant to catch up with other browsers and their development time has slowed down to a crawl, let's not lose sight of this fact. So technically, you could still say they are leading the browser war by a wide margin.
Score: 0
|Hahaha, Apple.com.
Makes you wonder, doesn't it.
Score: -2
|Heh...
It would *really* make you wonder if MSFT beat 'em *all* to Apple.com. Can you even imagine the hilarity that would ensue over that one? I swear, internetworld7 would go into seizures.
"See! They have to be first to Apple.com! Otherwise, how would they be the first to copy all their awesome features?!?"
;-)
Score: -7
|In my own tests I cannot repeat Apple loading that much slower on IE8. I've tried on three different OS's. perhaps the test is misinterpreting the requiest for a quicktime activex control?
Score: 0
|lol, definitely true.
Score: -1
|"It would *really* make you wonder if MSFT beat 'em *all* to Apple.com"
Actually, I think they already have, and for quite some time... in the form of being the world's largest Mac software developer...
Yup, they hate the Mac.
Score: -7
|Arez u serius?!
Score: -2
|Wow,
That .2 second difference will be really noticeable. Jesus.
Score: -3
|Visited every day for a year, it adds up to over a minute of your life waiting, just for that one test.
Score: 0
|Heh... IE8 slower on Microsoft, msn, and live.com...
I don't care who you are, that's funny.
Score: -4
|So not only is FF faster than IE (at Microsofts own cooked test LOL), it actually supports web standards. How about comparing IE to the latest Chrome and Safari builds? And include standards support. How about SVG support in IE?
Score: 2
|Not a bad idea. Someone should do that sometime.
http://www.betanews.com/...es-IE7-speed/1237494714
Score: -7
|*laughing*
That was awesome, Scott.
Score: -8
|That's funny, i don't see the IE8 release on any of those benchmarks buried under that pile of links. I also see little mention of standards support. Try again....
Score: -1
|Your inability to read or follow links is your problem, man. I found the info just fine. As much as I seem to enjoy giving Scott a hard time, you got nothing (as usual).
Score: -6
|uhh, does this take into consideration the constantly changing internet bandwidth?
Score: 1
|Well, certainly if these particular tests were run on different days or under very different circumstances, you could conclude that any number of mitigating factors could affect the results. In this case, the tests were conducted during the same hour of this morning, on the same virtual Vista-based test system. So what matters here is relative speed -- how fast is one browser compared to another in the same environment? Conceivably in any other system, using any other network, at roughly the same period of time, you should be able to see the same speed differential.
-SF3
Score: -1
|no it does not, testing locally is far more accurate
Score: 0
|Score: -1
|Score: -1
|If you take out the best and worst performance by internet explorer, the difference drops to 66%.
Bottomline, this is not the most scientific way to prove superiority of one browser vs. another. It does prove that Firefox is faster, but it will be hard to prove that 81% faster load times is statistically accurate.
Score: 0
|Superiority? No. What the test attempts to do is look at Microsoft's argument that performance is something that must be felt, and apply as reasonable and fair a test to Microsoft's own suggested criteria for judging that "feeling." It doesn't conclude IE8 is superior to Firefox 3.5 Beta 4; what it should conclude (provided the math comes out right) is that Microsoft has a very difficult case to make if its argument is that you "feel" the quality of a Web browser based on how soon it loads pages.
-SF3
Score: -2
|Fair enough.
The point of my argument though is you have to be careful about using this type of data, because it can be susceptible to a lot of manipulation either way.
One interesting element that has not been mentioned in regards to the whole "experience" angle is how engines present information. By this I mean I would probably feel I'm getting a better experience if the rendering engine on my browser allows me to view the text almost right away, even if I have to wait for some graphics to load, rather than a "faster" engine that leaves me staring at a blank page and suddenly shows me the entire thing at once. Without really knowing what Microsoft said, I kind of wonder if this is what they are referring to when talking about "feel" towards the browser.
Just a thought...
Score: 0
|Of course Betanews loaded faster.
Microsoft gave you a Betanews optimized version so you will be more pleased.
Is a common practice to give a little optimized products to reviewers.
Score: -1
|Prob should have cleared arp cache between runs. And throw away first results as router had to do lookups also.
Score: 0
|Scott:
Learn some math:
Taking the facebook example:
0.938 / 1.082 = .8669
The one thing you forgot to do is to add "1 -"
Meaning the formula should have been: 1-(.938/1.082)=.1330 or 13.30% which is a vast difference from 86.68%
Score: 1
|Lesson learned. For some reason that hadn't copied through the entire column, so I'm very glad you pointed it out. Thank you.
-SF3
Score: -5
|Then how come it says 15.35% for facebook? that doesn't match 13.30%. I didn't bother to add the total speed differential up, but I imagine that would be impacted as well.
Score: 0
|SF3 is just plain stupid. If he doesn't even know basic math he is incapable of writing a decent article on tech.
Score: 0
|It's strange.. aol.com: 5.118 vs 5.126 = +99.84%..
Score: 0
|Yeah this is probably expected, but unless you are really going to be timing how fast your fav site loads it doesnt really make much different to the browser people get. like the below poster has said, Firefox has a completely different functionallity compared to IE so there really is not suitable comparison (unless you go for speed :P)
Score: -1
|That is certainly fine and dandy, but the reason to use Firefox is that it does many other things better. You can choose the minimum font to display, and with Tab Mix Plus add-on, you can open a new tab for every click on of bookmarks, links, etc. There is simply a good long list of what is better with Firefox. Faster is but icing on the cake -- Go FF !!!
Score: 1
|I just don't understand about the true benefits of the speed test. To me what make me use a browser is compatibility with add-on's and reliability. Here and there I will go back to IE8 because a specific web site does not work well with Firefox. I am not "upgrading" to any of the 3.5 beta version because I like my add-on's and it works 99% of the time.
Score: -1
|@hkgfnt - The new Firefox beta browsers should now work for almost any website that is compatible with IE. And almost all of the addons for Firefox are now supported for them unless you have an old outdated program that offers a built in addon, as they aren't updated with Firefox's auto-updates. (.NET Framework Assistant 1.0 Addon for example.).
Score: 0
|sorry but you are wrong. "No Script" still screws up page layout. Talk about something you know and that would be sandboxes.
Score: 1
|Erm...I did say "almost" all of them, you dont have to mark down my posts just because you can't read. And of the NoScript addon well thats simple, if its not compatible then its not worth installing, you have an option of turning off javascript for selected pages using tools>options. Nothing a clear of the cache wouldn't sort out, and if you run into any badlly scripted pages that have a malformed script on page load, you might want to find a better warez site ;P
Score: 1
|what's the big deal? In the end, it's a personnal choice and a few seconds difference is within the margin of error. use whatever browser you want.
Score: 0
|Yeah what he said :)
Score: 0
|