Firefox 4 Beta 7: Faster than 3.6, but not 5x faster

Yesterday, after several weeks of delay due to continued heavy crash counts, Mozilla released public beta 7 of its Firefox 4 browser, with at least three more public beta cycles planned before the end of the year. Beta 7 is the first public release to contain the device Mozilla calls JaegerMonkey, which hybridizes the optimized TraceMonkey engine introduced in Firefox 3.5 with a new just-in-time native code compiler.

The organization touted Beta 7 as being three to five times faster than the stable Firefox 3.6.12 in executing well-known benchmark suites, including the organization's new Kraken suite, and Google's V8 test battery. In a newly revised series of tests conducted this morning by Ingenus LLC, there were limited instances of 300% acceleration, but not across the board. Firefox 4 Beta 7 posted speed scores that were 2.38 times those of Firefox 3.6.12 overall.

Most indicative of Firefox 4's gains were its scores in the SunSpider 0.9 battery. On Ingenus' test system (based on a Gigabyte motherboard running an Intel Core 2 Quad Q6600 processor at 2.4 GHz), total time elapsed for Beta 7 was 348 ms, versus 880 ms for 3.6.12, for about 253% the observed speed. Ingenus' tests, however, measure total performance on a deeper level: Here, each individual element of the battery is compared to how a much older browser performs each time - in this case, Firefox 3.0.19, which is the "index browser." Those relative scores are then averaged together, producing a figure that represents how much more performance a user should expect to see. With old Firefox 3.0 always represented by 1.0, Firefox 3.6.12's computed average is 7.023 - meaning that most of the time, the Mozilla browser in use every day performs just over seven times better than the one in use in 2008.

Firefox 4 Beta 7's speed average on SunSpider 0.9 is 16.308, or about 232% the score for 3.6.12. The speed boost that Beta 7 gets from JaegerMonkey is very noticeable: The previous public Beta 6 posted a relative score of 11.529. By comparison, Microsoft's public Internet Explorer 9 beta remains the SunSpider 0.9 speed champion, with a 14.319 relative score. The more recently released IE9 Platform Preview 6, though technically not a complete browser, does run SunSpider even faster with a 16.443 relative score.

For rendering a final overall speed score on all test batteries, however, Ingenus factors in not just speed but resource efficiency. There's two ways to make a browser faster. One is by simply throttling the accelerator, if you will - burning more gas costs more resources. We've seen this strategy from Google with Chrome quite often, where a new dev build is 30% faster than the previous one, but consumes 30% greater CPU cycles and memory to do it. Ingenus' strategy with its final score is to enable accelerator throttling and resource usage to cancel each other out, so that when the final score does improve, it's because architects found a way to use resources more efficiently.

Again, new browsers are compared against the old Firefox 3.0.19. In terms of efficiency, Firefox 3.6.12 is only 35% better at handling system resources than its predecessor in the SunSpider 0.9 test, scoring a 1.351 for efficiency. The last 4.0 Beta 6 scored a 2.331 for efficiency. Beta 7 with JaegerMonkey scored a 3.106, especially on account of turning down peak processor usage from 81.5% to 17.2% (which is actually 20.4% to 4.3% of available quad-core cycles). Beta 7 does use considerably more memory (52% more by Ingenus' estimate), but leveling off CPU usage to fewer than one-quarter the cycles more than made up for that memory increase. The IE9 public beta scores a 2.997 for efficiency in this test.

Two weeks ago, the WebKit developers unveiled a new test runner envelope for their SunSpider battery, dubbing it SunSpider 0.9.1. Instead of running each heat for five cycles with forced intervals in-between, it runs each heat for 21 cycles with much shorter intermediate intervals. Now, even though the battery runs more than four times the tests, it does so in as little as one-fifth the time. (Anyone who's been calculating elapsed time using a stopwatch should now realize how unrealistic their results have been.)

The per-heat elapsed times for SunSpider 0.9.1 differ very little from 0.9. However, the unveiling of 0.9.1 is fascinating to Ingenus for another reason entirely: Because of the way modern JavaScript interpreters optimize code, routines that are bunched together with shorter spacing intervals consume far more resources.

So while Firefox 4 Beta 7 tipped the CPU indicator only as high as 17.2% for SunSpider 0.9, the very same tests pushed the needle to 87.5% for version 0.9.1. These efficiency losses can be seen for every browser tested thus far at just about the same level (tests on other brands besides Mozilla and Microsoft have yet to be completed). As SunSpider picks up the pace, Beta 7's efficiency scores can't even match those of Firefox 3.0.19: Its efficiency score for 0.9.1 is just 0.968.

Next: Higher efficiency costs for higher workloads...

This article originally appeared in Net1News.

©Copyright 2010 BetaNews, Inc.