Thursday, March 12th 2009

Firefox 3.1 beta 3: Faster but Slower

A couple of days ago I tested how good the latest unreleased browsers from Opera, Microsoft, Mozilla, Apple and Google are at running Javascript.

Today Mozilla released Firefox 3.1 beta 3. So I gave it a spin on the Sunspider and V8 Javascript benchmarks.

I was obviously expecting it to do better than beta 2 in both tests, but that didn’t quite turn out to be the case. Below are averages from three runs of each combination of browser and benchmark, using a clean Firefox profile. (Not safe mode, since that will reduce Javascript speed in beta 3!)

Edit: I just added the data from my previous test for the non-Firefox browsers, for comparison. I also made a nice new graph.

To make the two benchmarks more comparable I’ve normalized the scores (inverting the time data from Sunspider, turning it into speed). The winner of each benchmark is given 100 and the others are given relative scores.

Graph comparing Javascript performance for Mozilla Firefox 3.1 beta 2, Mozilla Firefox 3.1 beta 3, Opera 10 alpha, Internet Explorer 8 Release Candidate 1, Safari 4 beta and Chrome 1.0

And here is the raw data (averages for three runs):

Browser Sunspider
(lower is better)
(higher is better)
Firefox 3.1 b2 1,737ms 133
Firefox 3.1 b3 2,150ms 201
Opera 10 a 6,188ms 137
IE 8 rc1 7,994ms 47
Safari 4 b 1,633ms 1,056
Chrome 1.0 1,761ms 1,136

That’s pretty weird stuff. Beta 3 runs the V8 benchmark 51% faster than beta 2. But it chugs through the Sunspider benchmark 19% slower. (Or taking 24% longer, if you wish … 2150/1737 = 1.238 and 1/1.238 = 1-0.192)

Does anyone happen to know why this is the case? Was this a strategic move from the Firefox developers? I mean, did they introduce this slight performance regression knowingly, intending to fix it for final release?

One Response to this post:

  1. Execvator says:

    Whatever the cause, it seems to be mostly fixed now. Bug 481103 – TM: Dromaeo v3 Sunspider massive slowdown after 2-28 merge