Browsers and Javascript performance revisited (30/05/2011 15:30)

I have previously written an article about browsers and their performance. If you don't think it matters which browser you use, think again and read the old blog to learn more.

The calendar tells me it's been a while since the old one was updated, so it was time to pick up the performance-tests and see how the different browsers are doing today. With all the improvements and versions that have come and gone since my last update, it is time to start measuring anew, as the old scales and points make less sense today.

I am going to use 4 different tests for each browser, of which 2 are public tests and 2 are my own. But how to end up with a fair benchmark score?

There are a few cases here that makes it hard to measure the times and sum them up. I also need to cater for the fact that browsers are becoming faster and faster as time passes.

I've decided to let each test be worth 1-10 points, the more the better. If a browser can perform the test, it gets 1 point. Then 1 point is added for 10% shorter time, up to a max of 10. The baseline for each test is decided by the slowest result (more or less) among these browsers. This should privde a total score somewhere between 0 and 40 points for each browser.

The numbers listed under each test is the number of milliseconds the test took, so that means lower is better. The score is the other way around, more means better.

	Sunspider 0.9	DOM Core	MySimpleBench	MyAdvancedBench	Score	Rank
Internet Explorer 9.0	246	14297	1088	1682	6,55	# 7
Firefox 4.0.1	264	303	184	944	18,55	# 5
Google Chrome 11	291	64	473	538	25,14	# 1
Apple Safari 5.0.5	330	52	527	939	21,43	# 4
Opera 11.11	288	82	415	666	24,46	# 2
Firefox 5.0	275	303	163	942	18,44	# 6
Google Chrome 12	288	91	481	575	24,04	# 3

 

 

Important notes:
The Sunspider Benchmark manages "almost" strangely close results. The other tests show a lot more diversity. Why is this?
Simply put, it is because Sunspider is the most known, famous and used benchmark. And all browser-makers do all kinds of cheats, hardcoded memory-lookups etc to secure a good score here.
My own two benchmarks are focused on more "real-world-usage" of the leetcode engine, with the kind of usage you will find in, for instance, ManagerLeague.
The DOM Core test is important as it focuses strictly on changing DOM-objects, which means elements on a web-page, rather than calculating the square root of arbitrary numbers a million times.

 

Conclusion May 30th 2011
Internet Explorer 9 was a HUGE improvement from it's previous versions. IE9 is more than twice as fast in everything when compared to IE8, and it supports a lot of cool CSS-features and HTML5. However, when it comes to pure leetcode performance, it is not secret that it is sadly still lagging a good bit behind. In these tests, it managed 6,55 points , of a max total of 40.
The fastest browser overall is Chrome 11. slightly faster than Opera 11.11, but not much, the two top dogs getting scores of 25,14 and 24,46.
Safari trails not too far behind, lightning fast with pure native DOM-access, but showing signs of weakness when the tests become slightly more complicated.
FireFox 4 was also a huge improvement over previous FF-versions, but falls slightly short of "the pack" in these tests. That said though, it shows more stable results that the others, which might count for something. Still, we hope that performance is a priority for the Mozilla team over the next months, so it will not start to lag too far behind.

All browsers have made significant improvements over the last 2 years. Chrome keeps getting better and better, and both Mozilla and Google deserves credit for focusing on this. It will only get better, so I am looking forward to test FF 5, IE 10, Chrome 12, Safari 6 and Opera 12.

Addition June 23rd 2011
It is slightly surprising to note that both FireFox 5.0 and Google Chrome 12 seems to be slower than their previous versions. I know of no reason why this would be, and it's not by much, but still it is a little surprising.

Stay tuned!