Browser Battle: the 5 most popular Android browsers benchmarked.

Normally I write about Cisco and network engineering related stuff. But today I decided, I'll do something different for a change.

Benchmarking a mobile browser is not a straightforward as it sounds. The developers behind the browser and/or components used in the browsers, develop their software to perform exceptionally well on their own benchmark. 

The five most popular browsers on Android, see table below:

Browser	Layout engine	JavaScrip engine
Chrome	Blink, WebKit fork	V8
Dolphine	Dolphine engine, WebKit fork	Unknown
Firefox	Gecko	SpiderMonkey
Opera	Blink, WebKit fork	V8
UC Browser	U3, WebKit fork	Unknown

As you can see both Chrome and Opera rely on the same layout engine and JavaScript engine, with only minor differences in versions, and proprietary patches. Since similar technologies produce similar results, it is reasonable to expect similar benchmark results.

Firefox is the only browser that uses a layout engine not forked from WebKit.

Both UC Browser and Dolphine, use their own WebKit fork. It is not public how much they have diverged from the original code. This means we only can make inferences based upon variations in benchmark results.

Benchmark methodology 

All test where done on a Sony Xperia Z: model C6603, running Android 4.4.2. All power management features where disabled and before every test the phone was power-cycled. For data only a stable WiFi connection was used. All browsers cloud based web acceleration options where turned off. The test page was loaded, and the cache was cleared, after which the test was done. All browser versions used where the latest at the time of writing.

The main three parameters tested are JavaScript performance, HTML5 performance, and CSS/HTML5 conformance.

Because of the before mentioned bias of any one particular test JavaScript performance was tested using three benchmarks: WebKit Sunspider 1.0.2 (originally developed, by Apple Inc), Mozilla Kraken 1.1 (developed by the Mozilla Foundation), and Google Octane 2.0.

HTML5 performance was tested using Futuremark Peacekeeper, and Rightware Browsermark 2.1. These test also use HTML feature conformance as part of their compound score metric. So to provide an insight into the CSS/HTML feature conformance as a standalone metric, Facebook's Ringmark 1.4.0 scores are also measured.

JavaScript Benchmark Results

The WebKit Sunspider benchmark metric is measured in milliseconds, lower is better. The original developer of this benchmark is Apple Inc. Apple's browser Safari uses Nitro as its JavaScript engine. Since non of the browsers tested use this JavaScript engine (as far as I am aware). It's result is reasonably unbiased, though somewhat synthetic. Since the executed test are separate pieces of JavaScript specifically developed for this benchmark. Also difference between the fastest and slowest browsers is just 115.3ms. Or to put it in other words UC Browser is only 8.7% slower, than Dolphine Browser. Not a noticeable difference for the user.

As with the previous benchmark, the metric is milliseconds, and lower is better. Also, it is not unexpected that the Mozilla browser obtains the best score on a Mozilla benchmark. The difference between the fastest and slowest browser is almost 2.5 seconds or 18.2% a pretty significant difference. 

The Google Octane 2.0 benchmark uses an entirely different metric, a Geometric Mean, as opposed to the previous two. Here, higher is better. Again we see that browser of the company behind the benchmark achieves the best score. 

JavaScript Benchmark Conclusion

As was expected the performance of  Chrome and Opera's are very similar in all tests. The results of the Kraken 1.1 and Octane 2.0 benchmarks are weighted two times heavier than the results of the Sunspider 1.0.2 benchmark. 

The browser overall JavaScript results:

1. Dolphine Browser 11.1.1
2. Firefox 29.0.1
3. Chrome 35.0.1916.138
4. Opera 21.0.1437.74904
5. UCBrowser v9.8.0.435 

HTML5 benchmark results

The Futuremark Peacekeeper seems heavily weighted toward fast rendering. Other components tested include: DOM operations, and text parsing. Notably Firefox scored highest in the categories HTML5 canvas drawing, DOM operations, and text parsing. But only was able to produce a rendering at an average speed 8.59 frames a second, and therefor the slowest at rendering of all browsers. UCBrowser the slowest browser overall, was able to produce 13.87 fps, and Chrome was fastest at 20.17 fps.

The highest speed at which HTML5 canvas drawing was done by Firefox at 4.02 fps, and the slowest was UCBrowser with 3.03 fps. On the desktop platform, the performance is at +30 frames, for most systems and ready to be applied on nextgen websites. On mobile however we are still some time away before the performance is good enough on a large enough portion of the platform, to be a useful addition to mobile websites.

Rightware's Browsermark 2.1 is more of a black box when it comes to understanding how it arrives at its score. But HTML5 canvas drawing and HTML5 conformity seem to be the major factors. As with the previous benchmark, a higher score is better. 

The best two browsers, Chrome and Opera have the same HTML5 conformity score of 81%, Firefox comes in at 74% and both Dolphine Browser and UCBrowser score 66%.

Facebook's Ringmark only examines the support for web standards like CSS and HTML5. It does not benchmark the speed of the browsers. All browsers, passed all the rings. This means that they support enough features to deliver a full web experience on nextgen websites.

The only difference between Opera and Chrome was that Opera supports the Ring 1 feature ''multiple audio playback'' and Chrome does not.

HTML5 Benchmark Conclusion

Since all browser had a passing score on Ringmark, and it does not measure performance, both Peacekeeper's and Browsermark's results will be weighted two times heavier.

HTML5 overall benchmark results:

1. Chrome 35.0.1916.138
2. Opera 21.0.1437.74904
3. Firefox 29.0.1
4. Dolphine 11.1.1
5. UCBrowser v9.8.0.435

Overall Conclusion

As expected Chrome and Opera performed very similar. Also the browsers seem to be optimized for their own benchmarks.

Since Google is most heavily invested in HTML5 technology, to replace Adobe Flash on the desktop and mobile. It is not unexpected that Google Chrome performs best. Only Firefox and UCBrowser still support Flash.

Since the reliance on HTML5 for mobile websites and even on most desktop websites is minimal, the JavaScript performance is weight twice as heavy.

Dolphine is the winner, followed by Chrome and Firefox on a shared second place. The performance difference is between first and second place is only minor. Opera comes in at third, and UCBrowser is last.