The Big Beta Browser Benchmark Bonanza!

[Ispira]

First off, I know there's already a couple of topics about browsers, but no benchmarks or that sort of thing, I made this post on another forum and decided to post here as well.
Also as one more note, I know speedtests aren't exactly relevant but I put them there anyway as they are a statistic.
Without further ado I bring to you:

The Big Beta Browser Benchmark Bonanza
So, Recently I've been experimenting with many different browsers
and wondered for myself exactly which one was the best, and how they all compared.
So I bring to you all, my results of equal testing between these top competitors:

(If you'd like a browser here, PM me or post below and I will test.)
* Internet Explorer 10 (Release) Version: 10.0.9200.16576
* Google Chrome Beta (Beta) Version: 28.0.1500.29 beta-m
* FireFox Aurora (Pre-Beta) Version: 23.0a2 (2013-06-02)
* Opera Next (Beta) Version: 15.0.1147.24
* Maxthon (Release) Version: 4.0.6.2000
* Avant (Release) Version: 2013 build 107


My system specs:

* Foxconn M61PMV
* AMD Athlon II x2 240
* Zotac nVidia GeForce GT210
* 3GB DDR2 RAM
* 500GB WD Caviar Blue

The results

Launch times (Cached)
Listed in order of fastest launch to slowest launch.

* 1st: IE10
* 2nd: Chrome Beta
* 3rd: Opera Next
* 4th: Avant
* 5th: Maxthon
* 6th: FireFox Aurora

Loading YouTube.com (Non-cached)
Listed in order of fastest loading time to slowest loading time.

* 1st: Avant
* 2nd: Maxthon
* 3rd: IE10
* 4th: Chrome Beta
* 5th: Opera Next
* 6th: FireFox Aurora

Bing search of "Locked Out of Heaven - Bruno Mars" (Non-cached)
Listed in order of fastest loading time to slowest loading time.

* 1st: Maxthon
* 2nd: Avant
* 3rd: IE10
* 4th: Chrome Beta
* 5th: Opera Next
* 6th: FireFox Aurora

Speedtest Averages


IE10 Speedtest averages:
Average ping: 37ms
Average download: 55.9 Mb/s
Average upload: 5.48 Mb/s

Chrome Beta Speedtest averages:
Average ping: 40ms
Average download: 53.10 Mb/s
Average upload: 5.51 Mb/s

Opera Next Speedtest averages:
Average ping: 42.6ms
Average download: 56.19 Mb/s
Average upload: 5.45 Mb/s

FireFox Aurora Speedtest averages:
Average ping: 38.6ms
Average download: 61.59 Mb/s
Average upload: 5.42 Mb/s

Maxthon Speedtest averages:
Average ping: 40.3ms
Average download: 57 Mb/s
Average upload: 5.39 Mb/s

Avant Speedtest averages:
Average ping: 36ms
Average download: 57.46 Mb/s
Average upload: 5.48 Mb/s

[All tests taken via speedtest.comcast.net, using the Atlanta, GA. server.
Three tests were taken on each browser, the data reflects the average of the three.]

BrowserMark Benchmark


IE10:
Score: 2859

Conformity:

Conformity CSS3 2.0: 52%
Conformity Flash 2.0: Yes
Conformity HTML5 2.0: 68%
Conformity Network 2.0
ISP: ()
Ping: 218 ms
Download: 729.58 kbit/s
Upload: 739.94 kbit/s
Variance: 48.92% (from 456.55 kbit/s to 893.77 kbit/s)

Conformity Silverlight 2.0: Yes
========================

Chrome Beta:
Score: 4366
Conformity:

Conformity CSS3 2.0: 48%
Conformity Flash 2.0: Yes
Conformity HTML5 2.0: 67%
Conformity Network 2.0
ISP: ()
Ping: 224 ms
Download: 672.79 kbit/s
Upload: 725.77 kbit/s
Variance: 62.55% (from 340.22 kbit/s to 908.57 kbit/s)

Conformity Silverlight 2.0: Yes
========================

Opera Next:
Score: 4371
Conformity:

Conformity CSS3 2.0: 48%
Conformity Flash 2.0: Yes
Conformity HTML5 2.0: 65%
Conformity Network 2.0
ISP: ()
Ping: 218 ms
Download: 706.75 kbit/s
Upload: 753.67 kbit/s
Variance: 53.44% (from 415.42 kbit/s to 892.31 kbit/s)

Conformity Silverlight 2.0: Yes
========================

FireFox Aurora did not complete the test.
It suffered 2 issues:
Halfway through the test it encountered a script error.
At the end of the test it froze at 100% (I gave it 5 minutes) and would not proceed.
========================

Maxthon:
Score: 2765
Conformity:

Conformity CSS3 2.0: 46%
Conformity Flash 2.0: Yes
Conformity HTML5 2.0: 68%
Conformity Network 2.0
ISP: ()
Ping: 232 ms
Download: 648 kbit/s
Upload: 713.34 kbit/s
Variance: 82.13% (from 148.84 kbit/s to 832.72 kbit/s)
Conformity Silverlight 2.0: Yes
========================

Avant:
Score: 2805
Conformity:

Conformity CSS3 2.0: 52%
Conformity Flash 2.0: Yes
Conformity HTML5 2.0: 68%
Conformity Network 2.0
ISP: ()
Ping: 224 ms
Download: 698.58 kbit/s
Upload: 721.71 kbit/s
Variance: 51.22% (from 405.6 kbit/s to 831.47 kbit/s)

Conformity Silverlight 2.0: Yes
========================

PeaceKeeper Benchmark


IE10: Score: 1598, HTML5 Capabilities: 3/7

Chrome Beta: Score: 3205, HTML5 Capabilities: 6/7

Opera Next: Score: 3076, HTML5 Capabilities: 5/7

FireFox Aurora: Score: 1875, HTML5 Capabilities: 6/7

Maxthon: Score: 2767, HTML5 Capabilities: 6/7

Avant: Score: 1322, HTML5 Capabilities: 3/7

Overall winner
Google Chrome Beta

And lastly but certainly not least, Winners by category!
--NEEDS UPDATE--​

 

[KSoD]

Hate to burst your bubble on this but the results are flawed for various reasons.

First off you are testing on your system, which doesnt match any other system out there. So to say Google Chrome is the winner, is wrong. As it is not the fastest opening browser on my setup. In fact the multiple extensions have caused Chrome to lag in opening speed compared to various others. So there is a factor that was not produced into your results. How many extensions and what purpose of the extensions.

Next comes in the fact that you could easily have Chrome already put into resources. Since this is something that Windows does with SuperFetch, if you use Chrome more often than the rest, it will already be cached in your system resources and allow your system to load it faster than say, Avant, which you just installed for this test. Therefor causing your results to be biased toward the result of Chrome for the fact that Windows knows you have a tendency to use Chrome and allow it to be stored, ready to use at all times. So naturally the loading times will be lower for something in cache than something just installed.

Following this pattern, how many resources are used on the system overall and by what? 3GB of DDR2 RAM is low end and slow by today standards. So if you have 60% already used, some of that by Chrome, your results are flawed.

Firefox, Opera and Maxathon all have various extensions. Which were in use for the test and how did they affect the browsers performance? You show results for the browsers by using a Search Engine searching for a specific item. Nothing of which involves the browser. It involves the search engine and internet connection. Not to mention depending on which browser was tested and when, search engines remember results for IP addresses. So if you tested with Aurora first and Maxathon last, of course Maxathon will be faster as the engine will remember the search by the fact you have done it 6 times previously from that IP Address.

I wont continue taking this apart. But the truth is that there is no such thing as the best browser. It revolves around to many factors and unless you can say you just installed Windows and have never used any programs before this was done, the results are already tarnished. It is also the same reasons why a program can run great on my machine but not on yours. The differences in our usage and behavior on the machine can determine everything.

Not trying to say what you have done isnt worth while. I am just trying to show you that much more goes into a browser and how it functions than what they would want you to believe. Especially with how Windows is setup to operate.

 

[maddangerous]

ill hop in and say this..

were any of the browsers 64 bit?

that would make a difference too

 

[Ispira]

As for the statements made above...

SuperFetch was cleared before each test.
My RAM usage was ~35% before each test.
No plugins were used, nor extensions. (Other than the obvious Flash and Java for the benchmarks/speedtests).
Each test was done in the EXACT same conditions (All other applications and nonessential processes closed).
I understand the results are based on my system, And I posted what my specs are for that reason. I understand people with better/worse PCs WILL get better/worse results, I posted this for the simple reason of giving a general idea, and what worked better for me.

If I missed something or you have anything else to say/ask feel free to post and I'll answer as quickly as I can, Thanks.

-Ryan

 

[maddangerous]

As for the statements made above...

SuperFetch was cleared before each test.
My RAM usage was ~35% before each test.
No plugins were used, nor extensions. (Other than the obvious Flash and Java for the benchmarks/speedtests).
Each test was done in the EXACT same conditions (All other applications and nonessential processes closed).
I understand the results are based on my system, And I posted what my specs are for that reason. I understand people with better/worse PCs WILL get better/worse results, I posted this for the simple reason of giving a general idea, and what worked better for me.

If I missed something or you have anything else to say/ask feel free to post and I'll answer as quickly as I can, Thanks.

-Ryan

64 bit too, or only 32...?

 

[Ispira]

Ah, 64-bit versions where applicable.
I'll add that to the version list, actually. I do believe all the browsers were 64-bit, but it's something I'll need to triple-check.

 

[KSoD]

How were the tests done? I see PeaceKeeper for 1 test but the other doesn't have any information.

How did you perform the launch testing? How did you calculate the time used for each browser to open?

I see you say that those results were Cached, but others were Non-Cached. How did you verify that the browsers were cached to perform that test and get those results?

Last thing I found is that going to sites like HTML5test.com gave me different results than what you state. Just as when I go to CSS3.info for their selectors test, I get different results. Which is again different than your results, but more importantly is different than the results I get for the same browser on CSS3test.com.

So now we have 3 different tests for CSS3, all of which produce different results for IE10. while that may not seem important, something like CSS3, which is a standard, should produce the same results no matter the test done. So to get various results for the same test within the same browser, begs to ask the question of which is correct?

I am not trying to argue about this whole thing. But since I show different results for tests that should be standardized only shows that without knowing the testing methods, the results are not valid. As the only one we know for sure is PeaceKeeper, which when done I got a different result than you as well. I got 1750, where you got 1598. So that gets me to ask why is IE10 showing different results for the same browser? The only thing that it could be would be the OS in use. So which OS are you using? I have Windows 8 Pro with Media Center installed. My version of IE10 is also different than the one you state. I have 10.0.9200.16850.

I know you state that this was done on your system and results will vary. But after finding results that vary for the same test with the same browser, it makes one wonder why. After all a browser should produce the same results for the same tests. The only thing that would cause a variation would be the OS in use, but not to such an extent that we see.

Actually now I see how there is variation. Since I just ran PeaceKeeper again, this time with a different result.

You can see my current result with my previous result under it. Now the thing that makes me wonder about this test, last time I ran the test nothing else was going on with my system. This time, with a higher result, I have files copying over to a SD card and my system is in use. So how does PeaceKeeper produce different results on the same system and why is it higher when the system is being taxed when it wasn't previously?

That doesn't make sense to me how I can get higher results with my system in use than not in use.

So even though the tests were performed with a given set of parameters, the results can vary even using the same test at different times gives different results. So I hope it makes you wonder, if you perform the same set of tests again, would the results vary from your first set? If so why? That is the major question that I have. Why can I run PeaceKepper on the same system at different times with different tasks going and get different results? Shouldn't the results be the same using the same program? I would think so. But it is clear that it isn't. But why?

I will have to do some research and see if I can find out. That is something that is interesting how that could happen.

 

[Ispira]

This is very interesting indeed...
I just ran peacekeeper for 2 hours straight on Avant (Constant retesting, And avant is my least-used browser installed). It seemed almost as it had a pattern of every 4th test being different.

1350,1350,1350,1312,1312,1312,1355, etc, etc, etc.

I would definitely like to know a more solid way to test...As of now my efforts seem effectively useless.

 

[KSoD]

Well I don't have an answer for a more solid way to test. As you can see just by going to various sites, even using something designed to specifically test browsers, the results vary with the same browser on the same system. I got different results back to back. You are getting different results every 4th time. You have a pattern going while I do not.

So from what I can see there really isn't an effective way to get verified results for every browser with a standard set of results. It is already clear that the CSS3 tests vary depending on the site, which shouldn't be happening.

The HTML5 test is the same way. Different tests, different results. There isn't a standard test made up for such things. So really it just comes down to which results you want to stick with and consider to be the most proper.

Just ran PeaceKeeper again and got yet another different result. Running it back to back to see if I can repeat the pattern you posted about or if I will get different results back to back again.

Even further is the fact that no where on the PeaceKeeper site can I find any real information about the tests performed. It gives a lot of generic information but nothing specific about what is tested, what it is tested with or how the results are generated.

The other thing I find drastically disturbing about the PeaceKeeper site is this single line:

Which is the best browser for you? Find out with Peacekeeper, the free universal browser test from Futuremark.

A test like this, which produces different results cant be praised to say that it can determine the best browser for a person. that is nothing more than a personal opinion. I used Firefox, Opera and Chrome for a period of time. Opera for a few years, then Chrome for the last couple of years. I used Firefox for time inbetween. I am now using IE10 only, just cause I want to test it out as a daily driver compared to my Chrome experience and yet PeaceKeeper is trying to tell me that its tests can tell me what is right for me to use? I don't think so.

So my results for back to back tests are in and yet again, different for the 2 tests.

Why is it that you get same results several times in a row, but I am getting different results test to test? It has to be something with the way it does the test, but without any information given by them, there is no way to verify any of the information. All we know for sure is that the results are flawed given by PeaceKeeper by the simple fact it can be shown that it produces different results.

Still trying to see if I can find out anything more. But so far nothing has come up.