Benchmark Results: S6 Obliterates M9, What Happened?

Benchmark Results: S6 Obliterates M9, What Happened?

Let’s start this with a disclaimer: I do not believe there is significant meaning behind synthetic benchmarks in correlation with real-world use. We’ve seen, time and again, some high-scoring phones output questionable performance. In this regard, Samsung is one of the usual culprits, and despite having opted for some of the highest spec configurations on each and every one of their flagships, many of them still managed to lag. Other top-tier devices, including the latest Nexus phone and tablet, managed to output inconsistent performance as well.

While hardware constraints, bottlenecks or throttling can all be causes, we all know Samsung’s mistake was in their software. In our MWC predictions discussion article I voiced my hunch that Samsung would make a big deal about performance on their S6 reveal, and I was right. During the Unpacked Event of yesterday, they mentioned performance enhancements multiple times, and the User Experience specialist of the event stated quite adamantly that there would be no lag at allThey did say that it was mostly because of their revamped TouchWiz and optimizations, but they didn’t make as big a fuzz as I expected them to about their new processing package powering up the S6. In fact, during the show they didn’t specifically utter the name Exynos 7420 nor its details, but a slide showing that the chip had 14nm FinFet processes made it very clear that it was in fact Samsung’s baby inside their S6.

Benchmarks

One of the first things many people, including reviewers and journalists at MWC 2015 grounds, is put devices to their paces to test their performance with synthetic benchmarks. This year’s round of flagships didn’t escape the lust for chip-measuring (and the post-measurement bragging) of the lucky ones who got their hands on the devices, and now we finally see the concrete (but early) benchmark results of both the M9, the insignia of Qualcomm’s Snapdragon 810, and Samsung’s Galaxy S6. The end numbers are as shocking as we had expected them to be:

The Exynos in the S6 Edge got an impressive AnTuTu output of close to 70,000, with the regular S6 sitting at a little over 68,000. What is more impressive about this is the fact that it is even better than what we had been expecting from the S6, as previous leaks placed the score at around 61,000. Where the extra 9,000 came from is beyond me, and it’d be nice to get to know why there was such an increase in performance compared to what we had heard of previously. The Galaxy Note 4 and Galaxy Alpha had been benchmark leaders for a while, as they approached the 50,000 barrier nearing the end of 2014, with the Note 4 Edge surpassing that with the Lollipop update. Phones released later did see better benchmarks, and the ones with the Snapdragon 810 show a marked improvement over these ones too.

But then we have the M9 which managed to score 55,333. While this is a a significant percentage boost over the latest of 2014 phones, and a big step-up from last year’s HTC flagship, we can’t help but wonder how come it is not as high as it was expected to be, and how come the Exynos inside the Samsung managed to output something much higher. What is most troubling is the fact that on-screen tests on the M9 benchmark were done at 1080p resolution, while as you can see on the side, the S6 renders those at 1440p. This means additional strain on the Mali GPU of the Exynos SoC in the S6, and a bigger workload too. With a higher pixel count typically comes higher GPU stress and lower framerate outputs… yet what we see would suggest an even bigger upperhand for the S6. Something to keep in mind, however, is that the S6 features better memory (LPDDR4 RAM as opposed to LPDDR3) and better storage (UFS 2.0 rather than eMMC2), which make it arguably superior in other aspects of the benchmark. Because of this, we can’t compare the two score outputs directly to determine the superior processor; instead, I urge you to consider the scores in comparison to previous iterations.

Snapdragon on Fire

Now there’s something else that bothers me about the M9’s benchmark output, and that is that the benchmark score is not much higher than that of the G Flex 2, which typically spits out around 54,000 from what I’ve seen around, and this score gets dramatically lower on subsequent benchmarks (I found a score as low as 37,000, but on another variant). The G Flex 2 was criticized for its benchmark output, in particular the subsequent throttled results, as they were indicative of bad thermal design which most likely prompted dynamic frequency caps to prevent overheating, something which was also rumored to be a flaw of the Snapdragon 810 but was reportedly fixed. Many analysts expected a delay yet to their surprise the chip fulfilled its schedule.

LG made the statement that performance improvements, especially regarding their “benchmark performance”, would come with future software updates for the G Flex 2. We still don’t know for sure if there were thermal constraints and if those were present in the G Flex 2; but if there were, not much was done to improve the performance in the M9, or the promised performance improvements didn’t make it through. It could also be that the Snapdragon does present inherent flaws in design that could cause it to have lesser performance than what was hyped for months. However, keeping in mind LG’s past G3’s overheating issues, it could very well be a design flaw, so we are compelled to give them the benefit of the doubt.

Design and Experience

The Snapdragon 810 and the latest Exynos had seen quite the leaked benchmark war these past months, and virtually all comparisons were in favor of the Exynos chipset. While synthetic benchmarks only tell half the story of UX, Samsung’s new TouchWiz seems to have gotten a revamp too, so we might be able to finally see the hardware in Samsung phones’ shine to the best of their ability, but also as the best in the industry. How come Qualcomm seemingly fell behind, when they were in such a good position?

If I had to guess, it was the radical change in their scheduling, and what they had been used to producing. They had relied on their Krait cores for a good few years, and the formula worked really well for them. More over, it allowed for the conservative strategy of incremental updates to core frequency and count to keep the marketing numbers easily apparent of the upgrades they provide – consumers intuitively infer that the 2.5 GHz in the Snapdragon 801 beats the 2.3GHz in the 800, even if there’s no absolute law that states that higher frequencies necessarily mean better performance: and this is something Samsung knows by heart, as their Exynos chips saw the same 1.9GHz maximum frequency for their past 3 flagships (Note 4, S5, Note 3) yet their performance improvements were noticeable each time.

From a design standpoint, I personally believe that Qualcomm was forced to radically change their game in order to account for the increasing demand for 64-bit chipsets introduced by Apple’s A7 and perpetuated by their A8 chipset – their schedule was altered and they had to rush forward to a 64-bit SoC. Moreover, the Snapdragon 810 dramatically changed their typical architecture to that of an octa core big.LITTLE arrangement of 4 powerful cores and 4 power-efficient ones for low-level tasks. So they had to change to big.LITTLE, they had to change their types of cores, from Krait to the ARM Cortex ones. That added to their incursion into flagship-level 64-bit. Moreover, Samsung’s new chipsets have 14nm processes as opposed to Qualcomm’s 20nm ones, which would give Samsung a clear advantage off the bat. Let’s not forget that Samsung was also the pioneer of big.LITTLE, and the first 64-bit smartphone chip on Android was also found on a Samsung phone (albeit the functionality wasn’t supported back then); Samsung has had manufacturing experience in the segment that Qualcomm suddenly jumped into, and during Qualcomm’s 32-bit days they were pushing technology forward with hardware innovations and refining their own strategy, all of which culminate on this chip.

Conclusion

While both phones will offer unprecedented performance, it is pretty clear that the Snapdragon 810 failed to deliver on the hype that it built for itself. Samsung’s Exynos, on the other hand, kept it subtle and showed the engineering prowess of the company. We covered the Exynos’ chipset history before and our conclusion was that Samsung had undoubtedly gained steam in the processor game, and that they were becoming a force to be reckoned with. At the same time, I believe that Qualcomm is becoming stagnant, even if momentarily so. The monopoly they once held over flagship chipsets is dissipating as more competitors join the top-game with new offerings. Samsung’s Exynos is a good example, but so is Intel  and Nvidia’s tablet-focused chips are also quite the game-changer on that front. With a higher emphasis on mid and low range devices, the dominance of Qualcomm’s flagship game takes hits on many fronts, and their royalty decline is a bit alarming, so we expect them to step their game back up soon anyway.

The S6 looks to be a promising device, and now with an AnTuTu score just 5k below the highest scored – by the Tegra X1, no less – and fervent promises of better performance, I think we can expect something really interesting on April 10th. I still remain skeptical, however: these are not release units, and the S6 in particular is still a bit away. There could be software cheating going on behind the scenes, as many OEMs have done it before, but we can’t know for sure and we can’t know who is or isn’t doing it as of now. At the same time, despite all the talk about no stutters nor lag, I managed to spot some anyway in some hands on videos, so that thesis was a little too ambitious and easily brought down with an early counterproof. And let’s not forget that Qualcomm’s Krait will see a 64-bit successor with their Kryos CPU in the Snapdragon 820, so that custom-design coupled with the experience they were missing might once again tip the scales in their favor – and whoever laughs last, laughs the best. It just so happened that this time Qualcomm couldn’t laugh loud enough, and Samsung CEO JK Shin commented that this was the reason they didn’t opt for a Snapdragon this time. Regardless of who wins, we consumers benefit, and so does technology. In the grand scale of things, it’s a win-win for us.

The real question here is: will we even get any good custom ROMs and ROM variety on this latest Exynos? My SM-N900 is skeptical.

 

 

About author

Mario Tomás Serrafero
Mario Tomás Serrafero

Mario developed his love for technology in Argentina, where a flagship smartphone costs a few months of salary. Forced to maximize whatever device he could get, he came to know and love XDA. Quantifying smartphone metrics and creating benchmarks are his favorite hobbies. Mario holds a Bachelor's in Mathematics and currently spends most of his time classifying cat and dog pictures as a Data Science graduate student.