It’s no big secret that benchmarks rarely tell the whole story when it comes to measuring system performance. As we’ve seen countless times, it’s easy for manufacturers (and aftermarket developers) to game the system and give a false sense of speed. This first became an issue in the Android world with the use of ramdisks to inflate Quadrant scores via boosted I/O scores. But then last year, Samsung upped the cheating ante by adding benchmark-specific CPU clock speed optimizations and other manufacturers quickly followed suit.
Not too long ago, we covered an Xposed Module that allows users to disable Samsung’s benchmark cheats, but not everyone wants or is able to install Xposed. But switching to a custom ROM is a sure-fire way of disabling these benchmark “optimizations,” right? Not anymore, thanks to some recent merges to CyanogenMod 11 that will affect upcoming builds.
These changes began innocuously enough with a new performance profile system, which allows for application-specific CPU and GPU scaling settings. But rather than making this an entirely user-configurable feature, upcoming builds will come preconfigured with Quadrant and Antutu whitelisted for the higher performance mode. Luckily, all is not quite as bad as the Samsung benchmark cheating scandal of last year for one main reason: CM11’s implementation will allow the user to manually enable the higher performance mode for any app(s) of their choosing, rather than simply benchmark apps.
What are your thoughts on all of this? Are you upset that the now for-profit company is resorting to the same benchmark “optimization” tactics as the large OEMs, or are you so jaded with benchmarks that this doesn’t even make a difference? Let us know in the comments below.
Want something on the XDA Portal? Send us a tip! -- Join us for xda:devcon 2014!