The State of Android Gaming — Interesting Takeaways From TestBird

The State of Android Gaming — Interesting Takeaways From TestBird

TestBird is a Chinese startup company established in 2013 that offers a cloud testing platform for mobile games. The company will be publishing the first English edition of their report, China’s Mobile Game Compatibility Test White Paper for the First Half of 2015, at Game Connection this coming Wednesday.

Game Connection is an international convention for video game professionals held twice a year in France. The report aims to show the current trends in the Chinese mobile gaming scene, as well as common compatibility problems and their causes. The report should be useful not only to those interested in the Chinese market, but also to the whole gaming industry as many of the issues discussed are not specific to China.

Let’s start with a quick idea of what the report covers: about 13,000 games were analyzed across over 2,000 testing devices, covering more than 90% of Android devices used worldwide, with the number of found compatibility issues exceeding 870,000 (showing no improvement from the situation in 2014).

Various points related to the mobile gaming market on Android are also discussed, and will be essential to analyzing some of the compatibility issues discussed later. For example, the cocos2D-X and Unity3D engines make up more than two thirds of the mainstream market. Custom built engines account to slightly less than 10%. It’s also interesting that 90% of tested games used an Internet connection, with 75% (in total) requiring one to be played properly. Qualcomm and Mediatek dominate the CPU chipsets market, taking up more than three quarters of the market together. Other popular competitors such as Samsung, HiSilicon or Intel don’t make up more than 10% of the market. Screen resolutions show a steady and rapid increase of 720P and 1080P devices respectively, which constitute more than half of the market.

When it comes to memory, more than three quarters of devices had 1GB or 2G of RAM. Most of this information isn’t surprising and comparable to the global market, as we’ve discussed some of these trends for major OEM flagships before. Android platform versions are also mostly comparable with Google’s Dashboards, with more than 90% of devices running Android 4 or newer. It’s still essential to know about all this when thinking about who your target audience is, depending on the type and requirements of your game: different Android versions and CPUs exhibit different compatibility issues, higher amounts of RAM make some issues less likely (when it comes to loading resources, for example), and so on.

Unsurprisingly, more popular mobile device brands show the least amount of problems

Android 5.x also showed almost twice as many compatibility problems as Android 4.x, in part due to the transition to the new Android runtime.

Tested compatibility problems include installation failure, startup failure, other force closing errors, stuttering, freezing, etc. Unsurprisingly, more popular mobile device brands show the least amount of problems. For example, Samsung and Xiaomi have a 10% failure rate (average incompatibility rate per game per device), compared to double that for less popular brands.

CPU brands also show different failure rates, with about 20% for popular brands like Mediatek and Qualcomm, 12-13% for Samsung and HiSilicon, and over 40% for Spreadtrum (which focuses on low-end parts).

These results make sense as higher-quality and more-tested (meaning more popular) brands are less likely to have bugs that haven’t been noticed and fixed already. It’s also no surprise that well-known engines such as Unity3D, cocos2D-X and Unreal Engine show less problems than custom built ones, for similar reasons, or that devices with more RAM show less compatibility problems (though the contrast might come as a surprise, with devices having 512M RAM or less having a rate of 40-60% of compatibility issues, which is twice or thrice as much as devices with at least 1G of RAM). Android 5.x also showed almost twice as many compatibility problems as Android 4.x, which can be explained in part by the transition to the new Android runtime.

Unsurprisingly, it’s not rare for game creators to opt for popular existing game engines rather than custom-built solutions.

Of course, a lot of this isn’t very surprising. However, as we’ve mentioned before, it’s still necessary to study your target audience and make sure your game will actually run for a large proportion, taking potential issues and their causes into consideration. The white paper may be of special interest here, as it also discusses leading issues: force closes that make up around 40% of compatibility problems (mainly because of insufficient memory), freezing at 15% (for reasons related to poor network connections, unoptimized game code or weaker capabilities of phones) and startup failures at 12% (also mainly due to insufficient memory, or an incompatible game version). The white paper also includes more detailed statistics about compatibility issues per game complexity and genre, as well as an analysis of various performance indexes for each (including CPU usage, startup time, memory usage and more).

Shrinking APK sizes to accommodate player’s mobile data plan sizes can be a good move

TestBird’s report also discusses some safety concerns, such as piracy and repacked APKs that show advertisements or contain viruses. Suggestions to developers on things to focus on in order to avoid common compatibility problems (such as installation and startup failures or force closes) are also offered, as well as some tips when optimizing for performance.

The white paper should be of use for game developers in general, and more so to those interested in the Chinese market, by making it clearer which configurations to target and test on, but also which genres are already well established (making your game less likely to get noticed among the competition) and how to differentiate your games. Other factors will also have to be taken into consideration, such as how users acquire games in China. The Google Play Store has a rather shy hold, with gamers using different stores instead (downloaded from QR codes in advertisements, or pre-installed stores such as the Xiaomi app market). A list of the top Chinese app stores was also compiled by TestBird, based on feedback from various developers.

Localization (which isn’t restricted to translation, but also includes targeting the audience’s culture and customs) will be another factor, as sources familiar with the Chinese gaming scene find that it leads to noticeably better sales for more advanced games. Even shrinking APK sizes to accommodate player’s mobile data plan sizes where WiFi isn’t popular can be a good move. Popcap did both for their Plants vs Zombies 2 game, for example, though that didn’t work out very well in the end due to a higher difficulty and price gouging. Simpler games can also do well, since their instructions and design isn’t as deep and can appeal to all audiences (Flappy Bird or 1024 are examples of that).

That’s not to say it’s always the case, since games like Clash of Clans and Hearthstone are still extremely popular in China, but it’s still a good move in order to stand out from the crowd and get your game noticed.

As we’ve mentioned at the start of this article, TestBird will present their report at Game Convention this Wednesday. The company will also be offering the white paper, along with a 100 device testing service (both free!) for interested game development companies. If you’re interested, you can register on TestBird’s website. If you’re not part of a company but are still an interested developer, independent or otherwise,  you can just fill in your name under the company field.

About author


Currently studying for a Master's degree in Computer Science at EPFL, Germain Z. is interested in all things related to technology, mobiles and programming. Previously, he was also a Forum Moderator and Recognized Developer on the XDA forums.