The Snapdragon Samsung Galaxy S9 has a GPU Stability Bug that can be Exploited to Trigger Remote Reboots
The new Samsung Galaxy S9 includes the latest Qualcomm Snapdragon 845 system-on-chip for several markets, including Hong Kong, the United States, Canada, and parts of Latin America. Samsung has often chosen the latest flagship Qualcomm Snapdragon SoC over their in-house Exynos SoC in some markets. This is usually the case in the United States, with the exception of the Samsung Galaxy S6. To the average consumer, there is little difference between the Exynos 9810 and Snapdragon 845 variants. Under-the-hood, however, the difference in chipsets can lead to wildly different experiences.
Anandtech‘s review of the Snapdragon Galaxy S9 and the Exynos Galaxy S9 revealed stark differences in performance and battery life between the two models, with the Snapdragon model easily outperforming the Exynos model. With these two vastly different chipsets, a difference in performance doesn’t seem to be the only concern this time around, according to a U.K. based company called GraphicsFuzz. GraphicsFuzz is a start-up firm that specializes in testing GPU reliability on devices. They develop tests to look for bugs in graphics drivers and offer help in diagnosing the root causes of any issues that they discover. For instance, the team discovered a security issue affecting the ARM driver for the Samsung Galaxy S6, for which they were awarded a bug bounty by Google. During their testing of the Snapdragon Samsung Galaxy S9, GraphicsFuzz found an error in the Adreno 630’s graphics driver that allows them to trigger a whole-phone reboot via a valid WebGL Internet page when browsing with the stock Samsung Internet browser.
In particular, there is an error in the Adreno 630’s rendering of a complex yet valid shader that can be exploited to make the device freeze then eventually reboot. A shader is simply a program that allows the GPU to render an image. GraphicsFuzz did not design the WebGL page with malicious intent to trigger this bug, and instead say it was incidentally discovered during their standard testing of GPU stability of devices. Once they discovered that this remote crash was reproducible, the company reached out to XDA-Developers to facilitate the disclosure process with both Qualcomm and Samsung.
WebGL Crash Reproduction on the Snapdragon Samsung Galaxy S9
Before we reached out to representatives from either company, we validated GraphicsFuzz‘s findings on our own device. GraphicsFuzz set up a special webpage for us to test against, and we chose the 5 most popular Internet browsers on the Google Play Store to see what would happen. The table below shows the effects of rendering the complex shader on 5 different web browsers.
Device Tested: Qualcomm Snapdragon 845 Samsung Galaxy S9+ (SM-G965U)
Operating System: Android 8.0.0 Oreo SM-G965U
|Google Chrome v65.0.3325.109||Freezes for ~2 seconds only|
|Samsung Internet v18.104.22.168||Freezes then eventually triggers a full reboot|
|Opera v45.1.2246.125351||Freezes phone|
|Microsoft Edge v22.214.171.1246||Freezes for ~3 seconds only|
|Firefox v59.0.2||Browser crashes|
Both Google Chrome and Microsoft Edge will freeze the phone for a few seconds and generate an WebGL error, but the device will ultimately be fine. GraphicsFuzz reported to us that they have been in discussion with the Google Chrome team for some time and have learned that Chrome implements a mechanism that ends the GPU process after a set period of time to prevent a full phone crash. Opera freezes the phone, but it doesn’t trigger a reboot. The Firefox app itself crashes but the phone is fine. Lastly, accessing the page via Samsung Internet causes the phone to slow down to a crawl before triggering the full phone reboot.
Here is a video demonstration of the crash:
Detailed Explanation of the Error
GraphicsFuzz performed a more in-depth investigation which indicates that the issue causing the phone to reboot is in the GPU driver for the Qualcomm Adreno 630 which is part of the Qualcomm Snapdragon 845 system-on-chip. GraphicsFuzz collected a log on the crash, which we have embedded below. To give a brief summary of what is happening, when the phone renders the complex shader, the GPU sets something called a “fence.” A fence is used to orchestrate the access to the shared memory between the CPU and GPU. A mobile GPU, unlike a desktop, has access to the same RAM as the CPU, so when a game is played or something else renders, it uses a fence to access that shared memory. On a device that has discrete graphics, the GPU itself has its own memory. All current mobile phones share video memory and random access memory with the RAM’s flash storage. The problem here is that the fence is not able to complete which triggers a kernel panic and causes the phone to reboot.
Complete Kernel Panic Prior to Reboot
[12681.035590] [2:crtc_commit:117: 433] kgsl kgsl-3d0: |a6xx_snapshot_gmu| set FENCE to ALLOW mode:0 [12681.035839] [2:crtc_commit:117: 433] kgsl kgsl-3d0: |kgsl_device_snapshot| snapshot created at pa 0x000000016e500000 size 927400 [12681.035993] [0: kworker/u16:5:27740] kgsl kgsl-3d0: |kgsl_snapshot_save_frozen_
objs| kgsl_snapshot_save_frozen_objs start [12681.036085] [2:crtc_commit:117: 433] Kernel panic - not syncing: !!!FENCE TIMEOUT [12681.036156] [2:crtc_commit:117: 433] CPU: 2 PID: 433 Comm: crtc_commit:117 Tainted: G W 4.9.65-13087505 #1 [12681.036248] [2:crtc_commit:117: 433] Hardware name: Samsung STARQLTE PROJECT Rev14 (DT) [12681.036319] [2:crtc_commit:117: 433] Call trace: [12681.036368] [2:crtc_commit:117: 433]  dump_backtrace+0x0/0x248 [12681.036438] [2:crtc_commit:117: 433]  show_stack+0x18/0x28 [12681.036509] [2:crtc_commit:117: 433]  dump_stack+0x98/0xc0 [12681.036578] [2:crtc_commit:117: 433]  panic+0x1e0/0x44c [12681.036646] [2:crtc_commit:117: 433]  sde_plane_wait_input_fence+ 0x174/0x28c [12681.036727] [2:crtc_commit:117: 433]  sde_crtc_atomic_flush+0x1c4/ 0x5e8 [12681.036807] [2:crtc_commit:117: 433]  drm_atomic_helper_commit_ planes+0x19c/0x1fc [12681.036891] [2:crtc_commit:117: 433]  complete_commit+0x74/0x6a4 [12681.036960] [2:crtc_commit:117: 433]  _msm_drm_commit_work_cb+0x48/ 0x1c4 [12681.037038] [2:crtc_commit:117: 433]  kthread_worker_fn+0x78/0x194 [12681.037108] [2:crtc_commit:117: 433]  kthread+0xd8/0xf0 [12681.037172] [2:crtc_commit:117: 433]  ret_from_fork+0x10/0x20 [12681.037239] [2:crtc_commit:117: 433] Kernel loaded at: 0x800a0000, offset from compile-time address 20000 [12681.037331] [2:crtc_commit:117: 433] SMP: stopping secondary CPUs
GraphicsFuzz believes that the reason this issue is only happening on the Samsung Internet browser is that of the GPU watchdog. Sometimes a GPU may hang on long-running shaders, in which case the browser or OS typically has a GPU watchdog that force-restarts an unresponsive graphics driver. The GraphicsFuzz test shader has several for loops that may make it take longer to render, but it is still a valid shader. Several other devices, including the Exynos 9810 Samsung Galaxy S9 with the Mali-G72 GPU, do manage to render this shader. Thus, the team at GraphicsFuzz came to the conclusion that this error is happening due to a faulty GPU driver for the Adreno 630.
The Google Pixel 2 XL with the Qualcomm Snapdragon 835’s Adreno 540 GPU running the same version of the Samsung Internet browser slows to a crawl as well—this means that this error could be an issue with the Qualcomm GPU driver rendering the shader and the Samsung Browser watchdog not ending the service.
Hopefully, Qualcomm can diagnose the underlying issue in their GPU driver that causes the reboot and provide a fixed driver to Samsung soon. Of course, it could be some time before this update propagates to end users. In the mean time, we expect that Samsung will push an update to the Samsung Internet web browser to mitigate the issue (at least preventing it from being exploited via a web page), matching the behavior of Google Chrome. Although this issue is known to affect the Qualcomm Snapdragon 845 Samsung Galaxy S9/S9+, it could also affect more devices with the Snapdragon 845.
If you are interested in testing the GPU reliability of your own mobile or desktop device, the team at GraphicsFuzz has put together a webapp demonstration that allows you to run some of their valid shaders on your device. You can access that webpage by following this link.
- March 28th, 2018: GraphicsFuzz reached out to XDA-Developers informing us of the issue. XDA-Developers reproduced the issue on our own Samsung Galaxy S9+ (SM-G965U).
- March 29th, 2018: GraphicsFuzz reached out with more details and set up special web pages for Qualcomm and Samsung employees to reproduce the bug
- March 30th, 2018: XDA-Developers reached out to both Samsung and Qualcomm with full details of the report. Our Qualcomm contact reached back to us acknowledging that our message was received.
- April 2nd, 2018: Our Samsung contact reached back to us acknowledging that our message was received.
- April 4th, 2018: Our Samsung contact recommended that we file a report at Samsung’s Security Reporting page. XDA-Developers filed a report, and a Samsung Engineer was assigned to the report.