Mobile VR and The Hardware Advancements it Requires
Virtual reality is set to make a big incursion this year, with many VR companies trying to seize a share of the pie in what will likely become a heated battlefield. Mobile VR, while somewhat underrated, is growing too.
While the HTC Vive and the Oculus Rift have advantages over the untethered Gear VR, IonVR and other mobile solutions, technology like Intel Real Sense and Project Tango are bound to bring convincing six-degrees-of-freedom to untethered VR, evening the playing field and allowing for many new experiences. But for mobile VR to advance, we also need to see various changes in the actual phones’ hardware. Whether we love some OEM decisions or not, I suspect that this year, we will see increments or additions in specification sheets that are aimed at improving VR experiences — and the best part about mobile VR is that, due to its modularity, each phone upgrade gives you both a new, better phone, and better VR hardware.
Most specifications that benefit VR thus benefits smartphone anyway, although some are not as useful for smartphones as they are for VR. One of such specifications is 4K displays, a resolution bump that many, including myself, find impractical for phone screens. But that and much more changes when you put on the headset.
The first likely improvement I want to mention is one I haven’t heard discussed, yet one I find integral to the mobile VR experience and immersion. After trying the Gear VR for a week of heavy use, I came to regret not getting a screen protector. My phone’s screen is mostly fine, but the tiny hairlines on the glass show up in VR, especially if your mind is aware they exist (‘cannot unsee’). While not completely immersion-breaking, it puts the need for higher-quality glass on our panels at a forefront, as we’d be pouring money (VR software is expensive right now) into a platform we might become unable to enjoy after a sudden phone drop, or even a tiny rogue pebble in our pocket.
The second specification that mobile VR requires to be upgraded is display resolution. This is, however, one of the trickiest points to improve upon as actual smartphone usage won’t see a practical improvement with the move to 4K displays. Just like 1440p was met with plenty of resistance, 4K will certainly not be well-regarded by those with no interest in VR given it brings constraints to performance and power efficiency. It is worth noting, though, that some manufacturers have managed to improve overall screen efficiency despite bumping the resolution, as Samsung did with the Note 4, which is said to be 14% more power-efficient than the Note 3’s 1080p panel.
If the panel itself becomes more efficient despite the extra resolution, then the next thing to worry about is performance due to the increased pixel count of the screen. Yet as Sony has shown, it’s not necessary for smartphones with 4K displays to run 4K, or even 1440p, at all times. Rather, they could dynamically change the resolution when it is needed, and as shown in the linked article, you can too. If I were to call the shots, I’d allow the user to change the resolution at will. You can already do this through adb or a terminal emulator, and I usually do it on 1440p phones as I don’t notice the difference unless they are being used for VR. I don’t see much of a battery gain, but performance sees a slight bump in the UI and a big bump on games that render at the native resolution.
A higher-resolution screen is ultimately better for convincing VR. At the current 1440p standard, the display doesn’t allow for truly convincing rendering of far-away objects, and the screen-door effect can be a turn-off for many people. Mind you, the latter can be improved upon through other means, but for convincing detail on faces and so on, there needs to be a better pixel density. Things like text can become seriously difficult to read depending on the distance (or, since it’s VR, text size), and there is noticeable aliasing in straight lines. But if that wasn’t enough, the base graphics of current VR experiences are not very good either.
I’ve tried some of the better-looking VR games like Gunjack (UnrealEngine 4) and for all their merits, they are as far from tethered VR graphics as mobile games are from PC graphics. Jumps in graphics performance are a must if we want mobile VR to offer better experiences. We’ll see hefty GPU gains in the upcoming SoCs from Qualcomm and Samsung, with the 820 offering up to 40% better graphics performance over the Adreno 430, and the Exynos 8 promising “console-quality graphics” (a rather vague assertion). Mobile VR will be behind tethered VR for a while if not always, but significant GPU gains like this will allow better graphics and higher resolutions. On a perhaps sad note, the necessary jump to 4K displays and their application in VR will likely mean that we won’t see computer-rendered graphics in VR evolve until the generation after 4K is tamed.
The SoCs in our phones would also have to be heat-efficient. The Galaxy S6, for example, sees heat issues after prolonged sessions of VR, while the Note5 does not (allegedly due to its bigger back surface). Even if the phone doesn’t get hot enough to reach an error message, the accompanying throttling can ruin the immersion by lowering the framerate, which is crucial in VR. This means that the phone has to be designed efficiently so that it does not accumulate heat. Finally, VR also consumes plenty of battery life, given that the chipsets are often pushed near their limits (hence the aforementioned heat). Powerbanks can help, but not when plugged into the phone during VR as that could make things worse — the Gear VR, for example, allows you to plug the headset so that the phone doesn’t consume nearly as much battery.
To summarize, there are many ways in which our phones can improve in order to deliver better VR experiences, and given that VR is rapidly coming into the spotlight of 2016, we should expect OEMs to pay more attention to their devices’ VR capabilities. This means improving upon the components that benefit VR (some more than others, but ultimately all of the above are significant), but sadly, this doesn’t always mean tangible, practical advancements for the general user.
Depending on how the transition to 4K is handled, we could either see the issues many fear or more choice and features without compromising battery life. While Gorilla Glass 4 focused on shock resistance, I wouldn’t be surprised if the next generation of glass in our screens is better at resisting minor scratches. Better chipsets are something we are always excited to see, and that’s where the improvements that benefit both VR and the general user are most solid. Finally, better sensors don’t hurt the user experience and allow for better head tracking and, perhaps with new technology, we could see positional tracking like on the Rift as well.
All in all, I believe that improving specifications and optimizing them for VR go hand-in-hand if done right. There is a lot of enthusiasm and investment behind VR, with predictions of unparalleled expansion. The fact that mobile is at the forefront of it alongside gaming is a way for us enthusiasts at the vanguard of smartphones to take a first-class seat through technologies that might just be ubiquitous in 10-years-time. And because you are bound to upgrade your smartphone every year or two, especially if you are a power user, you might find yourself with the best mobile VR hardware without even looking for it, with a whole new reality only a headset away.