Google open sources a tool used to enable Portrait Mode-like features from the Pixel 2

Google open sources a tool used to enable Portrait Mode-like features from the Pixel 2

Update 05:02PM CST: Google has reached out to clarify that the Portrait Mode technology itself is not being open sourced, but rather, that the technology that makes it possible—semantic image segmentation—is now open sourced. The title has been amended to reflect this correction.

Most people agree the Pixel 2 family has the best cameras on any smartphone right now. The camera hardware itself is great, but most of the magic is happening on the software side. For example, the HDR+ feature makes almost any camera better when it’s ported to other phones. A new software feature on the Pixel 2 is “Portrait Mode.” It identifies you and blurs the background to create a cool effect.


portrait mode tech

The camera is using semantic image segmentation to achieve this. Basically, it categorizes every pixel with a label such as “person” or “sky.” This helps the camera to differentiate between a person in the foreground and the sky in the background. Google has released this technology as open source, which means developers can use the same technology in their own apps. Portrait Mode is just one example of how this technology can be used. Developers can do even more cool stuff.

This release includes DeepLab-v3+ models built on top of a powerful convolutional neural network (CNN) backbone architecture [2, 3] for the most accurate results, intended for server-side deployment. As part of this release, we are additionally sharing our Tensorflow model training and evaluation code, as well as models already pre-trained on the Pascal VOC 2012 and Cityscapes benchmark semantic segmentation tasks.

Source: Google Research

About author

Joe Fedewa
Joe Fedewa

Former Managing Editor at XDA-Developers. Lover of all things with displays.

We are reader supported. External links may earn us a commission.