At last year's Google I/O, Google unveiled Live Caption, an accessibility feature that transcribes audio playing back on the device. Using Android's AudioPlaybackCaptureConfiguration API, Live Caption captures audio from the device and runs it through three on-device machine learning models to generate captions from any English-language speech that's recognized. The first devices to get support for Live Caption were the Pixel 4 and Pixel 4 XL, but Google later expanded support to the Pixel 2 and 2 XL, Pixel 3 and 3a series, Samsung Galaxy S20 series, OnePlus 7T series, OnePlus 8 series, the OnePlus Nord, and now, the new Pixel 4a. With the launch of the Google Pixel 4a, though, Live Caption gets its first functional upgrade: The ability to detect and transcribe speech over phone calls.
We first spotted hints for this new feature back in April during a teardown of Device Personalization Services, the app responsible for Live Caption. The strings suggested that the user will be able to opt-in to transcribe audio during a phone call, and if they choose to do so, their use of the feature will be announced to others on the call. Once the feature is enabled, other parties on the call will hear the following: "Hi, the person you’re about to speak with has call captions turned on. They’ll see captions of what you say to help them listen along." This will work for both voice and video calls and supports apps like Telegram, Facebook Messenger, WhatsApp, and more.
Other than introducing the ability to transcribe phone calls, Google hasn't announced any other improvements to Live Caption. It still only works with English-language speech and still doesn't work with all kinds of media. However, it's possible that Google will continue to add new features, and it's likely that Google will continue to expand support to additional devices in the future. Google says that transcribing phone calls will become available for other Pixel devices with Live Caption first (specifically, the Pixel 2, Pixel 3, Pixel 3a, and Pixel 4) but it'll likely roll out to the aforementioned non-Google devices that have the feature, but we don't have exact confirmation on the availability or timeline. We'll update this article once we find out, though.