Android Q's new feature Live Caption will bring a real time subtitle system

On the recent Google I/O 2019, Google showed a new feature called Live Caption. It is built into Android Q, which can convert the audio played on the mobile phone into subtitles in real time, suitable for podcasts, video websites, voice messages, local shooting videos and other scenes.

What’s more, the Live Caption process is all done locally on the phone, and it doesn’t need to be uploaded to the cloud, which guarantees speed and security. What are the other features of Live Caption?

Can also be used without the network, processing locally

The importance of subtitles may exceed the imagination of many people. In the eyes of the general audience, it is an optional option, and for the hearing impaired, subtitles are an important tool for them to understand the content, Live Caption is designed for them.

Prior to this, Google has added automatic subtitles to YouTube. However, there are many video content without subtitles in the world. For the hearing impaired, they are like silent movies.

So the question is Can I add real-time captions to all the audio on my phone? This became Google’s next goal. After several teams’ efforts, this program was finally realized. However, initially, the system needs to upload audio to the cloud and then run a 2GB machine learning model, which is time-consuming and unsafe.

Later, they implemented the same processing effect on the mobile, and only need to run an 80M model, which can be automatically updated when the mobile phone is connected to the network, and then run completely offline without the network.

There are three main advantages of Live Caption: first, it runs completely locally and is suitable for all scenarios. Second, data is not uploaded to the cloud to ensure data security. Third, content creators and application developers do not need to do any work. , you can let users experience real-time subtitles.

System-level real-time subtitles for all applications

In Google’s demo, Live Caption can be used in live broadcasts, video sites, podcasts, local video shots, social media, voice messages, and more. The way to open it is to click the volume button and click the live caption button.

As the audio plays, dynamic real-time subtitles are displayed in the centre of the screen. You can adjust the display size of the subtitles and drag them to any position. Even more amazing, Live Caption works fine even when muted.

Although Google mainly emphasizes the role of Live Caption for hearing-impaired people, it is also very useful for ordinary users.

By turning on subtitles while watching a video, you can better understand the content. Google has implemented this feature at the bottom of the system, which means you can open subtitles in any video. In addition to special scenes such as conferences and subways, you can understand video content without turning on the audio.

Launched later this year

Content creators and application developers don’t need to do any work, so users can experience real-time subtitles at any time.

This feature will be integrated into Android Q and can be easily accessed by a large number of users once it is popularized. Currently, Live Caption is not yet able to experience in the Android Q system by the users. Brian Kemler of the Android team revealed that it will be available later this year.

Due to memory and space requirements, Live Caption will first appear on high-end Android Q devices and will support more devices later.