Live transcription and captioning in Android are a boon to the hearing-impaired

Live transcription and captioning in Android are a boon to the hearing-impaired

2:57pm, 7th May, 2019
A set of new features for Android could alleviate some of the difficulties of living with hearing impairment and other conditions. Live transcription, captioning, and relay use speech recognition and synthesis to make content on your phone more accessible — in real time. Announced today at I/O event in a surprisingly long segment on accessibility, the features all rely on improved speech-to-text and text-to-speech algorithms, some of which now run on-device rather than sending audio to a datacenter to be decoded. The first feature to be highlighted, live transcription, was already mentioned by Google before. It’s a simple but very useful tool: open the app and the device will listen to its surroundings and simply display any speech it recognizes as text on the screen. We’ve seen this in translator apps and devices, like the , and the meeting transcription highlighted yesterday at Microsoft Build. One would think that such a straightforward tool is long overdue, but in fact everyday circumstances like talking to a couple friends at a cafe, can be remarkably difficult for natural language systems trained on perfectly recorded single-speaker audio. Improving the system to the point where it can track multiple speakers and display accurate transcripts quickly has no doubt been a challenge. Another feature enabled by this improved speech recognition ability is live captioning, which essentially does the same thing as above, but for video. Now when you watch a YouTube video, listen to a voice message, or even take a video call, you’ll be able to see what the person in it is saying, in real time. That should prove incredibly useful not just for the millions of people who can’t hear what’s being said, but also those who don’t speak the language well and could use text support, or anyone watching a show on mute when they’re supposed to be going to sleep, or any number of other circumstances where hearing and understanding speech just isn’t the best option. Captioning phone calls is something CEO Sundar Pichai said is still under development, but the “live relay” feature they demoed on stage showed how it might work. A person who is hearing-impaired or can’t speak will certainly find an ordinary phone call to be pretty worthless. But live relay turns the call immediately into text, and immediately turns text responses into speech the person on the line can hear. Live captioning should be available on Android Q when it releases, with some device restrictions. is available now but a warning states that it is currently in development. Live relay is yet to come, but showing it on stage in such a complete form suggests it won’t be long before it appears.
Microsoft shows off Project xCloud with Forza running on an Android phone

Microsoft shows off Project xCloud with Forza running on an Android phone

6:05am, 13th March, 2019
has shared some more and the first at . The company has been working on a cloud game streaming service for a while. Microsoft is preparing the future of gaming platforms with a device-agnostic service that lets you stream games made for the Xbox One. And the first demo is Forza Horizon 4 running in a data center and then streamed to an Android phone attached to an Xbox One controller via bluetooth. "Anywhere we have a good network connection, we'll be able to participate in Project xCloud,” Microsoft head of gaming cloud Kareem Choudhry said in the video. While Forza Horizon 4 is a demanding game and an Android phone is a tiny device, it won’t be limited to extreme scenarios like that. Choudhry compared Project xCloud to a music streaming or video streaming service. When you have a Spotify account, you can log in from any device, such as your phone, your computer or your work laptop, and find the same music library and your personal music playlists. You can imagine an Xbox-branded service that you could access from any device. Even if your computer has an integrated Intel GPU, you could log in and play a demanding game from that computer. Everything would run in a data center near you. It’s easy to see how Project xCloud would work with Microsoft’s existing gaming services. The company promises the same games with no extra work for developers. You’ll access your cloud saves, your friends and everything you’re already familiar with if you’re using an Xbox or the Xbox app on your PC. If you’ve bought an Xbox, an Xbox 360 and an Xbox One, there will be more Xbox consoles in the future. “It's not a replacement for consoles, we're not getting out of the console business,” Choudhry said. Other companies have been working on cloud gaming. French startup Blade has been working on , the most promising service currently available. Shadow lets you access a Windows 10 instance running in a data center. Microsoft wants to associate technology with content. The company already sells a subscription service. With the , you can play Xbox One and Xbox 360 games for $10 per month. Let’s see how Project xCloud and the Xbox Game Pass work together when Microsoft starts public trials later this year.