spatialcomputing.info

Apple announced it’s introducing a host of accessibility features to Vision Pro, which is set to include Live Captions for the deaf or hard of hearing. It certainly feels like an important first step in eventually kitting the device with real-time translation at some point down the road.

The visionOS update, which is said to arrive some point later this year, will also add adjustable captions for Apple Immersive Video, and support for ‘Made for iPhone’ hearing devices and cochlear hearing processors. Additionally, accessibility features will include options to Reduce Transparency, Smart Invert, and Dim Flashing Lights for users with low vision or those sensitive to bright lights and flashing.

Since the update is system-wide, things like Live Captions will allow users to follow along with spoken dialogue in live conversations, but also audio from all apps, such as FaceTime or streaming content that isn’t already subtitled.

Meanwhile, the most recent demo of ChatGPT-4o (seen below) could be a sign of things to come, as it showcases just how good real-time translation has gotten—probably better than you’d expect.

Provided Apple decides to push such a feature in the near-future, it could mean face-to-face conversations will soon have subtitles, like you might see on TV or in video games. It could also allow for content that isn’t already subtitled to benefit as well, letting you catch up on the latest episode of that Turkish soap opera, Italian Serie B football match, or even figure out what Scottish people are actually saying. That’s not to say it would be perfect, but it could be hypothetically implemented right now with some degree of success.

Granted, Apple wouldn’t be the first to highlight such a feature when the company does come around to the idea. Before Google shuttered its Project Iris AR glasses, Google was teasing real-time translation as an important add-on that does essentially what I described above.

This follows an ongoing push by Apple to make Vision Pro probably one of the most accessible headsets on the market. It already includes voice input in addition to both hand and eye-tracking for easily traversing UI, as well as features such as VoiceOver, Zoom, and Color Filters for those with low vision. Other accessibility features include Switch Control, Sound Actions, and Dwell Control, which Apple says helps those with physical disabilities.