Siri on Vision Pro is Getting Eye-tracked Activation and Visual Awareness Alongside New AI Features
Apple spent a major portion of its WWDC 2026 keynote this week talking about new AI features that are part of an enhanced version of Siri. While most of the features will be accessible across devices, Siri on visionOS 27 has some unique touches that take advantage of Vision Pro.
The News
An overhauled version of Siri—which Apple is now calling “Siri AI”—is headed to Apple’s version 27 operating systems. The company detailed a wide range of capabilities, both new and improved, which make Siri AI more useful and more capable than previous versions.
“Siri AI is an entirely new version of Siri deeply integrated into iPhone, iPad, Mac, Apple Watch, and Apple Vision Pro,” the company announced this week. “It can draw on personal context understanding to search across messages, emails, photos, and more, and get things done across apps with even more systemwide app actions. Additionally, Siri AI can answer questions related to the content on a user’s screen or go out to the web to get up-to-date information using broad world knowledge and generate a helpful answer. A dedicated Siri app allows users to revisit a past conversation or kick off a new one—all in one place—and uses iCloud to privately sync conversational history across a user’s products.”
Beyond the new capabilities that will work across most of Apple’s modern devices, Siri AI is getting some unique attention on Vision Pro. In VisionOS 27, the Siri ‘orb’ becomes a placeable widget that can sit in the room with you. And when you want to talk to it, simply look at the orb and start talking. It’s a seamless way to activate Siri using Vision Pro’s eye-tracking, without needing to tap anything or say a wake word.
Siri is also getting ‘see what you see’ capabilities on Vision Pro. If the user asks the system to look at something, Siri has visual context of the user’s view of both the digital and real world. So you can ask about something you see on a webpage floating in front of you just as easily as you can ask about a piece of artwork on your wall.
This is a much more natural way to use visual intelligence capabilities that have been part of earlier versions of Apple Intelligence but were not exposed in particularly obvious ways.
The new Siri app, which functions more like a traditional AI chatbot, is getting a native visionOS version.
Apple says Siri AI is compatible with both the original Vision Pro (M2) and latest Vision Pro (M5). It’s available as a developer preview within visionOS 27 which is available now. Apple plans to roll out Siri AI features as a “beta” to the public later this year.
My Take
The inclusion of high-accuracy eye-tracking on Vision Pro continues to pay dividends to Apple. Turning Siri on Vision Pro into a persistent widget that’s activated with eye-tracking is the kind of subtle but clever idea that could very well set the standard for interacting with voice assistants on immersive devices going forward. Using eye-tracking to add context about the user’s question is also a smart way to leverage the feature.
Siri’s newfound ability to see the user’s digital and real world brings it much closer in line with Gemini’s visual capabilities on Android XR, which I’ve previously pointed to as a standout advantage over the AI capabilities of visionOS and Meta’s Horizon OS.
It’s unclear at this time if Siri on Vision Pro will be fed a still image of the world around the user at the time of the query, or if it will get a live view of the world around the user (as we see with Gemini on Android XR). The difference between a static or live view could lead to a significant gap in the usefulness of Siri AI’s ‘vision’ on Vision Pro compared to Gemini. On Android XR, Gemini can continuously see what’s around the user, enabling ongoing conversations with Gemini that evolve as new things happen. We’ll have to wait to see if Siri on Vision Pro can do the same.
We also don’t know if Siri will be able to ‘see’ during every query or if only specific queries will cause the headset to consider the world around the user. For Apple’s part, the company says it continues to emphasize privacy in its AI features, and says that any data that leaves the headset is processed in an encrypted way that’s not accessible to Apple or third parties.
The post Siri on Vision Pro is Getting Eye-tracked Activation and Visual Awareness Alongside New AI Features appeared first on Road to VR.
Filed under: Allgemein - @ 9. Juni 2026 19:50