spatialcomputing.info

While Apple’s new Vision Pro headset isn’t going to satisfy the existing base of consumer VR users, it’s mastering the rest of the basics better than anyone else.

Probably 90% of what consumers are using VR headsets for today is entertainment, and of that entertainment, most of it is gaming. And if you’re among those people using such headsets today, you’ll reasonably be disappointed that Apple Vision Pro lacks controllers and isn’t going to be playing top VR games anytime soon. But for everyone else, it’s a back-to-basics approach that’s laying a sturdy foundation to build upon in the future.

Today at Apple’s headquarters I got to check out Vision Pro for myself. Unfortunately the company didn’t permit any photos or footage during the demo, but the clips below are a fair representation of what I saw.

Apple Vision Pro (APV, let’s call it) is doing what only Apple can: carving out a subset of what other devices do, and making sure that subset of things is done really well. And given the current state of UX on most other headsets, this is a reckoning that was a long time coming.

Look & Tap

It starts with the input. Apple is leaning heavily into using your eyes as a cursor, and a pinch gesture as a click. The headset has cameras on the bottom that face downward so that even subtle pinches from your hand in your lap are visible and detected. But you don’t see a floating cursor where your eyes are, nor do you see a laser pointer shooting out of your hand. You just look at the thing you want to press, then do a quick pinch.

On paper you might think this sounds shoddy. But remember, this is Apple. They’ve tested and refined this system six ways from Sunday, and it works so well that after a minute or two you hardly think about how you’re interacting with the headset, you just are.

The pinch input is responsive and reliable. It felt so natural that the two or three times the headset missed my pinch during a 30 minute demo it felt really weird because my brain was already convinced of its reliability.

This look-and-pinch system is so simple for the headset’s basic input that I won’t be surprised if we see other companies adopt it as soon as possible.

Reality First

So there’s the simple input and then there’s a passthrough-by-default view. This is an MR headset after all, meaning it can easily do augmented reality—where most of your view is of the real world, with some virtual content; or virtual reality—where all of your view is virtual content.

When you put AVP on your head, you instantly see the outside world first. In fact, the way that Apple defers to the passthrough view shows that they want to treat fully immersive experiences as the exception rather than the rule. Generally you won’t pop into a fully immersive scene unless you actively making the decision to do so.

The passthrough view is certainly best-in-class, but we’re still probably two generations away from it truly feeling like there’s nothing separating your eyes from the real world. Granted, I was able to read all the text on my phone with no issue, which has been the ‘bar’ for passthrough quality that I’ve been waiting to see exceeded.

Beautiful Virtual Displays

The imperfect passthrough resolution somewhat betrays the exceptional display resolution which exhibits not even a hint of screen-door effect. It may not be ‘retina resolution’ (generally agreed to be around 60 pixels per-degree), but it’s good enough that I won’t know how far off it is from retina resolution until I sit down with an objective test target to find out.

That’s a long way of saying that the headset’s display has excellent resolution with great clarity across the lens. Top of the class.

This clarity is helped by the fact that Apple has done its Apple-y thing and ensured that panels, text, and images consistently render with superb quality. The entire interface feels iOS-polished with animations and easy to use buttons and controls. The interface was so simple to use that the demo chaperones had a hard time keeping me on task as I wanted to flick through menus and move floating apps around the room.

But here’s the thing, probably 75% of what Apple showed me was essentially just floating screens. Whether it was videos or a floating iMessage app or the web browser, it’s clear that Apple wants Vision Pro to be first and foremost be great at displaying flat content to the user.

The other 25% of what I saw, while very impressive all around, felt like just the start of a journey for Apple to build out a broader library immersive experiences.

Record & Rewatch Memories

AVP might not be a VR gaming headset, but it does at least one thing that no other headset does: capture volumetric memories using its on-board cameras. Using the button on the top of the headset you can capture volumetric photos and videos with just a press.

Apple showed me a demo of a volumetric video capture of a group of kids blowing out candles on a birthday cake. It was like they were right in front of me. I’d never even seen these kids before but I could immediately feel their giddy emotions as they giggled and bounced around… as if I was sitting right there while it was happening. Not to mention that the quality was good enough, at least in this best-case-scenario demo capture, that my first thought had nothing to do with the famerate or quality or dynamic range, but purely of the emotion of the people in front of me.

That instant connection—to people I don’t even know—was a clear indicator that there’s something special to this. I can already imagine watching a volumetric video of a cherished memory, or of a loved one that has passed, and I know it would be a powerful experience.

Doing it Right

And here’s the thing; I’ve seen plenty of volumetric video demos before. This isn’t a new idea, not even close. The thing that’s novel here is that everyday users could potentially shoot these videos on their own, and readily watch, share, and store them for later. On other headsets you’d need a special camera for capturing, special software for editing, a player app, and a sharing app to make the same thing happen.

This is the ‘ecosystem’ part of XR that’s missing from most other headsets. It’s not about what’s possible—it’s about what’s easy. And Apple is focused on making using this headset easy.

Continue on Page 2: Immersion Isn’t Off the Table »