spatialcomputing.info

The 2D to 3D photo conversion feature coming to Vision Pro in VisionOS 2.0 makes a novel capability meaningful for the first time.

Cue “Apple didn’t even do it first!” in the comments.

You’re not wrong. There’s been seemingly a hundred different startups over the years that have promised to turn 2D photos into 3D.

Even Meta had a go at it when it added 2D to 3D photo conversion to Facebook several years ago. But they never really caught on… probably because seeing 3D photos on a smartphone isn’t that exciting, even if Facebook added a little ‘wiggle’ animation to show the depth on 2D displays.

When it comes to features that people actually want to use—it doesn’t matter who does it first. It matter who does it well.

This headline says “the real deal,” because Apple has, in fact, actually done it well with Vision Pro. The 2D to 3D conversion doesn’t just look good, the feature is actually implemented in a way that takes it beyond the novelty of previous attempts.

The feature is part of visionOS 2.0, which is currently available in a developer beta. Apple says the feature creates “spatial photos” from your existing 2D images (which of course just means stereoscopic ‘3D’).

Granted, even though it’s “just stereoscopic,” seeing your own photos in 3D really adds a layer of depth to them (figuratively and literally). While a 2D photo can remind us of memories, a 3D photo feels much closer to actually visiting the memory… or at least seeing it through a window.

In VisionOS 2.0, just go to the usual Photos app, then open any photo and spot the little cube icon at the top left. Click it and the headset analyzes and converts it to 3D in just two or three seconds. With a click you can also return to the original.

The results aren’t perfect but they’re very impressive. It’s unfortunate I can’t actually show them to you here—since I have no way to embed a 3D photo in this page, and 99.9% of you are probably reading this on a 2D display anyway—but it’s the best automatic 2D to 3D photo conversion that I’ve personally seen.

The speed and accuracy is doubly impressive because the conversion is happening 100% on-device. Apple isn’t sending your photos off to a server to crank out a 3D version with cloud processing resources and then sending it back to your headset. That makes the feature secure by default (and available offline), which is especially important when it comes to a dataset that’s as personal as someone’s photo library.

Across the photos you’d find in the average person’s library—pictures of people, pets, places, and occasionally things—the conversion algorithm seems to handle a wide range of these very well.

While the feature works best on real-life photography, you can also use it on synthetic imagery, like digital artwork, AI-generated photos, 3D renderings, and the like. Results vary, but I overall I was impressed with the feature’s ability to create plausible 3D depth even from synthetic imagery which itself never actually had any 3D depth in the first place.

The thing the algorithm seems to struggle with the most is highly reflective and translucent surfaces. It often ends up ‘painting’ the reflections right onto the reflecting object, rather than projecting them ‘into’ the object with correct depth.

The only major limitation at the moment is that 2D to 3D photo conversion doesn’t seem to want to work on panoramic images. On Vision Pro panoramas can already be blown up and wrapped around you in a way that feels life-sized, but they would still get another layer of emotional impact from being 3D-ified.

It’s unclear why this limitation exists at present, but it’s likely either because panoramas tend to be very high resolution (and would take longer than a few seconds to convert), or Apple’s 2D to 3D algorithm needs more training on wide field-of-view imagery.

Beyond that limitation, the thing that really makes this feature… a feature (not just a ‘technical possibility’), is that it’s built right in and works in the places and ways you’d expect.

Not only can you send spatial photos to other users who can view them in 3D on their own headset, you can also start a SharePlay session and view them together—an incredible way to share moments and memories with the people that matter to you.

And its easy to actually get the photos you want onto your headset for viewing.

Many people will have their iCloud photos library synced with their headset, so they’ll already have all their favorite photos ready to view in 3D. I personally don’t use iCloud photos, but I was easily able to select some of my favorite photos from my iPhone and AirDrop them, which automatically opened the Photos app so they were right in front of me in the headset.

Further, you can just save any old photo to your headset—be it from Facebook, a website, or another app—and use the 2D to 3D conversion feature to view them with a new layer of intrigue.

And this is what makes this visionOS 2.0 feature different than 2D to 3D conversion software that has come before it. It’s not that Apple has any groundbreaking quality advantage in the conversion… it’s the fact that they made the experience good enough and easy enough that people will actually want to use it.