spatialcomputing.info

Apple today announced its iPhone 15 lineup of smartphones, including the iPhone 15 Pro which will be the company’s first phone to capture spatial video for immersive viewing on Vision Pro.

While Apple Vision Pro itself works as a spatial camera, allowing users to capture immersive photos and videos, I think we can all agree that wearing a camera on your head isn’t the most convenient way to capture content.

Apple seems to feel the same way. Today during the company’s iPhone 15 announcement, it was revealed that the new iPhone 15 Pro will be capable of capturing spatial video which can be viewed immersively on the company’s upcoming Vision Pro headset. The base versions of the phone, the iPhone 15 and iPhone 15 Plus, won’t have the spatial capture capability.

Details on exactly how this function works are slim for the time being.

“We use the ultrawide and main cameras together to create a three-dimensional video,” the company said during its announcement. But it isn’t clear if “three-dimensional” means stereoscopic footage with a fixed viewpoint, or some kind of depth projection with a bit of 6DOF wiggle room.

Given that the iPhone 15 Pro cameras are so close together—not offering enough distance between the two views for straightforward stereo capture—it seems that some kind of depth projection or scene reconstruction will be necessary.

Apple didn’t specifically say whether the phone’s depth-sensor was involved, but considering the phone uses it for other camera functions, we wouldn’t be surprised to find that it has some role to play. Curiously, Apple didn’t mention spatial photo capture, but ostensibly this should be possible as well.

While users will be able to watch their immersive videos on Vision Pro, Apple also said they’ll be able to share the footage with others who can watch on their own headset.

While the new iPhone 15 lineup will launch on September 22nd, Apple says the spatial capture capability won’t be available until “later this year”—which is curious considering the company also said today that Vision Pro is “on track to launch in early 2024.” Perhaps the company plans to allow creators to access the spatial video files for editing and use outside of Apple’s platform?