spatialcomputing.info

Much ado has been made about Meta’s latest avatars which juxtapose impressive expression against, well… a complete lack of legs. While full-body tracking is both desirable and achievable today with outside-in tracking systems, Meta doesn’t think it’s viable with inside-out tracking on headsets like Quest 2. However, the company say it’s investigating ‘fake’ legs instead.

Andrew “Boz” Bosworth is the VP of Meta Reality Labs (the company’s XR division), and soon to be CTO of Meta overall. In his role heading the company’s XR division he’s made a habit of doing impromptu Q&A sessions via Instagram where he answers both personal and work-related questions.

In his latest Q&A he was asked about the potential for full-body tracking on future headsets from the company, but dismissed the idea as not viable with inside-out tracking.

While Quest 2 is presently capable of head and hand-tracking (which makes it relatively easy to estimate the position of the arms and chest too), the headset has no concept of where your legs, feet, or hips are, and that’s why Meta’s avatars are essentially chopped off below the waist when you see them in VR.

On the other hand, other VR systems using cameras external to the headset (known as outside-in tracking) are capable of full-body tracking to enable more lifelike avatars and some use-cases that simply wouldn’t be possible without it.

It’s been suggested that with some computer-vision trickery, perhaps tracking cameras mounted on the headset (known as inside-out tracking) could be used to estimate the position of the user’s feet. Bosworth reasoned that this is not only extremely difficult given the position of the cameras on Quest 2, but will become even harder as headsets shrink.

Body tracking is super tricky. Because from the camera that’s on your face it can’t see your legs very well. And as we want to make the [headset] smaller form-factor, it gets even worse—[the cameras] can’t even see past your cheek sometimes to your upper body. Now we can get away pretty well with the upper body because we can see your arms, elbows, hands, and we kind of have a sense of what the musculoskeletal structure must be doing behind it, but feet are tough.

So [using outside-in tracking as opposed to inside-out] is probably necessary for some of the [full-body tracking] use-cases people have in mind. So that’s one of the things that we’re looking at.

Now there’s been a leaked thing [about full-body tracking on Quest]… I think that’s actually probably a little premature… there’s nothing substantial behind that necessarily. But it’s something that we’re always looking at and have in mind for sure.

The “leaked thing” Bosworth mentioned is likely a hint about body tracking in Quest 2 documentation that was spotted by our friends over at UploadVR.

Later in the Q&A, Bosworth again addressed Meta’s legless avatars and suggested that the company will probably move forward with ‘fake’ legs, which would only be seen from a third-person vantage point, as a stopgap.

Tracking your own legs accurately is super hard and basically not workable just from a physics standpoint with existing headsets [that use inside-out tracking]. You could go outside-in body tracking—but that’s an extra component, an extra cost, an extra setup—it’s a lot. Or you could have no legs, but everyone else when they look at you could see that you have legs and we could fake it, and no one would know the difference. That’s a better direction.

Obviously we’ve read the jokes [about our legless avatars], we appreciate them; they are very funny and very fair. So we are looking at how we can do [some form of legs]… if you look at your own legs and you see them out of position, that is a very bad experience and you feel very dysmorphic, but if you look at someone else’s legs and we just made up their position but they seemed reasonable, you’d just be like ‘yeah that’s probably where [their legs] are.’

Naturally Bosworth understands that the fake legs approach wouldn’t enable use-cases like using your legs as game input or for things like dancing, but at least in simple social VR settings—where avatars are just hanging out and chatting—it would be nice to have avatars that aren’t awkwardly missing their lower half while floating in the air.

Elsewhere in the Q&A Bosworth was asked if Meta would consider subscription-based access to the Quest content library (akin to Xbox Game Pass or Viveport Infinity). He said he didn’t think the company “has a catalog yet that could sustain a subscription,” further saying that cross-platform content would probably also be necessary for that business model to work (which is not presently the case with the Quest store). He did indicate that the company is likely to introduce gift cards for the Quest store in the future.