spatialcomputing.info

Meta today announced “major improvements” coming to Quest 2’s controllerless hand-tracking capability. The ‘re-architected computer vision and machine learning approach’ is said to specifically improve reliability for overlapping or fast moving hands and specific gestures. The SDK and OS update to enable these improved capabilities will begin rolling out today.

Meta first introduced controllerless hand-tracking to the original Quest back in late 2019 where it remained an ‘experimental’ feature until mid-2020 when it began allowing developers to use the new capability in their apps.

Since then we’ve seen a handful of games incorporate hand-tracking into their apps and even the launch of some games that exclusively rely on hand-tracking, like Hand Physics Lab (2021) and Unplugged: Air Guitar (2021).

Now, a little less than a year later, Meta says it’s bringing “major improvements” to Quest 2’s hand-tracking capability (the company confirmed the original Quest will not receive these improvements).

The improvements come thanks to a ‘re-architected computer vision and machine learning approach’ which improves the robustness of hand-tracking in key ways.

With the 1.0 version of hand-tracking on Quest 2, the system had particular trouble recognizing the user’s hands when they obstructed or touched each other and when moving quickly. From the user’s point of view, their virtual hands would disappear momentarily during these lost tracking moments and then reappear once the system detected them again.

With the 2.0 version of hand-tracking on Quest 2, Meta says the system will handle those obstructed and fast-moving scenarios much better, leading to fewer instances of disappearing hands. The company calls it a “step-function improvement in tracking continuity.”

The update is also said to improve gesture recognition in the hand-tracking system. Gesture recognition looks for specific hand-poses which the system detects as unique and can therefore be used as inputs. For instance, pinching is one such gesture and it’s employed to allow users to ‘click’ on elements in the Quest interface.

In the demo below, a ‘grab’ gesture is used to hold the virtual object, and the improvement in robustness for clapping is demonstrated as well.

– – — – –

Meta says the hand-tracking 2.0 update on Quest 2 will begin rolling out via an SDK and OS update starting today. The company says developers who have already built hand-tracking into their apps won’t need to change any API calls in order to use the upgraded system, though it won’t be automatically enabled. The company says developers can reference “upcoming documentation” for enabling it in their apps.

The move should bring Quest 2’s hand-tracking a step closer to Ultraleap, which has maintained some of the best hand-tracking in the industry to date, though it isn’t clear yet how the two systems will stack up.