spatialcomputing.info

Vision Pro is built entirely around hand-tracking while Quest 3 uses controllers first and foremost, but also supports hand-tracking as an alternate option for some content. But which has better hand-tracking? You might be surprised at the answer.

Vision Pro Hand-tracking Latency

With no support for motion controllers, Vision Pro’s only motion-based input is hand-tracking. The core input system combines hands with eyes to control the entire interface.

Prior to the launch of the headset we spotted some footage that allowed us to gauge the hand-tracking latency between 100-200ms, but that’s a pretty big window. Now we’ve run our own test and more precisely find Vision Pro’s hand-tracking to be about 128ms on visionOS beta v1.1.1.

Here’s how we measured it. Using a screen capture from the headset which views both the passthrough hand and the virtual hand, we can see how many frames it takes between when the passthrough hand moves and when the virtual hand moves. We used Apple’s Persona system for hand rendering to eliminate any additional latency which could be introduced by Unity.

After sampling a handful of tests (pun intended), we found this to be about 3.5 frames. At the capture rate of 30 FPS, that’s 116.7ms. Then we add to that Vision Pro’s known passthrough latency of about 11ms, for the final result of 127.7ms of photon to hand-tracking latency.

We also tested how long between a passthrough tap and a virtual input (to see if full skeletal hand-tracking is slower than simple tap detection), but we didn’t find any significant difference in latency. We also tested in different lighting conditions and found no significant difference.

Quest 3 Hand-tracking Latency

How does that compare to Quest 3, a headset which isn’t solely controlled by the hands? Using a similar test, we found Quest 3’s hand-tracking latency to be around 70ms on Quest OS v63. That’s a substantial improvement over Vision Pro, but actual usage of the headset would make one think Quest 3 has even lower hand-tracking latency. But it turns out some of the perceived latency is masked.

Here’s how we found out. Using a 240Hz through-the-lens capture, we did the same kind of motion test as we did with Vision Pro to find out how long between the motion of the passthrough hand and the virtual hand. That came out to 31.3ms. Combined with Quest 3’s known passthrough latency of about 39ms that makes Quest 3’s photon to hand-tracking latency about 70.3ms.

When using Quest 3, hand-tracking feels even snappier than that result suggests, so what gives?

Because Quest 3’s passthrough latency is about three-and-a-half times that of Vision Pro (11ms vs. 39ms), the time between seeing your hand move and your virtual hand move appears to be just 31.3ms (compared to 116.7ms on Vision Pro).

– – — – –

An important point here: latency and accuracy of hand-tracking are two different things. In many cases, they may even have an inverse relationship. If you optimize your hand-tracking algorithm for speed, you may give up some accuracy. And if you optimize it for accuracy, you may give up some speed. As of now we don’t have a good measure of hand-tracking accuracy for either headset, outside of a gut feeling.