spatialcomputing.info

Efficient text entry without an actual keyboard remains an industry-wide challenge for unlocking productivity use-cases in XR headsets. Researchers have created a comprehensive catalog of existing text entry techniques to codify different methods and analyze their pros and cons. By making the catalog freely available, the researchers hope to give others a head start on creating new and improved techniques.

Guest Article by Max Di Luca

Massimiliano Di Luca leads the VR Lab at the University of Birmingham, UK, where he is an Associate Professor in the School of Psychology and in the School of Computer Science. He previously worked at Meta where he pioneered work on hand inputs and haptics for VR. His most recent collaboration with industry was recently recognized by the ACM SIGCHI 2025 awards for pioneering the interaction framework of Android XR through exemplary industry-academia collaboration, establishing foundational input methods and interaction guidelines for XR operating systems.

As immersive experiences become increasingly sophisticated, the challenge of efficient text entry remains a crucial barrier to seamless interaction in virtual and augmented reality (VR/AR). From composing emails in virtual workspaces to logging-in and socializing in the metaverse, the ability to input text efficiently is essential for the usability of all applications in extended reality (XR).

To address this challenge, my team from the VR Lab at the University of Birmingham (UK) along with researchers from the University of Copenhagen, Arizona State University, the Max Planck Institute for Intelligent Systems, Northwestern University, and Google developed the XR TEXT Trove—a comprehensive research initiative cataloging over 170 text entry techniques tailored for XR. The TEXT Trove is a structured repository of text entry techniques and a series of filters that aim at selecting and highlighting the pros and cons of the breadth of text input methods developed for XR in both academia and industry.

These techniques are categorised using a range of 32 codes, including 13 interaction attributes such as Input Device, Body Part (for input), Concurrency, and Haptic Feedback Modality, as well as 14 performance metrics like Words Per Minute (WPM) and Total Error Rate (TER). All in all, the number of techniques and extensivity of the attributes provide a comprehensive overview of the state of XR text entry techniques.

Several key takeaways can be surmised from our research. First and foremost, text input performance is inherently limited by the number of inputting elements (whether fingers, controllers, or other character selectors). Only multi-finger typing can lead to performance comparable to touch-typing speed with a keyboard on regular PCs. As visualized in the plots below, each additional input element (or finger) adds about 5 WPM speed on top users.

Words per minute using multiple fingers, and different input devices. (each dot represents one technique analyzed in the study).

Our research also indicates that haptic feedback, the presence of external surfaces, and fingertip-only visualization are preferable ways to improve typing performance. For instance, typing on surfaces (instead of in mid-air) contributes to a more comfortable and potentially more efficient typing experience. External surfaces also minimize sustained muscle strain, making interactions more comfortable and reducing the onset of Gorilla Arm Syndrome.

Finally, and more interestingly, as of today, no alternative has fully replaced the keyboard format, probably because it still delivers the highest words-per-minute. Perhaps because it also requires high learning curves. We believe that the main path for faster typing in VR than PC might lay on the need to reduce travel distances on a multi-finger keyboard via Machine Learning and AI. XR needs its own ‘swipe typing’ moment, which made one-finger typing on smartphones much more efficient.

In that regard, the deep dive from the XR Text Trove represents a significant step towards a more comprehensive understanding of text input in virtual and augmented reality. By providing a structured and searchable database, we aimed to offer a resource for researchers and developers alike, paving the way for more efficient and user-friendly text entry solutions in the immersive future.

As we explain in our paper, this work has the potential to significantly benefit the XR community: “To support XR research and design in this area, we make the database and the associated tool available on the XR TEXT Trove website. The full paper will be presented at the prestigious ACM CHI conference next month in Yokohama, Japan.

Several authors in our team are co-creators of the Locomotion Vault, which similarly catalogs VR locomotion techniques in an effort to give researchers and designers a head-start on identifying and improving various methods.