spatialcomputing.info

Ray-Ban and Meta’s second-generation smartglasses are slated to get a lot smarter starting this week, as the company launched an opt-in beta in the US that will hook in the device’s camera into AI-powered object recognition, powered in part by Microsoft’s Bing.

Ray-Ban Meta Glasses were released in October shortly after the company launched its latest VR headset, Quest 3. The glasses are a second-gen follow-up to its Facebook Ray-Ban Stories glasses released in 2021, introducing improved cameras, audio and more design options. The $300 glasses are admittedly pretty smart, although they don’t include displays, meaning user input is handled by voice assistant or by basic touch on the glasses’ struts for things like taking pictures, videos, and listening to music.

Now Meta CTO Andrew ‘Boz’ Bosworth announced via Instagram that the company is rolling out multimodal AI that will use the device’s camera to add some new functionality, allowing the device’s cameras to perform AI-powered object recognition.

Rolling out first to US-based users who opt into the Early Access feature, this expands the functionality of the AI assistant by allowing you to ask Meta to do things like “look and tell me what you see,” as demonstrated by Bosworth in the Instagram post.

This also lets you ask other questions about photos you take, which could be things like an English translation of a sign you’re looking at in a different language, the genus species of a houseplant, or to write a caption for the photo you just took. The company says these are only a few of the sorts of things it can do, as it expects to expand that list over time.

Meta has been working with Microsoft for a while now, so it’s no surprise the new object recognition tools are rolling out to Ray-Ban Meta Glasses. Back in September 2023, Meta introduced Bing-powered chatbot functionality to US-based users of WhatsApp, Messenger, and Instagram, letting users do basic text-based interactions and generate images with AI.

If you’re in the US, to join the Early Access program you can simply head to the Meta View app on your phone, tap the ‘gear’ icon in the bottom right. Swipe down and tap Early Access.