Here’s taking sensing to the next level: Meta is now testing “super sensing” AI glasses that use cameras and audio recording to continuously collect audio while taking photos every few seconds. The concept is being called a personal agent that’s with you all day long.
According to Financial Review and The Information, based on interviews with anonymous sources, a user would use AI to later query what they saw or heard or recall their day. Super-sensing could be activated on Meta’s existing glasses with a software update.
Quick reaction to the project has provoked worries about privacy, especially from non-wearers of smart glasses. Meta’s current AI smart glasses use an LED in the corner of the frame that lights up to tell others when a wearer is taking photos or filming. However, Meta is apparently planning not to activate the LED when super sensing features are used. Meta discussed its current use of capture LED in a recent post on its website that also discussed related privacy concerns.
Meta is arguably the most all-in on smart glasses, as CEO Mark Zuckerberg thinks glasses could replace the smartphone as the main device people use to access AI tools such as translation apps or chatbots. At a recent quarterly earnings call, he said he wants glasses to evolve from “being able to answer questions to being able to be a personal agent that’s with you all day long, helping you remember things and achieve your goals.”
Meta is reportedly trying out one proposal with the super sensing AI glasses not storing raw footage and audio and instead extracting the audio and image metadata to be uploaded to a server to be queried by Meta’s AI, presumably limiting the privacy concerns. A Meta spokesperson did not respond directly to the reports of super sensing being tested and instead said the company often files patents that "disclose concepts that may or may not be implemented, and a granted patent does not guarantee that Meta has pursued or will pursue the technology described." In other words, in one interpretation, Meta might not pursue super sensing at all.
Even as Meta conjures up new ways to enrich the smart glasses experience, some backlash has emerged against the growth of AI tools in apps and daily use, often by consumers, and not only related to smart glasses. While sensors for smart glasses might excite many sensors manufacturers, the ultimate use of their sensors in AI-related applications has already raised some privacy-related red flags.
In a Q&A with Fierce in April, Amar Abed, CTO at Invensense a TDK Group company, called smart glasses “one of the most natural ways for human to harness the power of AI while also being engaged in the world around us.” Yet, he also said users of smart glasses “want to ensure their privacy and the privacy of people around them are preserved.” OEMs need “to create stylish, wearable glasses while also addressing privacy concerns,” Abed added.
Abed believes consumer adoption of smart glasses is still in the early days, but added, “by the turn of the decade, I expect we’ll see the privacy and power targets needed to drive exponential growth beyond early adopters.”
JR Raphael, contributing editor to Computerworld, recently reflected on a more general AI backlash beyond smart glasses, noting: “As AI creeps into every corner of our lives, an absence of AI may soon be a premium experience worth paying for.” He noted the frustration and even exasperation some people are feeling about AI manifestations.
Jack Gold, principal analyst at J. Gold Associates, called the super sensing technology an example of "sensors gone wild." He said people generally take privacy for granted when interacting with others. "If people are suddenly having AI assistants recording and interpreting every interaction, how candid a conversation can we really have? I expect this type of system will fundamentally change the way we interact, since we can no longer expect any kind of 'it stays between us' conversations...This is not just about AI, it's about any kind of tech powering this kind of device."
Leonard Lee, executive analyst for Next Curve, called super sensing "an early phase effort by Meta" and suggested Meta might use the tech to apply multi-modal models such as small Video Language Moels to generate context information and not the actual content. Still, he said in response to questions from Fierce: "Privacy will be a major concern [with super sensing] especially in public spaces where the right to be forgotten and disregarded/blacked out will become a regulatory hurdle for any product that implement applications based on super sensing."
He said it will be interesting to see if Meta provides full transparencey for super sensing-based applications as they are rolled out. "Transparency will be demanded and enforced in markets outside the US," he said. In China, Meta's services and some products are already blocked or effectively banned.