MindMaze developed a cost-effective VR HMD upgrade called Mask that can detect biosignals in your face to bring your facial expressions and emotional queues into VR experiences to help improve virtual social interactions.
Social VR experiences, which let you communicate and share activities with other people from around the world, are incredibly compelling. Oculus proved that when it showed off Toybox for the first time, and platforms like AltspaceVR, VRChat, and Rec Room as well as co-op games like Arizona Sunshine and Raw Data reinforce the idea.
In the real world, emotional expression is a large part of face-to-face communication. You can tell a lot about a person’s demeanor and intent from the expressions on their face, irrespective of the words they are saying. In VR, you don’t have the luxury of seeing the face of the person you’re talking to, so you can’t pick up on such emotional cues. The developers of VRChat mapped several facial expressions to the thumbstick/trackpad on your controller to address this problem, but those expressions aren’t natural, which means they aren’t a reliable gauge of that person’s mood.
MindMaze has a solution called Mask that could bring emotional communication to VR, which could make social VR a whole lot more interesting.
“Humanizing virtual reality is a natural progression. What makes us human in social interactions is emotion, but that’s been missing in virtual reality so far,” says Tej Tadi, CEO, and founder of MindMaze. “For the industry to progress, all experiences, but especially those in transmedia VR, have to improve. Until now it’s been stunted by the lack of quality, immersive content. MASK brings emotion, the ultimate driver of engagement, with the goal of evolving VR beyond novelty and into the mainstream.”
MindMaze Mask is a combination of hardware sensors and software. Mask uses biosignal processing technology and machine learning from the healthcare industry to detect your facial expressions “tens of milliseconds” before they manifest on your face. MindMaze’s software then replicates your expressions on a VR avatar in real-time.
Mask can replicate a wide variety of facial expressions, including anger, disgust, happiness, or sadness. MindMaze said that Mask’s sensors detect electrical impulses in your face to determine your facial expression. The company developed proprietary algorithms to analyze the signals and build a “neural signature” of your expressions.
MindMaze engineered Mask technology to be an affordable upgrade for existing VR HMDs and to be compatible with any HMD on the market. The sensors that MindMaze uses for Mask are compact and don’t require a lot of power. The company said they would work well in mobile VR devices.
We spoke with Tej Tadi, MindMaze’s founder and CEO, via Skype shortly before the Mask announcement. During our call, Tadi showed us a remote demo of the Mask technology in action, and honestly, our jaw dropped to the floor when we saw the technology in action. Tadi had a volunteer with an upgraded OSVR headset demonstrate the facial expression tech, and to prove the demo was real, they asked us to request expressions. During the test, they showed us smiling, frowning, and a look of surprise, and the transitions from one expression to the next were seamless and instantaneous.
We also asked to see a single eyebrow raise, but Tadi said that wasn’t possible with the current number of sensors. Tadi said that a larger number of sensors would allow tracking of such an expression, but he wants to keep the costs down to a minimum. He didn’t specify how much a Mask upgrade would cost, but he said the setup costs “tens of dollars” to build and he plans to keep it that way.
Tadi said MindMaze is currently shopping for partners to bring the Mask product to market. The company is considering fostering direct relationships with HMD makers, but it’s also exploring an aftermarket solution that could work with any VR HMD. Whichever path the company takes, we just hope it doesn’t take long to bring Mask-enabled hardware to the market.