Not Perfect, But Promising
Mask wasn’t perfect by any means. I saw the occasional misinterpretation of data which could cause my avatar’s face to do something that I wasn’t actually doing. When it came to normal blinking (not winking), the prototype didn’t seem to pick up my blinks very well. However, Tadi claims that the algorithm gets smarter and more robust over time individually for each user as they use it.
How much better it can get over time is the big question. It’s going to need to be pretty darn close to 100% because users will need to be able to trust the system to represent them accurately. Facial expressions are a hugely important part of unspoken communication; you wouldn’t want to be in an important virtual meeting with a potential new client and look as if you’re sneering at them (due to misinterpretation of your facial expressions) when you’re actually smiling. And even outside of important meeting scenarios, face-tracking in VR still needs to be highly robust so that it doesn’t lead to awkward facial movements that end up dropping your avatar into the uncanny valley.
The ability to improve Mask’s robustness from 90% to 99% will make or break the applicability of the tech.
An Unfair Advantage
While computer-vision based face-tracking has the potential to be much more precise—capturing subtle changes in the face rather than just being able to jump between a few poses—even if the headset didn’t pose a significant challenge, this electrode-based approach has an advantage that computer-vision based face-tracking physically cannot: prediction.
Tadi claims that the Mask algorithm can, via the electrical signals being read from your face, sense the expressions you’re going to make “20 to 30 milliseconds” before you make them. If true, that potentially gives an advantage to this method, opening the door to essentially instantaneous mirroring of your expression in the virtual world.
The electrode/algorithm approach to face-tracking is also interesting because it doesn’t require much processing power, Tadi says, and the workload can easily be performance on a VR headset powered by a smartphone. That’s in contrast to the computer-vision approach which requires more heavy-duty processing to derive your facial expression from each frame.
So, Mask seems promising. What’s next? Tadi says that the underlying technology is compatible with any headset, and that the company is in talks with all the major players in the market. Ultimately the company seeks not to sell the hardware as an add-on, but offer a simple blueprint for direct integration and then license the algorithm. We could see a headset with this tech integrated as early as holiday 2017.
While I think that VR is destined to one day incorporate face-tracking, it’s not yet clear to me exactly how precise and accurate it needs to be in order to be a must-have feature (and one that doesn’t risk misrepresenting the user). Given the current state of the industry, bringing headset costs down is a greater priority right now than adding more features (and cost), which could mean tough timing for Mask.