--
FACEBOOK PROPOSES A BIZARRE LOOKING TRANSPARENT VIRTUAL REALITY HEADSET
It has a television camera trained on you.
Even if the results are somewhere between moderately unpleasant and horrific, Facebook Reality Labs aims to help people see your eyes when you’re in virtual reality. FRL published a report earlier this week on “reverse passthrough VR,” a method for making virtual reality headsets less physically isolating. Researchers found a method for projecting your face onto the front of a headset, although they emphasize that it is still a work in progress.
Passthrough VR is a function that shows a live video stream from a headset’s cameras while the user is still wearing the device, allowing them to see the actual world. When users leave the confines of their virtual reality realm, Facebook’s Oculus Quest platform, for example, displays a passthrough feed. It’s useful for swiftly exiting VR, and it can also be used to enable augmented reality.
It can be used to exit VR rapidly, and it can also be used to enable augmented reality by adding virtual objects to the camera stream. However, as FRL points out, people in the vicinity of a headset user are unable to make eye contact, even if the wearer can see them perfectly. Bystanders who are accustomed to seeing their friend or coworker’s bare face may find this awkward.
Nathan Matsuda, a scientist at FRL, sought to fix that. According to a blog post, Matsuda began in 2019 by mounting a 3D display onto an Oculus Rift S headset. The screen showed a virtual representation of Matsuda’s upper face, with custom-rigged eye-tracking cameras capturing where Matsuda was looking so that his avatar’s eyes could gaze in the same direction. Matsuda wore a telepresence tablet that displayed a duplicate of his own face as a result, which is probably just as embarrassing but with a more intriguingly postmodern twist.
According to the blog article, FRL chief scientist Michael Abrash didn’t think the proposal was very feasible, which is understandable. He says, “My first thought was that it was kind of a ridiculous idea, at best a novelty.” “However, I don’t tell researchers what to do since creativity requires the opportunity to try new things.”
Matsuda took the idea and led a team in designing a sleek design over the next two years. The team’s prototype headset adds a stack of lenses and cameras to a regular VR headset display, which it presented ahead of next week’s SIGGRAPH conference. Inside the headset, the stereo cameras record an image of the face and eyes, and their motion is translated into a digital model of the face.
The image is then projected onto a light field display that faces outward. That display gives the impression of gazing through thick goggles and seeing a pair of eyes, but you’re actually looking at a real-time animated duplicate. The display can go blank if the wearer returns to full VR, indicating that they are no longer engaged with the outside world. The end result is a pair of octagonal goggles that belong in a Terry Gilliam film.
Individual components of the system aren’t all revolutionary, according to FRL. HTC already offers a face tracking add-on for its Vive Pro headsets; it maps movement to an avatar inside VR rather than an outward-facing screen, but the concept is the same. The focus of this week’s paper is on the potential of light field displays and how they can improve in-person social interactions.
Although many of those glasses have darkened lenses, and as Road to VR points out, projected light on clear lenses can potentially block your gaze, HoloLens-style projection glasses theoretically leave your face considerably clearer than passthrough displays. However, while businesses such as Apple are said to be experimenting with passthrough designs, Facebook’s recent research indicates that a solid screen isn’t always a barrier to eye contact.
