Meta wants you to feel the metaverse with high-tech haptic gloves

rl-haptic-glove-1

A research prototype for haptic glove from Reality Labs Research.

Meta Reality Labs Research

VR visuals can only look so good, given the current display technology, and we may already be approaching their limit. But what about our other senses? Facebook parent Meta’s dream of one immersive metavers do big promises, but current VR interactivity is limited. That Oculus Quest’s controllers, for example, still feel more appropriate for games than work. After immersive video, mixed reality, face tracking and spatial sound, haptic feedback may be the next limit.

The idea of ​​slip-on vibrating gloves that let you feel the virtual world has been part of our sci-fi vision of VR for a long time. Meta’s research arm has been working on haptic gloves for seven years, and even now the gloves that have been developed are not yet portable. But that could be them one day.

I did not get the chance to try Meta’s prototype myself, but Sean Keller, the company’s Reality Labs research director, and Michael Abrash, its chief researcher, spoke to me through the latest developments. They explained why haptics are so important to Meta and where all of this fits into the future of AR smart glasses.

Gloves made of … blisters?

Meta’s prototype inflatable bladder gloves are a step towards what could eventually become a pair of consumer gloves. However, a new look at the technology shows that it will be some time before we see them bundled with a Quest VR headset sequel.

Recent glove designs use microfluidics to push air through a series of blisters across the gloves, which is less conceptually challenging than filling a pair of gloves with lots of small motors. It’s a weird idea, just like Meta’s version of a Dune stillsuit for your hands.

“We’re literally changing the stiffness of the material,” Keller says. “We have those that use air to move something across your fingertip up and down, or sideways and inward – that help you create that shear force – and those that are small blisters that create pressure.”

rl actuators

Miniaturizing pneumatic actuators (pictured) that use air pressure to create power.

Image courtesy of Meta

But Meta is convinced that haptics can do things at a level that has not been possible before. Facebook Reality Labs Research tried a demo in 2017 that created the feeling of dropping balls made of different materials on your outstretched fingers. The wooden ball felt different from the marble ball and the squishy foam ball. The illusion is a combination of hundreds of pressure bladders (pneumatic actuators) and the visual inputs from VR. According to Michael Abrash, it just would not work the same way on a 2D screen, making this technology specific to AR and VR.

This technology is found in a laboratory now, but Meta’s research indicates a need for an entire haptic reproduction engine. It’s reminiscent of how PlayStation 5’s DualSense controller has needed to develop new ways of creating immersive haptics, but on a larger and more complex scale.

“I have all these blisters and things on my hands, they have tubes. And the next one, I want 10 times more. It’s going to be a big problem from a system perspective, you can not solve it without one day building something like microfluidic microprocessors , “says Keller.

Meta is also working on ways to create the right materials for durability so that the gloves can actually be used in one place. But right now, the gloves are exclusively in research mode.

An experiment from 2017 used a fingertip to give the device haptic feedback as a series of virtual spheres made of different materials – wood, marble, foam – fell from the sky in VR.

Meta Reality Labs Research

Touch as a way to conjure up tools

Based on Meta’s demonstrated demo concepts, many of the uses of haptic gloves right now resemble simulations of hand-based activities: playing Jenga, thumb-wrestling, or picking up objects. Many of these ideas are already kind of possible with hand tracking, just minus the physical feedback.

Where Michael Abrash sees technology making a big difference is simulating virtual tools. Typing on a keyboard or holding a sculpting tool or brush that is actually not there may mean that haptics finally make the idea of ​​working in VR less awkward. For now, of course, it will also mean wearing large, strange gloves that are tethered with cables and tubes, which is just as awkward.

“The question becomes, how well can the gloves mimic the tools? And I do not know the answer to that yet,” Abrash says. “A virtual keyboard could move with your hands, so while typing, if you slide around a bit, we can still deduce what you meant to type.”

rl-haptic-glove-2

An earlier research prototype for haptic glove.

Image courtesy of Meta

Smart glasses do not use haptic gloves (not now at least)

Abrash does not expect to see haptic gloves being worn all the time with smart glasses; instead, the company has been working on electromyography. EMG user wrist-based straps which can sense motor neuron signals, make them gesture and controls. Meta’s already laid out a roadmap for where that technology may be heading. These bands will use wrist vibrations for feedback, but Meta’s full research into haptic gloves may collapse and lead to products that are a fusion of both.

Abrash sees the haptic gloves and neural inputs as being a feedback loop that can make invisible tools work. But that combination of technology is still theoretical: Right now, Meta is just about to make the haptics feel compelling.

“One hole with EMG is that it’s unidirectional you send signals, but there’s no feedback,” Abrash says. “What you can imagine is that you combine EMG and gloves. And the point is not to literally imitate your hand. The point is that your hand is now a surface to give feedback on.”

But EMG is now closer to emerging in true technology, while haptic gloves are still stuck in the test lab. “We probably want one input system that we just want to use all the time. I have no idea what it is, it could be that everyone puts on their gloves in the morning, just like the Victorians did – you wear your gloves all day , and it also has EMG, “says Abrash. “But EMG is this very powerful channel for your mind to express its desires in a way that is completely frictionless. And for me in AR, it’s just a little hard to imagine how we are going to beat it in a near time frame. It is technology that’s on its way to product – I can ‘t tell you when, but it’s not a 10 year thing. With haptic gloves it’s a research thing. And I can not tell you when it’s shipped: It can be a 10-year-old thing. And it really meets a different set of needs. It’s not a way of expressing commands, it’s a way of actually acting in the world. ”

Meta sees EMG as something that will soon find their way into people’s homes, In other words. But we will not see haptic gloves so soon.

rl-sean-keller

Sean Keller leads AR and VR interaction and input research at Reality Labs Research.

Image courtesy of Meta

The metaverse seems close, but it’s not quite here yet

Abrash says his research team is focused on far, far-reaching goals than what Meta’s consumer product team is working on, and we should not expect to see haptic gloves around the corner. But he sees much of Meta’s vision as a work in progress, and haptics and future input are a big part of it.

“For over seven years, we have built the pillars on which any version of the metaverse should be built,” says Abrash about his further work in relation to Mark Zuckerberg’s latest metaverse lifter. “If we had good enough avatars right now, you and I might do this [chat] in VR, right? We are just building the underlying technology on which the meta-verse can be built, and that is what we have always done. So the metaverse conversation came in public, and my feeling was, “Great, I’m glad the world is catching up, it’s a good thing we’re prepared for this.”

Leave a Comment