Why you can’t have legs in virtual reality (yet)

By Rachel Metz, CNN Business

Mark Zuckerberg stares at a cartoon version of himself in a virtual world and tries to pick an outfit for his avatar to wear. Zuckerberg flicks his hand to change the avatar’s apparel, switching from a black shirt, slacks and white sneakers to a skeleton costume to an astronaut suit.

“All right. Perfect,” he finally says, settling on the shirt, slacks and shoes combination for his avatar, which happens to be the exact same outfit Zuckerberg is wearing. Zuckerberg’s avatar then teleports to a virtual spaceship-type setting to meet friends — one of whom is floating, another who appears as a big red robot.

The clip, which Facebook presented in October to help explain its name change to Meta and its new direction as an architect of the still-not-real “metaverse,” is captivating. It envisions the ability to create true digital facsimiles of ourselves in a virtual world. But there’s just one problem: It’s still far beyond the capabilities of Meta’s current virtual reality. Today, if you gallivant around Meta’s flagship social VR app, Horizon Worlds, you do so without legs or feet. While you can customize your avatar to look somewhat like you, it will still have the ghostly appearance of a floating torso with only a head, arms, and hands.

The difference between the realistically responsive full-body avatar Zuckerberg imagines and the typical options currently available in VR apps is more than just aesthetic. Using VR without meaningfully mimicking all your real-life movements can limit the ability to feel fully immersed in VR services. And it makes the future of Meta, with its lofty promise of a metaverse — a wide-ranging virtual realm that people can explore via digital avatars — feel that much further away. But moving beyond the kind of legless avatars seen in the company’s Super Bowl ad on Sunday will be a challenge.

Meta, which declined to comment for this piece, has been considering for years how to make avatars more realistic. In an Instagram AMA (Ask Me Anything) session earlier last week, Andrew Bosworth, Meta’s VP of Reality Labs and incoming CTO, acknowledged the difficulty of the task while saying the company is considering how to solve it.

“Tracking your own legs accurately is super hard and basically not workable just from a physics standpoint with existing headsets,” Bosworth said.

Companies can track a person’s upper body reasonably well with a headset and controllers, but actual leg tracking is practically non-existent in virtual reality right now — at least when it comes to the kind of VR you’re likely to use in your living room. Some apps, such as VRChat, do let people have full-body avatars, but they tend to use software to approximate lower-body motions; it can be silly-looking at best and disconcerting (or even sickening) at worst.

Despite all the progress made in perfecting the technology behind VR headsets in recent years, it’s still tricky to perfectly track your legs in real life and recreate the same movements in VR without setting up an array of sensors on or around your body. Still, several VR experts told CNN Business they think it’s important to bring legs into virtual spaces.

“The reason you want legs is it grounds us,” said Avi Bar-Zeev, an Oakland, California-based VR and AR consultant for startups and a former employee at Apple and Amazon who also worked on Microsoft’s HoloLens. “It ties us to the world.”

The evolution

It’s not easy to create a sense of presence in VR, but the options have evolved a lot.

If you bought an Oculus Rift headset in 2016, for instance, you had to connect it to a powerful PC as well as to a sensor camera on a stand that tracked the headset. At first, the headset didn’t even come with tracked hand controllers; it initially shipped to customers with an Xbox controller and a small handheld remote.

In the past few years, tethered headsets and external sensors have mostly been replaced by what’s seen as a more user-friendly approach: a wireless, all-in-one VR headset with built-in sensors, such as Meta’s Quest 2, which is the most popular by far. (Market researcher IDC estimates the headset made up three-quarters of all recent headset shipments.)

These headsets are far more portable than their predecessors, and easier to set up and use (there’s no risk of literally getting tangled up while using one). But tracking largely remains limited to the body parts that are the most essential and can be tracked well with the headset’s sensors and a pair of controllers: your head and your hands.

If a company wants to represent a person’s legs in a realistic way in VR, it needs to find a way to keep tabs on what those legs are actually doing in real life. Adding more sensors to the headset itself — like cameras on the underside that point toward the ground — might seem like a potential solution, but, as Bar-Zeev said, it’s not that easy.

Bodies come in so many shapes and sizes, and they change over time. For many of us, that means those cameras wouldn’t have a great view of our legs and feet, making it hard to capture enough hints about your motion to infer what your legs should be doing in VR. Among other obstacles, if you were to tilt or turn your head, Bar-Zeev pointed out, a ground-facing camera in a headset would lose sight of the limbs it’s attempting to track.

“That’s not a very reliable way to get people’s legs, and it’s a pretty bad angle to capture the legs from,” he said.

There’s another option for bringing your legs into VR that’s already on the market: physical sensors that attach to your body. HTC, which sells several different Vive VR headsets mostly geared toward enterprise users, also offers $129 trackers that you can strap to your limbs or to objects (such as a tennis racket) to track them in VR. For now, though, the trackers only work with headsets that tether to a PC, and they require a base station as well.

HTC is moving in the direction of more powerful and portable tracking. It soon plans to start selling a $129 wrist-worn tracker that can track an arm from the fingertips to the elbow and which is meant to work with the Vive Focus 3 (a self-contained $1,300 VR headset currently marketed to companies rather than individual users). It’s a step toward giving people more realistic hands and arms in VR than handheld controllers currently provide, and it may help lead to more limb tracking.

Daniel O’Brien, general manager of HTC America, said that over time he expects the ability to track more points on the body, such as feet and hips. “I think full body immersion and tracking with an all-in-one headset is what everybody wants and everybody is working toward,” O’Brien said, though like Meta’s Bosworth he cautioned that it’s just not possible right now.

“VR is hard enough as it is”

More realistic tracking and body representations could help draw more mainstream interest in VR, but the technological improvements required to make that happen risk making the user experience more cumbersome, costly and less enticing to potential customers, at least in the near-term.

“VR is hard enough as it is; no one’s going to assume consumers will put [sensors] on themselves,” said Timoni West, vice president of augmented and virtual reality at game-development platform Unity.

West suggests that leg movements may eventually be animated with the help of AI: Motion could be predicted based on data from the headset about how a person’s head is moving. However, doing this well would require a massive amount of data about the ways people walk, for instance, and since it would not include tracking the legs specifically, it wouldn’t be true to how each person moves. (Meta already uses AI along with sensors to track headsets, controllers, and hands.)

Depending on how realistic the legs appear, this approach may also bring up the issue of the Uncanny Valley, or the creepy feeling some people have in response to human-like representations that aren’t quite human.

Not everyone wants a VR body that mimics the one they have in regular life. West prefers non-human avatars, like an anthropomorphized stick of butter that’s one of many options in VRChat. They also predict that, rather than seeing an increase in VR legs, there will be fewer people asking about VR legs over time.

“The problem is the closer it gets to real, the more it starts to bug people that it isn’t real,” West said.

The-CNN-Wire
™ & © 2022 Cable News Network, Inc., a WarnerMedia Company. All rights reserved.