That was a shock to the developers of the virtual reality headset, which Taiwanese smartphone company HTC and video game developer Valve released one year earlier, but it opened a whole new potential to the technology that has only just started to take root — a cost-effective approach to motion-capture technology that could revolutionize the way stories are told. Over the past year, as virtual productions have increased during the pandemic, that potential has become more prominent than ever.

When the initial HTC Vive headset was released, the trackers existed only inside a pair of controllers used to track hand movements, but the company fast-tracked the release of the HTC Vive Tracker, a wireless tracking device that can be attached to any object and mimic its movement in virtual environments. While other VR companies have released trackers of their own since then, Vive remains the industry standard, and more recently has shown new potential as a cheaper version of motion-capture technology used by studios. In March 2021, the company released the third edition of the Vive Tracker ecosystem, as well as the Vive Facial Tracker. Now, anyone looking to experiment with mo-cap filmmaking has access to a new set of options. According to HTC Vive senior director Shen Ye, the Vive Trackers were released without a specific practical need. They let the industry figure it out. “We had no idea what people would do with it,” Ye said. “We thought people might strap on a virtual gun or a sword.” Ye tried it out on boxing gloves, importing his jabs into VR. Instead, the motion-capture potential quickly took root.  “We didn’t expect developers to come to us and saying, ‘Hey, I think you can use this for full-body tracking,’” Ye said. But in retrospect, it made perfect sense: Trackers can be attached to hands, feet, and the waist to fully encapsulate the range of human motion. “There’s no limit to how many trackers you have have as long as you have a PC with enough outlets to plug them in,” Ye said. HTC Vive operates on the basis of two small, box-shaped “base stations” positioned on either side of the room, each of which contains a spinning laser. As long as a person is wearing trackers within that space, the full motion of the trackers are picked up. Little by little, developers began to tinker.

The VR game studio CloudGate Studio put trackers to work with its 2018 release “Island 359,” in which players could not only shoot dinosaurs but also kick them with trackers attached to their feet. Developer Croteam experimented with an actual minigun using Vive trackers for “Serious Sam VR.” And the U.K.-based graphics studio iKinema developed a complex mocap software called Orion powered by Vive Controllers that imported a full range of body movements into VR. The company was bought by Apple in 2019.

However, the most impressive use of Vive Trackers to date came from another unexpected place — breakdance battles. Earlier this year, as the pandemic continues to have a destructive impact on people in the performing arts, dance battles started cropping up on the VR social platform VRchat, with experienced dancers strapping on Vive Trackers to engage in communal dance battles with people from around the world. That eventually lead to the creation of the International Dance Association, the first-ever dance community in VR. “Stuff like that makes me really excited about what we can do for society with this technology,” Ye said, “to connect the world with real social interactions beyond a camera. It’s the actual body language that people are starting to miss.”

And then there’s the filmmaking side of things. Little by little, production studios have started to utilize virtual production technologies to pre-visualize scenes in VR. When Steven Spielberg assessed the virtual sets of “Ready Player One” ahead of production in 2017, he used a Vive headset. Now, trackers can be attached to cameras that translate their movements into a 3D engine (usually the Unreal engine, which was originally created for gaming). If an actor wears trackers, their movements translate to the 3D space as well. Polish-based commercial production company Blackfish Studio has been experimenting constantly with Vive trackers. While working on an animated commercial for the payment service GoPay, the company used a Vive tracker to capture more dramatic angles on its actors during an action scene.

That’s a far more simple and consumer-friendly approach to mo-cap in contrast to the complex technologies typically employed on film shoots. “Because the traditional mo-cap system is so incredibly expensive, people are realizing that a normal gaming PC and consumer-facing VR hardware can let you do something for couple thousand dollars, versus the hundreds of thousands of dollars that it costs for a big mo-cap stage,” Ye said. “This is 50 times cheaper than normal mo-cap studios.” Well, not so fast: “The important thing to know is that they do two different things at two different qualities,” said Ben Grossmann, CEO of Magnopus and a virtual production supervisor whose credits include “The Lion King,” among others. “Traditional mo-cap is capable of large areas, multiple objects, and are more appropriate to production levels of reliability and precision. Vive tracking is presently more suitable to small areas, one-person, lower precision, and quality — not the level of detail expected at movie quality.” In other words, don’t expect James Cameron to shut down his New Zealand mo-cap compound where he’s shooting his “Avatar” sequels in favor of a couple of consumer-grade Vive trackers. But the technology undeniably holds potential for innovative storytellers who can’t afford Hollywood-grade special effects. “I do believe that Vive tracking is a viable low-cost alternative that can empower low-budget filmmakers to do great things,” Grossmann said. “They complement professional solutions and even have a place on professional sets in some contexts.” A more realistic assessment of the cost structure for mocap storytelling with Vive trackers suggests the technology is a little less than 10 times cheaper than the industry standard. Full-body tracking with Vive trackers costs $2,100, while traditional outside-in mo-cap goes for $23,000. But use of Vive trackers can have broader ramifications for a film’s budget, as Ye points out, including the use of extras. “You can actually have one person act out all the movie extras,” he said. “You just have to record the body movements one at a time.” That’s a lot of work, but it opens up room for innovations for filmmakers who can’t afford massive crowds. Meanwhile, Vive’s tracking hardware continues to grow more complex with the release of its facial tracker this year. “As computing gets cheaper, these technologies will end up being more and more accessible by more and more people,” Ye said. Based on its current applications, it’s only a matter of time before cheaper film productions lean into the potential of Vive trackers, but the technology is already here for the taking. Additional reporting by Bill Desowitz. Sign Up: Stay on top of the latest breaking film and TV news! Sign up for our Email Newsletters here.