The VR center had no idea they were developing the next phase of motion capture technology.
Five years ago, when “Rogue One” was nominated for an Oscar for Best Visual Effects, the HTC Vive team was surprised to see that they were part of the story. In a BBC documentary about the execution of the film’s fast-paced space scenes, a member of the Industrial Light & Magic effects team showed how they used a virtual reality camera in real time to explore the imaginary environment before taking pictures inside. . This camera took the form of an iPad with a Vive controller attached to the back.
This was a shock to virtual reality headset developers, which Taiwanese smartphone company HTC and video game developer Valve launched a year earlier, but which opened up new potential for technology that just just starting to take root: a cost effective approach to motion capture technology that could revolutionize the way stories are told. Over the past year, as virtual productions have increased during the pandemic, this potential has become more prominent than ever.
When the initial HTC Vive headset was launched, trackers only existed within a pair of drivers that were used to track hand movements, but the company quickly tracked the version of the handset. ‘HTC Vive Tracker, a wireless tracking device that can connect to any object and mimic its movement in virtual environments. While other RV companies have launched their own followers since then, Vive continues to be the industry standard and, more recently, has shown new potential as a cheaper version of the motion capture technology used by studies.
In March 2021, the company launched the third edition of the Vive Tracker ecosystem, as well as the Vive Facial Tracker. Now, anyone who wants to experiment with making mo-cap movies has access to a new set of options.
According to Shen Ye, senior director of HTC Vive, the Vive Trackers were launched without a specific practical need. They let the industry find out. “We had no idea what people would do with it,” Ye said. “We thought people could put on a virtual weapon or a sword.” You tried it with boxing gloves, important your spikes in VR. In contrast, the potential for motion capture took root quickly. “We didn’t expect the developers to come to us and say,‘ Hey, I think you can use it for full-body tracking, ’” Ye said.
But in retrospect, it made perfect sense: trackers can be connected to the hands, feet, and waist to completely encapsulate the reach of human movement. “There’s no limit to how many trackers you have as long as you have a PC with enough sockets to connect them,” Ye said.
HTC Vive works on the basis of two small box-shaped “base stations” located on either side of the room, each of which contains a rotating laser. As a person carries crawlers within this space, the complete movement of the crawlers is collected. Gradually, the developers started playing.
The virtual reality game studio CloudGate Studio launched the crawlers with its 2018 version, “Island 359,” in which players could not only shoot dinosaurs, but also kick them with crawlers connected to the feet. Developer Croteam experimented with a real mini-weapon using Vive fans for “Serious Sam VR”. And the UK-based graphics studio iKinema developed a complex mocap software called Orion powered by Vive controllers that imported a full range of body movements into virtual reality. Apple bought the company in 2019.
However, the most impressive use of Vive Trackers to date came from another unexpected place: breakdance battles. Earlier this year, as the pandemic continued to have a destructive impact on people in the performing arts, dance battles began on the VR VRchat social platform, with experienced dancers facing Vive Trackers. to participate in community dance battles with people from all over the world. world.
This eventually led to the creation of the International Dance Association, the first VR dance community. “Something like that makes me really excited about what we can do for society with this technology,” Ye said, “to connect the world with real social interactions beyond a camera.” It’s the real body language that people are starting to lose. “
And then there’s the cinematic side of things. Gradually, production studios have begun to use virtual production technologies to preview scenes in virtual reality. When Steven Spielberg evaluated the virtual sets of “Ready Player One” before production in 2017, he used a Vive headset. Now, trackers can be connected to cameras that translate their movements into a 3D engine (usually the Unreal engine, which was originally created for gaming). If an actor carries followers, his movements are also translated into 3D space.
Polish-based commercial producer Blackfish Studio has constantly experimented with Vive fans. While working on an animated ad for the GoPay payment service, the company used a Vive tracker to capture more dramatic angles to its actors during an action scene.
This is a much simpler and simpler approach for the mo-cap consumer in contrast to the complex technologies commonly used in film shootings. “Because the traditional mo plug system is incredibly expensive, people realize that a normal gaming computer and consumer-facing VR hardware can let you do something for two thousand dollars, compared to hundreds of thousands of dollars. dollars that cost stage cover, “Ye said.” That’s 50 times cheaper than normal studies. “
Well, it’s not that fast: “The important thing is to know that they do two different things with two different qualities,” said Ben Grossmann, Magnopus CEO and virtual production supervisor who has among his credits “The Lion King”. “The traditional mo-cap is able to develop wide zones, multiple objects and is more adapted to the levels of production of reliability and precision. Currently, Vive tracking is more suitable for small areas, for a single person, with lower accuracy and quality, not because of the level of detail expected in the quality of the film. “
In other words, don’t expect James Cameron to close out his New Zealand mo-cap complex where he will be filming the sequels to “Avatar” in favor of a pair of consumer-level Vive fans. But, undeniably, technology has potential for storytellers to innovate who can’t afford Hollywood-grade special effects. “I think Vive tracking is a viable, low-cost alternative that can empower low-budget filmmakers to do great things,” Grossmann said. “They complement professional solutions and even have a place in professional settings in some contexts.”
A more realistic assessment of the cost structure for mocap narration with Vive followers suggests that the technology is a little less than 10 times cheaper than the industry standard. Full-body tracking with Vive trackers costs $ 2,100, while traditional external chapters cost $ 23,000. But the use of Vive fans may have wider ramifications for a film’s budget, as Ye points out, including the use of extras. “You can actually get a person to play all the extras in the movie,” he said. “Just record body movements one at a time.”
That takes a lot of work, but it opens up space for innovations for filmmakers who can’t afford massive crowds. Meanwhile, Vive’s tracking hardware continues to become increasingly complex with the release of its facial tracker this year. “As computing becomes cheaper, these technologies will end up becoming more and more accessible to more and more people,” Ye said.
Based on its current applications, it’s only a matter of time before the cheapest film productions lean toward the potential of Vive fans, but the technology is already here.
Additional reports by Bill Desowitz.