Currently, the annual PBS Short Film Festival runs on a virtual screen very close to you. Every day until August 31, 2021, the audience can see a series of stories selected by independent filmmakers through an immersive WebXR beta experience called Screen on the Green.
It includes 25 short films that are played back and are accessible via compatible VR headsets, including Oculus Quest or via a web browser. Up to 300 participants can access the experience at once and choose between one of two environments, a daytime setting overlooking the cityscape or an outdoor landscape in the moonlight.
In the center of each space is a large-scale outdoor cinema screen, where the 25 films are played consecutively. When participants come together, they can see the space filled with avatars and explore hidden Easter eggs, such as movie posters or ways to support the nonprofit American public broadcaster.
In previous years, the festival has held face-to-face screenings in the Washington, DC area, but pandemic restrictions made organizers consider it an alternative virtual reality experience.
To create it, the PBS digital innovation team leveraged a wealth of Amazon Web Services (AWS) technology, including AWS Media Services, Amazon Simple Storage Service (Amazon S3), and Amazon CloudFront.
“Screen on the Green celebrates stories that are often underrepresented in making films in a unique format that allows participants to experience those exciting and remarkable moments that filmmakers intended together,” says Mikey Centrella, director of product management, PBS digital innovation team.
“Cinema is a difficult art form to recreate in a linear format. Instead of broadcasting or transmitting the films through a social platform, we wanted to transport the public to the cinema ”.
“The pandemic provided us with a unique opportunity to try out a VR format for the festival, especially because remote work has helped the audience feel more comfortable with immersing themselves in a video experience and VR technology has improved,” he adds. .
Centrella is fully aware that not everyone can see the “screen” with VR hardware.
“Access to VR headsets can be a financial barrier. So we built an experience that could be available to anyone with an Internet connection and a computer, using WebGL and WebXR, ”he says.
“We are trying to introduce viewers to a new way of viewing PBS content in an immersive, virtual environment across the web and soliciting feedback to report on future releases or new VR / AR products. This will help us measure our appetite for view PBS content on XR and, if the community is interested, the team will be better prepared to evaluate future PBS digital products and technologies in this space. “
That said, the concept of virtual reality allows us to replicate the feel of a film festival in the physical world.
“You’ll be amazed at how far a hand wave or body tilt can go when communicating feelings in VR,” Centrella says.
“This experience is intentionally about watching movies together pretty much like in a movie theater, where complete strangers or best friends can be together to experience together.”
While this should not be seen as a phenomenon similar to the phenomenon of the “surveillance party” that has taken the storm into the world of sport, the concept could lead to greater public involvement. “Just like in a real movie theater, you can’t or shouldn’t really talk when the film is screened, so for now, participants can’t talk,” Centrella says.
“If the audience finds value in viewing PBS content in VR, the digital team may consider adding new features such as feedback, surveys, audio chat, or having a live host as a filmmaker at a later date or for a product future “.
Building the screen
The PBS Innovation team chose AWS Cloud Services to support the effort. “[We wanted to] making sure we can design a virtual environment to stream the PBS Short Film Festival that would be remarkable and stable, ”says Centrella.
“The team was already using AWS Cloud Services for their own sandbox to help us experiment, feed and deploy prototypes in artificial intelligence, voice assistants, machine learning, virtual reality, augmented reality, streaming direct and a recommendation engine, ”he adds.
“But recently, new services like Media Tailor and Channel Assembly have been made available and are suitable exclusively for the use case of the Film Festival VR Experience.”
“We wanted participants to have a sense of community and spatial awareness of others by their side,” Mikey Centrella, PBS
According to Dave Levy, vice president of the U.S. government, nonprofit and healthcare, the cloud giant worked closely with the PBS digital innovation team from creation to the final installment of its virtual reality festival . that ”.
“Early in the development of the project, PBS and AWS recognized that recreating the theatrical experience in person meant that PBS would need a way to continuously stream festival videos to the virtual world to hundreds of users in a synchronized manner.” , Levy continues.
“With an AWS cloud infrastructure based on services like Amazon S3 already established, the PBS Innovation team envisioned a file-based channel transcoding and editing solution that it used AWS Elemental MediaConvert i AWS Elemental MediaTailor“.
All of the green screen video content is in PBS’s proprietary Media Manager solution, which distributes every short MP4 file to Amazon S3 and Amazon CloudFront.
To create the movie festival stream, the MediaConvert file-based video transcoding service is used to transform MP4 files into HLS playlists. Channel Assembly within MediaTailor uses HLS playlists to create a linear stream ready for the distribution of the experience as a single output.
The stream is carried to the video player that supports the WebXR experience, which lives on a web page hosted from an Amazon S3 repository.
“Without MediaConvert and Channel Assembly at MediaTailor, converting short films into a linear channel in HLS format to make it possible would have been shown to require a lot of labor and is prohibitive in its costs,” says Levy.
“AWS MediaTailor technology [completely] it allowed us to see a linear channel deployed quite quickly that can reach hundreds of viewers potentially, ”Centrella agrees.
“Our Media Manager solution was built at home and is part of PBS’s broader service image; we used it to feed media to MediaTailor to create plays and provide metadata so that users can enjoy additional context about movies during their viewing experience.
“It’s also worth noting that the client, the part that users experience in their browser, is a component in itself made up of many smaller pieces that had to be built.”
The team also made sure that each time a participant logged in, they would experience the same moment together without delay.
”Amazon elastic container service (ECS) i Amazon ElastiCache for Redis allowed the team to expand its backend infrastructure and find the desired processing power and performance needed to deliver real-time viewer avatar stances, allowing for a more authentic experience as users navigate the world and see others. avatars in space, ”says Levy.
“ElastiCache was perfectly adapted to the goal of keeping hundreds of users in sync at once,” Centrella says.
“We wanted participants to have a sense of community and spatial awareness of others next to them. So we needed a way to collect and broadcast viewers’ positions in real time in a scalable way, but with performance. move to a configuration that would extend the calculations across multiple instances.
”During development and upload testing, we tracked the server’s ability to keep up to date with the 33 ms we expected it to manage in order to spread 30 updates per second to all users. As we added more users during upload testing, we monitored the cadence and identified that it was able to constantly meet this 33 ms tempo up to approximately 600 users / connections. To be safe, we set a limit of 500 users, so the experience will always remain stable. “
AWS provided support and guidance to PBS throughout the process, but there were challenges. According to Centrella, one issue was “finding the right combination of AWS services and leveraging them to provide a reasonably low latency real-time experience while we had a good idea of the traffic we would get.”
“Testing new emerging technologies doesn’t have to be complicated or costly prohibitive,” said Dave Levy, AWS
“Proper configuration of AWS services to be functional and provide the scale we needed probably took longer than the actual creation of the code,” he adds. “The AWS side of the experience was not particularly heavy. By context, the Node.js file [web app] server code is less than 100 lines “.
Other challenges were the components of virtual reality, such as controlling aspects of the media viewing experience, debugging in the context of virtual reality, as well as “developing for desktop and desktop users. virtual reality and try to reach a user experience that adapts to two very different ways of interacting in a 3D world ”, adds Centrella.
But the reaction has been positive.
“We’ve been asking participants to leave comments about their departure,” Centrella says. “75% of respondents said they would use this experience if it were available all the time to watch PBS content, not just movies. Other comments are: I love shorts and various avatars! ‘”
“Internally, the AWS team is thrilled with the result and we hope the PBS audience is satisfied as well,” Levy says. “The collaboration is representative of our ongoing commitment to helping M&E customers around the world innovate to transform the entertainment experience.”
“As PBS demonstrated with this project, testing new emerging technologies does not have to be complicated or costly prohibitive, ”he adds.
“Even with a small team of technology developers, the scalability and flexibility of the cloud provide a solid foundation for experimentation and innovation that can ultimately help drive the development of audience experiences. more attractive. “
Certainly, the PBS team continues to focus on creating and quickly testing ideas. “While PBS may not yet have a virtual reality channel, we will not override the idea of the future and much of the work we are doing with AWS will provide the foundation for future innovation in this field,” says Centrella .
And what advice would PBS give to similar organizations looking to try something like this?
“Start by determining the essence of the experience. Who is it for? What is the only thing you want them to do? says Centrella. “Then create a set of functions from here. But keep the list small. Make sure each function is connected in some way to the main essence and, if it doesn’t, col. · Place it on the back.
“The sky is the limit in VR, but that can be a problem. Being unlimited can often result in a product that is not delivered to the end user and has functions that just don’t make sense. ”