How Will Immersive Tech Transform Entertainment?

Media and Entertainment: The Show Must Go
On— Everywhere

An excerpt from The Augmented Workforce: How Artificial Intelligence, Augmented Reality, and 5G Will Impact Every Dollar You Make

Without a doubt, the media and entertainment industries have had an enormous influence on technology since the advent of television. Increasingly, they seem to be locked in a self-sustaining cycle of technology enabling new and better content and experiences, which requires better technology as it improves. This is not going away anytime soon. People in the media and entertainment industries should think about how to apply AI, XR, and 5G to the future of entertainment. Let’s talk about how these technologies are used in the media today.

AI for Media

We rely increasingly on recommendations with ever-increasing libraries of things to experience. Anyone who’s ever used YouTube or Netflix knows this. Whether you’re starting from the home page or finishing a final season, you face that choice of “what’s next?” Today, technologies not dissimilar from AI are making those recommendations based on what you’ve liked and other people like you have liked as well. In 2018 it was revealed that Netflix markets the same shows differently inside their app, changing show graphics based on users’ tastes. In the near future, this will extend to the influence of those shows’ content as well.

With the advent of more lifelike computer-generated characters, “deepfake” videos and lifelike synthetic voices, AI will customize narratives that can be “painted with data” to reflect what we’re more likely to enjoy. Whether it’s through your viewing history or facial expression, AI will tailor the stories you are told. Of course, our entertainment won’t be limited to linear media experiences like videos. AI enables us to speak to virtual characters through lifelike facial animation and voice assistant AI. Do you like Game of Thrones? You can spend an hour with a synthetic “Mother of Dragons.” Do you prefer sports? Ask famous athletes and coaches for their opinion on this weekend’s big game. Wherever there is repetitive labor, there is an opportunity for automation and AI.

Whether it’s through your viewing history or facial expression, AI will tailor the stories you are told.
Click To Tweet

XR for Media

The “family room” first appeared in the 1945 book, Tomorrow’s House, by George Nelson and Henry Wright. It was a place where American’s began to make room in their homes for their radio and eventual TV. For nearly eighty years, our connection to entertainment has been inching closer to a convergence between the cinema and a comfy couch. However, since the advent of mobile video and gaming, that trend has reversed, and quickly.

Today, we can have impactful media experiences everywhere, including immersive technologies like augmented and virtual reality. In popular titles like Star Wars: Vader Immortal, consumers use VR to live within the stories they love. The AR touchstone title Pokémon GO has shown how augmented reality brings stories into the real world. Creators use virtual reality production techniques to plan and “shoot” film in VR, as if they were on set with the animals. Examples include the live action remake of The Lion King and Jungle Book.

Most of season one of Disney’s popular streaming show Star Wars: The Mandalorian was scouted in VR on virtual sets shown on massive LED walls.

Filmmakers adapt these techniques for TV shows and even commercials. Broadcasters use a type of augmented reality to make more elaborate shots of talent on live-action stages. Sets, props, performances and more are captured in volumetric scans, allowing them to be played back from any angle. Increasingly, these technologies are being used beyond pre-production planning and into the productions themselves. Traditionalists expected that digital cameras would never replace traditional film, but many of them now shoot exclusively digital. Of course, this isn’t only for creating or consuming pre-produced content, it’s affecting live events too.

For nearly eighty years, our connection to entertainment has been inching closer to a convergence between the cinema and a comfy couch.
Click To Tweet

From 2016 to 2017, Cathy served as Chief Communications Officer for a cinematic VR studio, Future Lighthouse. During her time there, she worked with a very progressive and futuristic team that brought to life the first branded campaign ever to be nominated for a Tribeca X Awards, BeefeaterXO. The studio was ahead of its time. It brought three of the Venice Film Festival’s VR Selections to life: Melita, Snatch (in partnership with Sony Pictures), and Campfire Creepers (in partnership with Oculus and WME and featuring Robert Englund who played Freddy Krueger in the Friday the 13th horror franchise movies). Despite the amazing creative output from the studio, Future Lighthouse closed its doors in 2018. Its closure was very similar to many VR startups that have had a hard time raising funds. They were ahead of their time and many of the team has gone on to work with major VR studios and companies.

Flash forward to the summer of 2020. John and Cathy attended an award-winning play—a virtual performance of “Finding Pandora X” as part of The Venice Film Festival. Later that year, we joined revelers at electronic music legend Jean-Michel Jarre’s New Year’s Eve concert in a virtually reconstructed Notre Dame. These experiences are truly participatory, allowing the audience to travel to imagined worlds and shape events as they unfold. Customers will be expecting much more from live events, with examples like this, especially as an alternate option as the world meets face to face again.

XR Talks: Augmenting Hollywood

5G for Media

With more to see and do, we’ll need more bandwidth to experience it all. 5G already allows us to take higher-impact experiences to new places, whether it’s playing a multiplayer game from the highway, or watching a 4K movie while we’re camping in the wilderness. On the workforce side, there are many uses for 5G. Remote production crews can communicate more easily, previewing their work with stakeholders. Distributed teams can collaborate, guiding or shaping the work remotely. Complex computations like visual effects are done in the cloud, and in real time, allowing for companies to operate anywhere and perhaps even rent the expensive computer equipment required for this work. News crews capture ever more data for networks, allowing producers more options to tell the story, and viewers more ways to experience it, even in augmented reality where it’s blended over the real world.

“Virtual production” for film and TV has exploded in popularity due to social distancing and quarantine protocols. Using LED screens to represent a natural background, actors can perform in front of a screen. The television audience can’t tell whether the actors are on location or using a digital medium. A great example of this was Disney’s Star Wars series The Mandalorian, which used LED backgrounds to great effect. These stages are often located in places without high-speed Internet, so 5G can save the day and enable virtual production.

Who would have thought ten years ago that people could make a living live-streaming video games
Click To Tweet

The world of entertainment is changing. What constitutes entertainment today will be different tomorrow. Who would have thought ten years ago that people could make a living live-streaming video games, or play those games over the cloud on any device? Or that Tik Tok clips of just a few seconds would capture whole audiences and make people famous? The lines defining what is entertainment, gaming, music, and reality itself are merging. In the next ten years, games, subscriptions services, and other forms of digital entertainment will increasingly merge. The blurring of what constitutes a game, TV show, or movie will create new and exciting ways for people to express themselves.

Cathy Hackl is a leading tech futurist and globally recognized business leader specializing in augmented reality (AR), virtual reality (VR), & spatial computing. This article represents a chapter of Hackl and Buzzell’s Book, reprinted here with permission. See more or purchase the book here.

John Buzzell is an award-winning technology leader with experience spanning major technology transitions: AR/VR, Video Games, Mobile apps, Websites and more. This article represents a chapter of Hackl and Buzzell’s Book, reprinted here with permission. See more or purchase the book here.