Original article was published on Artificial Intelligence on Medium
Level Two: Dynamic Content
With the data fidelity needed to support frequent and reliable feedback loops, the next step is to optimize by making content changes, in real time. By combining historical knowledge of successful narratives with current audience reaction, narratives will change in real time to optimize for user outcomes. This difference is a fundamental paradigm shift, diminishing a foundational part of storytelling of the past — the role of the storyteller in crafting the narrative. Instead, the stories of the future will overwhelmingly focus on the consumer of the story and be delivered by artificial intelligence. It will determine what happens next virtually as it becomes visible on the screen. This means that while we are excited and on the edge of our seats to find out what happens next, so is the narrative!
Dynamically changing content through traditional methods is not feasible as filmmakers simply cannot produce enough content to keep up with the provided feedback loops. To fully take advantage of the engagement data, content must be produced quickly enough to keep up with the feedback loops. Thus, artificial intelligence technology, as described above, would need to advance considerably to continue the narrative and update the visual and auditory components accordingly.
This does not necessarily mean all TV shows and movies of the future will be animated. Some might, but photorealism is possible too. Initially, content will be created using photo & film references to develop augmented frames, where individual pixels are exactly as they are in real life. Today, Unity is able to take film references, model the scene, and augment the footage, producing video indistinguishable from reality.
For people, Soul Machines is using reference film and photos to create standalone artificial humans. Among their pursuits, Soul Machines is developing a virtual version of Black Eyed Peas star will.i.am. There could even be a future where we could conjure up characters and actors of the past, as Unity CEO John Riccitiello suggests. As these technologies advance we could develop completely new, photorealistic frames of humans in real time.
While this technology is inspiring, we are a far ways away from developing content that dynamically responds to feedback loops. For reference, it takes Pixar about 24 hours to produce one frame. Using supercomputers, it took two years just to render Monsters University! (For an otherwise fascinating look into animation and content production, check out how Pixar has evolved its capabilities over the past 24 years and the end-to-end process for animating Toy Story 4).
Producing new content on the spot, without sacrificing quality, might prove challenging. Just as difficult might be managing technology’s ability to use its in depth understanding of people to influence what they view. We are already seeing today that faster feedback loops and engagement optimization can unfortunately start to develop echo chambers, confirming political and social beliefs, and increase political strain as sides more deeply hold onto their beliefs. Therefore, this must be managed properly or dynamic content will be met with strong resistance.
Level Three: Immersion
Beyond just developing narratives that look to optimize for the viewer, the audience will also have the ability to intentionally influence the narrative. Before, they had no control over what they view, but in Level Three, the viewer takes control.
To help understand the impact of being an observer versus a participant, consider a key component driving the success of Disney’s theme parks. Disney World is about YOU, not Disney. As Matthew Ball writes in Digital Theme Park Platforms: The Most Important Media Businesses of the Future:
“Disneyland is an experience involving many moving parts in harmony, like an orchestra. Everything has to be tuned, what you hear, what you smell, what you see,” how you see it, the speed at which you assimilate all of that, just like a film, is choreographed.”
While an obvious difference between a film and theme park is that you are physically in the theme park, the key difference maker is that:
“[Disney doesn’t] control the camera, because the camera is you — it’s you when you come to Disneyland”.
In both films and theme parks, the characters and the scene are meticulously choreographed and perfected, but in films, the storyteller keeps control of the camera, and thus the narrative and potential experience. However, when you are the camera, you can optimize for the experience that is best for you. In its parks, Disney provides all of the important infrastructure for its visitors, but then the visitors can see what they want, allocate their time how they want, and immerse themselves as deeply as they want. TV shows and films of the future will offer what Disney does in its parks.
Optional story lines
Translating this to digital media content, instead of just being an observer of the dynamically changing world, the viewer will have direct influence over what content they are consuming. In a rudimentary way, media companies are already realizing the value of this and are acting on it by providing optional story lines for the user to dive into. For example, Star Wars has its main storyline in Episodes I-IX, but it also has a few side plots — Disney has produced the films Rogue One: A Star Wars Story, Solo: A Star Wars Story and the TV series Star Wars: The Clone Wars and The Mandalorian to name a few. They have also announced three new series to launch in the coming years. Fans can keep up with it only if interested, focusing their time and energy where they like.
A seamless choose your own adventure
In the future, the user will be able to materially impact storylines. Viewers will transform from passive observers to impactful agents.
In doing so, choose your own adventure books will effectively be brought to the digital world. Though premature, this exists today too. Netflix released, through the Black Mirror series, Bandersnatch, a story with decision points that produce different outcomes in the story.
Though certainly novel, the movie did not grip the media world as the new format for storytelling because it is still lacks full empowerment of the viewer as many decisions were trivial. There were instances where decisions would yield a “wrong choice” response, show the same video, regardless of which option you chose, and impact something relatively meaningless like the soundtrack. This prompts the question — were these really impactful decision points? To engage viewers, it must feel like they have meaningful influence on the storyline.
The main limitation here is content production — Bandersnatch needed to have a limited number of true decision points because they could only produce content for so many scenarios. Producing for more scenarios would have linearly increased their budget, rendering it unfeasibly large. Developing artificial intelligence technologies to produce video in an instant, as discussed in Level Two, will lower the marginal production cost of new content. The number of choices could become limitless. You can even make a decision whenever you want, rather than just when prompted, like in Bandersnatch.
Though extremely rudimentary and limited, technologies representative of a similar idea are being built. Talk to Transformer, based on the Open AI GTM-2 algorithm, shows how a modern neural network can create text. It can recognize inputs from news articles, stories, song lyrics, poems, recipes, code, and HTML — for example, it can know the characters from stories like Harry Potter and The Lord of the Rings. Using the same base algorithm, AI Dungeon is a text generation program that can take a user action and further the story line in a coherent way (though, it certainly has its limitations).
As these technologies begin to realize their full potential, imagine not rooting for a character in Game of Thrones to end up on the Iron Throne, but actually helping one of the characters rule the seven kingdoms. Better yet, take them all down and end up on the throne yourself! For those who want it, this can increase story engagement immensely. This has the opportunity to truly make media a first party experience — one where the viewers are as much a part of the story as they want to be.
This should not be confused with getting everything that you want — that is boring! Being challenged and surprised yields a much more rewarding and entertaining experience. Because of that, viewers will only be able to influence narratives, not dictate them.