Original article was published on Artificial Intelligence on Medium
Artificial Intelligence Goes to Hollywood (and Infiltrates the Movie Industry)
More and more machine learning applications find their way into various aspects of movie production and the acting trade
Setting the scene
Let us now venture into the land where big budgets and creativity meet: Hollywood, Tinseltown, the magical movie land.
There, increasing budgets are closely trailed by the increased use of more and more refined CGI. Movie production decisions do not only depend on budget, but also on projected profit. Those projections are based on viewer data. Budget allocation? Data. CGI? Data.
I can already hear AI rubbing its virtual hands.
A-list movie production is not a cheap investment. Modern blockbuster movies regularly top 200 million dollars. If you’re someone from a production company about to make (part of) such an investment, you’ll want to make as sure as possible that the movie is going to be a hit. No one invests without the expectation of profit.
How do you assess a movie’s potential for profit? Look to the past. Which (type of) movies did well? Are there characteristics that blockbusters share? What are viewers looking for, and what are they willing to spend money on?
The fear to lose money and deviate from known success formulas has led to the current ‘age of the sequel’.
But, as we all know, that’s no guarantee for success. Some sequels are great, others not so much.
What if we had a system that included as many relevant parameters as possible into the ‘blockbuster potential’ assessment, not — or at least less — hindered by human risk aversion?
Guess what? Big production companies are already doing that. 20th Century Fox uses a system called Merlin (which predicted the success of Logan), Warner Bros. recently began collaborating with Cinelytic, a company using machine learning for predicting movie success. Belgium-based company ScriptBook can predict a movie’s box office return with 86% success rate (and has potentially already co-written scripts for movies — we don’t know which ones due to non-disclosure agreements…).
No one invests without the expectation of profit.
Let’s only hope that the increasing use of these AI/machine learning systems will also lead to supporting hidden gems that would have been ignored otherwise. If the studios provide more data than just past successes, can these systems anticipate moviegoers’ sequel fatigue?
CGI, immortality and deep fakes
We have all heard about the magic of the green screen. Actors and actresses do their thing in front of a green screen, VXF artists sprinkle on some CGI magic et voilà, our heroes and villains are fighting with laser guns on top of a spaceship. (Bonus factoid: the screen is green because it was originally used by weathermen, who — back in the day — tended to wear blue suits. What color provides great contrast to that? Right, bright green.)
But separating the real actors/actresses from the virtual background is not always an easy task.
In fact, the best, smoothest results require an almost pixel by pixel assignment of pixels to actor/actress, foreground item or background. (And there even are pesky pixels that comprise a little bit of everything.)
MIT researchers recently presented a system called semantic soft segmentation that:
…analyzes the original image’s texture and color and combines it with information gleaned by a neural network about what the objects within the image actually are.
Long story short, the system expedities the process significantly and even though it’s currently working with static images, there’s little doubt that the movie industry is following its development closely.
What about the actors/actresses themselves?
One thing AI/ machine learning can already do is ‘tweak’ the actors/actresses. A well-known recent example is the Irishmen, the movie in which Robert De Niro, Joe Pesci, and Al Pacino were ‘de-aged’ with a combination of machine learning and innovative motion-capture techniques. No long hours in the make-up room required.
Another example is Thanos in Avengers: Endgame. A new machine learning system that goes by the apt name Masquerade painted Josh Brolin’s expression onto a high-resolution rendering of Thanos’ face, saving VFX artists many hours of painstaking work.
It’s not about the AI, it’s about how we develop and use it.
Those are actual actors, though. How about actual AI acting systems?
Well, we’re not there yet. But considering the current scarily good deepfakes and the growing ability of machine learning systems to make what almost seem like creative leaps suddenly makes a real life S1m0ne look less implausible.
Will this put actors/actresses out of business? I hope — and think — not. As with other AI artistic endeavors, the future could be hybrid. Semi-independent AI actors could give human actors/actresses a ‘partner’ to act with in front of a green screen, or perhaps even to improvise with.
A possible problem is that the data used to develop Hollywood AI might reflect or even reinforce existing biases. On the other hand, careful and judicious use of machine learning could help to identify said biases. It’s not about the AI, it’s about how we develop and use it. The use of AI in Hollywood should be accompanied by a diverse and inclusive cast of characters.