AI Gets Creative

Artificial Intelligence (AI) is spreading its tentacles across the globe. Put simply, AI systems are computer programs that sift through large volumes of data to produce sophisticated conclusions. Its cousins, Machine Learning (ML) and Deep Learning (DL), use special programs called neural nodes to look for and remember patterns in the data, auto-correcting to find a problem’s solution.

Applications of AI range from speech-to-text translation to financial trading, GO playing to Spotify or Netflix recommendations. Amazon, the world’s third most valuable company, has built its empire around AI, using it to guide robots in its warehouses and to underpin its home assistants. Google’s boss, Sundar Pichai, has said AI will have a “more profound” impact on society than electricity or fire.

Traditionally, AI adoption has been restricted to the tech or financial sectors: where vast quantities of data drive decisions, and AI systems can easily slot in. Although this is changing. Whole industries are on the verge of disruption through the development of autonomous vehicles or customer service bots powered by AI. In the last quarter of 2017, public companies mentioned AI and ML in earnings reports more than 700 times — seven times as often as in 2015 — as they scramble to get a piece of the action.

A human resource that has always been considered outside the realm of AI is creativity. Always one step removed from the robotic functionality of computer programs, the innate human ability to create and produce original material has always been considered unique. An inimitable attribute. However, as AI systems consume more data, they will start mimicking and surpassing human endeavours in fields from product design to speech writing, shattering any preconceptions about what AI can achieve.

This week at MTM, we look at how AI is getting creative — operating in a world developed by itself.

Telling the story

On the surface, Sunspring is like any other film entered into Sci-Fi-London, a film festival dedicated to science fiction and fantasy genres. It tells the story of three individuals in a futuristic dystopia entangled by love and murder. Our protagonist opens by warning of a future mired in unemployment, where young people have to sell blood to survive; a chilling vision from writer Jetson.

This thought may have been penned by Arthur C. Clarke or J.G. Ballard. In fact, it might have been directly plagiarised from them. Jetson is not a fledgling writer, but an AI system with the aptly cybernetic title, LSTM Recurrent Neural Network. It was fed tens of thousands of science fiction screenplays and stories and, after sifting through these large volumes of works and applying Natural Language Processing (NLP) (technology that detects and manipulates vocal and speech patterns in meaningful ways), Jetson was programmed to produce its own story. Sunspring is the result.

And it’s not the only one. A company called Greenlight Essentials is running a Kickstarter campaign to finance Impossible Things, a horror film co-written with AI software, with the ambitious goal of becoming the first feature length movie created by NLP. Like Jetson, the AI system will analyse existing plot structures, before cross referencing these with box office success, to determine, and ultimately attempt to produce, a successful horror movie.

It is not just writing where AI is pushing through. At the 2017 Cannes Festival of Creativity, advertising agency Saatchi & Saatchi created a video directed by AI technologies. This included decisions made by AI on specific cuts, the rendering of sound, and even the editing of emotional scenes. Whilst these projects are ambitious, the execution often falls short as AI systems still cannot replicate the nuance of human speech (Sunspring is decidedly difficult to follow). Although, as aids to human creation, AI systems are developing quickly.

Shaping the narrative

In the development of storytelling, McKinsey and the Massachusetts Institute of Technology have been asking whether AI systems could collaborate with writers to improve their stories. Through analysing Disney Pixar’s Up and its emotional arc (the shifts in tension and emotion that shape the narrative), the researchers developed the metric of visual valence — the extent to which an image/scene elicits positive or negative emotions.

The AI system was then able to cross reference the valence of a story with social media engagement — in this case the number of comments generated. The findings demonstrated, perhaps unsurprisingly, that positive stories elicit higher audience engagement. Though not producing a story independently, the researchers noted AI could play a supporting role in video creation, for example through identifying a particularly emotional scene and enhancing it with a comparable piece of music.

Disney is developing its own AI systems to predict narrative quality. By collecting large volumes of crowd sourced responses on Quora, developing neural networks modelled off the various ‘Up Votes’ and using these as quality indicators, the team have developed an metric for narrative value. These neural networks were then developed to evaluate individual parts of the story to assess if different segments could work well together.

Though nascent, this is the beginning of AI systems providing judgment on seemingly subjective questions — such as, what makes a good story? Similar to the emotional valence research, this probably cannot be let loose independently immediately, however there is clearly a lot of potential.

Exploring the archive

Applying AI systems in content management is an area developing quickly, particularly in sport. IBM, the official partner of Wimbledon since 1990, created an extensive 2017 highlights reel of the championships. Developed by its flagship AI program, Watson, the “Cognitive Highlights” fuse metadata and multimedia, identifying key points through the on-court statistician that provides data on things such as the speed of the ball, number of aces, saved breakpoints and length of rally.

The system then continues to capture exciting moments using a combination of audio and video AI tools, including using the roar of a crowd to determine pivotal points, and sequential player movement to capture unique or erratic behaviour. These moments are then ranked to provide a data driven highlights reel — saving an editor hours of time.

The value of this technology can be applied beyond sport. Media and entertainment companies have amassed huge archives of hundreds of thousands of hours of video footage that is currently unstructured and unused. Formatting this information in an easily searchable database, through AI-assisted content tagging, would produce a rich library of video content; a 21st Century Tower of Babel.

Not enough budget to shoot a sunset in the Himalayas? A monetised archive could solve your problems. Need a bustling New York City shot to frame a scene? Through exploring an extensive library, this could be found quickly. Broadcasters sitting on decades of archive will be given new leases, opening avenues of revenue thought locked in time. Young filmmakers on low budgets could immediately expand the scope of a project through creatively handling resources at their disposal.

A word of caution

AI is exciting. But it needs quality data to operate smoothly. These examples are happening on a small scale, often in research labs. Its scalability is yet to be proven. Undoubtedly though, broadcasters and media companies that invest early will reap the benefits of this ever-growing beast, positioning themselves alongside the progression of creativity.

If you are interested in talking about AI, please do not hesitate to get in touch.

Source: Deep Learning on Medium