AI Film Spotlight: “In the Robot Skies”

Original article was published on Artificial Intelligence on Medium

AI Film Spotlight: “In the Robot Skies”

In this series, I select and discuss AI projects that interact with cinema and filmmaking practice. These case studies range from narrative films through to experimental artworks that utilise AI technology as a creative tool.

“It is not our intention to make human film crews completely unnecessary.” — Professor Toon Goedemé

In 2016 the research group Embedded and Artificially Intelligent Vision Engineering (EAVISE) partnered with British filmmaker Liam Young and Channel 4’s Random Acts series to produce In the Robot Skies is the world’s first narrative shot entirely through autonomous drones. ITRS features the use of three AI-powered drones that have been trained to follow basic rules of cinematic composition (rules of thirds and framing techniques) whilst in-flight. Although drones are often used to capture footage, ITRS has evolved this process to create “specially developed camera drones each programmed with their own cinematic rules and behaviours” that perform without the need for pilots or user instruction.

The film is described as a story of “two teenagers each held by police order within the digital confines of their own council estate tower block in London. In this near-future city, drones form both agents of state surveillance but also become co-opted as the aerial vehicles through which two teens fall in love.”

Shot through the point-of-view of these drones, ITRS follows this story whilst exploring the concept of the “drone” as both cultural object and filmmaking device. The “drone” in this context is both a symbol of oppression and liberation, an object of totalitarian surveillance and youthful rebellion. Set on location in South London the film uses the urban environment to construct a dystopian backdrop in which the drone prowls the endless balconies and towers. In seeing through this POV the drone’s user interface becomes the primary source of contextual information as resident and technical data is displayed on-screen. This style of data-visualisation is synonymous with the representation of AI or technically advanced vision, seen throughout the history of cinema in classics such as The Terminator’s “termovision” HUD, or Robocop’s various vision models. ITRS embodies this aesthetic as a formal device to humanise and characterise these drones in order to distinguish the lover’s “good” drones from the spying police drones. Familiar emoji avatars and colourful typefaces contrast authoritative warnings as the film cuts between the perspectives of the various drones. In seeing drones as characters Young is able to construct a distinct “visual language” that combines previous visualisations of AI vision with its dystopian future aesthetic in order to represent these drones as autonomous agents

Screenshot: The Drone UI and Shot Framing Demonstration

The drone’s ability to transport and communicate messages and sentiments whilst generating its own speech seen in the instant-messaging style on-screen pop-ups visualise the scope and limitations of its own autonomy. Despite their abilities, the friendly drones are ultimately acting in accordance with the directions of the user. They may translate and deliver content autonomously but the motivation and intent behind such activity ultimately come from an external source. The same distinction can be made to address the AI-drone technology used in production as despite capturing this footage with little supervision (for the safety of the actors) these drones are similarity at the command of their creators and the parameters they set. Although an important point, this distinction does not strip ITRS of its achievement in successfully deploying these agents. Debates of “autonomy” within AI theory circles and amongst legislators will continue perpetually so it is important to recognise the magnitude of such achievements and admire the quality of the final product.

ITRS achieves a unique synergy, the drone is an actor in both a narrative and practical sense as it facilitates the logistical requirements needed in order to capture such footage whilst playing an essential part in the plot. The result of this meta-approach is a film that utilises and leverages AI beyond the realm of experimentation and curiosity. It showcases a meticulous collaborative process which has produced a rich engaging short that harnesses the creative and practical potential of AI and drone technology.

ITRS exists within a wider EAVISE project labelled “Cametron” which combines the AI-drone technology seen in ITRS with other methods in order to work towards a “virtual director”. This process involves cutting edge technology (including the use of quadrocopter drones) to equip this system with key characteristics which include “semantic awareness, cinematographic compliance, and (pro-)active processing”. The system is created from a combination of semantic observations gained from multi-modal scene analysis and pro-active continuous visual analysis from fitted cameras and microphones. Cametron is driven by a carefully crafted algorithm that aims at a system that can “operate in real-time, with high autonomy of the cam/mic units combined with a powerful, supervising, virtual director”.

ITRS offers a promising insight into the future of AI-drone technology, demonstrating the creative and practical use of deploying AI agents during filmmaking processes. The novelty of using AI within film production is potent given the lack of existing crossover and given the quality of early AI-driven films it is exciting to look towards the potential of a new AI-driven wave of film production and human-machine creativity.