Collective Vision of Synthetic Reality

Original article was published on Artificial Intelligence on Medium


In the following paragraphs, I would like to outline the structure of the workshop and share the outcomes from the past two workshops.

At the beginning, the participants receive a short introduction to the current state of AI-powered media synthesis and were given the set of brainstorming cards. The current version of the card deck is divided into 7 categories of most accessible AI models for image synthesis, video synthesis, audio synthesis, text-based models, models for image recognition, post-processing, and models useful in the media synthesis workflow. Some time is then given to the participants to browse the cards and acknowledge every AI model’s potential. The cards are specially designed to be easy to distinguish between different categories and understand the potential of each model within several seconds. Two brainstorming sessions follow after this initial phase of the workshop. Each group (equipped with their own card deck) has 10 minutes to speculate on the potential future use-cases of synthetic media and “prototype” the specific synthetic media example using the cards. The crafted scenario and a precise recipe for its production (which AI models were used, in which order, etc.) are documented on a workshop sheet, either by drawing or writing. We use matching color stickers for marking the categories of used AI models for easier further reference.

Speculative scenarios as workshop outcomes

In one workshop participants imagined a future where people will be able to visualize their memories. This could be beneficial in psychotherapy, but also in entertainment and advertising. The imagined workflow uses image and text generation based on a text input, combined with additional data input, voice synthesis, and talking-heads generation. Visualizing distant memories, which we no longer can imagine in our minds, is re-occurring during brainstorming sessions. No matter how stretched the actual process is from what’s possible, this speculation inspires us to think about the positive use of synthetic media synthesis with significant scale impact.

The world around us exists as far as there is someone to perceive it. Does this also apply to neural networks? Another re-occurring theme is artificial seeing and understanding of the outside world, which, once recognized, can be digitally translated, for example, to a blind person. This is often imagined as possible with the combination of depth estimation, semantic map creation, and translation to image synthesis, for instance, with GauGAN. The question of whether we could enhance a limited vision with AI-driven visual synthesis leads to more philosophical discussions about the relevance of sight in the future as such. What does it mean to see? Do we all see the same visual representation of the world, and is this representation changing along with the cultural and technological transformations? These speculations sparked an inspiration to prototype a device from a dystopian future that would deliver uniformed visual perception to every person’s brain and thus replace the natural sight, which would be banned. It’s not the same as using a camera in an eye implant or Google glasses because the camera image could still be perceived (and processed) in a subjective way. This idea goes a step further, making sure the visual processing is fully substituted with controlled media synthesis directly inside the brain. Once our technological tools can completely replace human senses, we might one day wake up in the future of standardized perception and lose the beauty of crafting a personal inner world.

How do you imagine synthetic reality?

During these workshops (and discussions with many professionals from diverse backgrounds), I realized how vital it is to be aware of what’s (already) possible and what will be possible very soon when it comes to generating synthetic media, no matter what profession or technical skills. Although not everyone is going to actively participate in experimenting with generative AI models, having some idea of what kinds of ML/AI models there are and what they can do makes it possible to imagine possible scenarios — and start envisioning them applied in our daily reality. AI technology can feel distant to people, who are not directly involved in AI research. But in the end, it’s all of us who will be (or already are) interacting with synthetic media in daily visual communication.

Most people go through the initial stage of shock from visual uncertainty synthetic media bring (“how are we going to know what’s real and what’s not?”) to the realization that this has already been happening on a massive scale long before deepfakes (remember Photoshop?). Creators usually go from the fear of losing their job and being replaced by AI algorithms to the realization of substantial creative potential methods of synthetic media tools bring. These realizations take time and require adjustments in thinking patterns. Shifting the perspective from a passive spectator towards an active player is the game-changer we need.

I have made my cards available for free as digital download, so now you can get acquainted with currently the most popular AI models and start brainstorming on your own. After reading the cards, try to imagine what could be done with each model or even a combination of different models. Free yourself from the constraints of what’s possible and allow yourself a far stretch in designing synthetic media scenarios. It does not matter how realistic the plot is — conceptualizing the given information from the cards already stretches your thinking and makes you ready for a critical discussion about the future of synthetic media. If you feel like sharing your ideas, join our survey, and contribute with your scenario to the Collective Vision of Synthetic Reality!

Meanwhile, I will continue collecting the responses and evaluate the data to drive more complex conclusion.

Let me know if you’d like to arrange an individual session with me, or facilitate a workshop for your class or team (online version also possible). In case you’re an educator yourself, you can order the physical card decks and use them in your own sessions.