Exploring Gender through the Lens of Computer Vision with Artist J.Rosenbaum

Original article was published by Beth Jochim on Artificial Intelligence on Medium

Exploring Gender through the Lens of Computer Vision with Artist J. Rosenbaum

Explorations of AI Art — Episode 02

[This interview has been previously published on Cueva Gallery’s blog on July 8, 2019]

“Many people feel that there is a ‘realness’ to painting that isn’t present in digital work and that is one of the things I hope to subvert.” — J. Rosenbaum

[Fig.1] Kinnari 3D Render: Kinnari use their wings to move between the mystical planes and the earthly. Credit: J.Rosenbaum.

Libre AI had the opportunity to organize another interesting discussion about generative art. This time we reached out to American artist J. Rosenbaum.

Rosenbaum is based in Victoria, Australia, and is a contemporary figurative artist working in 3D modeling, Augmented Reality (AR) and Artificial Intelligence (AI), investigating the intersection of technology and art.

An important part of the work of this artist is the exploration of the nature of Non Binary transness and their own genders and sexuality. This research is continuing in 2019 into computer perceptions of gender with a PhD at RMIT University in Melbourne.

Rosenbaum was very kind and accepted to have a chat with me, which I share in this piece.

Beth J.: You are a contemporary figurative artist. When did you start to incorporate AI (and other computer-based techniques) in your work and — above all — why?

J.R.: I first started with machine learning in art during my masters in contemporary art. I used 3D modeling to plan my paintings and provide models to paint from and became interested in Augmented Reality and the ways I could bring art to life and other ways to make art with my computer.

I saw a viral post about Style Transfer and while I was aware of other forms of machine learning such as DeepDream I couldn’t see a use for it in my work. But Style Transfer clicked for me. My work has archaeological influences and I really liked the idea of seeing my work translated into the archaeological style of my inspiration. It grounded the works in history and used Augmented Reality to transform them in the phone.

I was always very conscious of the meaning behind my work, so for me using Augmented Reality was a glimpse into the world of my 3D renders, and the machine learning interactions were like the artificial thought processes of my artificial people. But the meaning is about the complexity of internal monologue of your gender not always matching your exterior and how we are always so much more than we appear on the surface. The title of the app for this work was named “recursion” and called back to the programmatic nature of recursion but also to the concept of recursion of gender. We define our gender for ourselves and in so doing contribute to the known notions of gender.

Beth J.: You also paint nudes with your “signature ochre technique using layered washes and strategic removal of paint to create a wonderful three dimensional look and a strong emotional feel” . These paintings carry a deep conceptual meaning. They convey ideas. What is the difference between those and an artwork that you create with a machine?

J.R.: Honestly, I don’t paint as much as I used to! I was hitting a wall with my painting and didn’t feel like I had as far to go. There is a delicious meditative state while doing a painting and sometimes I miss the materiality of it. One day I suspect I will take up the brush again and bring the two together in a collaborative way, but for now I feel that my machine generated works have greater depth and potential waiting for me.

Many people feel that there is a “realness” to painting that isn’t present in digital work and that is one of the things I hope to subvert, that belief that painting is somehow more because it involves manual processes.

The meditative state of painting has been replaced by the meditative state of curation of datasets and results, of drawing into the works and working with the computer. I often draw in response to the machine generated artworks and find that I am doing that on a digital surface. It would be just as easy to print it and work into it manually but I don’t, the digital drawing, for me, is just as satisfying and has less cleanup!

I do find I have learned a lot from my painting, from composition and classic artistic knowledge and understanding to a sense of the material and the hidden depths behind the generated works. It is said you need to have an understanding of formal art techniques before you learn abstraction and I find that my knowledge of art creation and history has helped ground my digital practice.

[Fig.2] Hidden Worlds uses Artificial Intelligence and Augmented Reality to examine gender through the lens of computer vision. Courtesy of J.Rosenbaum.

Beth J.: The project “Hidden Worlds” uses AI and Augmented Reality to explore gender through the eyes of a computer. But the project is much more than this. Can you tell us about it? Do you think that AI and machine learning applied to art could help us go beyond the concepts of human and gender? Could we be free from bias?

J.R.: This is the main subject of my research going into my Phd! I believe that AI is a blank slate that we can influence very easily. It picks up our bias and amplifies them in most cases.

My “Hidden Worlds” works explore gender through the lens of computer vision by training a GAN with a mixed dataset of Greek and Roman statuary. The dataset was constructed by me as these sculptures were made to show men and women as perfect godlike beings, I wanted to see if the GAN would blend gender characteristics and what that would look like. I wanted to create godlike representations for those of us who never got to see themselves realized as gods. I was delighted to find that it did blend gender markers together. The dataset was small and the works were abstract, made more abstract by upsampling them. I then drew into the work on my iPad, just seeing where the works took me and with a few of those I built interactive 3D models to be explored in the Augmented Reality space. Each work was submitted multiple times to some classifiers to see what machines saw in the works. I used Ryan Kiros’ neural storyteller to generate passages of narrative based on still images at each stage of production and put everything together in the app, animated drawings, interactive 3D models, narrative and music so that the works could be further explored via Augmented Reality.

For me, as a non-binary artist it was important that these works reflected non-binary gender. The narrative flips pronouns at random and causes a disconnect in the viewer as they hear pronouns that don’t appear to fit. This echoes the disconnect we feel when misgendered. It also causes the viewer to question their pronoun assumptions and what meaning pronouns have.

Going forward in my Phd I will be building on this groundwork to see if I can subvert the binary training of most datasets. Gender minorities are not often included in dataset training and this can lead to issues for us as we move forward in a world increasingly governed by computer vision. I do believe we can create less biased AI, we just need to be less biased ourselves and have diverse teams so that everyone is considered at every step of the way.

I believe that we could learn from computers and machine learning and hopefully as we train them and consider the training data we will learn and grow alongside them.

Beth J.: Computers can be collaborators, machine learning can help us expand horizons. What is the future of art and what will it mean for the spectators?

J.R.: I believe in the future we will collaborate increasingly with machines to make amazing things! The future of art is wide open to experimentation and democratization, moving beyond institutions to being more decentralized and expansive. For spectators this means that they will have more to choose from, more ways to get involved with the process, and hopefully more ways to engage with and be moved by art. The future of art is open, open source, open to the public, open for business. The future of art is engagement, dialogue, exploration and interactive. The future of art is collaboration with machines, with people in the same room or on the other side of the world, it is about making the world smaller and more inclusive for everyone.

Beth J.: Is there a question nobody ask and you would like to answer?

J.R.: I’d like if more people asked each other for their pronouns! Mine are they/them.