Leonardo and the GAN dream

Source: Deep Learning on Medium


Explore the concept of a GAN network by travelling back in time to the renaissance, as Leonardo faces one of his most surprising challenges

Go to the profile of Javier Ideami

Leonardo slammed his fist on the table.
 — Outrageous! Porca Miseria!

The courtiers of the king and other royalty contemplated the scene from the sides of the great room. Clara, his cousin and confidant, tried to comfort him. 
 — What happened Maestro? What’s the matter?

Leonardo hesitated for a moment. Then sighed and raised his eyes towards Clara. Suddenly, his index finger made a massive leap that took Clara’s gaze all the way to the other end of the sumptuous room.

— Antonio? What’s the matter with your apprentice? I thought you were so very delighted, and..

Everyone whispered hurriedly. Leonardo raised his hand and the entire room fell silent. He stood up and spoke hurriedly while pacing around.

—Why did I hire Antonio?, muttered to himself.

— Antonio will be in charge of selecting the best works of the lead painter of the Genovese Studio, and bring them to me. In the meantime, Antonio shall be learning and improving his own technique, on his own time, by himself, slowly, carefully, methodically..

Clara pretended not to understand, although, in fact, she understood pretty well, as always. Leonardo went on.

— I soon became alarmed that the Genovese studio creations were nothing like what I was expecting.. the themes, the technique, all was wrong! He made a pause and looked up, and up.. as if he could see them again.

— They would contain some appalling mistakes.. terribile! and a total lack of craftsmanship. But Antonio.. he.. spoke sweet words to me, he said that.. the lead artist was sick, hallucinating, and was going through some personal turmoil.. would you believe it?

Clara was, in the meantime, trying to offer some water to Leonardo. But Leonardo was getting more and more agitated.

— You all!, do you wanna know the truth? davvero? la verita!
He pointed his finger towards Antonio, who was on the opposite end of the room, looking uncomfortably towards the floor.

— He was lying! those terrible paintings, they were his own! Antonio was.. making them! They were.. terrible. Leonardo pointed his finger again at Antonio.

— Terrible at the start, for sure. And I would tell him what was wrong, and then… well.., over time they became better.. and.. better.. until one day..

Almost falling, Clara kept trying to put a glass of water on Leonardo’s hand, but Leonardo would not notice.

— That day, I told Antonio that the Genovese painter had recovered his touch. I was so glad that.. all was back to normal!

Clara finally managed to put the glass on his hand.
 — Leonardo, drink per favore, take a seat, calm down carissimo.

Leonardo sat down and drank hurriedly, then looked at Clara with urgency.

— But that night Clara, that night.. I caught him!! Painting in secret, painting one of those..

As if activated by a hidden switch, Antonio rushed towards Leonardo, who looked at him surprised. The room became an ocean of whispers.

— Master Leonardo, I need you to understand, let me explain.

— Explain?, Leonardo replied irritated.

— Weeks ago, I had a dream, Master.

The ocean of whispers traveled back and forth, right and left, then gave three spins and came to rest on the nose of Leonardo, who vigorously pushed it aside.

— I know, confusing, I also don’t understand it Master, but.. I now believe that we are a.. GAN.

Everybody in the room fell silent, except Lady Sabina who shouted to the top of her lungs
 — Blasphemy!

Clara asked everybody to calm down. Leonardo looked at Antonio inquisitively.

— Ma che cosa?, tell me.. have you been drinking?

— Master, let me explain.

Leonardo turned somber.

— If you have not been drinking, you better explain right now.., or else..

Antonio’s face lightened up.

— Yes, I explain. Master, that afternoon, weeks ago, while you were resting, I received a letter. The Genovese studio owners decided to renovate their entire building and postpone all work till next year.

A courtier brought a chair and Antonio sat down.

— I was in panic, I knew you would be disappointed. The king trusted you and your reputation to bring the Genovese Studio creations to his court, and in exchange he was going to finance your next phase of research. Depressed, I went to sleep. Everything was going wrong. I so much wanted to learn to become a great painter. But as I had been hired to supervise the communication with Genova, I feared that you would send me back home once you knew that their studio had postponed all work until next year.

Leonardo looked intently at Antonio, trying to decipher what he would say next.

— I had fever. And then it happened.. I had such a strange dream. I was looking for a quick way to solve this riddle. How could I, simultaneously, work for you and learn to paint as well as the Genova masters, and yet not give you the bad news, the news that their studio was closing down for months…

As they listened, everybody was moving slowly towards the center of the room , where Antonio was speaking.

— I began to imagine.. What if I was to paint myself the Genovese works?. But, of course, I had no idea how to paint like them, or even what their paintings looked like. And yet, I decided to try.

Antonio made a dramatic pause, looked at everybody and then back to Leonardo.

— My first attempt looked.. weird.. almost random. Leonardo rolled his eyes.

Whispers were heard all around the room.

— I presented it to you. You became enraged. You said that this could not be the real thing. So I asked you: why not?.

Leonardo moved in his chair, uncomfortable.

— And you explained it to me, you explained how and why my creation differed from the real thing. Satisfied with your explanation, I assured you that I would complain to the Genovese Studio, and you were pleased with that.

Leonardo looked still somewhat annoyed but genuinely fascinated at the same time.

— I then created another painting with a different theme. But this time, I incorporated the feedback you gave me. Next day, I came back and before showing anything, I explained that the Genovese lead artist had replied saying that he had some health and personal issues that would last for some time.

As Antonio proceeded, the sun began to turn the room golden.

— However, I told you, the lead painter assured me that he would do his best to deliver great works to you and the king. He asked for patience and understanding. Then, I showed you my creation. You cursed and cursed again. You gave me a long speech about a mysterious condition called “improvvisa pazzia”, and then proceeded to enlighten me some more about the difference between my painting and what you were expecting.

Leonardo’s face was a mixture of pain and delight.

— Day by day, this process continued. It was clear to me that you truly believed the story of the Genovese painter’s situation, and that gradually, you were getting a bit less upset as my paintings got better, thanks to your feedback. Eventually, after some weeks, the key day arrived.

Leonardo frowned. Clara rushed by his side. The ocean of whispers rushed back.

— I showed you the painting of “Le cinque donne”. You were delighted. You laughed. You hugged me and declared that the Genovese lead artist was back in shape.

— Mamma mia!, Leonardo sighed towards the tall ceiling.

— I felt so proud. Within my dream, I celebrated with my wife, Marga, and 2 kids, Ana and Pietro. My daughter brought a wonderful cake full of strawberries. On top of the cake, in between the strawberries, there were 3 big bold letters: GAN.

— What is that, Ana?, I asked her.

She quickly began talking matter-of-factly.

— It’s so clear, dad. You and Leonardo became a network. You generated better and better versions of the Genovese style of painting. You did so by trying to deceive Leonardo into believing that your paintings were, in fact, coming from the Genovese Studio. Leonardo, however, is such a genius, and a great discriminator and critic. He would very accurately spot and explain the differences between your creations and what would have been genuine and authentic ones from the Genovese studio. With that feedback, you kept on improving. Until one day, Leonardo truly believed that your creation was a painting worth of the lead artist of the Genovese studio.

Ana made a quick pause to breathe, and proceeded.

— Your network reached stability and you became a great painter in the Genovese style. Dad, you did it all by creating an adversarial dynamic process between you two. That’s why your network is a GAN, a Generative Adversarial Network.

Antonio and Marga looked at each other, speechless.

— If you don’t believe me go ask Ian Goodfellow ok? He came up with all this stuff, that’s what my own dream inside your dream told me!

— has she been doing too much school work these days?, Marga whispered to Antonio.

— Are you ok, darling?, Antonio gave a cheeky smile to Ana. But Ana was not finished and kept on going.

— Well, well, but aren’t you going to listen properly to your daughter inside your own dream? There is much more, dad. Leonardo became gradually better and better at discriminating between your creations and the real stuff, and you kept on improving, until you reached mode collapse!

— Mode what?, Antonio was doing his best at following Ana’s enthusiastic and assertive delivery.

— when you kept painting over and over the very same thing, so obsessed with it. Leonardo was going mad!

Marga and Antonio exchanged a glance.
 — Those Minestrone soups are working, Antonio stated satisfied.

Pedro jumped right in.
 — Can we play GANs? please, please dad, can Anna and me go play GANs?
 — I wanna be the critic!, Anna volunteered.
 — I will be artist, I will fool you!, Pedro celebrated in advance.
 — Ok, children, go play, go play GANs. As for me.., I must get out of this part of the dream, I hear something…

Leonardo laughed, and like thunder, his laugh bounced off the richly decorated walls.
 — Extraordinary! Bellissimo! you clever boy, all right, you got me there Antonio, a GAN, we are a GAN!

Antonio was taken aback by Leonardo´s reaction.

Leonardo signaled Antonio to sit by his side.

— Antonio, this GAN thing is great. You knew that I would not have time to teach you to paint like a Genovese master. And yet, you needed my direct feedback to improve. So you tried to fool me, over and over. And as you kept getting better, I gradually became less able to detect problems in your creations. But Antonio, there is maybe an issue with this network.

— What would that be, master Leonardo?

— Lenta, molto lenta, very slow. You are gonna need lots of GPUs.

Antonio looked at Clara, and then back to Leonardo.

— Lots of what?

Leonardo laughed.

— Oh, clever Antonio, you think you are the only one coming up with obnoxious terms? I can also invent random funky ones, my son! GPU! TPU! and what about CycleGAN! uh! StarGAN! AttnGAN! BigGAN!

Antonio marveled at the sound of these new words.
 — Master, I get you. It took me a long time to get to this point, a lot of effort. Do you really think that this process could be made faster?

— Maybe

— How, Master?

— Let’s wait till Part 2, ha?

— Part what?

— I also had a dream Antonio. I dreamt that we live in -Medium- and that this is Part 1. Furthermore, I sense that it’s not a good idea to make each part too long beyond a small number of minutes. There are stats for this stuff, do you follow me, Antonio?

Clara rushed out of the room, anxiously requesting help.
 — the medicine, bring the medicine of Leonardo!!

— Va bene Maestro, we wait till Part 2, but tell me something please, should I worry if my daughter keeps asking me if we live in a simulation?