Generative adversarial networks (GANs): A new innovation of Machine learning

Original article was published by Hamza Abdullah on Artificial Intelligence on Medium


Artificial Intelligence is getting mature exponentially. Recent decade added a lot to wonder about its potential. Even though we are still far from human level intelligence but still statistical type of Artificial Intelligence or more specifically called as “Machine learning” is not less than a wonderland for experimenters and developers. Generative adversarial networks (GANs) are fascinating recent innovation in machine learning.

This article is sectioned into follow few headings

What are Generative adversarial networks (GANs)?

What it can do?

Challenges faced by GANs

Generative adversarial networks (GANs) were designed by Ian Goodfellow and his colleagues in 2014.

Ian Goodfellow: PHOTOGRAPH BY CHRISTIE HEMM KLOK

What are Generative adversarial networks (GANs)?

GANs are among one of the many types of generative models that create new set of data based on the training data provided that impressively looks close to the original. Its like a digital illusion. For example, recently, one of the application of GAN were gone viral. In which this super innovative model created images of human faces impressively like a real human face but in true reality that images of human faces doesn’t related to any human on Earth. Whoa! Isn’t it amazing?

Generative Adversarial Networks (GANs) are a marriage between two deep learning convolutional neural network (CNNs). Typically consists of two networks: generator and discriminator (aka critic) type of Convolutional neural networks (CNNs). A generator which learns to generate new set of data based on the training data provided. While the discriminator tries to differentiate between the newly generated data, actually fake a instance of data, from the training data. Once generator is trained, the discriminator is discarded.

Generative adversarial networks are based on a game theoretic scenario in which the generator network must compete against an adversary. The generator network directly produces samples. Its adversary, the discriminator network, attempts to distinguish between samples drawn from the training data and samples drawn from the generator.

— Page 699, Deep Learning, 2016

Faces generated by GAN

What it can do?

GANs comes with its true wondrous applications in many areas. Few of which are as follows:

  • Image to Image translation
  • Face Aging
  • Photo Blending
  • Super Resolution
  • Photo Painting
  • Clothing Translation
  • Video Prediction
  • 3D Object Generation

Recent few advances in GANs explored above-mentioned areas more deeply and presented reviewers with some impressive results. You can read a article by Jason Brownlee explaining applications of GANs.

One dark development of GANs could be in growing fake news. It has so much potential to infect the reality that if look in this way then they are truly daunting.

Challenges faced by GANs

Despite Generative adversarial networks (GANs) truly innovative potential they are still not so mature and face few challenges on the way. But on-going research can address all such challenges in near time.

Vanishing Gradients

Recent research shows that if discriminator is trained flawlessly and too good to be called as perfect then it might cause failure to generator while training due to vanishing gradients. In fact original GAN paper proposed a modification to minimax loss to deal with vanishing gradient problem.

Mode Collapse

Mode collapse is another failure faced by GANs. While doing optimization, each iteration over optimizes the generator and discriminator being so dump never understands it and stuck in this trap.

Failure to Converge

Another failure to GANs is failure to converge. Although, researchers have tried multiple ways to fix this problem of convergence. i.e. Adding noise to discriminator inputs, Penalizing discriminator weights