Pix2Pix : Image translation and how to use?

Source: Artificial Intelligence on Medium

All rights to rightful owner

Pix2Pix : Image translation and how to use?

Pix2Pix is a method for Image-to-Image translation, this includes (illustrated below):

  • Labelling of segmented images
  • Brightness change (Day to Night)
  • Aerial to map
  • Coloring of black and white pictures (includes contour images)
All rights to rightful owner

Some background knowledge:

First of all Pix2Pix employs a specific type of Adversarial Netowrk : a Conditional GAN. This implies that loss/objective function is also expressed as a function of a given observation x meaning that its actors do not rely only on the opponent’s feed. The loss function would therefore be:

All rights to rightful owner

The architecture of the generator (cGAN) relies in U-net type of architecture where both downsampling and upsampling layers are used, whereas the discriminator is using downsampling layers only.

How to use these features of Pix2Pix?

First of all, I personally use Tensorflow so I am using the Tensorflow version of Pix2Pix (bear in mind the library was originally written in PyTorch).

First of all clone the repository:

git clone https://github.com/miki998/pix2pix_image_translation

Access the cloned directory, and install the requirements

pip install -r requirements.txt

Now you are all set ! You should see three different scripts that you can run:

  • ./download.sh: to obtain the datasets you picked
  • before using test.sh, click on the links for the pre-trained model to download it, and move it to the directory

To try out the pre-trained model simply do the following:

./test.sh

And to train a model with a created folder with pictures inside:

./train.sh