Pix2Pix : Image translation and how to use?
Pix2Pix is a method for Image-to-Image translation, this includes (illustrated below):
- Labelling of segmented images
- Brightness change (Day to Night)
- Aerial to map
- Coloring of black and white pictures (includes contour images)
Some background knowledge:
First of all Pix2Pix employs a specific type of Adversarial Netowrk : a Conditional GAN. This implies that loss/objective function is also expressed as a function of a given observation x meaning that its actors do not rely only on the opponent’s feed. The loss function would therefore be:
The architecture of the generator (cGAN) relies in U-net type of architecture where both downsampling and upsampling layers are used, whereas the discriminator is using downsampling layers only.
How to use these features of Pix2Pix?
First of all, I personally use Tensorflow so I am using the Tensorflow version of Pix2Pix (bear in mind the library was originally written in PyTorch).
First of all clone the repository:
Access the cloned directory, and install the requirements
pip install -r requirements.txt
Now you are all set ! You should see three different scripts that you can run:
- ./download.sh: to obtain the datasets you picked
- before using test.sh, click on the links for the pre-trained model to download it, and move it to the directory
To try out the pre-trained model simply do the following:
And to train a model with a created folder with pictures inside: