Source: Deep Learning on Medium
AutoAugment: Learning Augmentation Strategies from Data (CVPR 2019)
AutoAugment is an augmentation strategy that employs a search algorithm to find an augmentation policy that will yield the best results on the model. Each policy has several sub-policies. One sub-policy is randomly chosen for each image. Each sub-policy consists of an image processing function and the probability that the functions are applied with. The image processing operations could be translation, shearing or rotation. The best policy is the one that yields the highest validation accuracy via the search algorithm.
During experimentation, reinforcement learning is used for the search algorithm. Learned policies are easily transferrable to new datasets. AutoAugment was tested on CIFAR-10, CIFAR-10, CIFAR-100, SVHN, reduced SVHN, and ImageNet.
The method has two components: a search algorithm, and a search space. The search algorithm is implemented as a controller RNN. It samples a data augmentation policy, which has the image processing operation information and the probability of using the operation in each batch. It also has information about the magnitude of the operation. This policy will then be used to train a neural network with a fixed architecture. The validation accuracy obtained from this will be sent back to update the controller, which is updated by the policy gradient methods.
In the search space, a policy consists of 5 sub-policies. Each sub-policy has two image operations that are applied in sequence. Every operation is linked to two hyperparameters. These are the probability of applying the operation and the magnitude of the operation.
Here are some of the results obtained with this method: