Transfer Learning

Source: Deep Learning on Medium

Since I was reading e-book called Deep Learning with Python by François Cholleton on my laptop , I decided to share nice topics I found interesting, described as Transfer Learning.


To eliminate the time-consuming process of building and fine-tuning Deep Neural Networks (DNNs) from scratch

A typical and effective way to do deep learning on a small amount of data will be using a pre-trained model. Well, what does it mean? The pre-trained model is the pre-saved model with its weights, trained on a huge dataset of large-scale classification tasks. For instance, if the initial model was trained on ImageNet dataset(where the vast majority of objects are animals and household items) it is possible to use this model to classification other types of objects as home furniture. It makes deep learning very effective in the case of a small available dataset. In other words, all convolutional neural networks extract low, mid, high-level features. For example, when it detects the face, it detects some edges on the first layer, some basic features on second and it detects more complex features as it goes deeper. The idea is that cars and faces have the same low levels features such as edges and colors, and only when we get high-level features such nose and eyes of person or tires on the vehicle, only then we can say that is it person or car.

There are quite a lot pre-trained models available. For example, on Keras framework you can find these following model using keras .applications:

NVIDIA’s transfer learning explanation:

Here’s how it works: First, you delete what’s known as the “loss output” layer, which is the final layer used to make predictions, and replace it with a new loss output layer for horse prediction. This loss output layer is a fine-tuning node for determining how training penalizes deviations from the labeled data and the predicted output.

Next, you would take your smaller dataset for horses and train it on the entire 50-layer neural network or the last few layers or just the loss layer alone. By applying these transfer learning techniques, your output on the new CNN will be horse identification.

Thank you for reading! Please hit clap 👏🏻 if you enjoyed reading. Let’s connect on Twitter!🐦