What is Transfer Learning? – A Simple Introduction.

Transfer Learning is a technique in machine learning where we reuse a pre-trained model to solve a different but related problem. It is one of the popular methods to train the deep neural network. It is generally used for image classification tasks where the amount of the dataset is small. 

In this article, we will go through what transfer learning is, how it works and the advantages it offers. Additionally, we will also cover the most common problems related to it. 

Transfer learning is a research problem in deep learning where a model trained on a task is reused to initialize another model for training a similar task.

Table of Contents:

  1. What is transfer learning?
  2. How does transfer learning work?
  3. Why should we use transfer learning?
  4. Further reading
  5. Summary

What is Transfer Learning?

Human beings have the ability to utilize the knowledge learned across different tasks. We are able to transfer the knowledge learned from a task to another similar task easily. 

For example:

  • If you know how to ride a bike, you can easily learn to ride a car.
  • If you know mathematics, you can easily learn physics.

In conventional deep learning, models are only trained to learn only one type of task. To learn another task, we have to build the model again from scratch. To overcome this limitation, we will learn the concept of transfer learning.

Transfer learning is a research problem in deep learning where a model trained on a task is reused to initialize another model for training a similar task. For example, you have an image classification model trained for wild animal classification. You can use this pre-trained model and use it for dog breed classification.  It is mostly used in computer vision and natural language processing tasks.READ MORE

In terms of deep neural networks, we use the weight of the pre-trained model and use it to initialize another deep neural network to create a new model. This new model contains the features learned by the previous pre-trained model. In this way, we exploit the knowledge gained from a task to improve the performance in another task

How does transfer learning work?

In a deep convolutional neural network (CNN) we have different layers which learn different features from data. These features ultimately form the entire deep neural network.  These features detect lines and edges in the earlier layers and shapes in the middle layers. The features in the last layers are task-specific.  In transfer learning, we use the features from the earlier and the middle layers and train the last layer from the scratch. This helps us to take the advantage of the features learned by the pre-trained model on the previous task.

A diagram explaining the concept of transfer learning.

The earlier and middle layers consist of general features which can easily be used for other similar tasks. While the last layers or classifier layers consist of task-specific features. These features need to be learned from the new dataset. 

Why should we use transfer learning?

Transfer learning offers several advantages over the existing approaches.

  1. Save training time.
  2. Require less labelled data.
  3. Better performance.
  4. Better generalization capability.

A deep neural network consists of a lot of parameters that require a lot of data. The training process takes a lot of time, sometimes it takes days or even weeks when the dataset is really huge. With the use of the transfer learning approach, we don’t train the model from scratch, so it requires less labelled data and takes less training time. 

As, we are using a pre-trained model, which is already trained on a wide variety of data. So, the newly formed model would have better performance even in the case of an extreme scenario. This would give the model a better generalization capability and allow it to be more robust.

Further Reading

Summary

In this article, you have learned what is transfer learning and how it is better than the classical deep learning approach. Specifically, you have learned:

  • What is transfer learning?
  • How does transfer learning work?
  • Why should we use transfer learning?

Still, have some questions or queries? Just comment below. For more updates. Follow me.

Nikhil Tomar

I am an independent researcher in the field of Artificial Intelligence. I love to write about the technology I am working on.

You may also like...

1 Response

  1. Anand says:

    Hey Nikhil,

    Very nice, to-the-point article on transfer learning.

    For a small size of dataset, an image augmentation along with transfer learning can give exceptional results for a vision task.

    I’ve put up a guide for image data augmentation that discusses beginners to advanced level image augmentation strategy. Hope this could be useful to many people: https://datamonje.com/image-data-augmentation/

    Kindly share your views Nikhil.

Leave a Reply

Your email address will not be published. Required fields are marked *