A Primer on Multi-task Learning — Part 2

Original article was published by Neeraj varshney on Deep Learning on Medium


Approaches for Multi-task Learning

In this section, we will look at the common ways to perform multi-task learning in deep neural networks.

Hard Parameter Sharing

Figure 1: Typical architecture for hard parameter sharing of hidden layers in MTL.

In the hard parameter sharing approach, the model shares the hidden layers across all tasks and keeps a few task-specific layers to specialize in each task.

Soft Parameter Sharing

Figure 2: Typical architecture for soft parameter sharing of hidden layers in MTL.

In the soft parameter sharing approach, each task has its own set of parameters. These task-specific layers are then regularized during training to reduce the differences between shared layers. This encourages layers to have similar weights but allows each task to specialize in specific components.