Original article was published by Neeraj varshney on Deep Learning on Medium
Approaches for Multi-task Learning
In this section, we will look at the common ways to perform multi-task learning in deep neural networks.
Hard Parameter Sharing
In the hard parameter sharing approach, the model shares the hidden layers across all tasks and keeps a few task-specific layers to specialize in each task.
Soft Parameter Sharing
In the soft parameter sharing approach, each task has its own set of parameters. These task-specific layers are then regularized during training to reduce the differences between shared layers. This encourages layers to have similar weights but allows each task to specialize in specific components.