Deep Learning — Open Problems

Source: Deep Learning on Medium

Deep Learning — Open Problems

In this post, I am sharing few open problems in the area of Deep Learning that needs attention.

  1. Explainability: This is one of the main concerns the community is currently facing. A huge amount of criticism from theoretical machine learning researchers is that neural network models are like Black box. The existing models fail to answer the question of “How a decision is being made”. DARPA has initiated a new program to look into this particular issue. I anticipate that in the next 2–3 years in deep learn this area will see new frontiers.
  2. Robust Neural Models: This is one other main concern the community is trying to address. This issue is very tricky though. If we add a small amount of white noise to a data point and feed it to a trained neural model at inference, the neural network will get easily fooled. This problem is famously called as adversarial example. Although the problem of robust learning prevails in other ML areas, a direct transfer of solutions from other areas is very difficult. The community should actively look into this open problem.
  3. Training a Universal model: This problem might be little far fetched given the present situation. From a philosophical progress point of view, “Can we train a neural network that can perform multiple tasks”. A little progress from transfer learning under a lot of constraint settings is available now. But generalizing this will have a profound impact on the way the area progresses.
  4. Theoretical understanding of error surfaces of Deep neural models: Again a burning question in the research community. Ever since the open problem about the landscape of neural networks is proposed in COLT 2015, a lot of traction is being gained in trying to address this issue. A little progress even under unrealistic assumptions will still be appreciated by the community given absolutely zero progress in this area.

The above 4 are the open problems I feel that need immediate attention to make significant progress in the area of Deep Learning.

P.S: If you feel there are any other open areas, please drop them as comments.