AutoML and AutoDL : Simplified

Source: Deep Learning on Medium


Take a peek into the beginning of a new phase in Machine Learning

What is AutoML?

As a data science research engineer, the most time consuming tasks for any problem statement are data analysis, experimenting with and choosing the right algorithm and parameter tuning. All of these require human expertise and is the reason why Data scientists have so much importance these days.

But what if this human intervention could be removed? What if such models can be made which can select the correct algorithm, do the data analysis and fine tune to get the perfect model all on their own? This is what AutoML is all about and is definitely the next step towards completely autonomous systems.

Other similar terms include AutoDL, AutoCV and AutoNLP to represent models in the domain of Deep Learning, Computer Vision and Natural Language Processing. In layman terms, for example, AutoCV aims to create a single system that can take on any Computer Vision problem, understand the data and choose the correct way to work on it, all of this without any human intervention.

Why AutoML?

Machine Learning and AI are in hype right now and there have been a lot of directions of research which are dubbed as the future of the field in recent years. AutoML, or as some people like to it call it meta-learning, is crudely speaking, a form of learning about learning.

Since it’s inception, the research in ML has always been on a rise and we have created a wide range of models and systems, each one for a specific problem statement. However we are now at an stage where we can combine these models and in some sense, move up in the hierarchy. Since our ultimate aim is to reach the top of the hierarchy and create an autonomous system, I would say that the hype of these meta-models is definitely worth their merit.

Google Cloud based AutoML services

How is AutoML different from ML?

In ML, data scientists first start with a problem statement and a dataset. The data is analysed and cleaned, a metric of performance is decided on and then a few models which might work on the dataset, according to the human intuition, are experimented with. There is a lot feature engineering and fine tuning involved before we finally reach an acceptable model.

AutoML is about trying to automate as much of this pipeline as possible. While some of these steps are easier to automate, like model fine tuning, some are extremely difficult, like choosing the correct architecture/model etc. To sum up, AutoML is about trying to create a single system that can remove the need of human intervention at every step of the modelling and training process.

How successful has it been?

There have been a lot of recent work done in the field of AutoML and AutoDL. I think a big portion of the credit for this should be given to multiple recent competitions in AutoML, trying to better the field and invite innovative ideas.

Google have created their own Cloud based AutoML platform that guarantees to help people use ML who know nothing about ML. However using their model is really costly and only suitable for businesses which are willing to spend the same.

There have also been a lot of successful open source platforms like Auto-Sklearn and Auto-keras, which have been successful in providing everyone with the recent advances in the field of AutoML .. for free!!

What’s next?

AutoML has seen successes that were not possible a few years ago and platforms like Google have been able to build systems that can remove Data scientists from the equation for common Machine Learning pipelines. But we are still far away from what can be called a truly Autonomous system.



References

[1] Guyon, Isabelle, et al. “Analysis of the AutoML Challenge series 2015–2018.” (2017).
[2] He, Yihui, et al. “Amc: Automl for model compression and acceleration on mobile devices.” Proceedings of the European Conference on Computer Vision (ECCV). 2018.
[3] Zoph, Barret, and Quoc V. Le. “Neural architecture search with reinforcement learning.” arXiv preprint arXiv:1611.01578(2016).