Original article was published on Deep Learning on Medium

# Formalizing Meta-Learning

This ‘learning-to-learn’ is very much aligned with human and animal learning in which learning methods incrementally improve over a period of time. This approach has advantages of data and compute efficiency.

Meta learning algorithm can be understood as made up of 2 levels of learning — inner and outer algorithm. Inner learning is similar to conventional learning algorithm such as improving image classification. During meta-learning, an outer (or upper, meta) algorithm updates the inner learning algorithm, such that the model learned by the inner algorithm improves an outer objective. This objective could be generalization performance or learning speed of the inner algorithm. Learning iterations of the base task can be thought of as providing the stimulus needed by the outer algorithm in order to learn the base learning algorithm.

# Terminology

Meta learning algorithm objective function can be mathematically expressed as