Original article was published by Luc Andrea on Artificial Intelligence on Medium
Jinx, Shiba Inu with golden hair, likes her little walking habits. Several times a day (I have the evening shift) we go for a walk. In order to optimize her life expectancy, we have tried to teach her not to cross the road alone. The goal being to compensate for a possible lack of attention on my part, even if it is highly unlikely :p. Every time she crosses the road, I firmly say “STOP” as soon as she approaches the curb. “STOP” is a previously learned command, which she has no (or almost no) trouble executing. The objective being that she identifies the moments when we have to cross the road and that she stops there naturally without my intervention. Gradually, she began to stop before each crossing. After a few walks, she began to assimilate, or at least that’s what I thought.
In reality, Jinx had not understood the exercise at all. She simply memorized all the passages where she had to stop. As a result, if we crossed the road at a place that Jinx had never seen, she wouldn’t stop.
Instead of generalizing with the crossing examples, she memorized each case individually. If she encounters a new example, she doesn’t know to stop.
It’s similar for a machine learning algorithm. The reason if a model is overfitting is that it memorizes instead of detecting relationships in the data to generalize.
Overfitting is learning by rote instead of generalizing.
Therefore, if we test its performance on examples already seen, it will be very good. But on new examples, it will be mediocre.
Here, our canine algorithm is not necessarily to be questioned. Indeed, if we assume that Jinx has sufficient mental capacity, several reasons can justify this behavior:
- She hasn’t seen enough examples.
- She has seen too many times the same examples.
If we take the analogy to its logical conclusion, it would be the consequence of a lack of diversity in the walks.
The cause of this problem is probably linked to memory. An intelligent system that has memory is susceptible to overfit (humans, animals, algorithms, etc.). A priori, memory would be more a support for understanding than a simple means of storage.
To learn by heart is not to understand
and don’t forget to diversify your walks 😉