Roughly one year ago, the article New Theory Cracks Open the Black Box of Deep Neural Network was posted on Wired. This is very significant as this article validates what those working with artificial intelligence (AI) already intuitively knew — that we are living through another age of groundbreaking technological and scientific progress.
The AI age shows uncanny resemblance to other profound historic events such as the Industrial Revolution and the advances in quantum physics, in that practice and experimentation raced ahead of theory. Maxwell, Boltzmann and Gibbs made their contributions to thermodynamics and statistical mechanics long after steam engines had already made the wealthiest group of people in history. And Schrodinger, Heisenberg and von Neumann developed mathematical formalism for quantum physics more than a century after Thomas Young performed the first double-slit experiment with light. As Larry Dossey puts it in his book One Mind: “I’ve never seem a patient, whom needed major surgery, refuse general anesthetic because the anesthesiologist could not explain precisely how it works.”
Specifically with AI, how deep neural networks are able to generalize so well (in some cases even better than humans) has been a nagging mystery. Despite that, industry pushed ahead and AI has so far been the playground of practitioners who have been advancing the field through experimentation. But with Naftali Tishby (the protagonist of the Wired article) presenting evidence founded in theory of how deep neural networks most likely work, it looks like theory is catching up!
I believe this gives us clues that we are indeed going through a momentous and groundbreaking age, once again. It’s very exciting to witness historic recurrence unfolding right in front of our eyes.
Source: Deep Learning on Medium