I’ll never forget the feeling of accomplishment I got when my first artificial neural network started working. Here’s me, a guy who’s just wanted to do comedy his whole life, studying alongside post-docs and PhD candidates. All of a sudden, this Jupyter notebook starts pumping out graphs and charts and I’ve completed my first project.
It was amazing.
Every neural network has a series of hyperparameters (tuners) you can use to optimize your outputs. The plain vanilla neural network relies on three that I can remember, number of hidden layers, number of hidden layer nodes, and learning rate. What’s amazing is that even as high-tech and precise as we are in late 2018, hyperparameter tuning is still mostly guesswork. I imagine one of many PhD candidates is working the mathemetical proofs to get it down to an exact science. My favorite of the three was the learning rate.
Intuitively, you’d think you’d want a fast learning rate. But this can lead to underfitting and overgeneralization and it shows in the learning curve. You have the opposite problem in a slow learning rate. My favorite analogy for this was climbing down a mountain. Think of the learning rates as strides. If your strides are too long, you may miss vital parts of the mountain, too short, you’re never getting down the mountain.
Artificial intelligence, machine learning, and deep learning are, by their very nature, multi-disciplinary studies. And I never miss an opportunity to apply what I’m learning in these neural networks to something else.
Consider a classroom of students a teacher is meeting for the first time. The teacher has no way of knowing everyone’s basic knowledge and/or how quickly they learn. If the teacher goes too fast, some students miss the material, too slow, students get bored. Many solutions have been tried and tested over the years, but none of them seem to have any long term success. And yet we still want every kid to run down a mountain at the same speed.
This is going to come up a lot in these articles, but I think what we’re going to see in the new economy is the emergence of individualized products and services. I think that’s a solution to our teaching problem. Going a little meta, I’d like to see neural networks that can test for a student’s base of knowledge, their learning rate, and their learning style. With the release of Tensorflow.js, I don’t think we’re too far off from that. I just wish I had the resources to make it myself, since that sounds like an awesome revenue stream.
On a high level, we need to start having discussions on what the future of teaching looks like. Is brick and mortar done? If so, what do you do to foster social development? If not, do only people who can afford these private lessons get ahead? Do you pipeline the kids that learn quickly into workers’ roles or leadership roles?
These are questions for someone a lot smarter than me or for me if I can learn political science in the next 2–4 years.
Until then, keep building the future.
Jimmy Murray is a Florida based comedian who studied Marketing and Film before finding himself homeless. Resourceful, he taught himself coding, which led to a ton of opportunities in many fields, the most recent of which is coding away his podcast editing. His entrepreneurial skills and love of automation have led to a sheer love of all things related to AI.
Source: Deep Learning on Medium