Source: Deep Learning on Medium
‘How do I get started with Machine Learning (ML)’ is the #1 question I get these days. There are way too many people asking this question mainly because most approaches are simply unapproachable and too scary for a novice. For every person that asks this question, I always reply with, ‘sadly, I can’t answer this question well’. Reason being, we are all unique individuals with unique learning requirements. However, I can definitely talk about what worked for me, although, this approach may work for some of you (keep that in mind).
My first failure with Machine Learning (WARNING: common failure in attempting to learn ML)
After recounting my failure story to others about the first time I attempted to learn Machine Learning, I quickly realized that this is a story A LOT people have experience. So, PLEASE PAY ATTENTION. Back in 2017, I saw a demo video of a neural network bot learning to play Super Mario world on its own. As the bot continued playing, it was learning to play smarter and smarter. That blew my mind… Then I saw other neural network bots playing games, like Starcraft, Dota, League of Legends, GTA5, and even a 17 year old kid who built a rap song builder that produces lyrics like Kanye West using LSTM. JEEEZZ.. I thought to myself, this stuff is way too cool, I must learn Machine Learning.
That night, I started installing Tensorflow, which is a deep learning framework mainly used for building neural networks. Shortly after, I started copying and pasting code from a famous tutorial that uses the MNIST dataset (a dataset of handwritten digits). Firstly, I had no idea what was going on in the script I was copying and pasting. Secondly, although the code gave me an 86% accuracy by following the tutorial, every single modification I’d attempt would throw an ERROR. Ugh… Super frustrating. These errors stemmed from the fact that I was new to python syntax and also had no idea how to write a neural network. YIKES!!
SO… I took a step back and did what any seasoned newbie would do, and I Googled it, ”How to write a neural network?”. Not to my surprise, I found a ton of mumbo jumbo math lingo and symbols that literally looked like alien language, like “use the sigmoid activation function to transpose the product of derivatives delta victimizations backpropagated through time..” WTF?! (I mean, this is a completely made up sentence, although, quite similar language used online). I felt defeated. BUT didn’t want to give up.
After talking to many folks in the industry, I finally found out about this famous Machine Learning course by Stanford University Prof. Andrew Ng on Coursera. WAHOUUU!! Finally … uh.. no. After lesson 8, Prof. Andrew Ng started talking Zerg Lore alien language. You see, the main thing that threw me off about learning Machine Learning was none other than the language shock of the ML vocabulary. I concluded that this course is only good if you have a base understanding of ML theories and the vocabulary — not for a straight beginner.
After another failure, this is how I succeeded at learning ML…
I finally decided to change my learning approach to focus on learning how to speak alien instead. For instance, there were many terms that recurs over and over again in ML, like activation function, back propagation, or even that strange Σ symbol (what is it and how is it calculated). Hmm… let me research that Σ symbol. Turns out it’s a Greek symbol for sigma, which is a mathematical way to indicate the summation of an array or matrix/vector. This is the exact process I went through for every term I didn’t quite get. Everything made more sense!
Whenever I had an ML term I was unclear about, for instance, activation function:
1) I’d isolate it
2) Ask a bunch of questions about what I don’t get about it
3) Then research those questions
After doing this repeatedly, I went back to the Andrew Ng course on Coursera and flew through the entire thing!
Things to consider when attempting to learn ML…
- Are you a Top-Down or Bottom-Up learner?
- Top-Down is when you best absorb information from the general explicit concept then specific implicit applications through deductive reasoning. For instance, you would, first, start by quickly skimming all the concepts and theories before unfolding and diving deeper “under the hood”. I’m definitely a Top-Down learner, so in ML, I prefer reading the theories and high level concepts before diving into the low level math behind it. This is the best way I learn or else, I’d be lost.
- Bottom-Up is when you best absorb information from the specific implicit applications then to the general explicit concepts through inductive reasoning. For instance, you would start with the detailed primitive blocks and combine them into larger and more complex function blocks. In other words, you would prefer diving straight to the math then mapping what is under the hood to these high level concepts, respectively. I’m not like this, however, many people are.
It is important for you to identify the type of learner you are so you can navigate through these learning approaches. This will make you more successful in learning ML. If someone says, “hey, this high level book for ML is the best for learning ML” and you know you’re a Bottom-Up learner, you can quickly dodge that suggestion as you know you won’t absorb the information.
2) What’s your goal for learning machine learning? Algorithm design or algorithm implementation?
- Algorithm design is when you’re more interested in the research side of ML. You’re designing algorithms and getting your hands dirty with the low level applications, unlike algorithm implementation.
- Algorithm implementation is using the algorithms others have designed to solve your problem that you are working on solving.
For me, I’m a bit of both. I prefer leveraging the work of others. However, at times, I have a unique problem to solve that I’d need to build my own algorithm from scratch. If you have a preference of what your ML goal for learning is, you can save yourself a lot of time prioritizing what you need to learn.
3) Do you learn best when you have a structured setting like a classroom or experimental so learn as you go?
- Classroom style learning is for those that need a curriculum to follow to keep engaged.
- Experimental style is for those that need a problem definition that needs to be solve. For ML, in my opinion, this approach definitely is more recommended. Learning the concepts is great and all, at the same time, starting from a problem and figuring your way out is the best for ML scenarios, particularly.
Math… How much math do I need?
From personal experience, I’d say ML is about 35% Linear algebra, 25% probability/stats, 15% calculus, 15% algorithms/complexity, and 10% data preprocessing. There are a few great courses to get you up to speed on the essential math tools.
Firstly, I want to give a HUGE shout out to Khan Academy and MPP, taught me math tools you need to know to write your own ML algorithms (downside to Khan Academy is that you have to build your own curriculum). That’s where I learned all the important math I needed. If you get to the point of writing your own algorithms, you’ll be given formulas and pseudocode most of the time. If you want to write your own algorithms, you have to get better at understanding pseudocode, how to calculate them, and all its various interchangeable indices. For instance, derivatives could be written like f’(x)=x² or dy/dx (both calculated the same way but could be confusing if you don’t know this). Finally, that is the basics of math I used for successfully learning ML.
What if I don’t want to build something from scratch? Can I use frameworks?
You don’t have to build neural networks from scratch. It’s like having the option of building a computer from scratch or buying a prebuilt one. Things like Tensorflow/Keras/Sklearn are like the prebuilt computer analogy. You can watch tutorials and jump right into using these (don’t have to worry too much about derivatives, matrix multiplications, activation function, back propagation) as the frameworks take care of most of the heavy lifting for you. Tradeoff is if something inside breaks you could have a harder time figuring it out or may lose control.
Do you have general advice?
I have additional tips for those interested in successfully learning ML.
- Compete in Kaggle competitions: this is a great way to not only make some money but also an incredible way to get your hands dirty with real life data that you care about.
- Follow the ML Subreddits: People in academia and industry folks post their findings on Reddit.
- Follow ML rockstars on Twitter: Many ML debates actually happen on Twitter. Here are a few people I follow on Twitter, like Trent McConaghy, Andrew Trask, and Pieter Abeel.
- Keep up with Hackernews: This channel is technically versed. Usually the best arguments rise to the top so it’s a great way to soak in some ML news.
- Subscribe to ML YouTube Channels: Siraj Raval is one of my favourites (yes, Canadian spelling lol).
- Read books on ML: Two of my favourite were Deep Learning with Python by Francois Chollet and Hands on ML with Sklearn & Tensorflow by Aurelien Geron.
- Attend hackathons or ML OpenHacks: Great opportunity to tackle a problem you care about. At first, I was using prebuilt ML model like Cognitive Services, then became frustrated that some of my use cases required more customizable ML models. This motivated me to learn how to build ML models from scratch.
- Start NOW: Trust me when I say this, trying to write your neural network from the very beginning makes a huge difference in your ML learning success. I’m talking about today, as soon as your done reading this post. Never mind, bootup your IDE right now!!! It’s cool to understand how to write a neural network in theory but writing them from scratch is a whole other rodeo. You’ll get to writing a neural network then you’ll get stuck, then you’ll officially have your first question to look up, then keep up this process. Start off with a simple feed-forward neural network project that predicts what color the background is of a page (white of black). Great simple project to learn the basics without a step by step tutorial like the MNIST scenerios.
The post was inspired by the HanselMinutes podcast I recorded with Scott Hanselman. I truly think there needs to be a blog post and podcast that is relatable to beginners that don’t know where to start, like myself at one point. If I can help one person on their journey to learning machine learning that success for me.