21 questions with Siraj Raval



AI Saturdays Lagos recently had the opportunity to have the Best selling Author and an amazing You tuber, Siraj Raval joined us over an hangout session on September 1st and It all started with a tweet :). Thank you for Joining us Siraj.


1. First of all, your hair fascinates us, what’s your secret hair product?

🙇 🙇 🙇 It’s nothing really, It’s just that I don’t put shampoo everyday.

2. What aspect of AI gets you excited all the time?

🙇 🙇 🙇 Right now, it’s anything that has to do with Reinforcement Learning because I’ve been doing supervised learning for so long and Reinforcement learning is a new challenge and to me, I get excited whenever I see a topic in Machine Learning and I feel like it’s really hard like I see an equation and I’m like — what’s going on here, that’s when I get excited and so, when I look at Reinforcement Learning and I see all these things like Policy Evaluation, Value Iteration, Bellmen Equation, these concepts that people don’t really talk about that often, I get excited. So Reinforcement, right now, is my topic.

3. What’s your favorite AI research paper of all time?

🙇 🙇 🙇 That’s a good one, uhm! can I say 2? The papers that excites me the most are papers that try to create better optimization strategy than back-propagation. Every neural networks uses the technique back-propagation so the two that I really like are Synthetic Gradients from DeepMind where they were trying to improve BP and another one is Bayesian Actor Critic Reinforcement Learning, I was just reading that but uhm! anything that tries to make a better algorithm than back-propagation — I’m still waiting, no one has done it yet, those are just some attempt but the second someone does it, I will be the happiest person ever.

4. What was your first ML model and how did you build it?

🙇 🙇 🙇 My first ML model is Linear Regression and I built it with scikit learn.

5. What’s a day like with Siraj?

🙇 🙇 🙇 I wake up at 7:30am, then I ride my bike to my co-working space. When I’m working, I like to be completely alone, I like to be in my own space and then I go home after like 6hours of working and then I will spend the rest of the day just kinda like alone, thinking of ideas so clearly, usually, I’m kinda in my own space but on weekends, I try to be more social.

6. What material would you recommend for NLP beginners?

🙇 🙇 🙇 That’s a great question and I coincidentally just worked on this NLP library, it’s called SPACY. It’s the best and fastest Natural Language Processing library in the world, not just in python but in the entire world. What is interesting about it is that it’s building data structures from pure C, not just in python, so it allows the NLP model to run much faster. So SPACY is the one.

7. Do you have any tricks you use to interpret mathematical equations?

🙇 🙇 🙇 It’s a process, I don’t ever consider myself to be a pro at Math. Pretty much every time I read a research paper, I would see something that I don’t understand but overtime, I would get used to certain symbols but in terms of tricks that I use — I think cheat sheets are very effective, whatever your topic is, if you have one page that has all the math for that topic, then you can just reference to it whenever you’re doing any kind of work and instead of having it in your brain because that’s a lot to keep at the same time so I like to have cheat sheet for all the Maths that I’m working with.

8. What book would you recommend for beginners AI students?

🙇 🙇 🙇 So, my friend Andrew Trask just released this book, it’s called Grokking Deep Learning, it’s on twitter. I would think that’s the one to read for beginners but to my most honest opinion about it, actually, is that I don’t even read books that much myself, I think it’s more effective to learn, obviously, yes, with video contents but also with blog posts and podcasts. I think It’s important to have a variety of different types of learning resources that you’re consuming and so yeah, textbooks in general, I’m not as into but you know, everybody learns in a different way so there’s not ‘one’ path is the best.

9. What was it like getting suspended during your freshman for stealing a laptop ?

🙇 🙇 🙇 I was 18 years old, this was more than a decade ago, yeah It was a pretty low point because I just got to Columbia University and my parents were very proud and that happened. I was suspended for a semester and I spent that semester in Europe, couch-surfing so I was basically homeless. For three and half months, I just visited one week per city and I spend like 10 USD/day but during that time, I learned a lot — I saw so many different people with different perspectives and when I got back, I decided, with everything that I learnt during my trip, that I’ve got a second chance, I’m going to study computer science and not finance and I’m going to try to do something positive. I didn’t know I was going to do AI, I had no idea, I thought I would get like a programming job and work with Google or something like that but it was only like 4 years later that I began to get into AI but yeah, I’ve been blessed with that opportunity to restart. I made a huge mistake and that could be considered a felony and I could have gone to jail for a long time and all my dreams would have been ruined — so, I felt very blessed and I think I worked harder than ever when I go back.

10. If you had graduated with an AI or ML degree, what career path would you have pursued?

🙇 🙇 🙇 That’s a good one. I was thinking about that, I was thinking, maybe be an AI Researcher, that would have been cool uhm like a full time researcher but you don’t need a degree to do that anymore. So, if I had a degree or in general, If I wasn’t doing what I’m doing right now, I would be doing research full time but yeah, this is so weird to say but I was actually one credit away from graduating.

11. What area of AI do you think African AI Researchers should look into?

🙇 🙇 🙇 That’s a great question, right now there’s a lot of interest in RL but that is kinda saturated as a research but what I think African AI could be really good at is a one-shot or zero-shot learning where its like you don’t necessarily have a lot of data but you can learn a lot from that and if anybody, not just African AI Researchers, solve zero-shot or one-shot learning where you can have only 20 training example and you can still learn something — then that would change the world because right now, you need a hundred of thousands examples to do any kind of serious machine learning and not everybody has that except for Google, Apple, you know. So, that would be huge contribution.

12. What’s your favorite operating system?

🙇 🙇 🙇 So everybody I’ve interviewed always said Ubuntu but I would say OSX and I like it because it’s UNIX based so I could use the terminal, the interface is very pleasing aesthetically and I do a lot of video editing so I use software like Final Cut Pro, Adobe Suite and all that works well with OSX. The only think I don’t like is this little touch bar that I’ve hated for 2 years but hopefully someday Apple removes it.

13. What do you envision as the future of your School of AI?

🙇 🙇 🙇 Right now, there’s a lot of oppoutunity to lean about AI so that you can get employed, like Nvidia has a deep learning Institute, Udacity exists, Coursera — these are all platforms that allow you to learn AI specifically for doing research or being employed by a company. With School of AI, the way it’s different is that we’re not just trying to teach people AI, we’re trying to create an environment of entrepreneurship, so we want our students to use AI to solve some problems in their local communities or in a larger ones by creating their own startups and ventures and just getting together as small groups because now it’s finally possible to make a big impact with a small number of people because all the resources you need to do AI are democratize and getting more democratize everyday like Google Colab for example.

14. Who do you think would achieve AGI first, OpenAI or DeepMind?

🙇 🙇 🙇 Uhm! right now, definitely DeepMind because no one even understands the full extent of what DeepMind is capable of doing for example, I’ve have a lot of people who I know that works at Google and stuffs and they don’t even know because they don’t have access to see what’s happening at DeepMind like my friend that I interviewed, Joel Shor with the 67 questions, later on I asked about DeepMind and he said oh just last week, DeepMind ran an experiment that used 80% of Google’s computer — that’s like the world largest super computer. Also, in terms of the quality of papers that are coming out of DeepMind like every few months are amazing. Open AI has OpenAI 5, great but also, that’s one thing compared to screen of accomplishment DeepMind has done. At School of AI, we want to have a whole research lab, so before the year ends, we’re gonna have at least a dozen papers hopefully some of them would be published at ICML. Before the year runs out, we would have tons of papers submitted to the top Machine Learning conference, I can promise you that.

15. Can you briefly explain what Singularity means to you?

🙇 🙇 🙇 For sure, you know when I started this, I was thinking we gotta solve AI so that eventually, it’s going to become smarter than all of us and hopefully solve all of our problems but the more I researched this, the more I learned about this, I realized the singularity is not just gonna happen because of an AI alone, it’s gonna be because of humans that have solved AI and now they can literally do anything that they want. They are the god, it’s not the AI that’s the god as they will be able to do anything. So what does the singularity looks like, the singularity looks like one person or a team of people has finally solved AI, they have the algorithm and then, whatever they choose to do with, that is going to be the singularity. They could choose to solve every problem in the world, they could choose to destroy the world — they could choose to do anything with it.

16. What is your favorite ML Algorithm?

🙇 🙇 🙇 My favorite one would be Neural Networks, in particular, it would be any of the stochastic models, the models that have unpredictable outcome — like you don’t know exactly what it’s gonna output so variational autoencoders would probably be my favorite model because if you generate things that you wouldn’t think you would generate before and I like that idea of randomness. I think we need to see more of randomness inside machine learning models because life is random and we wanna model that.

17. What are of AI/ML do you think is important but doesn’t get a lot of attention?

🙇 🙇 🙇 I think right now, there’s not a lot happening in Evolutionary Algorithms but that’s because no one’s really seen any success with evolutionary algorithms when it comes to industrial settings like for consumer applications but there’s gotta be something there that we’re missing because that is modelling evolution so there is something there but no one has figured out how to successfully apply evolutionary techniques to real world applications.

18. Can you suggest ways of overcoming frustrations when learning about a new concept?

🙇 🙇 🙇 There are two part, the first is stay healthy, I try to treat my body like a machine — you know, input and output. If I want the brain to be productive, I gotta be eating what I think would be fueling it best, I’m gonna try to do something physical everyday so I gotta make sure my biological machine is working properly so that I can feel the endurance to continue when it’s hard and not get brain fog, just to be as focused as I can. The second part is twitter, because on twitter you’re gonna see a lot of researchers releasing new algorithms and applications and that keeps me excited and that excitements help me push through the hard time — so as long as you’re seeing some of the new advancement in the field you’re gonna be excited because amazing stuffs comes out every other day basically and as long as you take care of yourself, nothing can stop you.

19. Can you tell us about two of your favorite AI Researchers?

🙇 🙇 🙇 Uhm! my favorite AI Researcher is Demis Hassabis. I’m still kinda like star struck a lil bit that he liked that video. I’m meeting with Deep Mind for the first time next week and it’s gonna be interesting how that conversation is gonna go — I’ve been waiting for this. Demis because he’s basically the mastermind, I mean, all of these research directions that Deep Mind is going in is because Demis had the intuition from having studied neuroscience, computer science, from having done this with game AI research for like the later part of a decade — all these experiences are the reasons that says okay, these group of people work on this and that and I think that intuition is very valuable and I don’t see anybody who has that high level of an intuition in the field. Also, Andej Karpathy because his blog post “The unreasonable effectiveness of Recurrent Neural Networks” is like a huge inspiration like I pulled so much information and inspiration from the blog post to make one or two videos back in the day and I always actually come back to it whenever I need a refresher — it’s like a cheat sheet, it’s got everything on one page about recurrent networks and why they work so well.

20. Can you talk about Move37?

🙇 🙇 🙇 uhm! No — yeah, it’s happening so fast but basically this is what I always talk about, about how Reinforcement learning is very important and I’m so excited to start talking about full time. I have so much content coming out about it in the next 10 weeks and by the end of it like I said before, we will have research papers, What I want first is for us to get a very solid understanding of this technology that is on the literally the fore front of all AI right now is deep Reinforcement Learning, that’s the edge of knowledge right now so if I can help students get there. Now I have six assistant instructors which is amazing so it’s super exciting, I have never done anything at this scale before — this is a whole new scale, this is beyond anything I’ve done with Udacity, previous courses — it’s gonna be sick and by the end of it, we want the students to be publishing research papers and we’re gonna guide them on how to do that in three parts and I hope to see all of you doing this course.

21. What’s you favorite AI joke (if any)?

🙇 🙇 🙇 My favorite AI meme or joke is probably anything that takes any of the OG AI researchers like Yann LeCun, Geoffrey Hinton and kinda twist them into like a cool version — so like bored Yann LeCun on twitter is hilarious, it says the most hilarious stuffs like ‘oh no, you just got torched like PyTorch’

That was it, and of course we sealed the afternoon with a freestyle from the AI Rap King himself ;) I chose one topic which was ‘unsupervised learning’.

We had an awesome time with Siraj and of course, a group photo 😀

Source: Deep Learning on Medium