Deep Learning Adventures chats with Laurence Moroney, AI Lead at Google

Original article was published on Deep Learning on Medium


Deep Learning Adventures chats with Laurence Moroney, AI Lead at Google

Laurence Moroney and Andrew Ng are pioneers in democratizing AI and Deep Learning

AI (Artificial Intelligence) and specifically Deep Learning are revolutionizing the way we use data, extract insight out of it and teach computers to perform tasks that apart from being cool, have many applications in industries across the globe. If you are new to AI or Deep Learning you might ask yourself where to begin your journey. That’s where specializations like AI For Everyone, TensorFlow in Practice and Deep Learning come into the picture. They vary in level of difficulty or prerequisites from easy to intermediate, are self paced, and most importantly very well developed by Laurence Moroney and Andrew Ng and their teams. If you scroll down you LinkedIn page you will see many posts of members taking similar specializations, especially during this unprecedented outbreak we are coming out of. But what if you didn’t get all the concepts covered in these courses? What if you wanted to take a deeper dive in some of their project but didn’t have the time to do so? Or what if you never thought of a concept or idea from a different perspective? What if you wanted to be part of larger community who are going through the same journey with you, support, uplift and learn from each other and have fun while learning something new?

This is exactly our goal with our meetup Deep Learning Adventures 😀 Since our inception in April 2020, we are meeting every week and have forged a strong community who support and uplift each other but also have fun while learning AI and Deep Learning. As our description states: “Deep Learning Adventures is a welcoming group for anyone interested in learning more about deep learning, its foundations, its strengths and weaknesses and ever growing applications that best serve humanity and help those in need throughout the world…”

Deep Learning Adventures Meetup Page

We strive to have a really high user engagement and foster a feeling of collaboration while we get to know each other better as a community. We have fun Deep Learning Trivia games that offer a fun learning experience as well as informal, happy hour sessions where we talk about anything and learn from each other. My friends and co-organizers David Patton, Robert Kraig and myself put together our presentations and host our weekly meetups, while taking a deeper dive into concepts we found interesting or challenging to our community including ourselves. Hyper-parameter tuning is built in our DNA and we explore many Deep Learning models and variations in an effort to increase their accuracy and performance.

Deep Learning Trivia Games

If you are reading this and would like to join us, I have some great news! All our sessions are recorded and available on our YouTube channel here: http://bit.ly/deep-learning-tf

We also have our meetup page: https://www.meetup.com/Deep-Learning-Adventures as well as our GitHub page: https://github.com/georgezoto/Deep-Learning-Adventures and https://github.com/georgezoto/TensorFlow-in-Practice as well as our Slack channel you can join us at 🎉

All our sessions are recorded and posted on our YouTube channel http://bit.ly/deep-learning-tf

By end of June we completed our TensorFlow in Practice specialization and we wanted to celebrate our journey by inviting Laurence Moroney, AI Lead at Google 🎉 I am thankful to Laurence you for being so approachable, humble and open minded. Our goal was to have an open discussion with him as a group, learn what new cool projects he is working on, provide feedback to Laurence on what we found most interesting so far in our TensorFlow journey and get to know each other better. This event was beyond awesome 😀 and is now available on our YouTube channel: https://bit.ly/deep-learning-tf

A fun discussion with Laurence
Cassava plant disease detection using TensorFlow

If you prefer reading my takeaways, I took some notes after watching it for a 2nd time (its addicting I guess and packed with content). and here are my notes from this session. This way you can search for keywords or have a high level idea of the interesting concepts we discussed 😀

  • Deep Learning Adventures
  • Thanks to all meetups for cross-posting this event
  • Get to know our community
  • Deep Learning trivia games
  • TensorFlow overview
  • TensorFlow in Practice Specialization
  • Prerequisites for TensorFlow in Practice Specialization
  • Colab as a Deep Learning development environment
  • Discussion with Laurence
  • Goals and challenges developing the TensorFlow in Practice Specialization
  • 300K AI practitioners vs 30M Software developers
  • Avoid another AI winter
  • Collaboration with Andrew Ng
  • TensorFlow Developer Exam
  • Computer Vision, NLP, Sequence Models
  • 10x increase in number of AI practitioners, Google’s strategy
  • MOOC — Massive Open Online Courses, Coursera, Udacity, Udemy
  • Knowledge sharing over profit making
  • User engagement in this remote setup
  • Program and code live, chat and collaborate with users/community
  • WWDC keynote, great user engagement
  • Other deep learning frameworks, PyTorch and TensorFlow
  • Academia and research, implementation and practice
  • TensorFlow ecosystem, TFX, ML model in production, model and data management, pipelines
  • Enterprise, edge devices, TF Lite, iOS, Android, Embedded systems, TF Serving
  • Other TensorFlow specializations, Data and Deployment specialization
  • Recording now, Advanced TensorFlow specialization, ML Engineering/TFX specialization
  • TensorFlow JS, TensorFlow Lite, Python, model conversion, Kotlin for Android, Swift for iOS, TensorFlow Serving, Federated Learning, Swift for TensorFlow
  • TensorFlow Developer Exam: PyCharm vs Colab
  • Deep Learning VMs from GCP or AWS
  • Setup PyCharm way in advance, prototype your code in Colab, Colab Pro
  • Tokyo University open source project that uses Colab to administer exams
  • Framework to connect AI developers with problems/projects/datasets similar to Kaggle
  • Demand for ML engineers, companies that don’t have ML engineers don’t know how to find them
  • Interview questions, wrong questions looking for the wrong skills brining in the wrong people
  • TensorFlow Certificate Network
  • Consultants for companies looking to introduce ML in their products
  • AI and Software development, collaboration, complement each other
  • AI development as part of Software development, building ML models will be the same as writing SQL or managing remote deployments as a Software development
  • Feedback to Laurence from our Deep Learning Adventures community regarding the TensorFlow in Practice Specialization
  • Course 1: Introduction to AI, computer vision, convolutional neural networks and real world images
  • Single neuron regression problem of fitting a ling through a list of data points while watching the model weight and bias update in realtime
  • TensorBoard in Colab
  • TensorBoard.dev
  • Share the results of your AI experiments with other people
  • Course 2: Convolutional Neural Networks in TensorFlow, overfitting, data augmentation, transfer learning, multi-class classification
  • Classic deep learning networks, L1/L2 regularization, batch normalization
  • Course 3: Natural Language Processing in TensorFlow, sentiment analysis, word embeddings, sequence models and text generation
  • Using RNNs to generate music, how does societal bias in our data affect our models
  • How to combine computer vision with text generation/sequence models, image captioning
  • Attention mechanism, AI explainability
  • Deep Learning for structured data, MPG dataset, classical techniques, random forest
  • Course 4: Sequences, Time Series and Forecasting/Prediction
  • Learning rate optimization, tf.keras.callbacks are very powerful, for consistent results set seed in Python/numpy and TensorFlow
  • Uptrend in time series causing our deep learning models to under-predict (lack of extrapolation feature in deep learning models in general)
  • Plotting observed vs predicted scatterplot of time series values, plateau visible causing our deep learning models to under-predict
  • Neural networks are often not designed to extrapolate, generate data outside their training dataset range
  • Assisting a neural network to extrapolate: experiment with activation functions, tanh, relu, leaky relu, linear
  • Ensemble model: combine statistical models with deep learning models for better forecasting
  • Taking our community feedback to improve future versions of TensorFlow in Practice Specialization, perhaps for TensorFlow 3.0
  • Questions from our community
  • AI winter, more AI developers, Laurence’s experience in UK in 1992, Lisp, Prolog, Windows OS, great papers but AI was out of reach of everyday people
  • Now big data, more processing power, new frameworks, train and use new AI developers to build new apps and fuel brand new industries, avoid another AI winter
  • 3 great revolutions in technology:
  • 1) web and distributed computing,
  • 2) mobile devices, iphone in 2007, new hardware and apps created, multi-billion dollar industries
  • 3) AI and machine learning, the next big thing/app of AI, more developers are needed to build more unique and innovative applications
  • Profit behind TensorFlow, new scenarios, new applications, Android, Google service like Google maps or Google Cloud, App store or Play store
  • Cool applications of AI: Cassava plant disease detection, early blight detection, edge Android AI app, helps farmers get local help, remove plants that weeks later might impact even more plants
  • AI takes root, helping farmers identify diseased plants
  • AI applications in the energy sector, Google Cloud uses AI to optimize energy usage and keep carbon footprint low
  • Google perks: free food in office, built an AI application that helps cafes minimize food waste
  • TensorFlow and Reinforcement Learning, TF agents
  • Being inclusive and accessible, stipends and paid training available for TensorFlow Developer Exam
How to prepare for and pass the TensorFlow Developer Certificate 🎉

We hope you enjoy it and we can’t wait for you to join our community 😀 In our upcoming session we will cover How to prepare for and pass the TensorFlow Developer Certificate 🎉

Best 😀,

George Zoto, David Patton and Robert Kraig