Original article was published by Arslan Ali on Artificial Intelligence on Medium
Complete Introduction About A Career Path In Al, ML, Dl, And DS
In This Tutorial, You Will Learn-
- What is Data Science ?
- What is AI?
- What is ML?
- Types of ML
- What is Deep Learning?
- Machine Learning Process
- Deep Learning Process
- Automate Feature Extraction using DL
- Difference between Machine Learning and Deep Learning
- When to use ML or DL?
- How do I learn machine learning on my own?
- What skills are needed for machine learning jobs?
- What Are The High In-Demand Programming Language Frameworks & Tools For Machine Learning Engineers ?
- 5 Best Machine Learning Books
- What are some algorithms that every machine learning engineer should know?
- How should you start a career in machine learning?
- Which are the best online courses/certifications for AI/machine learning?
- What are some common machine learning interview questions?
- What are the AI and ML starting salaries (WORLDWIDE)?
What Is Data Science?
Data science is the study of data. It involves developing methods of recording, storing, and analyzing data to effectively extract useful information. The goal of data science is to gain insights and knowledge from any type of data — both structured and unstructured.
Data science is related to computer science, but is a separate field. Computer science involves creating programs and algorithms to record and process data, while data science covers any type of data analysis, which may or may not use computers. Data science is more closely related to the mathematics field of Statistics, which includes the collection, organization, analysis, and presentation of data.
Lifecycle of Data Science
Capsulizing the main phases of the Data Science Lifecycle will help us understand how the Data Science process works. The various phases in the Data Science Lifecycle are:
- Data Preparation
- Model Planning
- Model Building
- Communicating Results
Discovery marks the first phase of the lifecycle. When you set sail with your new endeavor,it is important to catch hold of the various requirements and priorities. The ideation involved in this phase needs to have all the specifications along with an outline of the required budget. You need to have an inquisitive mind to make the assessments — in terms of resources, if you have the required manpower, technology, infrastructure and above all time to support your project. In this phase, you need to have a business problem laid out and build an initial hypotheses (IH) to test your plan.
Data preparation is done in this phase. An analytical sandbox is used in this to perform analytics for the entire duration of the project. While you explore, preprocess and condition data, modeling follows suit. To get the data into the sandbox, you will perform ETLT (extract, transform, load and transform).
We make use of R for data cleaning, transformation, and visualization and further spot the outliers and establish a relationship between the variables. Once the data is prepared after cleaning, you can play your cards with exploratory analytics.
In this phase of Model planning, you determine the methods and techniques to pick on the relationships between variables. These relationships set the base for the algorithms that will be implemented in the next phase. Exploratory Data Analytics (EDA) is applied in this phase using various statistical formulas and visualization tools.
Subsequently, we will look into the various models that are required to work out with the Data Science process.
R is the most commonly used tool. The tool comes with a complete set of modeling capabilities. This proves a good environment for building interpretive models.
SQL Analysis Services
SQL Analysis services has the ability to perform in-database analytics using basic predictive models and common data mining functions.
SAS/ACCESS helps you access data from Hadoop. This can be used for creating repeatable and reusable model flow diagrams.
You have now got an overview of the nature of your data and have zeroed in on the algorithms to be used. In the next stage, the algorithm is applied to further build up a model.
This is the Model building phase as you may call it. Here, you will develop datasets for training and testing purposes. You need to understand whether your existing tools will suffice for running the models that you build or if a more robust environment (like fast and parallel processing) is required.
The various tools for model building are SAS Enterprise Miner, WEKA, SPCS Modeler, Matlab, Alpine Miner and Statistica.
In the Operationalize phase, you deliver final reports, briefings, code and technical documents. Moreover, a pilot project may also be implemented in a real-time production environment on a small scale. This helps users get a clear picture of the performance and other related constraints before full deployment.
The Communicate results phase is the conclusion. Here, we evaluate if you have been able to meet your goal the way you had planned in the initial phase. It is in this phase that the key findings pop their heads out. You communicate to the stakeholders in this phase. This phase brings you the result of your project whether it is a success or a failure.
What is AI?
Artificial intelligence is imparting a cognitive ability to a machine. The benchmark for AI is human intelligence regarding reasoning, speech, and vision. This benchmark is far off in the future.
AI is categorized into the following levels:
1. Artificial Narrow Intelligence (ANI)
2. Artificial General Intelligence (AGI)
3. Artificial Super Intelligence (ASI)
Artificial Narrow Intelligence (ANI)
Artificial Narrow intelligence is also referred to as weak AI. These are the AI-based systems trained for performing a single narrow task. These machines have narrow capabilities restricted to certain specific tasks. These are the exclusive AI-based solutions available today. Most of the machines currently work on Narrow AI. Chatbots and virtual assistants are all based on machine intelligence with natural language processing capabilities (NLP). By recognizing human speech these machines are trained to converse with human beings using a natural and a personalized way.
Narrow AI machines are today being used for diagnosing cancer and treating other related illnesses. These machines have precision and cognition similar to that of humans and are able to accord medical professionals an improved accuracy.
– Machines for Speech Recognition.
– Machines that only work on voice commands to perform certain actions, such as Google Assistant, Alexa, Cortana.
Artificial General Intelligence (AGI)
Artificial General Intelligence (AGI) machines exhibit generalized cognitive abilities of human beings. This type of AI is competent enough to work like humans. Currently, AGI is still at a nascent stage. Apart from exhibiting human-like capabilities, these machines will be far better when it comes to intensive calculations. Combining human intelligence along with advantages that machines have over humans, AGI will be able to serve the perfect purpose. However, the technology will take time to arrive as these machines would be based on human brain functionality which is yet to be studied comprehensively. Moreover, the capabilities of AGI systems would be boosted to a level beyond human abilities so that they become competent to process data at an incredible speed.
Some of the properties of AGI machines include:
– Hypothesis testing
Well, some believe that replicating human brain function is likely to bring disastrous consequences.
– AlphaGo, a computer program for playing board game Go. The intelligence of this machine can be extended to be applied to other fields.
Artificial Super Intelligence (ASI)
Artificial Super Intelligence (ASI) commonly known as super-intelligence is a thing of the future. Currently, it is a hypothetical concept believed to surpass human intelligence. ASI technology is likely to remove the distinction between human intelligence and machines. ASI machines will be able to perform extraordinary things that only humans are capable of. They will be competent in decision making, art, and even emotional relationships. In fact, ASI machines will surpass the level of intelligence shown by human beings. However, some believe that humans have not yet achieved proficiency in emotional relationships and effective decision making. Thus, it can be said that in the next few years artificial super intelligence will excel in areas where humans have become unsuccessful.
AI VS Ml VS Dl
What is ML?
Machine learning is the best tool so far to analyze, understand, and identify a pattern in the data. One of the main ideas behind machine learning is that the computer can be trained to automate tasks that would be exhaustive or impossible for a human being. The clear breach from the traditional analysis is that machine learning can take decisions with minimal human intervention.
Machine learning uses data to feed an algorithm that can understand the relationship between the input and the output. When the machine finished learning, it can predict the value or the class of the new data point.
Types of machine learning
- Supervised and semi-supervised learning
- Unsupervised learning
- Reinforcement learning
What is Supervised Learning?
Supervised Learning is the one, where you can consider the learning is guided by a teacher. We have a dataset which acts as a teacher and its role is to train the model or the machine. Once the model gets trained it can start making a prediction or decision when new data is given to it.
What is Unsupervised Learning?
The model learns through observation and finds structures in the data. Once the model is given a dataset, it automatically finds patterns and relationships in the dataset by creating clusters in it. What it cannot do is add labels to the cluster, like it cannot say this a group of apples or mangoes, but it will separate all the apples from mangoes.
Suppose we presented images of apples, bananas and mangoes to the model, so what it does, based on some patterns and relationships it creates clusters and divides the dataset into those clusters. Now if a new data is fed to the model, it adds it to one of the created clusters.
What is Reinforcement Learning?
It is the ability of an agent to interact with the environment and find out what is the best outcome. It follows the concept of hit and trial method. The agent is rewarded or penalized with a point for a correct or a wrong answer, and on the basis of the positive reward points gained the model trains itself. And again once trained it gets ready to predict the new data presented to it.
Types of ML
What is Deep Learning?
Deep learning is computer software that mimics the network of neurons in a brain. It is a subset of machine learning and is called deep learning because it makes use of deep neural networks. The machine uses different layers to learn from the data. The depth of the model is represented by the number of layers in the model. Deep learning is the new state of art in terms of AI. In deep learning, the learning phase is done through a neural network. A neural network is an architecture where the layers are stacked on top of each other
Machine Learning Process
Imagine you are meant to build a program that recognizes objects. To train the model, you will use a classifier. A classifier uses the features of an object to try identifying the class it belongs to.
In the example, the classifier will be trained to detect if the image is a:
The four objects above are the class the classifier has to recognize. To construct a classifier, you need to have some data as input and assigns a label to it. The algorithm will take these data, find a pattern, and then classify it in the corresponding class.
This task is called supervised learning. In supervised learning, the training data you feed to the algorithm includes a label.
Training an algorithm requires to follow a few standard steps:
- Collect the data
- Train the classifier
- Make predictions
The first step is necessary, choosing the right data will make the algorithm success or a failure. The data you choose to train the model is called a feature. In the object example, the features are the pixels of the images.
Each image is a row in the data while each pixel is a column. If your image is a 28×28 size, the dataset contains 784 columns (28×28). In the picture below, each picture has been transformed into a feature vector. The label tells the computer what object is in the image.
The objective is to use these training data to classify the type of object. The first step consists of creating the feature columns. Then, the second step involves choosing an algorithm to train the model. When the training is done, the model will predict what picture corresponds to what object.
After that, it is easy to use the model to predict new images. For each new image feeds into the model, the machine will predict the class it belongs to. For example, an entirely new image without a label is going through the model. For a human being, it is trivial to visualize the image as a car. The machine uses its previous knowledge to predict as well the image is a car.
Deep Learning Process
In deep learning, the learning phase is done through a neural network. A neural network is an architecture where the layers are stacked on top of each other.
Consider the same image example above. The training set would be fed to a neural network
Each input goes into a neuron and is multiplied by a weight. The result of the multiplication flows to the next layer and becomes the input. This process is repeated for each layer of the network. The final layer is named the output layer; it provides an actual value for the regression task and a probability of each class for the classification task. The neural network uses a mathematical algorithm to update the weights of all the neurons. The neural network is fully trained when the value of the weights gives an output close to reality. For instance, a well-trained neural network can recognize the object on a picture with higher accuracy than the traditional neural net.
Automate Feature Extraction using DL
A dataset can contain a dozen to hundreds of features. The system will learn from the relevance of these features. However, not all features are meaningful for the algorithm. A crucial part of machine learning is to find a relevant set of features to make the system learns something.
One way to perform this part in machine learning is to use feature extraction. Feature extraction combines existing features to create a more relevant set of features. It can be done with PCA, T-SNE or any other dimensionality reduction algorithms.
For example, an image processing, the practitioner needs to extract the feature manually in the image like the eyes, the nose, lips and so on. Those extracted features are feed to the classification model.
Deep learning solves this issue, especially for a convolutional neural network. The first layer of a neural network will learn small details from the picture; the next layers will combine the previous knowledge to make more complex information. In the convolutional neural network, the feature extraction is done with the use of the filter. The network applies a filter to the picture to see if there is a match, i.e., the shape of the feature is identical to a part of the image. If there is a match, the network will use this filter. The process of feature extraction is therefore done automatically.
Difference between Machine Learning and Deep Learning
Machine Learning VS Deep Learning
When to use ML or DL?
In the table below, we summarize the difference between machine learning and deep learning.
Machine learningDeep learningTraining datasetSmallLargeChoose featuresYesNoNumber of algorithmsManyFewTraining timeShortLong
With machine learning, you need fewer data to train the algorithm than deep learning. Deep learning requires an extensive and diverse set of data to identify the underlying structure. Besides, machine learning provides a faster-trained model. Most advanced deep learning architecture can take days to a week to train. The advantage of deep learning over machine learning is it is highly accurate. You do not need to understand what features are the best representation of the data; the neural network learned how to select critical features. In machine learning, you need to choose for yourself what features to include in the model.
Artificial intelligence is imparting a cognitive ability to a machine. Early AI systems used pattern matching and expert systems.
The idea behind machine learning is that the machine can learn without human intervention. The machine needs to find a way to learn how to solve a task given the data.
Deep learning is a breakthrough in the field of artificial intelligence. When there is enough data to train on, deep learning achieves impressive results, especially for image recognition and text translation. The main reason is the feature extraction is done automatically in the different layers of the network.
Some Important Questions
1) How do I learn machine learning on my own?
Mastering machine learning enables you to become a Data Scientist, AI/Machine Learning Engineer. It also helps to use ML algorithms for development or add skills to your business analysis toolbox.
As a self-starter, you can follow these steps to excel.
Develop a strong foundation in Statistics, programming language like R, and a little Mathematics.
Read books, blogs, articles, and watch YouTube to understand the essential theory behind Machine learning.
Build practical skills the industry demands by using machine learning study packages and practice the essential topics.
Take hand-on projects into interesting domains, dive deeper into projects and build a strong portfolio along the way.
2) What skills are needed for machine learning jobs?
As a job seeker, you should be a potential prospect for the employer. You must have a deeper understanding of Algorithms, Applied Mathematics, Statistics, Probability, Programming languages, Analytical and problem-solving skills.
Here is a list of key skill-sets you must have.
You must have a clear understanding of fundamentals like –
Computability and complexity, and
Though you can start with one language, at one stage in your career, you should know all the languages. It is recommended to learn –
👉Probability and Statistics
Many ML algorithms are extensions of Statistics. Knowledge in probability and statistics makes your ML projects easy. You should know
Probability and its techniques
Measures, distributions, and analysis methods
👉Data modeling and evaluation
To apply standard algorithms, you should learn to choose an –
Appropriate accuracy/error measure
👉Applying machine learning algorithms and libraries
Applying machine learning algorithms is essential. For that, you should have a strong understanding of –
Partial differential equations
👉Software engineering and system design
At the end of the day, your deliverable is software. So, you must understand –
How different components fit together
Build appropriate interfaces
Avoid bottleneck with careful system design
Scale algorithms to an ever-increasing volume of data
Advanced signal processing techniques
3) What are some algorithms that every machine learning engineer should know?
Learning algorithms helps you solve real-world problems without or with minimal human intervention. Some of the popular algorithms you must know include –
Artificial Neural Networks
K Means Clustering Algorithm
Naïve Bayes Classifier Algorithm
Support Vector Machine Algorithm
4)What Are The High In-Demand Programming Language Frameworks & Tools For Machine Learning Engineers ?
If you’re wondering which of the growing suite of programming language libraries and tools are a good choice for implementing machine-learning models then help is at hand.
5)Top 5 Best Machine Learning Books
Author — Andriy Burkov
Latest Edition — First
Publisher — Andriy Burkov
Format — ebook (Leanpub)/Hardcover/Paperback
Is it possible to explain various machine learning topics in a mere 100 pages? The Hundred-Page Machine Learning Book by Andriy Burkov is an effort to realize the same. Written in an easy-to-comprehend manner, the machine learning book is endorsed by reputed thought leaders to the likes of the Director of Research at Google, Peter Norvig and Sujeet Varakhedi, Head of Engineering at eBay.
Post a thorough reading of the book, you will be able to build and appreciate complex AI systems, clear an ML-based interview, and even start your very own ml-based business. The book, however, is not meant for absolute machine learning beginners. If you’re looking for something more fundamental look somewhere else.
- Anatomy of a learning algorithm
- Fundamental algorithms
- Neural networks and deep learning
- Other forms of learning
- Supervised learning and unsupervised learning
You can buy this book here.
Author — Toby Segaran
Latest Edition — First
Publisher — O’Reilly Media
Format — Kindle/Paperback
Regarded among the best books to begin understanding machine learning, the Programming Collective Intelligence by Toby Segaran was written way before, in 2007, data science and machine learning reached its present status of top career avenues. The book makes use of Python as the vehicle of delivering the knowledge to its readers.
The Programming Collective Intelligence is less of an introduction to machine learning and more of a guide for implementing ml. The book details on creating efficient ml algorithms for gathering data from applications, creating programs for accessing data from websites, and inferring the gathered data. Each chapter features exercises for extending the stated algorithms and further improve their efficiency and effectiveness.
- Bayesian filtering
- Collaborative filtering techniques
- Evolving intelligence for problem-solving
- Methods for detecting groups or patterns
- Non-negative matrix factorization
- Search engine algorithms
- Support vector machines
- Ways to make predictions
You can buy this book here.
Author — Drew Conway and John Myles White
Latest Edition — First
Publisher — O’Reilly Media
Format — Kindle/Paperback
The Machine Learning for Hackers book is meant for the experienced programmer interested in crunching data. Here, the word hackers refer to adroit mathematicians. As most of the book is based on data analysis in R, it is an excellent option for those with a good knowledge of R. The book also details using advanced R in data wrangling.
Perhaps the most important highlight of the Machine Learning for Hackers book is the inclusion of apposite case studies highlighting the importance of using machine learning algorithms. Rather than delving deeper into the mathematical theory of machine learning, the book explains numerous real-life examples to make learning ml easier and faster.
- Developing a naïve Bayesian classifier
- Linear regression
- Optimization techniques
- Using R for querying data
You can buy this book here.
Author — Tom M. Mitchell
Latest Edition — First
Publisher — McGraw Hill Education
Format — Paperback
Machine Learning by Tom M. Mitchell is a fitting book for getting started with machine learning. It offers a comprehensive overview of machine learning theorems with pseudocode summaries of the respective algorithms. The Machine Learning book is full of examples and case studies to ease a reader’s effort for learning and grasping ml algorithms.
If you wish to start your career in machine learning, then this book is a must-have. Thanks to a well-explained narrative, a thorough explanation of ml basics, and project-oriented homework assignments, the book on machine learning is a suitable candidate to be included in any machine learning course or program.
- Genetic algorithms
- Inductive logic programming
- Introduction to primary approaches to machine learning
- Machine learning concepts and techniques
- Re-enforcement learning
You can buy this book here.
Author — Trevor Hastie, Robert Tibshirani, and Jerome Friedman
Latest Edition — Second
Publisher — Springer
Format — Hardcover/Kindle
If you like statistics and want to learn machine learning from the perspective of stats then The Elements of Statistical Learning is the book that you must read. The machine learning book emphasizes mathematical derivations for defining the underlying logic of an ml algorithm. Before picking up this book, ensure that you have at least a basic understanding of linear algebra.
The concepts explained in The Elements of Statistical Learning book aren’t beginner-friendly. Hence, you might find it complex to digest. If you still, however, want to learn them then you can check out the An Introduction to Statistical Learning book. It explains the same concepts but in a beginner-friendly way.
- Ensemble learning
- High-dimensional problems
- Linear methods for classification and regression
- Model inference and averaging
- Neural networks
- Random forests
- Supervised and unsupervised learning
You can buy this book here.
6) How should you start a career in machine learning?
Jumpstart your career in machine learning by following these set rules.
Gain a strong basic in Algebra, Calculus, and Statistics
Learn programming languages — R, Python, and Java
Attend to the exploratory project(s)
Create supervised and unsupervised models
Learn big data technologies
Explore deep learning models
Take online courses or certifications
Start participating in Kaggle competitions
7) Which are the best online courses/certifications for AI/machine learning?
The top three best online certifications are presented here.
1) Artificial Intelligence Engineer (AIE) — Artificial Intelligence Board of America (ARTiBA): In this certification program, you will learn the concepts of ML, supervised and unsupervised learning, Natural Language Processing, Cognitive computing, Reinforced Learning, and Deep Learning. With its global reach, it is the industry standard in professional credibility. You can create your career niche in AI functions across industries and countries.
2) Professional Certificate Program in Machine Learning & Artificial Intelligence — MIT Professional Education: This certificate program enables you to get acquainted with the latest advancements and technical approaches in AI. You will get well-versed in Algorithmics, Natural language Processing, Predictive Analytics, and, Deep Learning.
3) Deep learning specialization — Deeplearning.ai: It is a five-course specialization that helps you to get specialized in Deep Learning fundamentals and its applications. You will learn about neural networks, deep learning, convolutional learning, sequence models, and structuring ML projects.
8) What are some common machine learning interview questions?
It is necessary to understand the AI and ML concepts to clear your interview successfully. Some of the common interview questions are listed below.
What is the difference between machine learning and data mining?
When and why do a model exhibit poor performance?
What is a cross-validation technique?
In which algorithm techniques you are best at?
What is the function of unsupervised learning?
Where do you use pattern recognition?
What is model selection?
What is PCA, KPCA, ICA?
Explain ensemble learning
Which technique of machine learning do you use more, why?
How do you screen for outliers?
9) What are the AI and ML starting salaries (WORLDWIDE)?
According to PayScale, an entry-level ML engineer with less than one-year experience earns an average total compensation of $93,678. A machine learning specialist may earn up to 1 million.