AI and NLP for dummies

Original article was published on Artificial Intelligence on Medium

We live in a world where almost everyday we hear the word Artificial Intelligence, or AI for short. Since the invention of computers many years ago, people have tried to build machines that work like humans and think like humans. Great Sci-Fi movies have helped our imagination grow and at times have provided inspiration to some of the research work too.

1. What is Artificial Intelligence ?

People use different terms like Machine Learning (ML), Artificial Intelligence (AI), Augmented Intelligence etc. But what actually is AI? For any system to be called AI (in the current definition), it needs to have three main components:

  • Uses machine learning
  • Is Predictive
  • Is Adaptive

Let’s look at these three components with an example of an AI system: An Autonomous Vehicle (a car that drives automatically)

1.1 Uses machine learning

The AI system in the autonomous vehicle comprises high resolution cameras and radars that sense the surroundings. The input from these sensors are processed by machine learning models to do various tasks, such as accelerate, decelerate, turning and manoeuvring etc. Different machine learning models may be used to do different tasks. So, what exactly is a machine learning model? As the name suggests, machine learning means telling the computer (or machine) to learn from data and perform an intended task. It is different from traditional software which are not designed to learn from data. Traditional software only processes data to provide information. There are many machine learning techniques such as neural networks, support vector machines, decision trees etc. These techniques can be used to build machine learning models to do various tasks such as image recognition, speed controls etc.

1.2 Predictive

An AI system is predictive, meaning it predicts an outcome from a given set of inputs. A common misunderstanding is prediction refers to guessing future outcomes. If we use the term forecasting, which is essentially a subset of prediction, then that does talk about predicting an outcome in the future, such as: tomorrow’s stock market price, weather in the next three days, etc. But prediction can also mean processing credit card transaction information and predicting whether the transaction was fraudulent or not. Since the transaction has already happened, the outcome predicted will not occur in the future, but rather happened in the past. Some examples of the prediction tasks undertaken by autonomous vehicles are: to predict the speed the vehicle has to have next (and if it will need to apply the breaks), to predict what type of object is in front of the car (to avoid collisions), etc. In layman’s terms, one can say prediction is just another guess, but with data.

1.3 Adaptive

The last aspect of an AI system, to truly call it an AI system (but not true AI), is it has to be adaptive. This means, despite the initial training the AI system had, it will be able to adjust itself to be better at predicting as more and more data becomes available. In the autonomous vehicle example, while the vehicle is in operation, it collects data (through sensors) from the results of the predictive actions it has taken and these data are then used to adjust the possible outcome of a similar scenario in the future. Thus, as time goes by, the AI system becomes better and better.

1.4 What AI can do and cannot do ?

Today, AI can do various tasks such as;

  1. Play games such as chess, go etc. just like a grandmaster.
  2. Process and understand images just like a human
  3. Translate texts as well as humans
  4. Drive a vehicle just like a human

But today’s AI cannot do the following;

Discern the exact meaning of ambiguous sentences such as:

That parcel did not fit into my bag, it was too big.

In this example, if we ask the machine to tell us what “it” refers to, whether it is Parcel or Bag, the current AI systems cannot yet answer the questions like this.

1.5 Turing Test

Although the Autonomous Vehicle example meets the definition of AI, it is not a True AI.

But how do we know if a system is True AI or not (also known as strong AI or General AI) ? In 1950 the great mathematician Alan Turing introduced a simple test that involves a human player C trying to determine whether player A or player B is a machine and the other is a human, solely depending on the responses player C receives to the same question presented to both player A and player B. Player A and B are invisible to Player C. If player C cannot successfully distinguish the machine from the human, then that machine has achieved true AI status.

2. What is Natural Language Processing

“The biggest breakthrough in AI will be when computer can read and understand information like humans do.” ~ Bill Gates 2017.

Making the computers process human speaking languages is known as natural language processing. Why do we call it natural language? Aren’t all languages natural anyway? The answer is not quite. Since the invention of computers we have used another type of language known as programming language to give instructions to computers. These range from assembly language to object oriented programming languages to scripting languages. Contrary to that, natural languages are like English, Spanish, Chinese, Korean, Telugu etc.

Let’s take few examples of Natural Language Processing

2.1 Sentiment Analysis

Sentiment analysis is used to systematically identify, extract, quantify and study the affective states and subjective information of the things mentioned like products, services etc. Sentiment analysis helps businesses understand people’s emotions towards a certain thing (services, goods etc), and thus better understand that thing. This helps with making necessary adjustments to goods or services to match, the demand.

As shown in the above example, sentiment analysis means from the given texts a computer system can understand and extract the subjective information and quantify it to be Positive, Negative or Neutral. Sentiment analysis is challenging when it comes to processing larger texts such as documents or paragraphs, as they may have polarising expressions in different parts of the text. Hence generating a single sentiment value to represent an entire document may not be accurate. The other scenario where sentiment analysis usually underperforms is to identify subjective information (emotions) associated with different things mentioned in a single text.

“I really like the almond white chocolate drink from ABC cafe, but the cafe is always crowded which is a sad thing”

Now in this example, a person expresses two feelings, a positive feeling from the drink and negative feeling for the cafe. The new concept of Aspect Based Sentiment Analysis is used to identify such polarising sentiment associated with different things (aspects) in a text, but there is a long way to go.

2.2 Named Entity Recognition

Identifying named things such as persons, locations, countries, products, companies from an unstructured text is known as named entity recognition.

A named entity recognition model will help us identify first occurrences of “Apple” as a company, “Tim Cook” as a person, second occurrences of “apple” as a fruit, “Marina Bay Sands” as a location and “Singapore” as a country. An NER model can only identify predefined classes of entities and it is possible to train a model to identify business requirement specific classes such as Food Items, Brands, Needs/Wants etc.

As to keep this article short, below are some other systems that use NLP.

  1. Text translation from one language to another
  2. Transcribe — also known as voice/speech to text
  3. Text to speech
  4. Chatbots
  5. Automated personal assistants

Hope you enjoy reading this article. This article is an extracted version of Ai Palette’s internal AI Induction for new hires. Keep an eye on the next article on Knowledge Graphs.