Teaching your AI to Read

Original article was published by Supreet HVR on Artificial Intelligence on Medium


“In the lives of individuals and the societies, language is a factor of greater importance than any other”. A wise assertion by Swiss philosopher Ferdinand de Saussure. He proposed that Signs act as the basic unit of the language. His theories formed the basis for the discipline of Semiotics. Semiotics deals with the discovery of laws of sign formation and techniques of extracting their meaning. This is what we aim to do with Natural Language Understanding (NLU).

Image courtesy Pexels.com

Language is the basic and most important communication form for humans. Today most of the data that is generated is in the form of Natural language. Analysis of Natural language and extraction of understanding is one of the key capabilities for AI. So that AI could seamlessly integrate with applications and society.

This means we need to teach our AI to read.

High-level Approach

With this motivation let us look at the High-level approach we can take to building NLU ( Natural Language Understanding) for AI systems.

(Concept adapted from) Intelligent Systems Architecture, Design and Control by Alexander M. Meystel and James S. Albus

The knowledge can be defined as a pattern that is interesting and certain enough. Knowledge in the context is the understanding of the natural language.

Our approach to NLU will be to analyze the text. Generate the descriptive elements and tags about this text. Then to organize these elements of information to extract the meaning or understanding.

Process of Natural Language Analysis

The process of Natural language analysis is not different from how a kid would learn to read. They would first learn to identify some common words. Then they put these words into simple sentences. Further in to complete conversations.

The process of Natural language analysis broadly has the following parts,

Text Analysis

The first step of analyzing any text or information would involve identifying the different documents, sentences, and words.

Image courtesy Pexels.com

Document Triage

The documents (one or more) must first be prepared for analysis. This is the purpose of the Document triage phase. The document is first parsed to identify the language. This is called Language Identification. The next step in this phase is to extract all the text information from the document. This means omitting additional elements like Images, Tables, Links, and any formatting. This step is called Text Sectioning. Now the text is available to be processed further.

Text Segmentation

Sentence Segmentation

This is a process of identifying the sentence boundaries and breaking down the text into individual sentences.

Word Segmentation

This is a process of identifying the word boundaries and breaking them down into individual words. This can be used effectively in various algorithms. This process is also often referred to as Tokenization. Independent words are called the Tokens.

Lexical Analysis

The next step of the Natural language processing is to map or exact words from variations of the same base word. Consider we have a set of words {Connect, Connected, Connecting, Connection}. We can easily make out that all these words are dealing with the base word Connect. AI also needs to be empowered with this ability to link variations in the words to a base word.

Image courtesy Pixabay.com

Stemming

Stemming is a process undertaken to reduce the variations in the words to a single base word. This could be done by operations like slicing. Additional characters from the given variation are removed to reveal the base word. This process could sometimes lead to a situation where the base word may not be a real word in the language.

Lemmatization

Lemmatization is a technique where variations of the word are mapped to the base word or a “Lemma”. It is ensured that the base word is a real word and part of the language. Consider a set of words {selling, sale, sold}, here all 3 words should be mapped to the same base word Sell. This cannot be achieved by slicing. Through the technique of Lemmatization, the words can be mapped to the same base word.

Syntactic Analysis

The next step is to parse the structure of the texts using the grammar of the language.

Image courtesy Pixabay.com

Parts of Speech Tagging

The POS -Tagging is a process where the text is analyzed. Various parts of a text are identified and tagged with the relevant part of the speech. This could be Nouns, Verbs, Adjectives, Adverbs, etc. this useful in extracting the meaning from the text.

Syntax Tree

One of the standard ways of representing the text is in the form of a syntax tree. A syntax tree is an upside-down tree. This allows us to categorize the given text according to different rules of grammar. The syntax trees are useful in understanding what is the sentence referring to? What actions going on? And so on.

Image Simplified syntax tree for Reinhart 1983 by wikipedia.org

Name Entity Recognition

Named entity recognition is a part of information extraction. It identifies various named entities from the given text. Entities could be Person, Organization, Place, or Number based on the domain. This is one of the key techniques used in identifying the contexts of the text being processed.

Semantic Analysis

The main goal of the NLU is to extract the meaning of the text being processed.

Ontology Language

Ontologies are schema or structure of various objects, actions, and relationships in a given domain. Ontologies can classify objects in the text into the structure of the known domain. Enabling extraction of the meaning.

Take this sentence for example, “Roger gave a great hit”.

While using the ontologies of Baseball means that Roger has hit the ball very well. If we switch to ontologies of Movies, it means that Roger’s movie is viewed by many and has become popular.

Image courtesy Florian Thiery / CC BY-SA (https://creativecommons.org/licenses/by-sa/4.0)

Knowledge Graphs

In Knowledge graphs entities, actions, and relations in the text are mapped into a graphical structure. This allows the AI to process and navigate the graph to draw inference about the meaning of the text being processed.

Consider a sentence “Not charging your mobile phone, will lead to the phone switching off”. “Overcharging may lead to battery failure.” Sample knowledge graph generated form this sentence could be as shown below.

Now if we AI receives another sentence like “Roger is overcharging his phone”. AI can identify the risk and alert Roger. It can also be able to recommend a safer alternative by traversing the graph.

Implementation Approaches

Some possible approaches to implementation of the Natural language processing solutions,

Machine Learning Libraries

Use Machine learning and Text analysis libraries. We analyze, classify, predict, and extract information from the text using these libraries. These are sometimes also referred to as statistical techniques. Python is by far the most popular language for these implementations.

NLTK (Natural Language Tool Kit), spaCy, scikit-learn are some of the popular libraries.

For this approach, a good place to start would be, Natural Language Processing with Python Book by Edward Loper, Ewan Klein, and Steven Bird.

Image courtesy Pexels.com

Deep Learning

Use the deep learning algorithms to analyze and extract information from Natural language.

For this approach, a good place to start would be, Natural Language Processing with PyTorch: Build Intelligent Language Applications Using Deep Learning Book by Brian McMahan and Delip Rao.

TensorFlow (www.tensorflow.org) is also a platform worth considering.

Plug-n-Play

It is also possible to use Plug-n-Play components that provide various NLU solutions. This approach is especially useful when you want to build solutions NLU quickly. The other approaches will take considerable time and effort.

Microsoft Azure (https://azure.microsoft.com/en-us/services/cognitive-services/language-understanding-intelligent-service/) is a very reliable cloud solution.

One can also use Google Cloud (https://cloud.google.com/natural-language) or AWS (https://aws.amazon.com/comprehend/) based on preference and use case.

Image courtesy Pexels.com

Challenges with Natural Language Understanding

Natural language is not straight forward. There are a lot of challenges to understand the context, intent, and meaning of a given natural text.

There are several know ambiguities that one must address such as,

· Lexical Ambiguity (A single word has multiple meanings even the domain and context)

· Syntactic Ambiguity (A sentence can be interpreted in more than one way)

· Referential Ambiguity (A reference could be for more than one object)

Considering languages other than English only compounds the challenges further.

Concluding remarks

Jacques Derrida contributed to the work on Semiotics. He said even the simplest text is full of “aporia” meaning puzzles. He argued that the meaning of what is said depends on what is said after, the meaning also depends on what is not said. So this means text is always incomplete in the meaning. This interesting argument shows the complexities in the field of Natural language understanding.

Image courtesy Pixabay.com

There has been great progress made in the field of AI and Natural language understanding. Today we have systems that surprise us. The truing test is being passed often and by several AI agents.

After all, humans do not understand each other all the time. Having considered the nature of challenges. The progress in the field of Natural language understanding. I would like to be positive about what we can achieve with AI in this field.