ABCs of Natural language processing

Original article can be found here (source): Deep Learning on Medium

ABCs of Natural language processing

Ever wonder how Alexa, Siri work to assist you?

……. these virtual assistants leverage the power of AI and natural language processing. After converting voice data to text data, every time Alexa or Siri makes a mistake when responding to your request, it uses the data it receives based on how it responded to the original query to improve the next time.

What is Natural Language Processing?

Natural Language Processing (NLP), as the name implies, deals with processing the human languages. It deals with building computational algorithms to analyze and represent human language

At the core, NLP utilizes algorithms to extract meaning associated with every sentence and collect the essential data from them. The purpose of NLP is to read and understand human languages in such a way that valuable information can be extracted from unstructured text data.

Human languages are a bit complex to represent, understand and use linguistic (contextual, visual knowledge). It is difficult to apply a set of software program rules on the text data to get the pattern, so we leverage the Machine learning/AI approach as per the specific application type.

NLP gained popularity after the advancement in deep learning. Deep learning Model provides a way to handle and process large unstructured data

Application of NLP?

NLP algorithms have a variety of uses. project’s developers can use NLP algorithms for:

Text Summarization — make a short summary of a text block to extract the most important and central ideas.

Sentiment Analysis — the sentiment of a string of text or tweet or a customer review, from negative to neutral to positive.

Chatbot -Auto reply — Creating personalize chatbot for query and question answering to assist.

What are different types of NLP?

On the basis of applications, NLP can be divided into two broad categories–

1. Natural language understanding (NLU) –

NLU interprets the meaning of the language and comprehension of text. humans who speak the same language to understand each other, although mispronunciations, choice of vocabulary or phrasings may complicate this. NLU is responsible for this task of distinguishing what is meant by applying a range of processes such as text categorization, content analysis, and sentiment analysis, which enables the machine to handle different inputs.

2. Natural language generation (NLG)

NLG generates narratives that describe or explain input structured data like human ways. Some of the application of it is e-commerce product descriptions, written analysis for business intelligence dashboards, personalized customer communications via email and in-app messaging

Libraries available for NLP in Python –

Below are the very popular library available, A couple of these libraries already have trained models to directly use –


· Spacy

· Gensim

· Stanford NLP

· TextBolb

Recent Advancement — NLP Deep learning –

Why deep learning for NLP

For a long time, the majority of methods used to study NLP problems employed shallow machine learning models that are time-consuming and pick hand-crafted features. This lead to problems such as the curse of dimensionality since linguistic information was represented with sparse representations (high-dimensional features). However, the recent popularity and success of word embeddings (low dimensional, distributed feature representations) have open a path for deep learning models and do a better performance.

CNN, RNNs are popularly available methods

A Convolutional Neural Networks (CNN) is a type feedforward network, but with more layers, and where the forward connections have been manipulated, or convoluted, to achieve certain properties.