The Mighty GPT-3

Original article was published by Jaime Zornoza on Artificial Intelligence on Medium


Welcome my lovely reader! The goal of this post is to briefly explain what GPT-3 is, and then showcase a wide range of examples to highlight its potential as a general purpose Natural Language Processing Artificial Intelligence.

Some of these examples are truly amazing, with a few coming close to being scary, so I encourage you to keep reading if you really want to enjoy yourself and see the possible reach of Artificial Intelligence.

However, before exploring what GPT-3 is all about, I want you to take a look at the following video from one of the Iron Man movies, where Tony Stark openly and broadly speaks with the virtual assistant he has created: Jarvis.

In this short video, you can see that Jarvis is able to interact with Stark on just about any topic, have a perfectly human-like conversation with him, answer questions, and even help him with his research.

Up to know AIs have excelled in very narrow domain specific tasks: fooling humans on the Turing Test, performing sentiment analysis, beating the worlds champions at chess, or summarising texts.

Despite of how impressive these achievements might be, the software’s that carried out each task were built specifically for that task, with only that goal in mind, and were fine tuned to provide good results under certain specific conditions.

They were very far from what we call an Artificial General Intelligence (AGI), which is an Artificial Intelligence that can perform reasonably well on a wide range of tasks: something more or less like Jarvis from Iron Man.

GPT 3 is the probably the closest thing to this we’ve ever seen, and despite not being an AGI, it performs surprisingly well on a large variety of Natural Language Processing (NLP) related tasks.

Tweet by Andriy Burkov, author of The 100 Page Machine Learning Book.

After this brief introduction, lets really see what GPT-3 is all about.

What is GPT-3?

In July of 2020 OpenAI, the Artificial Intelligence company initially funded by Elon Musk amongst others, published the paper “Language Models are Few-Shot Learners”, which meant an upgrade from the previous year’s GPT-2 family of models, introducing the new, hyped, and more powerful one: GPT-3.

These family of models are Neural Language models, which means that they are trained trying to predict the missing or next word in a sentence, like an autocomplete from your phone or from the Google Search bar.

Language models can be trained in an unsupervised manner (without feeding the models a labelled set of data with the sentence, and the actual word that is missing) from a text corpus, by removing a word of a sentence from the rest of the text and trying to predict it using whats left.

You can find some awesome visualisations of how this works for the specific case of GPT-3 on the following link:

After this pre-training, the resulting NLP models are usually fine tuned for the specific task at hand (like text-translation for example), which means that labelled data for the specific task has to be collected, and that there always needs to be a posterior, second training.

With GPT-3, this need of posterior training for an specific task is eliminated, becoming unnecessary many times, while still achieving amazing performance on a wide variety of tasks with only this pre-training and no further fine tuning.

Alright, now that we know a little bit about GPT-3, lets see some examples of different uses of GPT-3 that will make your jaw drop to the ground.

Examples: GPT-3 in action

GPT-3 Coding

We will first see some examples gathered from the Twitter community about GPT-3 generating code. The following tweets from Sharif Shameen have gone viral, after showing how by giving the GPT-3 API some simple instructions, it can generate code according to those instructions.

Also, take a look at how GPT-3 can generate the code to build a Machine Learning model with just a few instructions and the description of the data set.

After seeing this, you might be asking yourself:

“Will GPT-3 replace programmers?”

It is too early to say, as these new models just came out, and they have not been excessively tried to build more complex applications, however, for relatively simple tasks like the ones above, they seem to work pretty well.

GPT-3 writing emails from a few bullet points

In the following example, GPT-3 is given an input email and a few bullet points to answer to such email, and writes a full, elegant response text.

With examples like this we can see just how versatile GPT-3 is: from coding to generating emails from bullet-points. Will it soon become a part of our day to day life, being used to simplify and automatise processes?

Conversation with GPT-3 using reasoning questions

Lets see another example. The following is a conversation of a user with GPT-3, after it gave it only the instruction to answer all the questions it was asked in an accurate manner. See how the flow of the conversation is smooth and completely indistinguishably from that of a human.