Let the machine write next..!

Original article was published on Deep Learning on Medium

6. Inference

For the inference part, we will use the best weights from the checkpoint. So let’s predict and see it.

I. LSTM model

We will take samples from random categorical distribution from logits. And convert them into words.

we will initialize our model with batch_size=1 and restore the trained
weights from the checkpoint.

Here, we are also passing hidden states from LSTM to maintain the state information according to the sequence of words. Let’s see some results!

# Prediction
seed = 'In the history of spaceflight , only one nation'
temperature = 1.0
temp = predicttion_fun_using_disb(seed, stacked_model, temperature, 20)
print(seed + ' ' + temp)
# input : 'In the history of spaceflight , only one nation'
# Output:
In the history of spaceflight , only one nation accord happy rubins
attainable accelerates writing violating contract topography finder
child walking concerns whitish barren equuleus generates undertakin
gs parasites females
seed = 'Next to actually sending another spacecraft there'
temperature = 0.5
temp = predicttion_fun_using_disb(seed, stacked_model, temperature, 50)
print(seed + ' ' + temp)
# input: 'Next to actually sending another spacecraft there'
# output: Next to actually sending another spacecraft there occasional noam famines scenic magnification mandated elliptic glow satellites scribbled permit tre warmer phases amateurs quotes meantime hooks loss lambda negatively appendages teams entailing terminate refrangibilities pella tarantula andrian selection ark dataset utterly mx enabled robot surya bands hangar sarah distinguishing etnos thunder jhuapl analogous interfering mathematicians tuesday writings uniting
  • From the above results, we can see that when giving some input seed words our model is generating sentences, though all generated words are not related to the meaning of the sentence. But the model is understanding about positions of verbs and adjectives.

II. Attention

Let’s see the output.

# Predictionseed = 'In the history of spaceflight , only one nation'
temperature = 1.0
final_string = predict_function_attention(seed, 20, temperature)
print(final_string)
# input : 'In the history of spaceflight , only one nation'
# output: In the history of spaceflight , only one nation infall deadly subsy
stems atkinson moderate himalayan lance abrahamic improve construct
ing scistarter streak insufficient viking east deistic earthguide s
hellfish especially repurposed megalodon pressures vac definition i
nstantaneously komsomol zingers astrobee effect propane cancelled p
rematurely icesheetmike sandstones application volcanoes cephalopod
s misty send combes

From the result, it can be said that it can understand little bit arrangement of verbs, nouns, adjectives.

But what about meaningful sentences? Well, our GPT-2 will take care of it.

III. GPT-2

# restore from drivegpt2.copy_checkpoint_from_gdrive(run_name='run1')
sess = gpt2.start_tf_sess()
gpt2.load_gpt2(sess, run_name='run1')
# Generateprefix = 'In the history of spaceflight , only one nation'
gpt2.generate(sess, length=50, temperature=0.5, prefix=prefix,
nsamples=5, batch_size=5 )
# input : 'In the history of spaceflight , only one nation'
# output :
In the history of spaceflight , only one nation has achieved the fe
at: the Soviet Union. But the Soviet Union did it. It was the first
nation to launch a rocket from a launch pad. It was the first natio
n to launch a rocket from a launch pad. It was the first to launch
====================
In the history of spaceflight , only one nation has managed to colo
nize another planet: the Soviet Union. The Soviets launched their f
irst lunar lander in November of 1972, and by the time it landed, t
he Moon was still in lunar orbit. They also made several attempts t
o colonize the Moon
====================
In the history of spaceflight , only one nation has held the title
of sole superpower – the United States. The Soviet Union, who were
the first nation to achieve space flight, was the only nation to ac
hieve space flight. And although they were technically still in the
process of achieving space flight, they
====================
In the history of spaceflight , only one nation has been able to la
unch satellites into space. The Soviet Union, for example, was able
to launch satellites into space on two separate occasions. However,
the Soviet Union was not able to carry out their first lunar base u
ntil the mid-1970s.
====================
In the history of spaceflight , only one nation has been able to ac
hieve the feat: the Soviet Union. Their first attempts were in Febr
uary of 1961, when the Soviet Union launched their first interconti
nental ballistic missile (ICBM) at New York City. They successfully
launched their ICBM on March 8
====================

Wooo! As we can see it generates sentences like it really knows about space!! Let’s see some other results.

gpt2.generate(sess, length=50, temperature=0.7,
prefix="ISRO has launched Chandrayaan II", nsamples=5,
batch_size=5)
# input :'ISRO has launched Chandryaan II'
# Output :
ISRO has launched Chandrayaan II at an altitude of only about 20 km
(15 miles) at a time from the launch pad in Chandrayaan, India. The
lander and its two-stage lander will be launched in the evening of
March 18th at 13:03
====================
ISRO has launched Chandrayaan II on September 18, 2019. This missio
n will launch from the Vastal Pad, the Space Launch Complex 41 (SLC
-41) in Chandrayaan, India. The satellite was launched by the India
n Space Research Organization (ISRO) under
====================
ISRO has launched Chandrayaan II as it reaches its second mission l
aunch from Pad 39A at the end of this month.This comes just a week
after the probe launched from its command module on April 18, and w
ill launch on Pad 39A at the end of this month. And
====================
ISRO has launched Chandrayaan II to its orbit earlier this month. C
handrayaan II is the third mission to orbit the Earth from home and
the first space mission to orbit the Moon. The probe’s main goal is
to study the Moon’s interior structure, analyze
====================
ISRO has launched Chandrayaan II mission on the orbit of DSO on Jul
y 5, 2019.The maiden blastoff of this mission on the Salyut-2 (D-2)
mission is slated for July 5, 2019. The launch window for DSO runs
from July
====================

We can see that it is also able to connect the full form of ISRO- Indian Space Research Organization. It can also relate the month July of 2019 as it is the month of launch. We can see that it is really powerful.

Now let’s give sentence which is not related to space and see what it will do.

gpt2.generate(sess, length=50, temperature=0.7,
prefix="Shah rukh khan has been offered a film",
nsamples=5, batch_size=5)
# input : 'Shah rukh khan has been offered a film'
# output :
Shah rukh khan has been offered a film called “Liftoff of Light“. I
n the film, a group of mission controllers (the crew members and th
e scientists) are shown how the probe will be launched en route to
the Moon. The mission team is also shown how
====================
Shah rukh khan has been offered a film about the events of the tim
e, and the circumstances leading up to it. In it, the filmmaker ask
s the reader to “walk on the Moon.” It’s a happy-couple film, for s
ure, but it�
====================
Shah rukh khan has been offered a film in the form of an interestin
g documentary film. “Khan’s ‘film’ is about the issues facing the
Asian people and the cause of human rights,” said Shah rukh, in rep
ly to a
====================
Shah rukh khan has been offered a film about the first lunar landin
g, called The Moon Under the Moon. This year sees the first opport
unity for Moon tourism in 2019, when the launch of the China Lunar
Exploration Program (CLEP) is scheduled to happen. The Chinese Luna
r Exploration Program (
====================
Shah rukh khan has been offered a film titled “Mysteries of the Uni
verse” that includes a holographic image of a star that is also the
subject of the film. The film will be released on May 29th, 2019.
A cow’s head has
====================

It can also generate a movie title related to space and can include details about the movie without teaching. Isn’t it amazing?