### What is Tensorflow?

When you are building neural networks, convolutional layers, or anything related to machine learning, it is recommended to use libraries. For beginners problem is amount of these libraries. “What I should use?” is a common question and there is as many answers as there is libraries.

Basically it doesn’t matter what library you are using because all works fundamentally similarly. I personally started using Tensorflow (tf) because there was a lot of examples and tutorials. I recommend it to beginners for two reasons. Firstly, as I already said there is a lot of material on web. It is also used in many companies so if you are interested to work for someone they might appreciate your tf knowledge. The second reason is that Tensorflow is Google’s open source library. As long as Google is behind this project it is updated often and there will be tutorials and other material from Google. Tf’s documentations are also very easy to read and I often find what I’m looking for.

Top of Tensorflow there is built a library called Keras. Keras is like drag-and-drop programming. You don’t learn anything by doing it and when you want to build something better it is more complicated. So I don’t recommend using libraries which are build top of other libraries. People say it is easier to read code when there is less lines but I think it is opposite. It is annoying to search from source code what method is doing when you could just write it to the main code.

### Tensorflow Hello World!

Python version:

a = 5

b = 6

c = a*b

print(c)

OUTPUT: 30

Tensorflow version:

import tensorflow as tf

# first we define the parameters

a = tf.constant(5)

b = tf.constant(6)

# then we define the calculation. It is not computed yet. This is called defining graph. In tf we first define graph and then later calculate it.

c = a*b

# this starts Tensorflow environment.

with tf.Session() as sess:

# This is calculating the calculation which we made earlier.

result = sess.run(c)

# And finally we print the result.

print(result)

OUTPUT: 30

From these examples we saw how Python and Tensorflow is calculating the same result differently. Tensorflow calculated the sum of two number in GPU and Python in CPU. In this calculation it doesn’t matter because this is so simple but when we are calculating big matrix multiplications it is a lot faster to calculate in GPU. So basically we could use Python to calculate neural networks but we don’t do it because it is much slower.

### Simple example

import tensorflow as tf

We set our input and output data

x = tf.constant([1,3,8,5,20,4],tf.float32)

y = tf.constant([2,6,16,10,40,8],tf.float32)

Randomly chosen starting points (so 1 and 2 is just random numbers)

a = tf.Variable(1.0)

b = tf.Variable(2.0)

Defining the graph. This is linear function.

y_hat = a*x+b

Then we calculate our loss function and take gradient of it. In the second line we define learning rate which is not normally something less than 1 but now we have linear function and we are using integers so using 1 as learning rate works better than smaller number.

loss = tf.losses.mean_squared_error(y,y_hat)

optimizer = tf.train.AdamOptimizer(1).minimize(loss)

Finally, we calculate our graph. Last line we shows the values of a and b which should be a = 2 and b = 0. Why these values? Well if you remember we defined our function to be `a*x+b`

. `2*x+0=2x`

. Test to multiply every `x`

value with `2 `

and you should get the `y `

value.

model = tf.global_variables_initializer()

with tf.Session() as sess:

sess.run(model)

for i in range(100):

sess.run(optimizer)

print(“a:”,sess.run(a),”b:”,sess.run(b))

OUTPUT: a: 2.0047016 b: -0.005181805

(Your output can be a little bit different)

Again:

- First we load our data and save it to
`tf.constant`

- Then we randomly choose our parameters which we add to our function
- After that we make our loss function and feed that into optimizer
- Finally we loop through
`optimizer`

and every loop it changes parameters to the direction which reduce loss between`y_hat`

(predicted result) and`y`

.

Now you might wonder what are **constant** and **Variable**.

**Variable**: These are same thing as normal variables in Python. We can change these and we have to define something to them first. Our parameters are variables because our optimizer have to be available to change those.

**Placeholder**: This is third type of parameter method which Tensorflow have. Placeholders are like variables but you define these inside session by writing `feed_dict={variable_name=value}`

. These are very popular way to feed in the data. I use placeholders later examples so you will understand then better.

**Constant**: This is like variable but you can’t change the values. So you can think of this like static variable which is once set and can’t be later modified. In our example I made data to be constant because we never changed it.

### Predicting the result of football game

First we import all important packages.

import pandas as pd

from math import floor

import tensorflow as tf

import numpy as np

from sklearn import preprocessing

Then we get the data from four different csv file and save those to `data`

which type is DataFrame.

data1 = pd.read_csv('premier_14_15.csv')

data2 = pd.read_csv('premier_15_16.csv')

data3 = pd.read_csv('premier_16_17.csv')

data4 = pd.read_csv('premier_17_18.csv')

dataCon = [data1,data2,data3,data4]

data = pd.concat(dataCon)

These functions just modify our data making it easier to use.

def normalization(raw_data):

for col_num in range(raw_data.shape[1]):

if raw_data.iloc[:,col_num].dtype == np.float or raw_data.iloc[:,col_num].dtype == np.int:

raw_data.iloc[:,col_num] = (raw_data.iloc[:,col_num] - raw_data.iloc[:,col_num].mean()) / (raw_data.iloc[:,col_num].max() - raw_data.iloc[:,col_num].min())

return raw_data

def embedding_matrix(column):

labels = []

embeddings = np.array([])

num_of_uniques = len(np.unique(column))

for i in range(num_of_uniques):

if embeddings.size == 0:

embeddings = np.random.uniform(low=-0.01,high=0.01,size=(min(50,(num_of_uniques+1)//2),1))

else:

embeddings = np.append(embeddings,np.random.uniform(low=-0.01,high=0.01,size=(min(50,(num_of_uniques+1)//2),1)),axis=1)

labels.append(np.unique(column)[i])

print("embeddings.shape:",embeddings.shape)

print("labels.shape:",len(labels))

return pd.DataFrame(data=embeddings,columns=labels)

em = embedding_matrix(data['HomeTeam'])

data = normalization(data)

We take Ys to other DataFrame and Xs to other.

x_data = np.column_stack((np.transpose(em[data['HomeTeam']].values),

np.transpose(em[data['AwayTeam']].values),

data['B365H'][:,None],

data['B365D'][:,None],

data['B365A'][:,None]))

y_data = data['FTR']

y_data = pd.get_dummies(y_data)

Then we split our data to train, test and validation sets.

train_size = 0.9

valid_size = 0.3

train_cnt = floor(x_data.shape[0] * train_size)

x_train = x_data[0:train_cnt]

y_train = y_data.iloc[0:train_cnt].values

valid_cnt = floor((x_data.shape[0] - train_cnt) * valid_size)

x_valid = x_data[train_cnt:train_cnt+valid_cnt]

y_valid = y_data.iloc[train_cnt:train_cnt+valid_cnt].values

x_test = x_data[train_cnt+valid_cnt:]

y_test = y_data.iloc[train_cnt+valid_cnt:]

print("x_train:",x_train.shape)

print("x_valid:",x_valid.shape)

print("x_test:",x_test.shape)

OUTPUT:

KIRJOITA

Now we define `x`

and `y`

variables to be `tf.placeholder`

so we can set the values later. We already have the data so we could define x and y right now but this is more common approach.

x = tf.placeholder(tf.float32)

y = tf.placeholder(tf.float32)

Hyper parameters. You can play with these and get a better result than I got. I just randomly chose these so choosing at least better learning rate (=alpha) might help.

# Parameters

ALPHA = 1e-3

num_epochs = 50

batch_size = 128

display_step = 5

More parameters…

# Network Parameters

num_input = x_data.shape[1]

num_classes = y_data.shape[1]

num_hidden_1 = 50

num_hidden_2 = 50

KEEP_PROBABILITY = 0.9

Then we make function which will build our model.

def neural_network(x,weights,biases,keep_prob):

layer_1 = tf.add(tf.matmul(x,weights['w1']),biases['b1'])

layer_1 = tf.nn.relu(layer_1)

layer_1 = tf.nn.dropout(layer_1,keep_prob)

layer_2 = tf.add(tf.matmul(layer_1,weights['w2']),biases['b2'])

layer_2 = tf.nn.relu(layer_2)

layer_2 = tf.nn.dropout(layer_2,keep_prob)

layer_out = tf.add(tf.matmul(layer_2, weights['out']), biases['out'])

return layer_out

Weights and biases for the model.

# Store layers weight & bias

weights = {

'w1': tf.Variable(tf.random_normal([num_input,num_hidden_1])),

'w2': tf.Variable(tf.random_normal([num_hidden_1,num_hidden_2])),

'out': tf.Variable(tf.random_normal([num_hidden_2, num_classes]))

}

biases = {

'b1': tf.Variable(tf.random_normal([num_hidden_1])),

'b2': tf.Variable(tf.random_normal([num_hidden_2])),

'out': tf.Variable(tf.random_normal([num_classes]))

}

keep_prob = tf.placeholder("float")

Then we build our model and save it to `predictions`

. After that we define the cost and the optimizer.

predictions = neural_network(x, weights, biases, keep_prob)

cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=predictions, labels=y))

optimizer = tf.train.AdamOptimizer(learning_rate=ALPHA).minimize(cost)

And finally we calculate the graph.

with tf.Session() as sess:

sess.run(tf.global_variables_initializer())

for epoch in range(num_epochs):

avg_cost = 0.0

total_batch = int(len(x_train) / batch_size)

x_batches = np.array_split(x_train, total_batch)

y_batches = np.array_split(y_train, total_batch)

for i in range(total_batch):

batch_x, batch_y = x_batches[i], y_batches[i]

_, c = sess.run([optimizer, cost],

feed_dict={

x: batch_x,

y: batch_y,

keep_prob: KEEP_PROBABILITY

})

avg_cost += c / total_batch

if epoch % display_step == 0:

print("Train: Epoch:", '%04d' % (epoch+display_step), "cost=", "{:.9f}".format(avg_cost))

_, valid_cost = sess.run([optimizer, cost], feed_dict={x: x_valid, y: y_valid, keep_prob: 1})

print("Valid: Epoch:", '%04d' % (epoch+display_step), "cost=", "{:.9f}".format(valid_cost))

correct_prediction = tf.equal(tf.argmax(predictions, 1), tf.argmax(y, 1))

accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

print("Accuracy:", accuracy.eval({x: x_test, y: y_test, keep_prob: 1.0}))

print("______________________________________")

print("Optimization Finished!")

correct_prediction = tf.equal(tf.argmax(predictions, 1), tf.argmax(y, 1))

accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

print("Accuracy:", accuracy.eval({x: x_test, y: y_test, keep_prob: 1.0}))

OUTPUT:

Train: Epoch: 0005 cost= 14.639687252

Valid: Epoch: 0005 cost= 7.217744350

Accuracy: 0.45794392

______________________________________

Train: Epoch: 0010 cost= 8.123380804

Valid: Epoch: 0010 cost= 3.757234097

Accuracy: 0.39252338

______________________________________

Train: Epoch: 0015 cost= 7.011940718

Valid: Epoch: 0015 cost= 2.546033621

Accuracy: 0.44859812

______________________________________

Train: Epoch: 0020 cost= 6.122287321

Valid: Epoch: 0020 cost= 2.085423470

Accuracy: 0.48598132

______________________________________

Train: Epoch: 0025 cost= 5.233317709

Valid: Epoch: 0025 cost= 2.182467699

Accuracy: 0.57009345

______________________________________

Train: Epoch: 0030 cost= 4.904307556

Valid: Epoch: 0030 cost= 2.305070400

Accuracy: 0.5420561

______________________________________

Train: Epoch: 0035 cost= 4.445039535

Valid: Epoch: 0035 cost= 2.415178537

Accuracy: 0.5607477

______________________________________

Train: Epoch: 0040 cost= 3.930668139

Valid: Epoch: 0040 cost= 2.189312696

Accuracy: 0.5607477

______________________________________

Train: Epoch: 0045 cost= 3.636997890

Valid: Epoch: 0045 cost= 2.088778257

Accuracy: 0.5607477

______________________________________

Train: Epoch: 0050 cost= 3.385137343

Valid: Epoch: 0050 cost= 1.808984399

Accuracy: 0.5420561

______________________________________

Optimization Finished!

Accuracy: 0.5607477

### Classifying images with Tensorflow

Imports…

importnumpyasnpimporttensorflowastffromsklearn.cross_validationimporttrain_test_splitimportosimportrandomfromPILimportImage

We import our data. This is from Tensorflow and it is just hand written numbers from 1 to 10.

fromtensorflow.examples.tutorials.mnistimportinput_data mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

Hyper parameters.

img_shape = np.array([28,28])

num_input = img_shape[0] * img_shape[1]

num_classes = 10

drop_out_prob = 0.5

learning_rate = 0.001

epochs = 5

batch_size = 128

display_step = 100

Parameters.

X = tf.placeholder(tf.float32, [None, num_input])

Y = tf.placeholder(tf.float32, [None, num_classes])

drop_out = tf.placeholder(tf.float32)

Then we made function which will build the model.

defconv_net(x, drop_out):

x = tf.reshape(x, shape=[-1, img_shape[0], img_shape[1], 1])

conv1 = tf.layers.conv2d(

inputs=x,

filters=32,

kernel_size=[5,5],

padding="same",

activation=tf.nn.relu)

pool1 = tf.layers.max_pooling2d(inputs=conv1,pool_size=[2,2],strides=2)

conv2 = tf.layers.conv2d(

inputs=pool1,

filters=64,

kernel_size=[5,5],

padding="same",

activation=tf.nn.relu)

pool2 = tf.layers.max_pooling2d(inputs=conv2,pool_size=[2,2],strides=2)

pool2_flat = tf.reshape(pool2,[-1,7*7*64])

dense = tf.layers.dense(inputs=pool2_flat,units=1024,activation=tf.nn.relu)

dropout = tf.layers.dropout(

inputs=dense,

rate=drop_out)returntf.layers.dense(inputs=dropout,units=num_classes)

Again we build the model, calculate the cost, and finally define the optimizer.

logits = conv_net(X,drop_out)

prediction = tf.nn.softmax(logits)

cost = tf.reduce_mean(-tf.reduce_sum(Y * tf.log(prediction), reduction_indices=[1]))

optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cost)

correct_prediction = tf.equal(tf.argmax(prediction, 1), tf.argmax(Y, 1))

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

Then we calculate the model.

withtf.Session()assess:

sess.run(tf.global_variables_initializer())

test_images,test_labels = mnist.test.next_batch(batch_size)forepochinrange(epochs):

num_steps = int(len(mnist.train.labels)/batch_size)forstepinrange(num_steps):

batch_x, batch_y = mnist.train.next_batch(batch_size)

sess.run(optimizer, feed_dict={X: batch_x, Y: batch_y,drop_out:drop_out_prob})ifstep % display_step == 0orstep == 1:

print("step:",step,".",epoch)

pred = sess.run(prediction,feed_dict={X: test_images})

print("prediction:",np.argmax(pred,axis=1)[:5])

print("y predicts:",np.argmax(test_labels,axis=1)[:5])

print("Cost:",sess.run(cost,feed_dict={X:test_images,Y:test_labels,drop_out:0}))

print("Accuracy:",sess.run(accuracy, feed_dict={X:test_images,Y:test_labels,drop_out:0}))

print("________________________________________")

print("Accuracy:")

print(num_steps, sess.run(accuracy, feed_dict={X:test_images, Y:test_labels,drop_out:0}))

print("Optimization Finished!")# Calculate accuracy for 256 MNIST test images

print("Testing Accuracy:",

sess.run(accuracy, feed_dict={X: mnist.test.images,

Y: mnist.test.labels,

drop_out:0}))

And finally we got 99.1% accuracy.

I recommend saving my Github to your bookmarks because most of these codes you can just copy from there. I hope this helped and if some examples wasn’t explained well leave a comment and I will explain those to you.

~Lankinen

Source: Deep Learning on Medium