Tensorflow model to learn a constant function.

Original article was published by Pramod Athani on Deep Learning on Medium


Tensorflow model to learn a constant function.

Tensorflow is an open source platform for machine learning developed by Google. In this post we will create a neural network model to learn a constant function. i.e., y = k (where k is a constant real number) using the tensorflow’s python library.

This can be achieved by using just one layer containing a single neuron. We use the Dense layer of tensorflow library for this example.

Just for kicks lets set the value of the constant we want to learn to π.

The following code imports the necessary packages.

Set up training data

Now, we set up the training data. Training data simply contains numbers between 0, 5000 and the corresponding output is just π. Obviously the output does not depend on the input training data at all!. In the neural network this is learnt by the bias term.

Set up and learn the model

With training data setup, now lets construct the neural network model. We use the Sequential model of keras containing as single Dense layer. Dense layer is one of the most basic layers in Tensorflow. It connects all inputs to all outputs.

In our case though we just have one neuron that ‘implicitly’ takes the training input (specified using the input_shape parameter to the Dense object constructor).

We use the ‘rectified linear output’ or ReLU activation function. An activation function of a neuron operates on the weighted sum of its inputs. Rectified linear output just passes on this weighted sum if the sum is greater than zero. Otherwise the ReLU passes on 0 as the output of the neuron.

We compile the model by calling the compile method on the Sequential model object and specify the optimizer and the loss function parameters. The loss function specifies how far off the model output is as compared to the given training output. The job of the optimizer function is to minimize this loss as the model is trained by repeatedly going through the training data.

The fit function of the Sequential model does the actual ‘training’. We pass the training inputs in the x parameter, training outputs or ‘labels’ in the y parameter. We train the network by going through 500 hundred rounds of the training data. This is specified using the epochs parameter.

Check the weights of the dense layer

Our model is trained now. By ‘trained’ we mean that the weight and bias of the neuron on the dense layer have been set so that the loss function is minimized. To view, what these weights are we can use the code below.

We expect that the weight and bias to be 0 and π respectively.

Prediction

Our model is ready. We can use it to make predictions. When we enter any data to the model, we expect it to output π more or less.

That’s it. That’s our tensorflow model to learn a constant function.

Below is the complete code that you can try on your own.