Deep Learning is at the forefront of the AI revolution, and for good reasons — incredible advances in natural language processing, image recognition, and even computer playing Go, have all been made thanks to the help of deep neural networks.
Because of the complexity of the tasks most neural networks have to accomplish, most people would expect that it’s hard to set up a neural network. Yeah, maybe 5 years back that would’ve been completely true, you would’ve had to set up your neural network from scratch. But today, with open-source frameworks such as Keras, it can be quite simple to get started. Here’s how. (Here’s the full script if you want to skip ahead!
The problem: predicting a salary using demographic variables
First, let’s find a toy problem to get started with. Here’s one I found: US Adult income data from the 1994 US Census Database. We will try using several fully connected neuron layers to predict a given person’s salary in the census.
To be honest, deep neural networks are probably a tad overkill for this task. Traditional data science techniques such as logistic regression are perfectly suited for this and infinitely easier to interpret. Still, this is a good way to get started with Keras.
To follow with this code, first install Python 3. When this is done, open your OS terminal, and type the following:
pip install pandas
Then install the following packages after pandas: sklearn, tensorflow, and finally, keras.
First, we’ll start with our imports. This script will need:
- pandas for the data manipulation,
- numpy for some quick operations,
- sklearn for data scaling and performance measurements,
- and finally, keras, our deep learning framework.
Loading the dataset
Our dataset contains variables such as the age of the worker, his or her education level, the number of years of education, marital status, job title, etc. The last value included in this dataset is the salary range, either lower or higher than $50,000. We will be using the former variables to predict this target.
Preparing the dataset
The next step is to prepare our dataset for Keras. We need to do the following:
- Balance the classes. Unbalanced classes will interfere with training.
- Transform the categorical variables (occupation, for example) into one-hot encoded variables. Basically making a new column for each label with a value of 0 or 1.
- Extract the X (variables) and y (targets) values for the training and testing datasets.
- Scale the variables.
The model in Keras
Now for the good part! Building the Keras model. There’s two interfaces to build models in Keras, the sequential model API and the functional API. Today, we will be using the sequential model API since it’s easier to follow and to build.
Here’s what we need to do.
- First, we will be setting up a ModelCheckpoint object to allow us to save the best results during the training process.
- Then, we’ll create a Sequential model object unto which we will add the different neural layers.
- We will need to choose the appropriate activation functions, optimizer and loss functions to build our binary classification model (we are trying to predict values of 0 or 1 for one variable).
- Finally, we will train our network.
Evaluating the model’s performance
How did we do? Let’s measure our model’s performance! We will measure the precision, a value of the amount of exact classifications.
- We will need to load the weights we saved during the model training and compile the model with those weights.
- Predict the values in the test dataset using the model.
- Compare the predicted salaries to the actual salaries in the test dataset.
In the end, this model gets a precision of about 87% for salaries below 50K and a precision of about 72% for salaries higher than 50K.
Not bad for a model with no feature engineering whatsoever…
The whole script is available here on GithubGist.
I hope you enjoyed my tutorial!
Source: Deep Learning on Medium