Understanding Hyperparameters to Increase Optimal Performance of Machine Learning Model.

Original article was published on Deep Learning on Medium

Understanding Hyperparameters to Increase Optimal Performance of Machine Learning Model.

let us understand what is Hyperparameter and why it is a important to know about it and how it hepls to Increase the performance of a model.

  • A Hyperparameter is a variable we need to set before applying a learning algorithm into a dataset .
  • we can say this Hyperparameters are like tuners we set or adjust during the learning process to obtain the optimal performance.
  • They are fixed before the training process begins. and we can change or adjust them to see if our model is properly trianed or not.
  • if there is some errors like training process is not performing good and there is increase of validation error and learning rate is not stable we have to set our own Hyperparameters.
  • first of all let us understand what is validation error and learning rate.
  1. Validation Error-

In this scenario or context validation error is nothing but the rate of increasing/ decreasing of neurons or functions in a current training process which further can tells us the accuracy of the ongoing training process.

2. Learning Rate-

This Learning rate is nothing but the adjustment of the weights of a loss function during a training process.

in figure (A),

Validation Error- Decreasing during Train

Learning Rate- Good

in figure (B),

Validation Error- Decreasing Slowly

Learning Rate- Bad

  • now we know what are Hyperparameters, now let us deep dive into the types of Hyperparameters and learn more about learning rate.
  • Hyperparameters can be further divided into two types:-

Optimizer Hyperparameters 》

  • Variables are related to more to the optimization and trianing process.
  1. Learning Rate
  2. Minibatch Size
  3. Epochs

Model Hyperparameters 》

  • This are more involved in the structure of the model.
  • let us learn more about optimizer Hyperparameters,

A. ‘Learning Rate’ –

  • During a learning step, we do a forward pass through the model, calculate the loss, then find the gradient.
  • to nudge the current weight so that our predictions become more accurate by gradient.
  • to learn more about gradient decent click below link,


B. ‘Minibatch’ -

  • as from name you can infer that minibatch is nothing but the number of examples or rows from that dataset we want to include in our training dataset.
  • thia include a online ( Stochastic) method which takes only one example to do a forward pass.
  • and then on that one example all that calculate error, backpropagation and set values are adjusted for all parameters.
  • but this online method we can say that is not efficient and not ideal for large model with big datasets.
  • So it is better to feed the batches from the datasets to the training dataset for better efficiency and results
  • it is never fixed how much of data we should feed we have to make assumptions and take care of one thing that it should not be too small or too big otherwise it can create bias.

C. ‘Epochs’-

  • some of us are familiar with epochs.it is number of passes a training dataset has completed.
  • epochs can help us understand the acccuracy of the model because we are passing the same training dataset multiple times and it is either increasing its accuracy or decreasing.this helps us to get more from our training data and we can make changes accordingly.
  • hope this get you start with Hyperparameters. in upcoming articles implementation of this Hyperparameters and how are they increasing performance will be showcased with the help of the real world example.
  • also we are going to deeply discuss about model hyperparameters.