Original article was published on Deep Learning on Medium

# Loss Function For Regression Problem in Machine Learning — Python Implementation Using Numpy and Tensorflow

## Regression Analysis:

**Regression Analysis **is basically a statistical approach to find the relationship between variables. In **machine learning**, this is used to predict the outcome of an event based on the relationship between variables obtained from the data-set.

Different types of **Regression Algorithm **used in **Machine Learning. Some are:**

- Linear regression
- Polynomial regression
- Logistic regression
- Stepwise
**Regression** - Ridge Regression

**Cost Function or Loss Function or Error**

In Machine Learning, the Cost function tells you that your learning model is good or not or you can say that it used to estimate how badly learning models are performing on your problem. This is typically expressed as a difference or distance between the predicted value and the actual value.

There are many types of Cost Function area present in Machine Learning. These are the following some examples:

- Cross-entropy cost
- Mean Squared Error
- Root Mean Square Error
- Hellinger Distance

Here are I am mentioned some Loss Function that is commonly used in Machine Learning for Regression Problems.

# Let’s Get Start

Before I get started let’s see some notation that is commonly used in Machine Learning:

**Summation: **It is just a Greek Symbol to tell you to add up a whole list of numbers. For example, **summation** of [1, 2, 4, 2] is denoted 1 + 2 + 4 + 2, and results in 9, that is, 1 + 2 + 4 + 2 = 9.

**Y: **Actual Value or Real Value

**Y-hat: **In Machine Learning, we y-hat as the predicted value

## 1. M**ean Square Error or L2 Loss:**

**Equation:**

- n = number of data points
- y = actual value or real value (labels)
- y-hat = predicted value

The average squared difference or distance between the estimated values (predicted value) and the actual value. M**ean Square Error** is almost always strictly positive (and not zero) is because of randomness or because the estimator does not account for information that could produce a more accurate estimate.

Python Implementation using Numpy:

import numpy as np

Y_true = [1,1,2,2,4] # Y_true = Y (actual values)

Y_pred = [0.6,1.29,1.99,2.69,3.4] #Y_pred=Y-hat (predicted value) # Mean Squared Error

MSE = np.square(np.subtract(Y_true,Y_pred)).mean()

print(MSE) # .21606

## 2. Mean Absolute Error or L1 Loss:

**Equation:**

- n = number of data points
- x = actual value or real value (labels)
- y = predicted value

Mean Absolute Error is the sum of **absolute** differences between our target and predicted variables. It measures the **average** magnitude of errors in a set of predictions, without considering their directions. It is a common measure of forecast error in time series analysis.

Python Implementation using Numpy:

import numpy as np

Y_true = [1,1,2,2,4] # Y_true = Y (actual values)

Y_pred = [0.6,1.29,1.99,2.69,3.4] #Y_pred=Y-hat (predicted value)# Mean Absolute Error

MAE = np.abs(np.subtract(Y_true,Y_pred)).mean()

print(MAE) # .398

## 3. Huber Loss or Smooth Mean Absolute Error:

The Huber loss can be used to balance between the MAE (Mean Absolute Error), and the MSE (Mean Squared Error). It is therefore a good loss function for when you have varied data or only a few outliers. It is more robust to outliers than MSE.

Python Implementation using Numpy and Tensorflow:

import tensorflow, numpyy_true = [[0, 1], [0, 0]]

y_pred = [[0.6, 0.4], [0.4, 0.6]]

h = tf.keras.losses.Huber()

h(y_true, y_pred).numpy()

# output 0.155

## 4. Log-Cosh Loss:

From TensorFlow docs: log(cosh(x)) is approximately equal to (x ** 2) / 2 for small x and to abs(x) — log(2) for large x. This means that ‘logcosh’ works mostly like the mean squared error, but will not be so strongly affected by the occasional wildly incorrect prediction.

Python Implementation using Numpy and Tensorflow:

import tensorflow, numpyy_true = [[0, 1], [0, 0]]

y_pred = [[0.6, 0.4], [0.4, 0.6]]

l = tf.keras.losses.LogCosh()

l(y_true, y_pred).numpy()

# output 0.108

## Some Extra Loss Function:

**Root Mean Squared Error**: It is just a Root of MSE.

Python Implementation using Numpy:

import numpy as np

Y_true = [1,1,2,2,4] # Y_true = Y (actual values)

Y_pred = [0.6,1.29,1.99,2.69,3.4] #Y_pred=Y-hat (predicted value)# Root Mean Squared Error

RMSE = np.sqrt(np.square(np.subtract(Y_true,Y_pred))).mean()

print(RMSE) # 0.398

**Mean Absolute Percentage Error**: It is just a percentage of MAE.

Python Implementation using Numpy:

import numpy as np

Y_true = [1,1,2,2,4] # Y_true = Y (actual values)

Y_pred = [0.6,1.29,1.99,2.69,3.4] #Y_pred=Y-hat (predicted value)# MeanAbsolute Percentage Error

MAPE = 100 * np.abs(np.subtract(Y_true,Y_pred)).mean()

print(MAPE) # 31.800000

**Mean Squared Logarithmic Error (MSLE)**: It can be interpreted as a measure of the ratio between the true and predicted values. As the name suggests, it is a variation of the **Mean Squared Error.**

Python Implementation using Numpy and Tensorflow:

import tensorflow, numpyy_true = [[0, 1], [0, 0]]

y_pred = [[0.6, 0.4], [0.4, 0.6]]

msle = tf.keras.losses.MeanSquaredLogarithmicError()

msle(y_true, y_pred).numpy()

# output 0.240