Luis Otavio Martins

On the field of Artificial Neural Networks, the sigmoid funcion is a type of activation function for artifical neurons.

The most basic activation funciton is the Heaviside (binary step, 0 or 1, high or low):

The Sigmoid function (a special case of the logistic function) and its formula looks like:

You can have several types of activation functions and they are best suitable for different purposes. In the specific case of the Sigmoid:

  • Real-valued and differentiable (you need this to find gradients);
  • Analytic tractability for the differentiaton operation;
  • It is an acceptable mathematical representation of a biological neuron behaviour. The output shows if the neuron is firing or not.

There is a table with the various types of activation functions:

Activation function

Most of the time you will be concerned about the following points when choosing the activation function:

  • continuity of the function
  • computational power to process all neurons of the network
  • type of the desired output (logistic/continuous variables or classification/categorical data)

Always upvote answers that you find useful. Everyone can be wrong so be respectful and polite.

Originally published at www.quora.com.

Source: Deep Learning on Medium