On the field of Artificial Neural Networks, the sigmoid funcion is a type of activation function for artifical neurons.
The most basic activation funciton is the Heaviside (binary step, 0 or 1, high or low):
The Sigmoid function (a special case of the logistic function) and its formula looks like:
You can have several types of activation functions and they are best suitable for different purposes. In the specific case of the Sigmoid:
- Real-valued and differentiable (you need this to find gradients);
- Analytic tractability for the differentiaton operation;
- It is an acceptable mathematical representation of a biological neuron behaviour. The output shows if the neuron is firing or not.
There is a table with the various types of activation functions:
Most of the time you will be concerned about the following points when choosing the activation function:
- continuity of the function
- computational power to process all neurons of the network
- type of the desired output (logistic/continuous variables or classification/categorical data)
Always upvote answers that you find useful. Everyone can be wrong so be respectful and polite.
Originally published at www.quora.com.
Source: Deep Learning on Medium