The sigmoid function is a smooth monotonic nonlinear function that maps real numbers to the interval $(0,1)$. It converts arbitrary real-valued scores into values interpretable as probabilities.

Examples:

• Logistic function
• Hyperbolic tangent
• Arc-tangent-based sigmoids

Logistic function

The most common sigmoid function in ML is the logistic function:

$$ \sigma(x)=\frac{1}{1+e^{-x}} $$

where:

• $x\in\R$
• $\sigma(x)\in(0,1)$

Derivative

Its derivative can be written in terms of the function itself:

$$ \frac{d}{dx}\sigma(x) =\sigma(x)(1-\sigma(x)) $$

Symmetry

$$ \sigma(-x)=1-\sigma(x) $$

Log-odds interpretation

The sigmoid is the inverse of the logit function:

$$ \text{logit}(p)=\log\frac{p}{1-p} $$

so that: