Sigmoid Function cheatsheet

• It is used to convert a real number to a number between 0 and 1.
• It is used in classification problems.
• https://en.wikipedia.org/wiki/Sigmoid_function Wikipedia

• Logistic Regression
• Neural Networks
• Gradient Boosting
• Principal Component Analysis (PCA)
• Reinforcement Learning

• x is the input.
• e is the Euler's number. Approximately equal to 2.71828.

• The sigmoid function is always positive.
• The sigmoid function is always less than or equal to 1.
• The sigmoid function is equal to 0.5 when the input is equal to 0.
• The sigmoid function is equal to 1 when the input is equal to infinity.
• The sigmoid function is equal to 0 when the input is equal to negative infinity.

• Advantages
  • The sigmoid function is differentiable.
  • The sigmoid function is monotonic.
  • The sigmoid function is bounded.
  • The sigmoid function is easy to understand.
• Disadvantages
  • The sigmoid function is not zero centered.
  • The sigmoid function is not sparse.
  • The sigmoid function is prone to saturation.
  • The sigmoid function is computationally expensive.

import numpy as np

def sigmoid(x):
    return 1 / (1 + np.exp(-x))

sigmoid <- function(x) {
    return (1 / (1 + exp(-x)))
}

function sigmoid(x)
    return 1 / (1 + exp(-x))
end

import tensorflow as tf

x = tf.constant([-1, 0, 1, 2, 3], dtype=tf.float32)
y = tf.nn.sigmoid(x)

print(y)

import torch

x = torch.tensor([-1, 0, 1, 2, 3], dtype=torch.float32)
y = torch.sigmoid(x)

print(y)

from scipy.special import expit

x = np.array([-1, 0, 1, 2, 3])
y = expit(x)

print(y)