machinelearn.dev

Tanh Function

The tanh function is a common activation function used in neural networks. It is also known as the hyperbolic tangent function.

#Getting Started

#inroduction

#algorithms using tanh

  • Neural Networks
  • Deep Learning

#formula

$$\sigma(x) = \frac{e{x}-e{-x}}{e{x}+e{-x}}$$

  • x is the input.

#properties

  • The tanh function is always positive.
  • The tanh function is always less than or equal to 1.
  • The tanh function is equal to 0 when the input is less than 0.
  • The tanh function is equal to the input when the input is greater than or equal to 0.

#Advantages and Disadvantages

  • Advantages

    • The tanh function is differentiable.
    • The tanh function is monotonic.
    • The tanh function is bounded.

  • Disadvantages ## looks into this

    • The tanh function is not differentiable at 0.
    • The tanh function is not differentiable at negative values.
    • The tanh function is not bounded above.
    • The tanh function is not bounded below.
    • The tanh function is not symmetric.
    • The tanh function is not centered around 0.

#Implementation

#Python


import numpy as np

def tanh(x):
    return np.tanh(x)

#R


tanh <- function(x) {
    return(tanh(x))
}

#Julia


function tanh(x)
  return(tanh(x))
end

#Tensorflow


import tensorflow as tf

def tanh(x):
    return tf.math.tanh(x)

#Pytorch


import torch

def tanh(x):
    return torch.tanh(x)