Q
Q
qovalenko2019-06-01 12:34:33
Mathematics
qovalenko, 2019-06-01 12:34:33

How are two types of derivative obtained, using the example of a sigmoid?

Let's find the derivative of the function
dC-sSEyML_A.jpg
f(x) = 1 / (1 + e^(-x))
and I saw that its derivative is
dy/dx = f(x)' = f(x) * (1 - f(x))
The derivative is considered so in machine learning
Where to study this topic in detail, help to understand the second kind of derivative.
Thank you!

Answer the question

In order to leave comments, you need to log in

1 answer(s)
D
Denis Zagaevsky, 2019-06-01
@qovalenko

The derivative of the quotient is equal to the difference between the product of the derivative of the numerator and the denominator and the product of the numerator and the derivative of the denominator, divided by the square of the denominator.
(1/(1 + e^-x))'
= (1' * (1 + e^-x) - 1 * (1 + e^-x)')/(1 + e^-x)^2 #according to the formula for the derivative of the quotient
=( 0 * (...) - 1' + (e^-x)' )/(1 + e^-x)^2 #according to the formula for the derivative of the sum
= (- e^-x * (-x)')/(1 + e^-x)^2 #according to the formula of the derivative of the exponent
= (e^-x)/(1 + e^-x)^2
Then
f(x) = 1/(1 + e^-x)
(e^-x)/(1 + e^-x)^2
= (e^-x)/(1 + e^-x) * 1/(1 + e^-x)
= (e^-x)/(1 + e^-x) * f(x)
= (-1 + 1 + e^-x) / (1 + e^-x) * f(x)
= (- 1 + (1 + e^-x)) / (1 + e^-x) * f(x)
= ((-1 / (1 + e^-x) + (1 + e^-x)/(1 + e^-x)) * f(x)
= (1 - (1/(1 + e^ -x))) * f(x)
= (1 - f(x)) * f(x)
That is, it's not "another way to calculate the derivative in ML", it's just transformations of the derivative of this particular function
. in mathematics for grades 8-11.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question