Why do we need activation functions and nonlinearity in neural networks?

Question

Neural networks

Why do we need activation functions and nonlinearity in neural networks?

Answer 1

If we look at how a deep neural network works without activation functions, we get the following:
y \u003d W "W'Wx (W"-weights of the third layer, W'-weights of the second layer ...)
Since we do not have nonlinearity, then each layer is a simple linear transformation, and from lin. we know algebra AB=C, which means we can replace W "W'W with one matrix W *. That is, adding a new layer without an activation function does not give us anything.

Answer 2

To ensure the stability of the result with different input parameters.