Incorrect multiplication result or logical error?

Question

Python

Good day! I decided to do neural networks for myself. I read Tarik Rashid's book "Make Your Own Neural Network", there at the end there is an example of the implementation of a three-layer perceptron for classifying handwritten digits using matrices. In short, with this approach, the output of the layer of the neural network is the product of the matrices of weights of connections with the previous layer by the matrix of inputs of this layer. And in that, and in the other matrix, the range of values ​​is from zero to one. For the sake of interest, I decided to rewrite the implementation from the book in C ++ using Boost. In the C++ implementation, the signal of all neurons of the output layer (which is, in fact, the probability of belonging to any class) is for some reason equal to 1.0, it turns out that what was submitted to the input is 100% related to all classes, which cannot be . Then rewrote from Python to C++ with Eigen, which is used in TensorFlow is the same. Rewrote to Go + Gonum, same thing. The first discrepancy is already after the first matrix multiplication: the multiplication of the matrix of weights of the connections of the input layer with the hidden one. The result is a matrix with values ​​like 3.27362482790, 5.4859804, 6.4859834, and so on. But how is this possible if the maximum values ​​in the multiplied matrices do not exceed one? Links to implementations:

• C++: https://gist.github.com/nickfromXXII/09837f2c2953f...
  
• Python: https://github.com/makeyourownneuralnetwork/makeyo...
  

Answer 1

The first discrepancy is already after the first matrix multiplication: the multiplication of the matrix of weights of the connections of the input layer with the hidden one. The result is a matrix with values ​​like 3.27362482790, 5.4859804, 6.4859834, and so on. But how is this possible if the maximum values ​​in the multiplied matrices do not exceed one?

take a 2x2 matrix with values ​​0.9(<1). And multiply it by itself. You will get a 2x2 matrix with values ​​1.62(>1). Magic, right?