Where is my mistake in neural network training?

Question

Neural networks

Good day! Please tell me if I understood the backpropagation algorithm correctly.
Network example. Input = Sample
784x1 matrix Input layer
= Input, 10x784 matrix
Hidden layer = Hidden, 10x10 matrix
Output layer = Out, 10x10 matrix = B * Out Self backpropagation . 1) To update the weight on the last layer - Out we need delta =(Expected - Result of the network) * by the derivative of the activation function of the layer Out gradient = (transpose the matrix of layer C and multiply by the delta matrix) * learning rate
new_weight = layer Out - gradient
2) To update on the hidden layer and subsequent ones, I need
delta = error of the previous layer ( delta Out) * on the derivative of the layer activation function Hidden
gradient = (transpose the layer B matrix and multiply by the delta matrix) * learning rate
new_weight = layer Hidden - gradient
I implemented this in the code, but when I update the weights, I get different matrices and I can't do the last operation. I deliberately omitted the multiplication by the activation function.

Answer 1

And so the error in the algorithm. There is a wonderful book "Create your own neural network" from the author Tariq Rashid, which explains the algorithm perfectly. Not for advertising.

Answer 2

Well, where? It is possible in the code of a program that we do not see.
PS Just please don't dump your code here. And even more so - control data.