Does the neural network give a different answer for different initial weights?

Question

Neural networks

What can be said in a multilayer parceptron using the backpropagation method, when the result of running a working example fluctuates quite a lot. And only the random weights change. But after all, in theory, they should strive for some values ​​after a hundred epochs.
Found the Perceptron Convergence Theorem. I have been analyzing the implementation for 2 days, what could be the reasons.

Answer 1

Yes, you find yourself in different local minima. You can play around with the parameters L1, L2, or choose a different optimizer for gradient descent.