Is the jumping error phenomenon normal when training a neural network and can the error be negative?

Question

Machine learning

Tested a neural network with various model parameters: the number of full-connected layers, their size, the size of the training package. The network validates according to Loss, which decreased, but the error began to fluctuate with an increase in the amplitude of oscillations.
Hence, 2 questions:

1. Is the phenomenon of "jumping error" from plus to minus normal in training?
   
2. Do I understand correctly that in the end, the error should be positive?