The CNN does not start to recognize images better after adjusting the weights and biases on the last fully connected layers. Is it necessary or is it a mistake?

Question

Neural networks

I am writing a convolutional neural network in C++ from scratch (only for educational purposes). Wrote the implementation of direct propagation of all layers used. I wrote backpropagation and weight adjustment with offsets of the last fully connected layers, but even after several full passes of the entire MNIST dataset , the grid does not start to recognize images better (as it was 9-10% , 9-10 % remains ).

Question. Did I make a mistake in the implementation of something, or is it not enough to train only the last layers in order for the CNN to get at least a little better at recognizing images?

PS Architecture of the CNN used :

• The input layer is 28x28x1 .
  
• Convolution layer 24x24x20 .
  
• The merging layer is 12x12x20 .
  
• Convolutional layer 8x8x50 .
  
• The merging layer is 4x4x50 .
  
• Fully connected layer 500 .
  
• Outgoing layer 10 .
  

Answer 1

If the weights on the convolutional layers were random, then they also need to be trained, and not just the last layers. The last layers are trained only when the neuron is already pre-trained and you would like to add some new class.