Neural network training - how to interpret the result?

Question

Python

Train the network. There are such results. It is clear that everything is bad, but it is not clear how, by how much and why.
I tried a different number of layers, neurons in a layer. I even tried the dropout, although it doesn’t smell like retraining. The figures do not differ from those indicated below (plus or minus one percent).
Epoch 1/100
200000/200000 [=======] - 7s 37us/step - loss: 1.0738 - acc: 0.4021 - val_loss: 1.0683 - val_acc: 0.4061
Epoch 51/100
200000/200000 [===== ==] - 7s 33us/step - loss: 1.0289 - acc: 0.4711 - val_loss: 1.0816 - val_acc: 0.3950
Epoch 100/100
200000/200000 [=======] - 6s 32us/step - loss: 0.9823 - acc: 0.5193 - val_loss: 1.1003 - val_acc: 0.3998
On training examples, usually the accuracy in the first epochs soars to 70-90%, but I get 40% right away, and then it teleports like a plum in an ice hole. What is the network missing? What are the options?
And a bonus question. It is often said that the network needs to be trained for a long time, sometimes a week or more. What is it usually connected with - with the sample size or with the number of epochs?
Is there a typical number of epochs after which it is clear that it will not take off? For example, if we randomly select network hyperparameters, then after how many epochs can we already draw conclusions?

Answer 1

Since there is overfitting - try to unscrew the dropout a little more. Between standard Dense layers 0.5 is the traditional value. I saw articles where the dropout was done as much as 0.8-0.9 just because there was little data.
Also try Adam as an optimizer.
Networks that are trained for weeks are all sorts of monstrous constructions, where there are 20 layers of 500 neurons each. And tens of millions of examples for learning.