What is batch_size in training models?

Question

Machine learning

For example here batch_size: 16. What does this mean? How is batch measured?

Answer 1

Usually, a batch is understood as two things - 1) during data preparation before training - the amount of data (number of rows) loaded into memory at a time, 2) during training - the amount of data (number of rows) fed to the model between loss function calculations. In the first case, the term chunk (chunk) is also used. Such a small value is the second case. Moreover, it is mentioned in the model training section. Those. for a single training step - we feed the model 16 rows of data.

Answer 2

How long to accumulate the gradient before changing the weights. It is measured in pieces of training examples.
You can learn from one example, but then it turns out that first the network will rush to the right by 10, then to the left by 7, and again to the right by 2. Or with a batch of 3 it will immediately go to 5 to the right. Those. instead of shying back and forth, it will gradually move little by little in the right direction. The value is selected by a scientific poke.