How can you train a neural network using data from a sliding window?

Question

Machine learning

I have a 150x2,000,000 dataset where rows represent consecutive times. In this case, tables of 150x10,000 will be fed to the input of the neural network.

I would like to train the model on the strings [0, 9999], [1, 10000], [2, 10001]... of the original dataset, but obviously I can't store them in their pure form.

How can I train my model consistently on these "windows"?

At the moment I'm using Keras.Sequential, but in general I would like to know the principle that applies not only to this model.

Answer 1

The question is not about machine learning, but about the choice of data storage mechanisms and their efficient reading.
Do you have a dataset of 150x2000000 numbers?
for a float, this is a little more than a gigabyte , just keep it in RAM as a two-dimensional array