Where to save the same type of large dataset in Python?

Question

Python

For example, I need to process 1 billion integers. Where to store them? The list will be long.
Or, for example, save 1 billion dictionaries, but where?

What is used to process large amounts of data?

Answer 1

Database on amazon, a couple of mower bucks and store whatever you want.

Answer 2

Well, for example, numbers (and not only) - it is convenient to store in arrays that numpy can handle. And with very large volumes, you can work with h5py, which works with data in HDF5 format

Answer 3

sqlite3 .