O
O
olamedia.2015-02-19 11:44:10
PostgreSQL
olamedia., 2015-02-19 11:44:10

Which database to choose for a large number of products?

Requirements/Features:
* Batch insert speed: 10-30 thousand rows per second, preferably < 5-10 sec.
* the number of entries can be approximately several tens of millions. (although theoretically it is possible to cut up to tens of thousands of positions or up to hundreds of thousands of positions, but in this case the number of tables will go off scale and there will be difficulties with selections)
* the database is used exclusively to store the fields key, value, modification time, indexing time.
* database selection is performed by key (for reading full information or making changes) and by time (for indexing in the search database)
In addition, what features should be considered when using the selected database in terms of batch insertion.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
R
raiboon, 2015-02-19
@w999d

Any of the databases listed will handle this without any problems. Eat Cassandra, and add Tokyo Tykoon - also suits your needs.
But are you absolutely sure that you will still have the original restrictions on sample types? If not really, then take postgres - you won't be mistaken, it will also digest all this, albeit more slowly, but there will be a significant field for maneuver in expanding the functionality.

O
olamedia., 2015-02-19
@w999d

In general, I thought, why not store all the rows of one price in one row of the table or even in a json file like this:

[{},{},{},{ '№ п/п': '2301',
    'Наименование товара': 'Шоколад НЕСТЛЕ батончик  Натс 50 гр',
    'Ед Изм': 'шт',
    'Цена': '21' },{},{},{}]

Indeed, for indexing, you will have to pick them up in the same pack, and the name with the price will still be in the index. Moreover, you can store the previous price list nearby and create/update a diff-file when loading to facilitate the work of the indexer.
The only downside is that you can’t save metadata like that, you still have to create a table for them. But the metadata table probably won't change that often.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question