J
J
Junior0072016-10-28 07:18:48
Database
Junior007, 2016-10-28 07:18:48

What are the nuances in big data?

Good morning everyone!
Such a question, for the purpose of training, I am going to develop a web parser - it will collect certain information from the Internet, I assume that the table will have about 20 columns with different parameters.
But the records, presumably, are about 500,000 - 1,000,000.
And, of course, the question arises - how to do all this correctly?
I’ll figure it out with the parser, I’m interested in exactly how to optimize data retrieval from the database? To make it work quickly and optimally even with a large number of requests, say, 20k - 50k per day (~10 hours a day)?
What can you immediately advise? What DB to use? What are the settings? DBMS? Books, articles?
By the way, I'm going to write in C++.
Experience in this area so far:
I just know how to create a database in MySQL, respectively SQL itself, i.e. I don’t know any subtleties of working with BIG DATA. While at the level of Select, Insert, etc.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
D
D', 2016-10-28
@Denormalization

500,000 - 1,000,000 is not BIG DATA. Any DB with any settings won't even notice.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question