V
V
Valeriy Solovyov2014-12-10 07:24:31
MySQL
Valeriy Solovyov, 2014-12-10 07:24:31

Where and how to store recommendation service data?

Good day to all
I'm making a recommendation server: rating, reviews, recommendations.
(Thousands of recommendation objects + reviews) * (different languages). Read operations heavily outnumber write operations (>>5x).
Question: how to build a fault -tolerant service.
Oftop:

For some sites (php+mysql) there was no normal hosting. That technical work, then the server hung up. As a result, I made Master-> multi Master mysql replication + dynamic dns (SP is returned to "live" servers). 3 VPS allowed to work without a big downtime + when the hosting server "burned out", nothing was damaged.
That's just MMM replication worked with problems due to the remoteness of servers in data centers. I had to write bone scripts =(

Back:
At first I wanted to do everything on a SQL database using Master-> Salve replications:
  • a separate daemon for adding comments / descriptions, which puts everything into a queue, which is processed on the master.
  • The site itself will read data from the slaves
Cons for me:
  1. SQL architecture
  2. configuring Master->Salve replication
  3. If the master falls, then you need to keep the queue = how to ensure the safety of the queue
  4. If the master is absent for a long time?
Then I thought that it could be implemented on MongoDB:
  • There is replication
  • No DBA needed =)
  • Speed ​​up
  • You do not need to implement work with the queue first, you can leave it for later
From what I don't understand in this case:
  • If VPS in different data centers, then what are the consequences?
  • If the nodes fall off, will there be performance degradation?

Answer the question

In order to leave comments, you need to log in

4 answer(s)
A
Alexander Nikitin, 2014-12-10
@padla2k

What for to you for such task SQL ? Look towards NoSQL or engines like Elasticsearch...

L
lega, 2014-12-10
@lega

If VPS in different data centers, then what are the consequences?

It should work fine, in one project I made a node on my home computer with a dynamic ip (via OpenVPN) for backups, it worked stably.

V
Vlad Zhivotnev, 2014-12-10
@inkvizitor68sl

Mysql Percona Galera is your answer. Just don't forget to back up and write _always_ only to one node (don't care which one, just on synchronous writes to different hosts, the galley is noticeably dull).
3 DCs - well suited [email protected], [email protected], [email protected]

G
Gleb Palienko, 2014-12-10
@ftdgoodluck

In general, mongo fits well with your case.
But in the paragraph "DBA is not needed" I can argue with you - with mongo you will sometimes find yourself in situations "the request slows down, but I don't understand why, but everything was cured by restarting the daemon." And Google, due to the youth of Mongi, will not always be able to help.
And now for your questions:
1. There should be no problems. Just keep in mind that when the master falls, one of the slaves takes on the role of the master, and the load on it will change, which must be taken into account in planning the server architecture.
2. Well, when the slave crashes, your reading performance will drop accordingly). And when the master falls somewhere in 1-4 minutes (depending on the configuration), a new master is selected. Here, the performance may drop a little at the moment when the old master turns on and synchronization begins, but again, it depends on the configuration and data volumes.
If necessary, I can consult in more detail on Skype)

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question