K
K
kazmiruk2012-07-11 14:47:06
NoSQL
kazmiruk, 2012-07-11 14:47:06

How to organize the relationship of NoSQL storages on the server?

There is a project for which a high load is planned. Therefore, initially its architecture is planned in such a way that further horizontal scaling is possible without significant code refactoring. For this purpose, NoSQL MongoDB was chosen (it attracted autosharding and other goodies). Now All data is stored in this DBMS (including sessions). The question arose about transferring sessions to another, more productive storage along with other data (such as hit counters, etc.). Of the options (chosen by benchmarks and available description), I consider memcached and redis. Redis attracts by the fact that the data is flushed to disk and restored upon reboot (ideal for statistics and sessions), there are transactions (which is sorely lacking in mongo), but it seems to be lower in performance. Memcached - faster but less reliable (you cannot be sure that the session is not invalidated unexpectedly when the user makes, for example, an order) and all data must be duplicated. All this is planned to be done in the form of a cluster using a consistent hash. On the one hand, redis is ideal for sessions and statistics, and memcached is ideal for data cache, but does it make sense to start such a zoo (mongo, memcached, redis), will there be any tangible gain? Maybe you should use redis for the data cache (it seems to be able to do everything that memcached) and store sessions in it, or vice versa, store sessions in memcached along with the cache? Or maybe leave sessions in mongo altogether, write statistics there, and use memcached for cache? Or maybe there are some other solutions? I ask for advice. order) and all data must be duplicated. All this is planned to be done in the form of a cluster using a consistent hash. On the one hand, redis is ideal for sessions and statistics, and memcached is ideal for data cache, but does it make sense to start such a zoo (mongo, memcached, redis), will there be any tangible gain? Maybe you should use redis for the data cache (it seems to be able to do everything that memcached) and store sessions in it, or vice versa, store sessions in memcached along with the cache? Or maybe leave sessions in mongo altogether, write statistics there, and use memcached for cache? Or maybe there are some other solutions? I ask for advice. order) and all data must be duplicated. All this is planned to be done in the form of a cluster using a consistent hash. On the one hand, redis is ideal for sessions and statistics, and memcached is ideal for data cache, but does it make sense to start such a zoo (mongo, memcached, redis), will there be any tangible gain? Maybe you should use redis for the data cache (it seems to be able to do everything that memcached) and store sessions in it, or vice versa, store sessions in memcached along with the cache? Or maybe leave sessions in mongo altogether, write statistics there, and use memcached for cache? Or maybe there are some other solutions? I ask for advice. but does it make sense to start such a zoo (mongo, memcached, redis), will there be any tangible gain? Maybe you should use redis for the data cache (it seems to be able to do everything that memcached) and store sessions in it, or vice versa, store sessions in memcached along with the cache? Or maybe leave sessions in mongo altogether, write statistics there, and use memcached for cache? Or maybe there are some other solutions? I ask for advice. but does it make sense to start such a zoo (mongo, memcached, redis), will there be any tangible gain? Maybe you should use redis for the data cache (it seems to be able to do everything that memcached) and store sessions in it, or vice versa, store sessions in memcached along with the cache? Or maybe leave sessions in mongo altogether, write statistics there, and use memcached for cache? Or maybe there are some other solutions? I ask for advice.
Also, in order not to ask a second question, I'll ask here. There are no transactions in mongo. There are two collections, with counters. It is necessary to atomically decrement the counter in one collection and increase it by N in another. Is there any normal solution? While we are using crutches: we set the blocking flag on documents, save the current state, perform operations, remove flags. In case of an error, we return the old state, remove the flags. In general, it works, but it looks like a crutch.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
P
petrkozorezov, 2012-07-12
@petrkozorezov

1. Starting a zoo is evil, it can only be done if you clearly understand why and why, and it will still be evil. The simpler the system, the fewer errors it has and the cheaper it is.
2. Radishes (like memcache) are primarily a cache, and secondly, storage. Therefore, it must be used for its intended purpose (session and runtime statistics are good, counting that it will be dumped to disk is bad).
3. Why NoSQL? Just for the sake of performance and horizontal scaling? Maybe then it will be easier to use PostgreSQL or MySQL as Key-Value? lionet.livejournal.com/98362.html here is well written about this.
4. Before stopping at something, it's better to write tests, run them, and draw a conclusion from this :)
Drawing a conclusion from all of the above, it is better to think it over better, test and decide what is needed and why. The simpler the system, the better it is.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question