K
K
Kirill Makhonin2014-04-27 20:25:39
Database
Kirill Makhonin, 2014-04-27 20:25:39

Which database to choose for the event monitoring system?

When developing a system for collecting event statistics, I came across the fact that I need to choose a database that will solve the following tasks:

  • Fast write speed data can come quite often
  • Easily expandable (most of all I like cassandra's flexibility)
  • Desirable support out of the box (well, or with small dances) MapReduce
  • Possibility to select not only by keys
  • Possibility to select not only by indexes
  • High speed of large selections (the selections themselves are not complicated, in two parameters, but they return a large set of rows)
  • Normal storage > 10^9 records (in the whole cluster, but not manual, but automatic balancing of records)
  • Desirable built-in replication (for example, like the same cassandra)
  • Highly desirable free license
Considered options:
Cassandra
+ Simple extension
+ Replication
+ Can be friends with Hadoop - Sampling
limits
Memcached
- Sampling only on
MySQL/MSSQL/PostgreSQL
keys - Scalability
? Normality of replication for this task
PS I understand that the ideal tool can be found, but still I want to make the most correct decision

Answer the question

In order to leave comments, you need to log in

4 answer(s)
J
jj_killer, 2014-04-28
@jj_killer

Cassandra, she is the fastest to write, she is good with replication and Hadoop. One minus, it is not dev-friendly, every new developer will delve into even the basics for a long time. At the expense of sampling, here you need to think in advance about the scheme.
You can also try Amazon RedShift (with COPY insert), so you can save the convenience of the selection, but this is a completely closed solution.
CouchDB is not at all clear what has to do with it.

S
slaykovsky, 2014-04-27
@slaykovsky

CouchDB

A
afiskon, 2014-04-28
@afiskon

See if it suits your needs or not - eax.me/riak

D
Dan, 2014-07-01
@golotyuk

MySQL + replication + sharding will do. Pros - the technology has been tested for years, it is used in a huge number of cases (Facebook uses MySQL as a key-value store). Cons - balancing must be done by hand.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question