S
S
Solvik742017-05-17 20:53:35
Database
Solvik74, 2017-05-17 20:53:35

How to unload the statistics table for such a project?

Hello!
Now I am developing a large project that keeps records of unique visitors, adding products from system users, placing orders, accepting payments, building reports and detailed statistics on all data.
The expected load on the record in the table of visits is at least 150 thousand records per day.
It is necessary to give users detailed statistics on visits and orders placed with a minimum delay.
For statistics, there is a separate table that summarizes all the necessary indicators from other tables (by insert triggers) up to the day. But even this table will have at least 350 million rows in a year.
In statistics, users can compile reports, both in total and by day/week/month.
Now MySQL 5.6 with InnoDB is used as a database. The tables are indexed.
The engine is written in Laravel.
Tell me, is it worth splitting the statistics table by partitioning, for example, by months, or is it better to start separate tables?
PS: I'm not strong in database architecture, so I'll be happy to listen to any of your suggestions! Thanks to all!)

Answer the question

In order to leave comments, you need to log in

2 answer(s)
T
terrier, 2017-05-18
@solvik74

Yes, partitioning by time is quite suitable here. There is not much difference between partitioning and separate tables in your case, but with partitioning, most likely it will be less trouble.
It is better to read statistics and analytics from a separate replica, not from the one that is being actively recorded.
There are improvements in 5.7 that are relevant to you, but in fact, after a minimal tweak, everything will take off quite to itself.
PS
Almost 2 records per second. You can even drive the base on an iPhone :)

D
Dimonchik, 2017-05-17
@dimonchik2013

you take something like that , fill it in, build business logic, you realize , for example, Clickhouse
is suitable for reports (but not for storing basic data) , and PostgreSQL as the main one, and there are no problems where to store and quickly get it, the same Monga -quick come down, or Elastic

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question