What is the best way to make a server architecture for a HiLoad site?

Question

Nginx

The bottom line is this: there is a site, attendance on which is growing weekly. For analytics, big data is collected about the logs of user actions on the site.
Now it's all done on the 1st server, the MySQL database. The database itself in compressed form occupies 500 megabytes, in unpacked 8 GB.
To upload a database dump to the server, you have to wait 4-5 hours.
This database has several tables that contain logs. So they occupy the lion's share (7.5 GB).
The load on the server is growing, there have already been failures on hard drives.
What can be used to improve website performance? its fault tolerance?
I see a solution to do this: remove tables with logs from the database, take a separate server, deploy PgSQL there and write all the logs there. Pictures are also placed on a separate server, or just take a storage for pictures. MySQL database is replicated to a separate server in master-master mode. But how to make it so that if one server fails, all requests go to another?

Answer 1

Your scheme lacks a balancer that will monitor which of the masters has died and switch to another.
For example: habrahabr.ru/company/bitrix/blog/146490
We ourselves use pgsql with wal logs (we have an old version of pg), we are testing elephants in the new version and maybe we will switch to them soon

Answer 2

According to the logs: there are many options, in addition to those already voiced, I can add:
1. Use AMQP (rabbitmq + federation) and federate the logs anywhere on any server, there on the receiving server you can already write anywhere you can even aggregate something on the fly and etc.
2. Use Scribe and use it to send messages to the log to any convenient collector-aggregator server.
Both solutions allow you to deliver messages to the log and, if one receiver server is unavailable, re-deliver to the backup. You can write anywhere, you can in mongodb, you can in mysql + tokudb (it has an excellent write speed and you can save space due to compression)
Regarding HA: redirecting requests in case of failure of one server, there are also many options:
1. Additional balancer in front of servers processing requests, nginx, haproxy, etc. (but here's the moment - this balancer will become SPOF)
2. Use solutions, for example heartbeat and when the server drops out - re-raise to the backup interface and extinguish on the first one (after a short time all requests will go to the new server)

Answer 3

try to look for a proxy under the database,

Answer 4

Logs do not become outdated? I'm in the plan, maybe it makes sense to delete logs older than 1 month, or dump them and pack them. And then along the crown, say, remove the excess.

Answer 5

For logging and analytics, it is beneficial to use NoSql, the same MongoDB has good write performance if transactions are not so important (which is usually true for analytics). For database replication, you can use MySql Galera habrahabr.ru/post/253869
Well, Nginx can distribute queries between servers, depending on availability and load.