Y
Y
Yaroslav2019-11-03 23:04:46
Amazon Web Services
Yaroslav, 2019-11-03 23:04:46

Do I need to work out scaling / sharding when using cloud databases?

I have my own web application that (currently) stores data in MySQL on the same server. It works great on a single server scale with the current (rather low) load, but there is an interest in how it will work if suddenly there are a thousand times more users on it (and all the tables in the database become much larger, each query will be more laborious yes and the amount of requests to the database will increase) ?
In principle, each user works with their own data, and there is very little interaction between users (to simplify the question, we will assume that there is none at all). Therefore, now the working idea is to use horizontal sharding, on each server to keep as many users as it pulls out (and all scaling is to transfer it to the correct server when a user logs in).
If you switch, for example, to AWS RDS (or advise some other options?), will you have to somehow use similar schemes (separate databases / shards for each user group) or you can stupidly not worry about the load, it is any reasonable load pull out? (Would the amount you pay for this be more than if you rethink the scaling scheme)?
In general, there are some maybe good guides on easy scaling for my case (when each user works exclusively with his own data). In each table (of which there are many), have a user_id field and index it, and in each query indicate WHERE user_id=NNN ? But it seems quite laborious and there are many chances to miss this condition somewhere in the code. A simple solution suggests itself, so that each user has all his data in his database (very small) and the application works with it. But then we will have a lot of databases, and it will be a little ugly somehow. Maybe somehow you can get a similar effect while maintaining simplicity and reliability and a low risk of errors?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
I
Ivan Shumov, 2019-11-03
@xenon

I’ll be pretty sharp, but without water:
- cloud providers can’t do magic, only create resources
- relational databases even today can only vertical scaling
- yes, you have to think on your own
- you can use Read Replicas to make life easier, but get ready for delays
- relational databases are not able to withstand spike load (and the rest do it very badly)
Recommendations:
- plan scaling
- learn to understand how your data is used
- learn to microservices (and not because it is popular, but because it happens data isolation)
- learn other types of databases, for example, DynamoDB, although if you don’t delve into it, you can get even more problems
- remember that there is caching
- read that there are patterns like CQRS
- learn how to design PWA (Progressive Web Applications)
- throw it all away because it will be very expensive in the end

V
Vitaly Karasik, 2019-11-04
@vitaly_il1

Works great on a single server scale with the current (rather low) load, but there is an interest in how it will work if suddenly there are a thousand times more users on it

The answer is very simple - check with a stress test. That is, analyze the current queries to the database and simulate x1000 number of users. 90% that you can do without sharding, but again it will show.
Plus - from the very beginning, write the application so that read-only requests can be sent to a separate DB server, this will reduce the load on the master.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question