Answer the question
In order to leave comments, you need to log in
Do I need to work out scaling / sharding when using cloud databases?
I have my own web application that (currently) stores data in MySQL on the same server. It works great on a single server scale with the current (rather low) load, but there is an interest in how it will work if suddenly there are a thousand times more users on it (and all the tables in the database become much larger, each query will be more laborious yes and the amount of requests to the database will increase) ?
In principle, each user works with their own data, and there is very little interaction between users (to simplify the question, we will assume that there is none at all). Therefore, now the working idea is to use horizontal sharding, on each server to keep as many users as it pulls out (and all scaling is to transfer it to the correct server when a user logs in).
If you switch, for example, to AWS RDS (or advise some other options?), will you have to somehow use similar schemes (separate databases / shards for each user group) or you can stupidly not worry about the load, it is any reasonable load pull out? (Would the amount you pay for this be more than if you rethink the scaling scheme)?
In general, there are some maybe good guides on easy scaling for my case (when each user works exclusively with his own data). In each table (of which there are many), have a user_id field and index it, and in each query indicate WHERE user_id=NNN ? But it seems quite laborious and there are many chances to miss this condition somewhere in the code. A simple solution suggests itself, so that each user has all his data in his database (very small) and the application works with it. But then we will have a lot of databases, and it will be a little ugly somehow. Maybe somehow you can get a similar effect while maintaining simplicity and reliability and a low risk of errors?
Answer the question
In order to leave comments, you need to log in
I’ll be pretty sharp, but without water:
- cloud providers can’t do magic, only create resources
- relational databases even today can only vertical scaling
- yes, you have to think on your own
- you can use Read Replicas to make life easier, but get ready for delays
- relational databases are not able to withstand spike load (and the rest do it very badly)
Recommendations:
- plan scaling
- learn to understand how your data is used
- learn to microservices (and not because it is popular, but because it happens data isolation)
- learn other types of databases, for example, DynamoDB, although if you don’t delve into it, you can get even more problems
- remember that there is caching
- read that there are patterns like CQRS
- learn how to design PWA (Progressive Web Applications)
- throw it all away because it will be very expensive in the end
Works great on a single server scale with the current (rather low) load, but there is an interest in how it will work if suddenly there are a thousand times more users on it
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question