How to organize the development process of a scalable system?

Question

Distributed Computing

Hello.
There is an understanding of the architectural structure of web applications oriented to high loads. However, there was no experience in developing any serious projects, and I wanted to know how to organize the initial stage of this development? What is the required software stack and/or other tools for this? By stack, I don’t mean in what language to back up, what database to use and what to use for caching, but I’m more interested in what the development process environment should be like - is it necessary to support “versioning” from the very beginning and if so, how to do it ? Where do you initially deploy the system - locally or not? if yes, then I would like to know in more detail what tools are needed for this and how, for example, then from the local area to project on a combat server without dancing with a tambourine? Is it necessary to use several nodes from the very beginning - for the simplest case, one for the base, one for the back, one for the front, or can you do everything on one and then somehow relatively simply scale to other nodes? In general, I would like to receive recommendations based on your experience of participating in the development of such systems that will simplify life.
In case it matters, the main bundle is: Python(Django) + PostgreSQL + Nginx. If there are reasons why the bundle is not the most successful, I will also be glad to "hear" them. The purpose of the event: to get a practical idea and experience in building a scalable system at all levels, albeit on a simple "home" example. Thanks

Answer 1

Your question as a whole does not make sense
Highload appears only on successful projects
If you are just jittery before the term
Take your project, any one
Put yourself locally on a virtual machine
And set yourself the task of filling it up
Then fight off your DDOS without losing legitimate users

how to organize the initial stage of this development? What is the stack of necessary software and / or other tools for this?

completely dependent on

By stack, I don’t mean in what language to back up, what database to use and what to use for caching, but I’m more interested in what the development process environment should be like - is it necessary to support “versioning” from the very beginning and if so, how to do it ?

Git should always be used
Versioning data is desirable
How to store them? - Make consistent backups

Where do you initially deploy the system - locally or not?

you have janga. When developing, use the built-in server, when deploying -nginx/uWSGI/postgreqsl

if yes, then I would like to know in more detail what tools are needed for this and how, for example, then to project from a LAN to a combat server without dancing with a tambourine?

ansible

Is it necessary to use several nodes from the very beginning - for the simplest case, one for the base, one for the back, one for the front, or can you do everything on one and then somehow relatively simply scale to other nodes?

no. At first, they simply buy more power, then they move the subd to a separate server, and only after this they think how to separate the backend.
Or in your task it is directly stated that you will have strictly more than 10k RPS

Answer 2

> How to organize the development process of a scalable system?
As for the organization of the process. Describe the sequence of activities for each role in the project. For example, a dev, having received a task, will transfer it to the status "in work", then transfer it to tl-a with the status "in review", etc.
Technically: start with standardization. This is a codestyle, this is a common environment for all girls, look towards vagrant, or docker. These are the rules for naming branches and commits. Think over and describe a diploma system without downtime, flow of migrations and rollback if something happens.