D
D
Dave2017-07-07 19:49:35
System administration
Dave, 2017-07-07 19:49:35

The data is constantly growing. How to be?

Developed a CRM system and successfully implemented it into operation. The volume of data began to grow constantly. There is only one VPS server so far. I am not strong in administration, I have to study and delve into it on the go.
In this regard, there are questions:
- How to calculate the maximum allowable load on the server?
It is clear that the withstand load depends on the server and software settings. But is there a common formula?
- The data is constantly growing. Soon the place on the hard server will run out.
What to learn to figure out how to distribute data between multiple servers and hards?

Answer the question

In order to leave comments, you need to log in

4 answer(s)
V
Vladimir Zhurkin, 2017-07-20
@djay

- How to calculate the maximum allowable load on the server?
It is clear that the withstand load depends on the server and software settings. But is there a common formula?

Alas, there is no such formula. Everything always has to be approximately by eye and coming from experience, if any.
There will always be a reason that the server / servers do not have enough power.
- The data is constantly growing. Soon the place on the hard server will run out.
What to learn to figure out how to distribute data between multiple servers and hards?

Here, in general, there is no clear answer, again, everything is based on how you write your CRM and its capabilities.
The easiest thing to go for is vertical scaling. Those stupidly increase the power of iron, put new HDDs, collect RAID, etc. This is the easiest way.
Now for the complex ones. As far as I understand, there are no problems with the database, at worst it can be completely transferred to a separate machine, so as not to mess with database clustering.
Now let's move on to files. At the moment, you can use distributed file systems such as ceph and glusterfs, ocfs2, etc. You can quite easily add pools of machines, unfortunately I'm not ready to say right now whether they have CEPH FS ready for use in production. You can use something like Hdfs from hadoop, but then you will need to pull the whole hadoop, mapreduse, etc., etc. In general, this is just an example, albeit a bad one.
Can be stored as blobs using eblob (made by Russian people) reverbrain.com/eblob
In general, there are many options, how to go depends only on you.

S
sim3x, 2017-07-07
@sim3x

How to calculate server load limit?
Stress Testing
The data is constantly growing. Soon the place on the hard server will run out.
Volume testing
What to learn to figure out how to distribute data between multiple servers and hards?
Sending clients to different servers

P
Puma Thailand, 2017-07-07
@opium

If you are used to current resources, then for a start you can consider current users equal to current resources, if your load grows arithmetically, then there are just twice as many users, then you need twice as many resources

R
Roman Mirilaczvili, 2017-07-08
@2ord

Uploaded files can be stored on an external server. It can be Amazon S3 or S3 compatible storage . Fortunately, there are a million components for working with S3 for different languages.
In addition, perhaps the structure of the database can be improved. Without the details of the scheme, I can not offer anything concrete.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question