What application architecture to use for monitoring?

Question

Monitoring

I want to write an application that would be a manager's workplace, in which he could get the information he needs on metrics, see graphs and the dynamics of metrics for different periods. There are about 50 managers in the company.
I need it to work without brakes. So that everyone can log in and see sales statistics and parameters derived from it.
How can I organize it correctly? Do I need to constantly make direct queries to the database to calculate each metric or data range? Or think about caching? Will users overload the server? What calculations should be done locally?
There are quite complex calculated parameters that can decently load the machine. Unfortunately, I cannot preliminarily calculate all the calculated parameters and write them down in a table, since the selected period is arbitrary and is not a multiple of, for example, a week.
Is there any literature I should read on this subject? I'm still a "green" developer.

Answer 1

I suggest storing data in the Prometheus time series DBMS and giving managers access to the admin panel in Grafana. Someone alone has to create panels with graphs by connecting to the data source's DBMS.
Added
In order not to slow down the main DBMS of the application, create a slave replica and connect Grafana to it.
Scheme:

App (W) -> DB master
Grafana (R) -> DB slave
   \
    (R)
   Prometheus

Added
I forgot that Prometheus itself takes metrics from devices that have an HTTP endpoint sticking out and issuing data at a strictly specified sampling rate (say, once every 5 seconds). Then maybe a slave replica is needed only for web services that take data from it.

Answer 2

Choose and use a BI reporting system that suits your needs and budget. Together with the Customer, go through the market of reporting systems. The decision on the acceptable budget, the list of necessary "beautiful things" and the methods for calculating indicators is to be taken by him.
In the question, write that there are "50 managers" in the company. Much depends on what you have in the company invest in the concept of "manager". If these are really people who make management and production decisions, then they most likely already have some kind of indicator and reporting system that they can maintain manually, in Excel, in the reports of accounting systems.
To reduce all these indicators into a more or less coherent system, and then force them to use it ... usually this task turns out to be more difficult than technical implementation.
Think about the intermediate storage of the NEEDED information in the necessary and sufficient volume. Implement data replication / collection of indicators into it from productive systems.
I advise you to forget about direct requests to productive work bases and work services. Otherwise, "manager reports" will lay down the main production processes, and the developer who created the system will be to blame.
In the most neglected case, it will be blamed for "failure to prepare reports for making managerial decisions", and "stop shipping / production / sales", and "reputational costs" both inside and outside the company.
Although if the main product of your company is managers and their beautiful reports, then it is the way to go.