Y
Y
Yaroslav2019-11-21 13:13:30
MariaDB
Yaroslav, 2019-11-21 13:13:30

What is the best way to store logs in a database?

The application now stores logs in the database. "Write only" logs are written quite actively, but read occasionally. The base is convenient in that:
1. The necessary records are easily filtered (relating to one of the many logging objects of the desired type)
2. It is quite important to always have records for the last time (day, week) at hand. with logrotate, this is all more complicated, because after rotation, the log becomes empty, and the logs are read through the web face, and not from the shell. With the database, the task of clearing everything that is older than a week and guaranteed to leave everything that is less than a week is made easier.
However, I'm a little worried about the load on the base. After all, she does not know that the logs are not needed (will not be required in the near future). Therefore, it can keep them in memory for a long time, cache them, consuming resources that will not get to the really important records in other tables (which I, the programmer, know that they are much more likely to be required from memory).
How good is the idea to store the logs in the database, or is it better to rewrite them to text files and occasionally, when someone came to look at the logs, parse the text?
Is it possible to somehow correctly "instruct" the DBMS (mariadb) that the data from the log table is "unimportant" and should be immediately dumped to disk, and from some other tables - "important", and should be kept in memory if possible?
Is it possible to somehow see how many resources the log table in the RAM (not on the disk) "eats" now?

Answer the question

In order to leave comments, you need to log in

4 answer(s)
_
_, 2019-11-21
@xenon

Look towards ELK, graylog and others like them.

I
Ivan Shumov, 2019-11-21
@inoise

The most logical thing is not to store logs in the database, damn it) You need to use your own tools for each task, for example, the same notorious ELK

Z
Zolg, 2019-11-21
@Zolg

it is better to store logs in a separate database,
even better - in a more suitable one (for example, elasticsearch)

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question