Answer the question
In order to leave comments, you need to log in
How to analyze the logs for the correctness of the execution of the list of actions in tasks?
There is a large amount of logs (about 2 TB). The logs contain records about the execution of certain processes. Each process has its own log file. Each process periodically performs some kind of task, which has activity 1, activity 2, and so on. Sometimes these processes crash. It is necessary to analyze the logs and find out that the actions in the tasks were performed in the correct sequence. By what means can this be done?
I tried Apache Spark, but I can't figure out how to group actions by task in it. The matter is that until some time in logs identifiers of tasks in actions were not written down or were written down only for some.
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question