Answer the question
In order to leave comments, you need to log in
How to properly set up logstash processing via grok for dynamic logs?
I would like to set up the processing of logs from mikrotik, in logstash, with line breaking through GROK, if the logs come dynamic of this kind
1,2,3 text, etc.
1.2 text, etc.
1 text, etc.
To do this, add in patterns like this
TOPIC1 %{NOTSPACE:TOPIC1}
TOPIC2 %{NOTSPACE:TOPIC1},%{NOTSPACE:TOPIC2}
TOPIC3 %{NOTSPACE:TOPIC1},%{NOTSPACE:TOPIC2},%{NOTSPACE:TOPIC3}
TOPICALL (?: %{TOPIC1}|%{TOPIC2}|%{TOPIC3}) %{GREEDYDATA:Log}
in the logstash config, write patterns from our patterns
grok {
match => { "message" => "%{TOPICALL}"
}
}
logs like "1 text, etc", "1,2 text, etc" and "1,2,3 text, etc",log output is obtained like this
{
"Log": "text, etc",
"TOPIC1": "1"
}
----
{
"Log": "text, etc",
"TOPIC1": "1,2"
}
--- -
{
"Log": "text, etc",
"TOPIC1": "1,2,3"
}
but if you do not use patterns in the config itself, but register direct processing of logs
grok {
match => {"message" = > "%{NOTSPACE:TOPIC1},%{NOTSPACE:TOPIC2},%{NOTSPACE:TOPIC3} %{GREEDYDATA:Log}"}
match => {"message" => "%{NOTSPACE:TOPIC1},%{NOTSPACE :TOPIC2} %{GREEDYDATA:Log}"}
match => {"message" => "%{NOTSPACE:TOPIC1} %{GREEDYDATA:Log}"}
}
then the logs already take the required form
{
"Log": "text, etc",
"TOPIC1": "1"
}
---
{
"Log": "text, etc",
"TOPIC2": "2",
"TOPIC1": "1"
}
---
{
"TOPIC3": "3 ",
"Log": "text, etc",
"TOPIC2": "2",
"
TOPIC1": "1
"
them together depending on the type of log.
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question