D
D
dick19972019-04-05 09:42:17
Regular Expressions
dick1997, 2019-04-05 09:42:17

How to properly set up logstash processing via grok for dynamic logs?

I would like to set up the processing of logs from mikrotik, in logstash, with line breaking through GROK, if the logs come dynamic of this kind
1,2,3 text, etc.
1.2 text, etc.
1 text, etc.
To do this, add in patterns like this
TOPIC1 %{NOTSPACE:TOPIC1}
TOPIC2 %{NOTSPACE:TOPIC1},%{NOTSPACE:TOPIC2}
TOPIC3 %{NOTSPACE:TOPIC1},%{NOTSPACE:TOPIC2},%{NOTSPACE:TOPIC3}
TOPICALL (?: %{TOPIC1}|%{TOPIC2}|%{TOPIC3}) %{GREEDYDATA:Log}
in the logstash config, write patterns from our patterns
grok {
match => { "message" => "%{TOPICALL}" }
}
logs like "1 text, etc", "1,2 text, etc" and "1,2,3 text, etc",log output is obtained like this
{
"Log": "text, etc",
"TOPIC1": "1"
}
----
{
"Log": "text, etc",
"TOPIC1": "1,2"
}
--- -
{
"Log": "text, etc",
"TOPIC1": "1,2,3"
}
​​but if you do not use patterns in the config itself, but register direct processing of logs
grok {
match => {"message" = > "%{NOTSPACE:TOPIC1},%{NOTSPACE:TOPIC2},%{NOTSPACE:TOPIC3} %{GREEDYDATA:Log}"}
match => {"message" => "%{NOTSPACE:TOPIC1},%{NOTSPACE :TOPIC2} %{GREEDYDATA:Log}"}
match => {"message" => "%{NOTSPACE:TOPIC1} %{GREEDYDATA:Log}"}
}
then the logs already take the required form
{
"Log": "text, etc",
"TOPIC1": "1"
}
---
{
"Log": "text, etc",
"TOPIC2": "2",
"TOPIC1": "1"
}
---
{
"TOPIC3": "3 ",
"Log": "text, etc",
"TOPIC2": "2", "
TOPIC1": "1
"
them together depending on the type of log.

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question