Answer the question
In order to leave comments, you need to log in
How to parse json in logstash?
Hello.
I am fetching data from a table from a database using the JDBC Source Connector in a Kafka topic and fetching it with logstash.
data is obtained in the following form:
"{\"schema\":{\"type\":\"struct\",\"fields\":[{\"type\":\"int64\",\"optional\":false,\"field\":\"FIELD_1\"},{\"type\":\"int64\",\"optional\":false,\"field\":\"FIELD_2\"},{\"type\":\"int32\",\"optional\":true,\"field\":\"FIELD_3\"}],\"optional\":false,\"name\":\"TABLE\"},\"payload\":{\"FIELD_1\":20,\"FIELD_2\":4,\"FIELD_3\":52}}"
{
"@timestamp" => 2021-09-06T09:34:45.914Z,
"message" => {
"payload" => {
"FIELD_2" => 2,
"FIELD_1" => 26,
"FIELD_3" => 57
},
"schema" => {
"fields" => [
[0] {
"field" => "FIELD_1",
"optional" => false,
"type" => "int64"
},
[1] {
"field" => "FIELD_2",
"optional" => false,
"type" => "int64"
},
[2] {
"field" => "FIELD_3",
"optional" => true,
"type" => "int32"
}
],
"name" => "TABLE",
"optional" => false,
"type" => "struct"
}
},
"@version" => "1"
}
filter {
json {
source => "message"
target => "message"
}
mutate {
add_field => {
"FIELD_1" => "%{[message][payload][FIELD_1]}"
"FIELD_2" => "%{[message][payload][FIELD_2]}"
"FIELD_3" => "%{[message][payload][FIELD_3]}"
}
convert => {
"FIELD_1" => "integer"
"FIELD_2" => "integer"
"FIELD_3" => "integer"
}
remove_field => [ "message" ]
}
}
{
"FIELD_3" => "52",
"FIELD_2" => "4",
"@timestamp" => 2021-09-06T13:01:58.448Z,
"FIELD_1" => "20",
"@version" => "1"
}
{
"@version" => "1",
"FIELD_1" => 20,
"FIELD_2" => 4,
"FIELD_3" => 52
"@timestamp" => 2021-09-06T09:34:45.914Z
}
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question