Answer the question
In order to leave comments, you need to log in
How to configure Filebeat to send data to a specific pipeline?
There is a web server (Ubuntu) running Nginx + PHP.
This server has Filebeat, which sends web server logs to the Elastic Ingestion node. That is, without Logstash, directly from Filebeat to Elastic. There isn't much traffic, so it's ok.
When I installed Filebeat, I tweaked the default pipeline a bit so that Russian letters in the URL are displayed correctly (urldecode).
Works all several months, stably. But every time I do an "apt upgrade" and if filebeat is updated, it starts writing to a new index, in which a default pipeline is created without my edits.
Q: How do I configure the Filebeat Nginx module to use my custom pipeline?
Now when I look at all my pipelines, here is how many there are:
elk.slavikf.com:9200/_ingest/pipeline/filebeat-*
filebeat-nginx-access-default: {},
filebeat-7.3.1-nginx-error-pipeline: {},
filebeat-7.4.1-nginx-error-pipeline: {},
filebeat-7.2.0-nginx-access-default: {},
filebeat-7.3.2-nginx-error-pipeline: {},
filebeat-7.4.1-nginx-access-default: {},
filebeat-7.3.1-nginx-access-default: {},
filebeat-7.3.2-nginx-access-default: {},
filebeat-7.2.0-nginx-error-pipeline: {}
- module: nginx
# Access logs
access:
enabled: true
# Set custom paths for the log files. If left empty,
# Filebeat will choose the paths depending on your OS.
var.paths: ["/var/log/nginx/*/*access.log"]
# Convert the timestamp to UTC
var.convert_timezone: true
# The Ingest Node pipeline ID associated with this input. If this is set, it
# overwrites the pipeline option from the Elasticsearch output.
output.elasticsearch.pipeline: 'filebeat-nginx-access-default'
pipeline: 'filebeat-nginx-access-default
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question