How to collect logs from Docker containers outside of these containers?

Question

elasticsearch

There are microservices described in docker-compose
Everyone knows how to write logs in File, Console, ...
I want a centralized collection of logs. Let's say the same ELK (ElasticSearch, LogStash, Kibana)
If you configure the logging configuration of each microservice to log directly to Elastic or LogStash - there are no problems here, but you want to beat configuration and hard binding in a specific container for collecting logs.
I would like this scheme:

To remove logs from files that are registered as docker volume, or from stdout to pick them up somehow, and then send them to the same LogStash or Elastic. Pick up logs outside the container, not inside!
How can this be done? Does docker itself have a mechanism for sharing log files between containers, or can it read the console, because the same Kitematic shows logs from the console without problems, then it’s really possible to pull it out.
Can anyone share a working docker-compose.yml file, who has already done a similar scheme?

Answer 1

You can add Filebeat to your docker-compose container , for example. And use it to redirect all logs to ELK

Answer 2

Docker has a great ability to log messages and send them using different drivers.
https://docs.docker.com/config/containers/logging/...
One that works great with Logstash is gelf Docker
run example
run -d \
-p 80:80 \
--name nginx-logs \
--log-driver=gelf \
--log-opt gelf-address=udp://ELK-host:12201 \
--log-opt tag=site.com.ua \
nginx
If this is docker-compose, then so :

services:
  nginx:
    image: nginx
    hostname: nginx-log
    logging:
       driver: gelf
       options:
         gelf-address: "udp://ELK-host:12201"
         tag: "site.com.ua"

In turn, the Logstash config for the input directive is

input {
  beats {
    port => 5044
  }
  gelf {
    port => 12201
    type => gelf
  }
}

The only thing, now I'm looking for how to separate these containers using Logstash in Kibana using tags, when there are a lot of them. I'm looking for the construction syntax for the output directive.