How can I configure logs to be sent to specific containers and not all running on the host?

Question

elasticsearch

Configuring filtering for docker containers doesn't work.
I started Elastic Stack without any additional settings.
The problem is that the pattern filter is not working. I've tried various methods: by container name, adding labels, but I still see logs of all running containers on my host, not a specific one.
How can I still set up logs to be sent for specific containers and not all running on the host?

And here are my settings:

filebeat.yml

filebeat.autodiscover:
  providers:
    - type: docker
      labels.dedot: true
      hints.enabled: true
      templates:
        - condition:
            contains:
              container.labels.collect_logs_with_filebeat: "true"      # label name in service docker-compose file
              docker.container.name: "test_golang_app_2"
          config:
            - type: container
              format: docker # auto, docker, cli
              # stream: stdout # all, stdout, stderr
              #containers.ids:
              #  - "${data.docker.container.id}"
              paths:
                - "/var/lib/docker/containers/${data.docker.containers.id}/*.log"

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1

setup.kibana:
  host: "localhost:5601"

output.elasticsearch:
  enabled: true
  hosts: ["localhost:9200"]

output.logstash:
  enabled: false
  hosts: ["localhost:5044"]

processors:
  - drop_fields:
      fields: ["agent.ephemeral_id", "agent.hostname", "agent.id", "agent.name", "agent.version", "docker.container.labels.com_docker_compose_config-hash", "docker.container.labels.com_docker_compose_container-number", "docker.contain>
      ignore_missing: false

monitoring.enabled: false
logging.metrics.enabled: false
logging.level: debug
logging.selectors: ["*"]
logging.to_files: true

and for testing i have written a simple application in docker container which constantly sends data to stdout stream
docker-compose.yml

version: '3.7'

services:
  simple_golang_app:
    image: simple_golang_app
    container_name: simple_golang_app
    build:
      context: app/golang/
      dockerfile: Dockerfile
      args:
        TEST_ENV: $TEST_ENV
    networks:
      - net

  test_golang_app_2:
    image: test_golang_app_2
    container_name: test_golang_app_2
    build:
      context: app/golang/
      dockerfile: Dockerfile
      args:
        TEST_ENV: $TEST_ENV
    networks:
      - net
    deploy:
      labels:
        docker.container.labels.description: "collect_logs_with_filebeat"
        co.elastic.logs/enabled: "true" # for Filebeat
        collect_logs_with_filebeat: "true"
    labels:
        docker.container.labels.description: "collect_logs_with_filebeat"
        co.elastic.logs/enabled: "true" # for Filebeat
        collect_logs_with_filebeat: "true"

networks:
  net:
    driver: overlay

main.go

package main

import (
        "fmt"
    "io"
        "os"
    "time" // https://pkg.go.dev/time
)

func main() {
        fmt.Println("Print from the Go program")
        fmt.Println(os.Getenv("TEST_ENV"))

    io.WriteString(os.Stdout,"This is the line to standard output.\n")
    io.WriteString(os.Stderr,"This is the line for standard error output.\n")

    // print every 5 seconds how long the program is running
    for range time.Tick(time.Second * 30) {
      go func() {
        fmt.Println(os.Stdout, time.Now())
      }()
    }
}

Dockerfile

FROM golang:1.17-alpine

ADD main.go /home

WORKDIR /home

RUN \
    apk add --no-cache bash git openssh && \
    go get -u github.com/minio/minio-go

CMD ["go","run","main.go"]

I also asked a question on the official elk page, maybe there will be an answer.
https://discuss.elastic.co/t/filtering-setup-for-d...

Answer 1

use container autodiscovery hints .
The thing is that auto-discovery of containers reads the logs of all containers that are running on a host that listens to filebeat.
To filter out only the containers we need, we add a label to all containers:
docker-compose.yml

services:
  app:
    labels:
      co.elastic.logs/enabled: "false" # for Filebeat

in this case filebeat will not pick up logs from containers that have this label.
OR !!!! CIVILIZED METHOD :
filebeat.autodiscover:
providers:
- type: docker
hints.enabled: true
hints.default_config.enabled: false
and add
labels:
co.elastic.logs/enabled: "true" to the tracked container