J
J
jenya77712020-01-02 15:29:48
Node.js
jenya7771, 2020-01-02 15:29:48

How to get messages from a topic in batches of Kafka?

Hello, I need to write all the API logs in ClickHouse, but I need to send them in batches, so I want to first collect the logs in Kafka, and then pick up the logs from you in batches once a minute and are already in ClickHouse.
Tell me how to take messages from the topic in batches and, after taking it, delete these messages?
I will work with Kafka through NodeJs

Answer the question

In order to leave comments, you need to log in

1 answer(s)
M
Maxim Babichev, 2020-01-03
@jenya7771

In Clickhouse, you can set up integration with kafka.
https://clickhouse.tech/docs/en/engines/table-engi...
Clickhouse itself will take data from kafka and write it down.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question