Answer the question
In order to leave comments, you need to log in
How to get messages from a topic in batches of Kafka?
Hello, I need to write all the API logs in ClickHouse, but I need to send them in batches, so I want to first collect the logs in Kafka, and then pick up the logs from you in batches once a minute and are already in ClickHouse.
Tell me how to take messages from the topic in batches and, after taking it, delete these messages?
I will work with Kafka through NodeJs
Answer the question
In order to leave comments, you need to log in
In Clickhouse, you can set up integration with kafka.
https://clickhouse.tech/docs/en/engines/table-engi...
Clickhouse itself will take data from kafka and write it down.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question