J
J
jenya77712018-11-23 20:26:46
Node.js
jenya7771, 2018-11-23 20:26:46

How to properly work with ClickHouse in nodeJs?

Hello, there is a site where I need to save all the transitions on the site, and since this is a lot of data on recommendations from the network, I decided to use ClickHouse . From the initial study of this database, I learned that the data needs to be inserted in batches. And the question arose, how to form and store this bundle of data, so that after, for example, once a minute, write to ClickHouse, and which library for nodeJs is better to work with?
I thought it was possible to synchronously write all transitions to a file, after saving in ClickHouse and deleting the file, this method is suitable, and how can it manifest itself if the site has a million transitions per day?
Thank you.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
A
Andrey Tsvetkov, 2018-11-24
@jenya7771

Write transitions to some temporary storage, file, radish, memcache, memory. Once every N seconds, transfer data from temporary storage to the clickhouse. With your loads (1 million per day), you can use absolutely any temporary storage, since this is not a load at all

B
bytdnel, 2021-08-24
@bytdnel

There is a library that out of the box provides caching of requests to a radish or node memory and returns an event when a batch of requests resolves. I think it will solve your problem

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question