How to build a tree?

Question

Programming

I have a table that receives 2-10k events per second. All this with virtualization is shown on ui, but the problem is different.
The data that arrives in the collection needs to be combined in real time and a prefix tree built (after all, there is no id).
The data looks like this:

$SYS/scheme/node/beacon
$SYS/info/enum/beacon
$SYS/organization/production_calendar/beacon
$SYS/system/event_archive/beacon
$SYS/archive/search/beacon
$SYS/sys/services/beacon
$SYS/sys/broker/state/MQTTnet.LogViewer-23ac9203-50ad-4d90-8832-9ab0fb0f4a51
$SYS/sys/schedules/event/00:00:01

I must say right away that $SYS is not a node, there are many of them, but this is just an example.
I would like to understand the logic of how this is all done in real time, when the list with event names is expanding all the time. And how not to catch outofmemory or stackoverflow errors

Answer 1

I would queue. A line came, shoved it into the task, forgot. But here the question is how much memory is needed to process 10k per second + realtime. There is already a cache (in the database or on disk), which prevents the memory from overflowing. Empirically, we find out how many tasks fit into memory. If there are already a lot of tasks in the queue, we write to the cache. If not enough, then put in a queue.
Or was the question about parsing and building a tree? Then it depends on the AP.

Answer 2

Dear INTERNALINTERFERENCE , I didn't quite understand your question.
Are you asking about visualization ideas for 10k events?
Or how to build statistics on the occurrence of nodes (leaves) in a branch?
Or something else...
Explain, please.