How to draw large data in a tree structure with dynamic loading on scroll?

Question

JavaScript

Guys, give advice.
There is data in a tree structure (about 100-150 thousand objects in the array, I unload it completely and store it in indexeddb)
They are drawn through a virtual scroll with the ability to hide / open branches and checkboxes.
Implemented by building a tree map, where in each node there are links to all descendants
(so as not to bypass recursively each time)
For 10-15 thousand, the flight is normal, but I think that from 50+ thousand it will slow down.
I want to embed dynamic data loading into the virtual scroll, but this is not easy to do with a tree.
If someone has implemented something similar, please hint in which direction to think. Now I have several options in my head and I don't know which way to go.
Now I'm thinking of picking up, for example, 1000 objects and calculating the delta, upon reaching which it will be loaded from the back.
But what to do when hiding a node? I think to keep a certain children_count in each node and increase the offset by it.

On the other hand, 150 thousand objects are not so many and you can basically
work with them by turning off reactivity and optimizing algorithms, adding web workers, etc.

Answer 1

If you don’t need to keep it immediately in reactivity, well, or just in visible processing everything at once, then intersectionObserver or a similar crutch stray can help (before, you had to describe behavior differently).
This will make it possible not to keep in memory and not process all the data and nodes at once, but it will be necessary to hurry up so that what is now in the viewport with a gap is processed correctly and what is not included in the DOM and memory is removed.
I'm just working with something similar now, but I have to keep all objects in reactivity at once. Fortunately, there are only decl, a couple of thousand.

Answer 2

Show descendants of the first level. If there are a lot of them, then load them gradually. When expanding one of the descendants, load its descendants (of the first level, without content) and display according to the same scheme. When closing - delete everything. Thus, the browser will only have the necessary minimum for rendering.
As for reactivity, it puts spokes in your wheels. As soon as you take the first task, you will start to take a wild drawdown in performance due to unnecessary redraws.