How to make a cross-platform library independent of Qt, but what would be the graphical component?

Question

Qt

this library itself will be implemented as a plugin. For example, if a plugin is loaded, then the Filter form will be from the plugin. And this window should be modal.

Answer 1

Your program must provide the plugin with methods with which it can build an interface.
Those. the whole gui in the main program, or in the main program, a method like "Draw a window according to the markup" sticks out. The plugin pulls this method and passes its markup there.

Answer 2

Well... if qt cannot be used for financial reasons (I see no other reason, because everything else will be more difficult anyway), then you can take the lightweight https://github.com/andlabs/libui and link it directly to each plugin - still won't add much weight. In fact, there are very few components.
There is also such a thing under it https://github.com/mimecorg/vuido
If you build your c++ application as a native nodejs module, then you can screw up a plugin system based on js scripts with rendering native windows.
In qt, an analogue of such a thing works out of the box through qml.