What to use to automate actions in the browser in parallel or multithreaded mode?

Question

Python

Hello! Please tell me which system or library is better to use to automate actions with the work of many browsers, the programming language is not so important, the main thing is that it is convenient to work with it.

And so, where I haven’t read it, they talk about Selenium , but it doesn’t work well for me in parallel or multi-threaded mode, it uses a lot of RAM and CPU power. I also read about Selenium Grid and Selenoid , but did not know what to choose and how to configure.

Please tell us what is better to use to automate actions in browsers. And how to run it so that everything works ( better with examples in C# or in Python ).

I would be grateful if anyone could help me with these questions.

Whoever finds it difficult to explain will throw at least a link where you can read and find the answer.

And also do not blame me if I wrote something wrong.

Answer 1

but it does not work well for me in parallel or multithreaded mode uses a lot of RAM and CPU power

Exactly. It just needs to be taken into account when calculating the required capacities. It's getting better and better these days. SSD drives and powerful multi-core CPUs.
Personally, my scheme for the minimum consumption of resources looked like this. This is PhantonJS 1.8 (now no longer supported) which ran with 30 webdrives. That's exactly right the program was launched in this amount. Each copy resident ate ~ 200-900 MB and constantly leaked from memory (at > 1.5 GB, the copy was hard killed). Each copy listened to its own port with webdrive commands. Everything was spinning on, if not mistaken, 32GB of RAM and 16 CPU. I gave out such a cluster something about 10RPS.
Those. you will have to look towards cluster solutions with a more or less serious approach to such a task. Selenium Grid as a whole is certainly a bit old. Selenoid looks fresher. Look towards ready-made cluster solutions based on these technologies, or write your own binding. When working on webdrive, the language is basically not important.

Answer 2

You need headless chrome and interfaces to work with it from what you need
. In general, I recommend to hesitate and write all your multithreading in manual mode, it will be more efficient and more expensive

Answer 3

I use CefSharp in C#, I don't know how it will suit you.

Answer 4

c# or rather .net comes with a WebBrowser control, it is a full-fledged browser (it used to be an Internet explorer, now it’s xs edge or already on webkit, if you use mono it will be firefox), it’s incredibly easy to work with it, although in practice instead of poking around in the DOM of a document using .net is better to do this by embedding javascript inside the page.
The work looks something like this, you throw a control on the form, you get access to the webControl.document document on the onload event, then these are already browser dom methods. Tip, instead of adding a script to the page, it's better to run it on the .setInterval timer with a zero interval, then the script will be launched in the context of the page and not some kind of .net sandbox, although the latter may be suitable for multi-threaded access (carefully the control does not imply multi-threading, but you can experiment by opening several controls and working with them from different threads, write back about the results)
This method is an order of magnitude more efficient than Silenium, since there is no intermediary, but still the resources will be used as a full-fledged browser (though a little less than the original one)