How to emulate a browser without a browser?

Question

selenium

Hello, the task arose to emulate the work of a browser without a browser, because it is very resource-intensive, but so that requests to the site are sent in exactly the same way as if it were a browser. Is it even possible?

Answer 1

headless browser

Answer 2

what is a request?
These are headers + request body.
All you need to do is ensure that your headers and request body are no different from your browser. There are also cookies, just turn them on. So that when the server sends you a set-cookie in your software, this would also work.
Although, of course, different sites have different protection, but using the whole browser directly and not sending requests is, to put it mildly, labor -intensive
ps if you take puff GuzzleHttp is great, a lot, including cookies out of the box.
And don't forget the timeouts between requests.

Answer 3

There is no universal answer, there are only two browsers left in the world that developers pay attention to - chromium and all its derivatives (microsoft edge, google chrome, opera, ...) and thanks to mozilla firefox (and even then I often see that lazy people even ignore it but such sites and I ignore the principle, if I can).
All other solutions will not give a guaranteed answer or will be comparable in complexity to the reverse engineering of the site to which you need to send requests. Those. you parse the communication protocol of the site and the server backend, and simulate requests, the popular curl library.
The web server developer has a bunch of ways to make this job harder.
psadvice, firefox headless (used as the basis for the standard WebBrowser component in mono) consumes unlike less resources than chromium , by the way, this component still works in .net windows, it is based on internet explorer, you know what kind of site support.
Some 3-4 years ago, I would recommend the webkit library (components, for example, javafx WebEngine, work with it (it used to require about 50 MB of RAM per instance, now everything is 150-200), but still try.