How to disguise yourself as a browser in node js?

Question

Node.js

Good afternoon, I have such a problem, I am writing a parser for one of the largest online stores, for personal purposes. I figured out the search mechanism in the browser, everything works as expected and predictable, but in the code it throws me a page with a captcha for any request to the api. Installed user agent:

axios.get(url, { headers:{'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36 '} })
    .then(response=>{
        console.log(response.data)
    })

to no avail, same page with captcha. How else can you disguise yourself as a browser? Perhaps there are other ways to solve this problem.

Answer 1

open the site under study
open devtools in chrome on the network tab
make an action that launches the request
copy yourself the request 1 to 1 from the network tab and try running it on your own
if it doesn’t help, then you can load the site in a real headless chrome through puppeteer and reproduce the user’s actions there