Why do I get a 404 when I send a request?

Question

JavaScript

Hello! I send a request to the site using the GET method, I get a 404 error in response. I pass all the headers, the link is correct. I also sent a request via postman and received the following response. Does anyone know how this can be bypassed? How do they understand that I'm not making a request through the browser?

Answer 1

Most likely, captcha protection is enabled on the site from third-party services like Cloudflare, which need to use JavaScript to implement captcha so that the user passes the captcha and goes to the site itself, or the site simply has protection that sends a JavaScript challenge to the client before rendering the site, which solved by the banal eval () when, after sending the response, the site sends the page content.
A more simplified scheme of work: the site sends a challenge on the first connection, after which it writes it to the "session" cell and sends this session or any data in the form of a Cookie that will identify that this particular user passed the captcha or challenge, subsequent JavaScript requests do not required because the site has identified you by the cookie.
It is unlikely that they understand that the request is not made through the browser, as an alternative to the request, it remains only to use libraries like Puppeteer that emulate the browser or execute JavaScript code; or you will have to write your own algorithms for bypassing protections.