O
O
Ocean_772019-05-20 22:42:11
Python
Ocean_77, 2019-05-20 22:42:11

Why does a 404 error occur when parsing data?

I'm trying to pull data from one site.
Access to data is available only after authorization, respectively, you have to remember cookies.
But, when you try to send a request, this page is returned.
5ce3007c358b3065764388.png
However, as soon as I open the same link in the browser, the request passes quietly and returns what is needed.
In the parser, all headers are exactly the same as in a normal request from the browser, including the user agent.
Does anyone know what could be the issue here?

PS Sample request code
import requests

def main():
    h = {'Cookie': 'cookname=fsink; cookid=9a465629c4b5460bfe473034a6ec03e0; php_session_id_real=3lqqoquddfbjeosj7qfuilgon',
         'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0',
         'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
         'Accept-Encoding': 'gzip, deflate',
         'Accept-Language': 'ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3',
         'Cache-Control': 'no-cache',
         'Connection': 'keep-alive',
         'DNT': '1',
         'Host': 'somesite.ru',
         'Pragma': 'no-cache',
         'Upgrade-Insecure-Requests': '1'}

    r = requests.get('http://somesite.ru/store/index.php', headers=h)

    with open('question.txt', 'w') as f:
        f.write(r.text)


if __name__ == "__main__":
    main()

Answer the question

In order to leave comments, you need to log in

2 answer(s)
D
Dimonchik, 2019-05-20
@Ocean_77

take a fidler
or test your code on https://httpbin.org/ , see what the server receives
such headers

'Cache-Control': 'no-cache',
         'Connection': 'keep-alive',
         'DNT': '1',
         'Host': 'somesite.ru',

are never configured in the code at all
, well, maybe 1-2 of them can theoretically be configured (but not necessary), but this is definitely not Host
just noticed:
about cookies in headers - very funny

F
Fedor, 2019-05-20
@Fedor19

cookid=9a465629c4b5460bfe473034a6ec03e0;
It looks like the cookie is encrypted. Could this be the issue?

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question