Answer the question
In order to leave comments, you need to log in
Why does a 404 error occur when parsing data?
I'm trying to pull data from one site.
Access to data is available only after authorization, respectively, you have to remember cookies.
But, when you try to send a request, this page is returned.
However, as soon as I open the same link in the browser, the request passes quietly and returns what is needed.
In the parser, all headers are exactly the same as in a normal request from the browser, including the user agent.
Does anyone know what could be the issue here?
import requests
def main():
h = {'Cookie': 'cookname=fsink; cookid=9a465629c4b5460bfe473034a6ec03e0; php_session_id_real=3lqqoquddfbjeosj7qfuilgon',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0',
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Encoding': 'gzip, deflate',
'Accept-Language': 'ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3',
'Cache-Control': 'no-cache',
'Connection': 'keep-alive',
'DNT': '1',
'Host': 'somesite.ru',
'Pragma': 'no-cache',
'Upgrade-Insecure-Requests': '1'}
r = requests.get('http://somesite.ru/store/index.php', headers=h)
with open('question.txt', 'w') as f:
f.write(r.text)
if __name__ == "__main__":
main()
Answer the question
In order to leave comments, you need to log in
take a fidler
or test your code on https://httpbin.org/ , see what the server receives
such headers
'Cache-Control': 'no-cache',
'Connection': 'keep-alive',
'DNT': '1',
'Host': 'somesite.ru',
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question