Answer the question
In order to leave comments, you need to log in
Python requests post pagination, how to navigate to the next page?
All good evening
I am making a parser and the task was to navigate through the pages of the site. When I click on the page number in pagination, the page reloads, but when I navigate through requests.post(url), I get a 404 output.
In Chrome DevTools, it issues two requests.
First
example of my code
rom bs4 import BeautifulSoup
import requests
import config as conf
headers = {
'user-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0',
"Accept-Language": "ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7"
}
data = {
'authenticity_token': '',
'email': conf.LOGIN,
'password': conf.PASS
}
session = requests.Session()
session.get('https://www.strava.com', headers=headers)
def get_token():
response = session.post(conf.URL_AUTHORIZATION)
soup = BeautifulSoup(response.text, "lxml")
token = soup.find('input',{'name':'authenticity_token'}).get('value')
return token
def authorization():
print(get_token())
data['authenticity_token'] = get_token()
session.post(conf.URL_AUTHORIZATION, headers=headers, data=data)
def get_response():
url = 'https://www.strava.com/segments/9926585/leaderboard?club_id=225082&filter=overall&page={}&per_page={}&partial=true'.format(2, 25)
response = session.post(url, headers=headers)
print(response)
# Точка входа
def main():
authorization()
get_response()
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question