Answer the question
In order to leave comments, you need to log in
How to set cookies or go to the next page in python requests?
Good afternoon, I am writing a parser in python and I ran into a problem:
- there is a site.ru site with subdomains by city
- when you go to spb.site.ru, you are assigned cookies (name: chjs, value: 1, domain: spb.site.ru , path: / etc.)
- if you immediately pass r = requests.get('spb.site.ru/page'), then the site will display data for your city, and not for St. Petersburg.
There are two ways to get the page page - substitute cookies immediately or first go to the page with the city, I tried both methods, but it did not work out. (it seems to me)
def get_page(url):
r = requests.get(url.rsplit('/',maxsplit=1)[0])
r = requests.get(url)
return r.text
def get_page(url):
jar = requests.cookies.RequestsCookieJar()
jar.set(name='chjs', value='1', domain='site.ru', path='/')
r = requests.get(url)
return r.text
def get_all_page(html):
soup = BeautifulSoup(html, 'lxml')
search = soup.find('title')
return search
Answer the question
In order to leave comments, you need to log in
Navigating to another page with requests?
with requests.Session() as session:
session.post(url_0, data) # изменяешь свой город
response = session.get(url_1) # получаешь данные с другой страницы сайта
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question