Answer the question
In order to leave comments, you need to log in
Pagination when parsing Avito in python?
The problem with parsing subsequent pages, in the version to which I finished, I can only pull out the first page, if you use this option pagination = soup.find('div', class_='pagination-root-2oCjZ'), then everything seems to be fine , the entire list of pages is displayed in html, but I don’t understand how to pull them out ... The pages themselves - https://www.avito.ru/murmanskaya_oblast/avtomobili...
import requests
from bs4 import BeautifulSoup
def get_pages_count(html):
soup = BeautifulSoup(html, 'html.parser')
pagination = soup.find('span', class_='pagination-item-1WyVp').findNext('span')['data-marker']
if pagination:
return int(pagination[-2])
else:
return 1
def parse():
html = get_html(URL)
if html.status_code == 200:
cars = []
pages_count = get_pages_count(html.text)
for page in range(1, pages_count + 1):
print(f'Парсинг старницы {page} из {pages_count}...')
html = get_html(URL, params={"p": page})
cars.extend(get_content(html.text))
print(cars)
else:
print('Error')
parse()
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question