Answer the question
In order to leave comments, you need to log in
How to parse from multiple pages?
Look, now I'm parsing the result from only one page, let's say - https://www.google.ru/search?q=%D0%BA%D0%B0%D1%80%... https://www. google.com/search? from this page.
I need to parse the result from all the pages that exist, that is, start=10, so that after the page has been parsed, 10 more will be added to start=10.
code example:
def bf_parse(base_url, headers):
session = requests.session()
request = session.get(base_url, headers=headers)
if request.status_code == 200:
soup = bs(request.content, 'html.parser')
links = (d.find('a') for d in soup.find_all('div', attrs=('goodsDescription')))
users = {a.get('href') for a in links if a is not None}
f = open("user_baza.txt", "a")
f.write('\n'.join(users))
f.close()
print('\n'.join(users))
else:
print("ерор")
bf_parse(base_url, headers)
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question