Answer the question
In order to leave comments, you need to log in
Why are only the first four goods parsed into ozone?
It is necessary to parse all the products on the page, but for some reason only four are parsed. help me please
import requests
from bs4 import BeautifulSoup
from fake_useragent import UserAgent
#Mozilla/5.0 (Linux; Андроид 11; SM-G981W) AppleWebKit/537.36 (KHTML, как Gecko) Chrome/88.0.4324.152 Мобильный Safari/537.36
#
url = 'https://www.ozon.ru/category/bytovaya-himiya-36861/'
response = requests.get(url, headers={'User-Agent': UserAgent().chrome})
src = response.text
soup = BeautifulSoup(src, "lxml")
blocks = soup.findAll(class_ = "bh6 bi")
for block in blocks:
product_url = block.find("div", class_ = "bi1").find("a").get("href")
print(product_url)
Answer the question
In order to leave comments, you need to log in
Because the rest are there in json. Which can be pulled out from there - yes, parsed ...
import json
import re
json_dump = re.search(r'JSON.parse\(\'({.*})?\'\)', src)
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question