X
X
xaxa_durak2021-11-10 22:31:15
Python
xaxa_durak, 2021-11-10 22:31:15

Why are only the first four goods parsed into ozone?

It is necessary to parse all the products on the page, but for some reason only four are parsed. help me please

import requests
from bs4 import BeautifulSoup
from fake_useragent import UserAgent
#Mozilla/5.0 (Linux; Андроид 11; SM-G981W) AppleWebKit/537.36 (KHTML, как Gecko) Chrome/88.0.4324.152 Мобильный Safari/537.36
#
url = 'https://www.ozon.ru/category/bytovaya-himiya-36861/'

response = requests.get(url, headers={'User-Agent': UserAgent().chrome})
src = response.text

soup = BeautifulSoup(src, "lxml")
blocks = soup.findAll(class_ = "bh6 bi")
for block in blocks:
    product_url = block.find("div", class_ = "bi1").find("a").get("href")
    print(product_url)

Answer the question

In order to leave comments, you need to log in

1 answer(s)
V
Vladimir Kuts, 2021-11-11
@xaxa_durak

Because the rest are there in json. Which can be pulled out from there - yes, parsed ...

import json
import re
json_dump = re.search(r'JSON.parse\(\'({.*})?\'\)', src)

618cb4dc72b5e774768476.jpeg

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question