9
9
99sun662022-03-23 16:39:58
Python
99sun66, 2022-03-23 16:39:58

Parser error, only the first item is parsed, why?

Parses only the first product, if there are several pages, then the first product from each.
How to fix?

from bs4 import BeautifulSoup
import requests


def parse():
    URL = 'https://www.sea.com.ua/ua/istochniki-pitaniya/v-korpusekozuhe/'
    HEADERS = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36'
    }

    response = requests.get(URL, headers=HEADERS)
    soup = BeautifulSoup(response.content, 'html.parser')
    items = soup.findAll('div', class_='product-list')
    comps = []

    for item in items:
        try:
            comps.append({
                'description': item.find('div', class_='description').get_text(strip=True),
                'title': item.find('div', class_='item-title').get_text(strip=True)
            })
        except:
            pass

    for comp in comps:
        print(f'Description: {comp["description"]}  Title: {comp["title"]}')


parse()

Answer the question

In order to leave comments, you need to log in

1 answer(s)
D
Dinili, 2022-03-23
@99sun66

items = soup.findAll('div', class_='col-sm-9 col-sm-pull-3')

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question