V
V
Vasily Sheshunov2022-01-16 16:12:38
Python
Vasily Sheshunov, 2022-01-16 16:12:38

How to get data from a website using BeautifulSoup?

Below is the page code that needs to be parsed by a loop from a in the div block (Italy, white, semi-dry, etc.), for further writing to a CSV file.

wrote this code:

product_data = soup.find(class_="product-snippet__info").find_all('div')
        for item in product_data:
            product_tds = item.find_all("a")
            title = product_tds[0].find("a").text
            title = product_tds[1].find("a").text
            title = product_tds[2].find("a").text
            title = product_tds[3].find("a").text
            title = product_tds[4].find("a").text
            title = product_tds[5].find("a").text
            title = product_tds[6].find("a").text
            title = product_tds[7].find("a").text


But an error occurs that the data was not found ( title = product_tds[0].find("a").text
AttributeError: 'NoneType' object has no attribute 'text')

Tell me where I went wrong?

#<div class="product-snippet__info">
#                                                    <div class="product-snippet__info-item">
#                                <span class="product-snippet__info-title">Страна:</span>
#
#                                                                                                   <a href="/catalog/vino/filter/country-italiya/">Италия</a>
#                                                          </div>
#                                                 <div class="product-snippet__info-item">
#                            <span class="product-snippet__info-title">Цвет:</span>
#
#                                                                                                   <a href="/catalog/vino/filter/color-beloe/">белое</a>
#                                                            </div>
#                                                    <div class="product-snippet__info-item">
#                                <span class="product-snippet__info-title">Сахар:</span>
#
#                                                                                                    <a href="/catalog/vino/filter/sugar_type-polusukhoe/">полусухое</a>
#                                                            </div>
#                                                    <div class="product-snippet__info-item">
#                                <span class="product-snippet__info-title">Объем:</span>
#
#                                                                                                    <a href="/catalog/vino/filter/volume-0_75/">0.75&nbsp;л.</a>
#                                                            </div>
#                                                    <div class="product-snippet__info-item">
#                                <span class="product-snippet__info-title">Виноград:</span>
#
#                                                                <a href="/catalog/vino/filter/grape-muskat_zheltyy/">мускат желтый 100%</a>                                                            </div>
#                                                    <div class="product-snippet__info-item">
#                                <span class="product-snippet__info-title">Производитель:</span>
#
#                                                                                                    <a href="/catalog/vino/filter/manufacturer-colterenzio/">Colterenzio</a>
#                                                            </div>
#                                                    <div class="product-snippet__info-item">
#                                <span class="product-snippet__info-title">Регион:</span>
#
#                                                                                                    <a href="/catalog/vino/filter/region-trentino_alto_adidzhe/">Трентино-Альто Адидже</a>
#                                                            </div>
#                                                    <div class="product-snippet__info-item">
#                                <span class="product-snippet__info-title">Стилистика:</span>
#
#                                                                                                    белое - яркое, из ароматических сортов винограда                                                            </div>
#                                            </div>
#                    <!-- #INFO -->
#                
#                            </div>
#            <!-- #DETAIL -->
#        </div>
#    </div>
#    <!-- #TOP PART -->

Answer the question

In order to leave comments, you need to log in

1 answer(s)
S
soremix, 2022-01-16
@SoreMix

Because in this block

product_tds = item.find_all("a")
title = product_tds[0].find("a").text

You have already found the tag aand inside it you are again looking for the Plus tag, in principle, all cycles are confused. will always contain one element. Because it's inside one of eight nested divs. So we remake the logic in principle a
title = product_tds[0].text
product_tds

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question