A
A
animanshnik2019-05-27 22:52:08
Python
animanshnik, 2019-05-27 22:52:08

How to tell the parser what to look for?

import shutil
import requests
from bs4 import BeautifulSoup


count = 1
while count <= 3:

    url = "http://education.simcat.ru/school25/news/page{}/".format(count)
    page = requests.get(url).text
    soup = BeautifulSoup(page, 'html.parser')
    divs = soup.findAll('div', {'class': 'maintext'})
    print('Count', count, url)

    def find_content():
        for div in divs:
            div_title = div.find('div', {'class': 'menu'})
            link_text = div_title.find('a').text
            print(link_text)

            img = div.find('a')
            img_src = img.find('img').get('src')
            img_src = 'http://education.simcat.ru/school25/' + img_src

            response = requests.get(img_src, stream=True)
            with open(link_text.replace('/', '') + '.jpg', 'wb') as out_file:
                shutil.copyfileobj(response.raw, out_file)
            del response

        return

    count += 1

    print(find_content())

There is such a code. The problem is that he is not looking for the right one.
The result should be:
Count 3 education.simcat.ru/school25/news/page3
I ate my grandfather and so on and so forth, as a result the result is
Count 3 education.simcat.ru/school25/news/page3
None.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Andrey Burov, 2019-05-27
@BuriK666

your find_content returns nothing, print(find_content())hence None.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question