Answer the question
In order to leave comments, you need to log in
Why is BeautifulSoup returning an empty list to me?
When I did such manipulations outside the class (procedurally), there were no problems with extracting links.
As soon as I created a class, everything stopped working.
After executing find_all I get an empty list.
Before that, I parsed 200 pages without any problems, but here you are! out of the blue :(
PS: soup.prettify gives the html code of the page. that is, the problem is in the find_all method
import requests
from bs4 import BeautifulSoup
import json
from datetime import datetime, timedelta
from textwrap import shorten
class SSLV:
def __init__(self):
self.root_url = r'https://ss.lv'
self.vacancies_url = r'https://www.ss.com/ru/work/are-required/filter/page1.html'
def make_request(self, url):
r = requests.get(url)
html = r.content
return html
def make_soup(self, html_code):
soup = BeautifulSoup(html_code, 'html.parser')
return soup
def calculate_pagination(self, url):
pass
def find_links(self):
# total_pages = self.calculate_pagination(self.vacancies_url)
# for p in range(total_pages):
# pass
html = self.make_request(self.vacancies_url)
soup = self.make_soup(html)
# print(soup.prettify())
links = soup.find_all('a', class_='am')
print(links)
ss = SSLV()
ss.find_links()
Answer the question
In order to leave comments, you need to log in
there is no find_all() method... where is it?
or should there be findall() here?
like this it works:
import re
links = re.findall(r'<a.*>.*</a>', str(soup))
print(links)
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question