Answer the question
In order to leave comments, you need to log in
Did the parser give out None?
Made a parser, produces None. I assume that this is due to the huge amount of javascript on the site.
You need to parse the photo. It is possible to parse the price and name, but when trying to parse the src of a photo, it returns None.
Will one BS do it, or should I use Selenium? I would be very grateful :)
PS This is my first parser.
The code:
import requests
from bs4 import BeautifulSoup
HOST = "https://irr.ru/cars/passenger/"
URL = "https://irr.ru/cars/passenger/lexus/"
HEADERS = {
'user-agent': 'tyt user-agent',
'accept': 'tyt accept'
}
def get_html(url, params=''):
r = requests.get(url, headers=HEADERS, params=params)
return r
def get_content(html):
soup = BeautifulSoup(html, 'html.parser')
items = soup.find_all('div', class_='listing__item')
wgg = []
for item in items:
wgg.append(
{
'title':item.find('div', class_='listing__itemInner').find('div', class_='listing__itemColumn listing__itemColumn_main').find('div', class_='listing__itemTitleWrapper').find('a', class_='listing__itemTitle').find('div', class_='js-productListingProductName').get_text(),
'link':item.find('div', class_='listing__itemInner').find('div', class_='listing__itemColumn listing__itemColumn_main').find('div', class_='listing__itemTitleWrapper').find('a').get('href'),
'foto':item.find('div', class_='listing__itemInner').find('div', class_='listing__itemColumn').find('div', class_='listing__imageWrapper').find('img',).get('src') ,
'cena':item.find('div', class_='listing__itemInner').find('div', class_='listing__itemColumn listing__itemColumn_price').find('div', class_='listing__itemPrice').get_text(),
}
)
return wgg
html = get_html(URL)
print(get_content(html.text))
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question