How to parse the price of a skin on Steam?

Question

Python

Hello everyone, please help me with the parsing issue.
I need to pull out the price and name of the skin through bs4.


from colorama import init
init()
from colorama import Fore, Back, Style
from bs4 import BeautifulSoup
import requests

gun_name1 = 'M4A4'
skin_name1 = 'Desolate Space'
wear_name1 = 'Field-Tested'
headers = {'User-Agent' : 'Mozilla /5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36OPR/68.0.3618.197'}
steam_link = (' https://steamcommunity.com/market/search ?appid=730&q= ' + gun_name1 +'|' + skin_name1 + '%28' + wear_name1 + '%29'
soup = BeautifulSoup(full_page.content, 'html.parser')
skin_desc = soup.findAll('span', {'class':'market_listing_item_name'})
print(Back.CYAN + '{0:*^60}'. format('steamcommunity.com'))
print(Back.RESET + skin_desc[1].text)

This code gives an error:

*********************steamcommunity .com*************************
Traceback (most recent call last):
File "beta_marketChecker.py", line 95, in
print(Back.RESET + skin_desc[ 1].text)
IndexError: list index out of range

Here is the html itself from steam:

<span id="result_0_name" class="market_listing_item_name" style="color: #D2D2D2;">M4A4 | Безлюдный космос (Поношенное)</span>

PS I understand the essence of the error, but I don’t know how to fix it. I’m new to python

Answer 1

Be sure to pass Accept and Accept-Language in the headers (specifically in this case), since without these headers the block with goods is not returned:

spoiler

Here is the working code, though without color, I think you'll figure it out:

from bs4 import BeautifulSoup
import requests

gun_name1 = 'M4A4'
skin_name1 = 'Безлюдный космос'
wear_name1 = 'После полевых испытаний'
headers = {

    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36',
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
    'Accept-Language': 'ru-UA,ru;q=0.9,en-US;q=0.8,en;q=0.7,ru-RU;q=0.6',
}
steam_link = ('https://steamcommunity.com/market/search?appid=730&q=' + gun_name1 +'|' + skin_name1 + '%28' + wear_name1 + '%29')
full_page = requests.get(steam_link, headers=headers)
soup = BeautifulSoup(full_page.content, 'html.parser')

skins = soup.find_all('a',class_='market_listing_row_link')

for skin in skins:
  name = skin.find('span',class_='market_listing_item_name').text
  counts = skin.find('span',class_='market_listing_num_listings_qty').text
  price = skin.find('span',class_='sale_price').text.replace('От','').strip() #HACK


  print(f'{name}: {counts} - {price}')

Result

M4A4 | Безлюдный космос (После полевых испытаний): 461 - $11.48 USD
StatTrak™ M4A4 | Безлюдный космос (После полевых испытаний): 44 - $32.53 USD
M4A4 | Азимов (После полевых испытаний): 52 - $117.83 USD
M4A4 | Зірка (После полевых испытаний): 172 - $11.86 USD
M4A4 | Рентген (После полевых испытаний): 82 - $10.78 USD
M4A4 | Грифон (После полевых испытаний): 264 - $5.10 USD
M4A4 | Магний (После полевых испытаний): 6,243 - $0.28 USD
M4A4 | Преобразователь (После полевых испытаний): 434 - $2.01 USD
M4A4 | Смерч (После полевых испытаний): 154 - $1.90 USD
M4A4 | Неонуар (После полевых испытаний): 206 - $18.22 USD

Just pay attention, there are several prices for skins - sale price, regular price, etc. I don't understand steam.
And for the future, always check the response you get as a result of a request. Just not the status code (200,404, etc.), but save the response to a file and view the contents in notepad or in the browser. Example:

with open('index.html','w') as file:
  file.write(full_page.text)

This code will write the response to your request to the index.html file.

Answer 2

See how many elements are returned in skin_desc.
Swears that there are less than two of them.
There is either one element, then you need to write skin_desc[0]
Or there is nothing there. Then you need to figure out why nothing got into the sample