A
A
abrcdns2022-01-05 20:11:39
Python
abrcdns, 2022-01-05 20:11:39

How to parse a website in Python with authorization through Steam?

Hi all !

Recently, I had an idea to create a balance parser for a site for the exchange of skins in CS GO.
I started looking for information and decided to use Python for this with the BeautifulSoup and requests libraries.
As a result, I reached a dead end, as the site requests authorization through Steam.
As far as I understand, in order to authorize on the site, you need to send him a POST request. I logged in to this site through Steam, and I looked at the POST request called "login" that he receives after I log in. I tried to repeat it - to no avail. When you try to parse - the site issues a version for an unauthorized user.
My code :

from bs4 import BeautifulSoup
import requests

s = requests.Session()

url = "https://old.cs.money/"

headers = {
    "Accept": "*/*",
    "user-agent": "Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Mobile Safari/537.36"
}

postData = {
    '_ga': '**************************************',
    'steamMachineAuth76561198135087873': '**************************************',
    'browserid': '**************************************',
    'recentlyVisitedAppHubs': '**************************************',
    'cookieSettings': '**************************************',
    'timezoneOffset': '**************************************',
    '_gid': '**************************************',
    'strInventoryLastContext': '**************************************',
    'sessionid': '**************************************',
    'steamCountry': '**************************************',
    'strResponsiveViewPrefs': '**************************************',
    'steamLoginSecure': '**************************************',
    'sessionidSecureOpenIDNonce': '**************************************'

}


responce = s.post(url, data=postData, headers=headers).text
test_resp = s.get(url, headers=headers).text

with open("index.html", "w", encoding='utf-8') as file:
    file.write(test_resp)

soup = BeautifulSoup(url, "lxml")
info = soup.find("span", id_="balance")
print(info)


The postData dictionary is the cookies that the post request sends.
I am new to this business and just studying this direction, maybe one of you could give me advice or throw off articles / books on this topic? On the Internet, I did not find almost any information about parsing sites with authorization through third-party services. I will be grateful for any help!

Answer the question

In order to leave comments, you need to log in

1 answer(s)
K
Kadabrov, 2022-01-05
@abrcdns

You log in, go to the developer tools,
find the network user_info and
take the cookies and user agent from there, pack it into a dictionary and send a get request, get json
url = 'https://old.cs.money/user_info'

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question