How to parse the prices of games from the Steam store?

Question

Python

Good day, habr users, I decided to delve into the topic of site parsing, I watched a video on YouTube. I decided to try to write the code myself, but ran into a problem. That I can’t find an article, no one parses a site where items are loaded down the page. In general, I want to parse titles / prices in the Steam store. Ordered the search so that only ~215 products remain. But the parser records only the first 50 elements of the store (because the products are loaded in 50 elements).

Climbed into the Network, into XHR requests. I see that every 50 elements a query is sent with the parameter start=50, 100, 150, 200...

But I don't understand what to do with it. Tell? Thank you! What library / resource to use, can you throw an article? I didn't find anything

Answer 1

You need something that can render js. For example PhantomJS, Selenium, Splash

Answer 2

Steam stupidly blocks for a large number of requests. If you need names \ prices, it is better to use services that provide such information ... https://hexa.one for example have an API.