I'm trying to parse all links from the Steam market, although the number of items on the market exceeds 10, the maximum number of links issued is 10?

Question

Python

import requests
from bs4 import BeautifulSoup

collections = [
    'community_5 - Vanguard',
    'bravo_ii - Alpha',
    'gods_and_monsters - Gods and Monsters',
    'bravo_i - Bravo'
]

def link_main(c):
    url_main = 'https://steamcommunity.com/market/search?q=&category_730_ItemSet%5B%5D=tag_set_' + collections[c].split(' - ')[0] + '&category_730_ProPlayer%5B%5D=any&category_730_StickerCapsule%5B%5D=any&category_730_TournamentTeam%5B%5D=any&category_730_Weapon%5B%5D=any&appid=730'
    info = requests.get(url_main)
    soup = BeautifulSoup(info.content, 'html.parser')
    for link in soup.find_all('a', class_='market_listing_row_link'):
        print(link.get('href'))

for c in range(0, len(collections)):
    link_main(c)

I'm parsing links, but I'm only getting ten in the output, from each collection, how to fix this?

Answer 1

you still need to go through the paginator:
https://steamcommunity.com/market/search/render/ ? query=&start=10&count=10 &search_descriptions=0&sort_column=popular&sort_dir=desc&appid=730&category_730_ItemSet[]=tag_set_community_5&category_730_ProPlayer[]=any&category_730_StickerCapsule[]=any&category_730_TournamentTeamry[]=any3&categogo