Why empty output when parsing in python?

Question

Python

I have a simple parser:

import requests
from bs4 import BeautifulSoup

URL = 'https://www.dns-shop.ru/search/?q=%D0%BA%D0%BE%D0%BD%D1%81%D0%BE%D0%BB%D0%B8&stock=soft'
HEADERS = {
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.61 Safari/537.36',
    'accept': '*/*'
}

def get_html(url, params=None):
    r = requests.get(url, params=params, headers=HEADERS)
    return r
def parse():
    html = get_html(URL)
    soup = BeautifulSoup(html.text, 'html.parser')
    items = soup.find_all('div', class_='catalog-product')
    print(items)

parse()

But it outputs [ ]

Where did I go wrong?

Answer 1

Content is generated by js, use Selenium or look for the request that the browser sends to the server and repeat it

Answer 2

In the headers, you need to set the cookies generated by the script.
If you enter without cookies, then the server at any address gives a page with a cookie generation script.