How to bypass verification when parsing Aliexpress?

Question

Python

Greetings, you need to parse goods from aliexpress, but when the parser goes to the aliexpress page, it requires verification and, as a result, the parser gives an empty list:

Success!
https://aliexpress.ru/af/category/202000054.html?trafficChannel=af&catName=cellphones-telecommunications&CatId=202000054&ltype=affiliate&SortType=total_tranpro_desc&groupsort=1&isrefine=y&page=1
[]

The code itself:

import requests
from bs4 import BeautifulSoup
import csv

URL = 'https://aliexpress.ru/af/category/202000054.html?trafficChannel=af&catName=cellphones-telecommunications&CatId=202000054&ltype=affiliate&SortType=total_tranpro_desc&groupsort=1&isrefine=y&page=1'
HEADERS = {
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36', 'accept': '*/*'}

def get_html(url, params=None):
    r = requests.get(url, headers=HEADERS, params=params)
    return r


def get_content(html):
    soup = BeautifulSoup(html, 'html.parser')
    items = soup.find_all('li', class_='list-item')
    ali = []
    for item in items:
        ali.append({
            'Test': item.find('a', class_='item-title').get_text(strip=True),
        })
    print(ali)



def parse():
    html = get_html(URL)
    if html.status_code == 200:
        print("Success!")
        print(URL)
        get_content(html.text)
    else:
        print(":(")
parse()

How to get around this? Thanks

Answer 1

Where is the class