B
B
bfesiuk2020-07-21 09:58:22
Python
bfesiuk, 2020-07-21 09:58:22

Why does it return status code 400?

Hello everyone, I'm trying to collect some data

from the OLX site
In order to get some information, I need to make a request, but for some reason I can't do it. As far as I understand, the error is in request_header

Source code:

import requests
import re
from bs4 import BeautifulSoup

headers = {'user-agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 \
(KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36"}


session = requests.Session()


def get_html(url):
    r = session.get(url, headers=headers)
    return r.text


def get_request_parameters(content):
    soup = BeautifulSoup(content, "html.parser")
    scripts = soup.find_all("script")
    phone_token = str(re.findall("var phoneToken = \'(.*)\'", str(scripts))[0])
    referer = soup.find('link', {'rel': "canonical"}).get("href")
    id = re.findall("-ID(.*).html", referer)[0]
    request_url = 'https://www.olx.ua/ajax/misc/contact/phone/' + id + '/?pt=' + phone_token
    print(request_url)
    print(id)
    print(referer)
    print(phone_token)
    request_header = {'user-agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 \
(KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36",
                      'authority': 'www.olx.ua',
                      'method': 'GET',
                      'path': '/ajax/misc/contact/phone/' + id + '/?pt=' + phone_token,
                      'scheme': 'https',
                      'accept': '* / *',
                      'accept-encoding': 'gzip, deflate, br',
                      'accept - language': 'ru - RU, ru;q = 0.9, en - US;q = 0.8, en;q = 0.7',
                      'referer': referer,
                      'sec-fetch - dest': 'empty',
                      'sec-fetch - mode': 'cors',
                      'sec-fetch - site': 'same - origin',
                      'x-requested - with': 'XMLHttpRequest'
                      }
    query_string_parameters = {'pt': phone_token}
    response = session.get(request_url, params=query_string_parameters, headers=request_header)
    print(response.status_code)
    

get_request_parameters(get_html('https://www.olx.ua/obyavlenie/prodam-videokartu-radeon-hd-5450-IDGBIF3.html#f9f01ff837'))


XHR tab:
5f1691157786a520584680.png
5f16911e848ee569311646.png
5f169128eb67c739703409.png
5f169131e320c034356784.png

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question