V
V
Vitaliy Zhovnirchik2017-05-22 17:00:13
Python
Vitaliy Zhovnirchik, 2017-05-22 17:00:13

Python requests not receiving correct HTML - page code, how to solve?

I'm new to Python I'm writing a parser for the site list.if.ua
for educational purposes. I almost did everything: I got a list of all categories, a list of all pages by category, but I can't get the HTML code of pages with a list of organizations in categories with more than one page. For example, the link http://list.if.ua/Avtomyyki-Ivano-Frankivska/page/3 in browsers (Opera, Mozilla, Chrome) opens well and you can see the list of organizations, and when requests are made to this page, there are no organizations in the received HTML code . Here is part of the Python code I use for testing:

import requests

def test():
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; rv:12.0) Gecko/20100101 Firefox/12.0',
        'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
        'Accept-Language': 'en-en,ru;q=0.8,en-us;q=0.5,en;q=0.3',
        'Accept-Encoding': 'gzip, deflate',
        'Connection': 'keep-close',
    }

    url = 'http://list.if.ua/Автомийки-Івано-Франківська/page/3'
    response = requests.get(url, headers=headers)
    print(response.text)

Tell me how you can get the correct HTML code through Python?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Alex Zaharchuk, 2017-05-22
@VitProff

For example, I also do not have organizations (from the browser). Writes "Nothing was found behind your inquiry." But this link works: list.if.ua/%D0%90%D0%B2%D1%82%D0%BE%D0%BC%D0%B8%D0...

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question