Bing SERP parsing. Why is the list empty?

Question

Python

I wrote Bing results, the script works, but for some reason nothing is written to the json file - it is empty. The code is the following:

from requests_html import HTMLSession
from time import sleep
import json

session = HTMLSession()
number_result = 50
data = []

with open('keys.txt', 'r', encoding='utf-8') as keys_file:
    # Обработка для каждого ключа из файла
    for line in keys_file:
        keyword = line.rstrip('\n')

        # Выдача bing по запросу ключа
        response = session.get(f'https://www.bing.com/search?q={keyword}&count={number_result}')

        # Выделяем блоки результатов поиска в выдаче
        blocks = response.html.xpath('//*[@id="b_results"]/li[@class="b_algo"]')

        # В каждом блоке находим title, description + высчитываем domain
        for i, block in enumerate(blocks, 1):
            title = block.xpath('//h2/a')[0].text
            link = block.xpath('//h2/a/@href')[0]

            description = block.xpath('//div[@class="b_caption"]/p')
            description = [x.text for x in description]
            description = ' '.join(description)

            if link.startswith('http'):
                domain = link.split('/')[2]
            else:
                domain = 'Error = NOT http'

            # Все данные подготовлены - сохраняем
            data.append({
                'domain': domain,
                'link': link,
                'keyword': keyword,
                'position': i,
                'title': title,
                'description': description
            })

        print(f'Готово = {keyword}')

        # Задержка по следующему ключу
        sleep(5)

with open('data-8.json', 'w', encoding = "utf-8") as file:
    json.dump(data, file, ensure_ascii=False)

Perhaps xpath does not copy the information correctly, but I check with xpath helper - everything is correct. If I run the script on my home computer, then I have an empty json, and if at work, then it is filled. How can this be? That is, it works on one computer, but not on the other.

Answer 1

decompose
save the response from bing into a file
with headers,
and finally you should be happy, bing loves to give parsers left information

Answer 2

I only managed to get a bing of reciprocity through selenium) if necessary, I'll throw off my script for an example