How to parse responses from Mail.ru in Python?

Question

Python

You need to write a script that will search for answers from Mail.ru, select a random one from the first, then go to this question and select a random answer. Sounds easy. But the problem arose at the very beginning. I do it like this:

import requests
from bs4 import BeautifulSoup as BS

url = 'https://otvet.mail.ru/search/'
response = requests.get(f'{url}мой вопрос')
soup = BS(response.text, 'html.parser')

But instead of questions, there are a lot of scripts on the site, so choosing them from soup will not work. How to parse questions in this case?

Answer 1

Press F12, open the network tab and see what XHR requests are there.
Spoiler:
Endpoint:
https://otvet.mail.ru/go-proxy/answer_json
Request example:
https://otvet.mail.ru/go-proxy/answer_json?q=%D0%B...