How to get html VK on search request (it is generated by JS)?

Question

Python

Have a request

https://vk.com/feed?c%5Bq%5D=*id1234&c%5Btype%5D=1...

with which you can get a list of comments of a person.
The nuance is that the html page in this case is generated and my parser will receive a search page in response, without a response:

import lxml.html
import requests

login = '1234'
password = '1234'
url = 'https://vk.com/'

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0'}
session = requests.session()
data = session.get(url, headers=headers)

page = lxml.html.fromstring(data.content)

form = page.forms[0]
form.fields['email'] = login # авторизируюсь
form.fields['pass'] = password

response = session.post(form.action, data=form.form_values())

r = session.get('https://vk.com/feed?c%5Bq%5D=*id1234&c%5Btype%5D=1&section=search', cookies={'from-my': 'browser'})
#print('onLoginDone' in response.text)
print(r.text)
with open('test.html', 'w') as output_file:
  output_file.write(r.text.encode('cp1251'))

I read on the Internet that these libraries are not quite suitable for such tasks. Which is better to use and how?

Answer 1

You don't need to parse. There are libraries for working with VK API .
For example, you can take comments from the wall as follows