Post authorization in Python, how to get the required html?

Question

Python

Hello, I need to log in to the site https://userator.ru , and then parse data from the page https://userator.ru/cgi-bin/soft.cgi here is my code:

import requests

with requests.Session() as session:
  url_login = 'https://userator.ru/cgi-bin/login_ajax.cgi'
  
  params = {'Login':'****', 'Password':'****'}

  session.get(url_login)
  session.post(url_login, data=params)
  

  url_next = 'https://userator.ru/cgi-bin/soft.cgi'

  r = session.get(url_next)

print(r.text)

The problem is that after sending a post request to the authorization site, I get not html, but headers:

{"BeginPage":null,"hash":"*********************************************aky7XKl5E6ZGpXvfUMs84GAiYrbHpEfXxj1I72HzqaEViZlrwEtNscDBkd","dispname":"Алексей","temp_hash":null,"status":"gotoaccount","id":"393696"}

The data is correct, which says that I was authorized successfully, but how to get the html page?

Answer 1

Log in to the site in the browser and click on the lock icon and see what is written in the cookies. You will be surprised, but there will be valuable information for you that must be sent in the headers on subsequent requests.