Python3 requests parsing problem with https. Why character set instead of HTML?

V

Vyacheslav Correct2019-10-27 23:12:42

Python

Vyacheslav Correct, 2019-10-27 23:12:42

As I understand it, the page is encrypted, but I don’t understand how and with what ...
Here is the code itself

import requests as r

headers = {'accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
      'user-agent':'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0'}

base_url = 'https://vprognoze.ru/'

def vp_parse(base_url, headers):
  session = r.Session()
  requests = session.get(base_url, headers=headers)
  return requests

r = vp_parse(base_url,headers)
print(r.text)

And accordingly the answer comes something like

<!DOCTYPE html>
<html>
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  <meta http-equiv="refresh" content="10;URL=/ciez2a">
</head>
<body>
  <script type="text/javascript">
    var JSEncryptExports={};
(function(w){function e(a,b,c){null!=a&&("number"==typeof a?this.fromNumber(a,b,c):null==b&&"string"!=typeof a?this.fromString(a,256):this.fromString(a,b))}function m(){return new e(null)}function D(a,b,c,d,f,g){for(;0<=--g;){var h=b*this[a++]+c[d]+f;f=Math.floor(h/67108864);c[d++]=h&67108863}return f}function Y(a,b,c,d,f,g){var h=b&32767;for(b>>=15;0<=--g;){var k=this[a]&32767,e=this[a++]>>15,z=b*k+e*h,k=h*k+((z&32767)<<15)+c[d]+(f&1073741823);f=(k>>>30)+(z>>>15)+b*e+(f>>>30);c[d++]=k&1073741823}return f}
function Z(a,b,c,d,f,g){var h=b&16383;for(b>>=14;0<=--g;){var k=this[a]&16383,e=this[a++]>>14,z=b*k+e*h,k=h*k+((z&16383)<<14)+c[d]+f;f=(k>>28)+(z>>14)+b*e;c[d++]=k&268435455}return f}function T(a,b){var c=J[a.charCodeAt(b)];return null==c?-1:c}function A(a){var b=m();b.fromInt(a);return b}function K(a){var b=1,c;0!=(c=a>>>16)&&(a=c,b+=16);0!=(c=a>>8)&&(a=c,b+=8);0!=(c=a>>4)&&(a=c,b+=4);0!=(c=a>>2)&&(a=c,b+=2);0!=a>>1&&(b+=1);return b}function E(a){this.m=a}function F(a){this.m=a;this.mp=a.invDigit();

Google did not give anything on JSEncrypt requests, as I understand it, the page is encrypted ... I don’t want to pull selenium, can’t I get html with a request?

Reply

Answer the question

In order to leave comments, you need to log in

1 answer(s)

X

xmoonlight, 2019-10-28
@Zonor

You need to implement the JS execution logic on your side on your own , if you don't want to pull libs that execute JS.