A
A
Alexey Poloz2017-04-29 14:30:35
Python
Alexey Poloz, 2017-04-29 14:30:35

SSL Python How to get a webpage?

You need to get the encrypted web page in its normal form
. There is such a code:

def get(src):
  context=ssl._create_unverified_context()
  with urlopen(src,context=context) as site:
    return site.read().decode()

With ordinary web pages, everything is fine, it receives and displays
But with encrypted problems:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xcf in position 102: invalid continuation byte

If you remove decode(), it will be output in the format b'...'
How to fix this? Or is there another way to properly output the received text?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
D
Dimonchik, 2017-04-29
@kosyachniy

the encoding may not be utf8
look at which browser autoselects or try chardet

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question