How to convert to Python 3 encoding?

Question

Python

Good time of the day!
I'm trying to communicate with the server via WebSocket
The client part looks like this

#!/usr/bin/env python3

import json
import socket
import random
import hashlib, base64

HOST = '192.168.2.208'  # The server's hostname or IP address
PORT = 8088        # The port used by the server

raw_key = bytes(random.getrandbits(8) for _ in range(16))
key = base64.b64encode(raw_key).decode()

s=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))

request = ("GET /asterisk/ari/events?api_key=test_ari:test_ari&app=ari HTTP/1.1\r\n"
"Host: 192.168.2.208:8088\r\n"
"Upgrade: websocket\r\n"
"Connection: Upgrade\r\n"
"Sec-WebSocket-Key: %s \r\n"
"Origin: http://192.168.2.208\r\n"
"Sec-WebSocket-Version: 13\r\n"
"\r\n"
) % key
s.send(request.encode())

while True:
    data = s.recv(2024)
    if not data:
        s.close()
        break
    data=data.decode('utf-8')
    print (data)

When shaking hands, the server responds

b'HTTP/1.1 101 Switching Protocols\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nSec-WebSocket-Accept: KYGe09BSgoqVr/5ACVqEjZ9Igyo=\r\n\r\n'

Decoding is correct

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: KYGe09BSgoqVr/5ACVqEjZ9Igyo=

But further, with events on the server, it responds in JSON format and issues

b'\x81~\x02`{\n  "type": "Dial",\n  "timestamp": "2019-04-23T20:59:29.627+0500",\n  "dialstatus": "",\n  "forward": "",\n  "dialstring": "1002",\n  "peer": {\n    "id": "1556035169.26",\n    "name": "PJSIP/1002-0000000d",\n    "state": "Down",\n    "caller": {\n      "name": "",\n      "number": ""\n    },\n    "connected": {\n      "name": "",\n      "number": ""\n    },\n    "accountcode": "",\n    "dialplan": {\n      "context": "from-internal",\n      "exten": "s",\n      "priority": 1\n    },\n    "creationtime": "2019-04-23T20:59:29.625+0500",\n    "language": "ru"\n  },\n  "asterisk_id": "00:0c:29:b2:91:db",\n  "application": "ari"\n}'

and there is an error

Traceback (most recent call last):
  File "./ari.py", line 38, in <module>
    data=data.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x81 in position 0: invalid start byte

I understand that characters like x81~\x02 do not understand utf-8 encoding, of course I can put another encoding, for example data=data.decode('cp1251'), then there is no error, and these characters are converted to Ѓ~

Ѓ~`{
  "type": "Dial",
  "timestamp": "2019-04-23T21:09:56.471+0500",
  "dialstatus": "",
  "forward": "",
  "dialstring": "1002",
  "peer": {
    "id": "1556035796.30",
    "name": "PJSIP/1002-0000000f",
    "state": "Down",
    "caller": {
      "name": "",
      "number": ""
    },
    "connected": {
      "name": "",
      "number": ""
    },
    "accountcode": "",
    "dialplan": {
      "context": "from-internal",
      "exten": "s",
      "priority": 1
    },
    "creationtime": "2019-04-23T21:09:56.469+0500",
    "language": "ru"
  },
  "asterisk_id": "00:0c:29:b2:91:db",
  "application": "ari"
}

But how do I get a pure JSON string so I can convert it to an array and work with it? I thought to get rid of these characters by replacing the substring with a regular expression, but these characters are found not only at the beginning but also in the middle of the string. What do you advise, maybe I have the wrong approach and the task is solved much easier?

Answer 1

Don't reinvent the wheel (unless you're facing a learning challenge)
https://websockets.readthedocs.io/en/stable/