A
A
Alex Lukyanets2021-04-05 18:16:43
Parsing
Alex Lukyanets, 2021-04-05 18:16:43

How to parse websites using web socket technology?

I am engaged in collection of open data.
The sites below are powered by web socket technology.

  • https://dallas.tx.publicsearch.us/
  • https://denton.tx.publicsearch.us/
  • bexar.tx.publicsearch.us

Has anyone encountered parsing on a similar technology? If yes, I will be grateful if you send articles or projects.
So far I don't know how to do it.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
S
StiflerProger, 2021-04-06
@mynameiswisethanyesturday

const WebSocket = require('ws');

const ws = new WebSocket('wss://dallas.tx.publicsearch.us/ws', {
  headers: {
    "Origin": "https://dallas.tx.publicsearch.us",
    "Cookie": "authToken=51f69ac5-67c5-41ac-8943-cace7ebc6b3c; authToken.sig=x9S_G8Lwy6KHaB2Ek4PT1PR7jhE;"
  }
});

ws.on('open', () => {
  console.log('Соединение открыто');
});

ws.on('close', (code, reason) => {
  console.log('Соединение закрыто', code, reason);
});

all sites will have their own cockroaches) for example, in what I threw off above. You must pass Origin and Cookie in the request header . If everything is clear with the first, then the first problem will be with cookies. As I understand it, they can be obtained by making a regular GET request to the main one. These cookies will be sent to you in Response Headers
606c101d381a7125759663.png
https://www.npmjs.com/package/ws
After a successful connection, do not forget to send pings later) You can already see the ping format in the Network tab in the browser

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question