How to stream video from a computer screen to a web page?

Question

JavaScript

It is necessary to broadcast video from the computer screen to a web page in real time. Keystrokes, clicks, etc. are also transmitted from the web page. it turns out such a collective farm RDP (This part is implemented and does not raise questions).
How it is now implemented
1. The C# console program is executed on the computer whose screen is to be translated.
2. The program takes a screenshot in jpeg format
3. The program sends a screenshot in binary form via WebSocket to the browser.
4. Next, js renders this image to the canvas.

var _canvas = document.getElementById("player");
var _ctx = _canvas.getContext("2d");
    
var _image = new Image();
_image.onload = function() {
  _ctx.drawImage(_image, 0, 0);
};

socket.onmessage = function(event) {	
  URL.revokeObjectURL(_image.src);		
  _image.src = URL.createObjectURL(event.data);
};

It works and is fast enough but not real-time, there is a small delay of 100-150ms.
Having collected performance statistics in the browser, I got the following.

From the timeline, you can see that in "Other" everything is occupied by URL.createObjectURL calls. It is impossible to display a byte array with a picture in js otherwise. Actually here is a dead end.
Respectively questions:
1. How in С# it is possible to capture a video stream from the screen of the computer and to transfer it to the browser.
2. Is it possible to somehow speed up the existing image transmission scheme? if so how?

Answer 1

you reinvented MJPEG. it is used in all kinds of video surveillance systems. due to the minimum amount of compression, it gives good still shots. the main drawback - for the same reason, it consumes a huge amount of traffic (really huge, a video for 5-10 minutes with FPS 20+ in this format will take a couple of GB). Above in the comments it was advised to use webrts. So, just in case - he's not fucking needed here. But you can pay attention to H.264 which is used there. That the video is in mjpeg, that h.264 can be simply and without unnecessary gestures opened in the browser as a stream. And native support for the video codec in the browser will work faster than rendering frames with Javascript.

Answer 2

2. The program takes a screenshot in jpeg format
3. The program sends a screenshot in binary form via WebSocket to the browser.

Somehow not optimal. I think the program should make a delay, form an optimal flow and send.

Answer 3

I thought about this topic.
To begin with, you are correctly told about webrtc, an open source project developed by Google.
Video streaming is more convenient, provided that there are many users. But it is not known how much your crutch with jpeg will fall, especially if you suddenly need to sharply increase FPS / image quality ...
Ps It is more stable to transmit video via webrtc, but now process keystrokes through normal request / response.

Answer 4

For transmission and display -- WebRTC

Answer 5

https://github.com/radioman/WebRtc.NET