C
C
Coder3212017-09-04 01:14:44
Node.js
Coder321, 2017-09-04 01:14:44

How to continue downloading after a crash?

There are two servers. One downloads from a large file, so 10-15 GB. Having admitted the swinging server has fallen. Is it possible to somehow resume the download after restarting from the stop point if That is, if at the time of the crash the downloading server downloaded 5GB, then do not download them again, but simply continue downloading. At the moment, the file is given like this (short version):

(req, res)=>{
const stream = fs.createReadStream(url);
stream.pipe(res);
}

Can someone advise something?
I found a not very beautiful solution, can someone tell me how to improve it
const url = `http://localhost:3589/file/download?fileName=music.7z&startFrom=${fileStat.size}`;
    const readStream = createReadStream('./file.7z')
    const writeStream = createWriteStream(`${Date.now()}.7z`);
    readStream.on('data', (data) => {
        writeStream.write(data)
    })
    readStream.on('close', (data) => {
        const req = request.get(url)
        req.on('data', (data) => {
            writeStream.write(data)
        })
        req.on('close', (data) => {
            write.close()
        })
    })

Answer the question

In order to leave comments, you need to log in

1 answer(s)
I
Ivan Koryukov, 2017-09-04
@MadridianFox

You can make such a resume in any language, because. all that is needed is to add headers to the request and determine the size of the file in bytes.
File resume is a property of the http protocol. If the http server supports resuming, then it returns the Accept-ranges. When downloading a file, you can interrupt the download, and when you decide to resume downloading, send a request to download the same file, but with the title Range: bytes=<count_loaded_bytes>-, then the server will start sending the file from the specified location. Google more about these headings.
This applies to ready-made http-servers that fully comply with the standard. In your case, you probably need to teach the sending server to send the file according to the standard. Those. if a request to download a file is received, it must be taken into account that an offset can be passed in the header, and the file should be sent starting from this offset.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question