A
A
amirkhonov2016-08-15 10:49:38
Python
amirkhonov, 2016-08-15 10:49:38

How to download only changed file with curl or wget?

Hello!
I need to download archives from a repository (eg https://downloads.wordpress.org/plugin/disable-com... only when the file on the server has a different size (Content-Length). The script is written in
Python.downloads. wordpress.org gives all the necessary data to find out when the file was modified and its size:

Status: HTTP/1.1 200 OK
Server:	nginx	
Date:	Mon, 15 Aug 2016 06:30:46 GMT	
Content-Type:	application/zip	
Content-Length:	80828	
Connection:	close	
Cache-control:	private	
Content-Disposition:	attachment; filename=disable-comments.zip	
Last-Modified:	Fri, 12 Aug 2016 09:46:22 GMT	
X-Frame-Options:	SAMEORIGIN	
X-nc:	BYPASS lax 189	
Accept-Ranges:	bytes

Tried wget, aria2c , curl with python os library: such a file already exists in the local directory. (I use the -N key for wget)
2. C curl does not work for me at all. It stupidly downloads the file and overwrites (curl url --output /folder/1.zip).
It seems to me that I am using the capabilities of these utilities incorrectly. Is it possible to implement such a task with python libraries without calling os?
Please help me figure this out. Thanks in advance.
PS I would like to solve the problem with wget, curl, so that they do not download the actual file on the local.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
V
Vadim Yakovlev, 2016-08-15
@amirkhonov

Well, there are many options.
As far as Python is concerned, here is an example algorithm for solving your problem:
1) Take the requests module (for simplicity)
2) request headers for your file through it (r = requests.head(' example.com '))
3) Compare content- length with the size of your file
4) If different, download the file
5) ...
6) Profit
PS: Wget and Curl also support getting only headers
for wget - this is the "--server-response"
parameter for curl - this is the '-head parameter '

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question