D
D
Dmitry Amirov2015-08-23 18:42:31
Nginx
Dmitry Amirov, 2015-08-23 18:42:31

How to proxy and cache static for a long period?

There is a certain remote service, similar in functionality to a product catalog. It has an API thanks to which we can display this list of products on our project. But bad luck, the remote resource often falls.
We learned how to cache product data through the API, and in fact, we work with cached data on our project. But the trouble is in the pictures of each product. As soon as the remote service crashes, the pictures of the products disappear, which greatly affects the usability of the project, and generally looks miserable.
To understand volumes. There are about 10,000 products in the catalog. Each has an average of 20 pictures. Plus 3 sizes of each picture, for a total of about 600,000 different images. This is about 80GB of data. The list of products is dynamic, i.e. Items can be added or removed. But the URL of each image is unique, i.e. the situation that the picture suddenly changed at such a URL cannot be.
There is an idea - to make a php script that proxy images from the service, simultaneously adding them to our server, and then giving them directly to nginx. In principle, this is feasible. But to be honest, I would really like to exclude php and in general any kind of dynamics in this place.
Is it purely theoretically possible to do something similar using nginx? Or Varnish? Which one suits me better?
My idea in general terms is the following - for statics, register a remote service as a backend, and set the cache time for a month. But here the problem arises, if the service falls, is it possible to force nginx to continue to give out the stale cache?
Or still throw bad thoughts out of your head, and make, as I said, a script. Again, the question arises of how to delete obsolete pictures. Those. Ideally, we would delete images from our storage that have not been accessed for, say, a month.
In general, please tell me in which direction to dig.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
D
Dmitry Amirov, 2015-11-27
@InSys

If anyone is interested in how the problem was solved in the end:
https://intsystem.org/server/proxy-resize-cache-im...

S
sim3x, 2015-08-23
@sim3x

Is it purely theoretically possible to do something similar using nginx?

https://www.digitalocean.com/community/tutorials/u...

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question