G
G
Grigory Vasilkov2017-04-22 00:45:08
Programming
Grigory Vasilkov, 2017-04-22 00:45:08

How to architecturally solve the problem of multiple sources of country-city-indexes?

There is a source of cities and available postal codes in them
. But the source does not have direct access to the "download all" type - there is only the ability to send a request from "more than two characters" to pull out the results by the index or part of the city name + country code in the system, say a thousand pieces
The list of cities also depends on the country, and there are about 400 countries in general (and each has a hundred other cities, and for each city a couple of thousand indices)
The list may change, since the geopolitical situation is now unstable, one city has disappeared, a new one has appeared, countries change, in the end, the service may simply stop serving certain cities or indexes in them.
The frequency with which the data changes is unknown, according to technical support - it is unpredictable - sometimes once a week, sometimes every three months, sometimes every day.
Does any caching solution make sense or is it better to leave everything as follows:
- enter two letters of the city
- ajax request for a script by keyup
- the script ripped data from a remote server with a regular request
- the data was displayed The
only minus is that the request for ajax takes well a quarter of a second, then a request to a remote server, plus its processing for another half a second - in general, a second for constant jerking off requests from a remote server, even considering that you can bring down the user's pride by setting timeouts and displaying an animation like "wait, there is a request".
That is, I'm thinking about making a daily cumulative cache from the requested cities in order to save the request, or will I not save anything other than hemorrhoids in this way?
Here's another surprise - that tomorrow another similar service is connected, which has its own database of cities and indexes, and codes (or even names) of cities may differ in the presence of blunts, upper apostrophes, hyphens, spaces, etc. - let's say even a common code, nevertheless less a bunch of sources and multiply by the cache - the head is already boiling.
What's the best way to destroy? Cache the results of queries in a common table, and hang up the cache, for example, for a day? And how to fill such a base? Or the best solution is to leave everything as it is - let yourself be ajaxed?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
R
res2001, 2017-04-22
@res2001

Cache.
Perhaps it's better not to cache, but to make your own database, where information will get from requests (if it is not in the database). In this case, you will need to make an autonomous service that, for example, will update the information in the database once a day.
In the case of your database, if the source fails, you will still remain in a healthy state.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question