W
W
we12017-06-17 11:25:14
SQL
we1, 2017-06-17 11:25:14

How to implement a database structure for a list of objects of the same type from different places?

Good afternoon.
I have been puzzling over the structure of the database for quite a long time so that it is convenient to use it. The task, it seems to me, is quite standard. The easiest way to describe it is by the example of an average weather forecast from different sources:
We have a list of cities, and for each of them, one to several sources are indicated. A regularly run script should get one link to the source and get information from there. That is, you need to in turn first query all links to city A, then all links to city B, and so on. in turn. However, I don't like the simplest idea of ​​storing all the links in one entry and adding a special field that would indicate the "finished" survey of this city with the help of a flag.
There is an option with timestamps, when the time of the last check is indicated and the oldest date is selected from the list next time. This option is good for everyone, except for the case when random failures lead to a violation of the order (after all, the sequence of different source links for one city is in no way connected in this case, and you can get a situation where the last source is checked many hours after the first, that is, the relevance the weather is different). I would like to get a simple system when one city object was polled in a “package” in the shortest possible time (when the script starts once a minute, four links for one city will be received in a little over three minutes), and not stretched for an indefinite time, when different situations can lead to
I would like to know an elegant scheme for solving this problem. Perhaps someone came across and has options?
Addition. All links are entered initially, and not generated on the fly, that is, they can be very different, they cannot be formed by identifiers or some rules and must be stored somewhere.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
R
Rsa97, 2017-06-17
@Rsa97

`cities` (`id`, `name`, `lastCheckTimestamp`, ...)
`urls` (`id`, `city_id`, `url`, `lastCheckTimestamp`)

SELECT `c`.`id` AS `city_id`, `u`.`id` AS `url_id`, `u`.`url` AS `url`
  FROM (
    SELECT `id`, `lastCheckTimestamp` 
      FROM `cities` 
      ORDER BY `lastCheckTimestamp` 
      LIMIT 1
  ) AS `c` 
  LEFT JOIN `urls` AS `u` ON `u`.`city_id` = `c`.`id`
    AND `u`.`lastCheckTimestamp` < `c`.`lastCheckTimestamp`
  LIMIT 1

If the request returned a url, then process it and change `lastCheckTimestamp` to urls
If not returned, then change `lastCheckTimestamp` to cities and execute the first request again

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question