A
A
AnryLacroix2020-03-09 06:41:16
Parsing
AnryLacroix, 2020-03-09 06:41:16

How to implement the removal of a product withdrawn from sale in the aggregator?

I am making a commodity aggregator, for about 100k-200k goods. The problem is that the availability of goods remains relevant, i.e. the parser collects data, gives me csv, I compare it with the original database, select the changed data and load the changes into the database. But what if the item was taken off the market? The parser will not collect data about it, no changes will be loaded, it will hang like a dead weight.
The only option that I see so far is a complete database update after each data collection, but this is an impossible load on the server. Ideas?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
D
Dr. Bacon, 2020-03-09
@bacon

write down the update time, if N hours have passed after it, do not show this product.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question