How realistic is it to build a product search system in python?

Question

Python

Hello.

• There is a project where people leave reviews for certain products.
  
• There is an admitad partner platform where companies pay publishers for leads.
  
• The project is monetized by the fact that people watch reviews (review articles), then follow the link to the store and make a purchase there.
  

The problem is that partner stores leave from time to time, close, new ones appear, update their products, and so on. As a result, updating all the data (and this is about 20,000 products) takes a month of work (or even more).
Is it possible to write a system that would take data from our feedback catalog, take some arbitrary online store and correlate

1. URL
   
2. price
   
3. Availability
   

My attempts to write such a system ran into the fact that:
1) Products are difficult to correlate, as webmasters call them whatever they want. As a result, even if the store has a product, it is not certain that it will have an official name. (even if you drive in the official name of the product and a specific site in Google / Yandex, it’s not a fact that they will find exactly what they need).
2) Often different sizes, different colors are displayed in separate lots, as a result there is confusion.
3) For each store, you need to make a separate html analyzer - but this is the lesser of the evils. This is where Beauty Soup does the trick.
Or is such a system already from the category of rocket science and this is not done in one?

Answer 1

If there is micro-markup - consider yourself lucky. You can find exactly where the price is. And it will be completely automatic.
Everything else can only be done indirectly (search ANYWHERE on the page for mentioning the price). Or in semi-manual mode - here Scrapy will help you, it's just in Python.