content downloader. how to parse goods?

Question

Parsing

I'm the first time)
the question is whether it is necessary to parse by product categories or take, for example, "boots", parse everything that is. rubber, suede, etc. and after already on the site add them manually to the appropriate categories?

Answer 1

Parse everything that is and then add it to the category.
To make a fast parser, you need to separate the operations of obtaining data (sending a request to the server, parsing content, etc.) from the operations of reification (representation in the form of a model). They have different cost of machine time. Parallel execution saves a lot of time.
In your case, there is no need to manually add in the categories. When parsing the received content, get the category and other data, save it in the database. Sample after the end of parsing.