Answer the question
In order to leave comments, you need to log in
Django + Scrapy: which approach to take?
Task: it is necessary to save data from Scrapy parsed data through Django ORM.
Searching, I found several approaches:
1) Django Item
2) django-dynamic-scraper
3) Initialize Django from under Scrapy settings
Which approach should I choose? Anyone have experience with either of these approaches (pros or cons)?
Answer the question
In order to leave comments, you need to log in
Try everything
With low RPS and good data validation, you can immediately put into djanga
The problem with Scrapy data is that it can be incomplete, hence - only NoSQL DB during initial saving, the data then needs to be processed one way or another - get rid of duplicates, empty ones, determine the required, but missing ones,
I would not contact such a combine, scrapy separately - django processing and presentation - separately
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question