What tools for administering site parsing exist?

Question

Parsing

Hello!
Tell me, please, in which direction to dig, what to read to organize the administration of many scripts for parsing sites (more than 50)?

Answer 1

We already wrote about scrapy here
The team of this project launched the platform https://scrapinghub.com - control panel and monitoring of spiders

Answer 2

there is a classic an alternative from Chinese comrades, but it’s better with Scrapy

Answer 3

Crawl and monitor +1000 websites
https://stackoverflow.com/questions/19618735/crawl...
And here is something similar, judging by the description - The Crawler dashboard view shows you a much more in depth view of your current Scrapy Crawlers . Here you see breakdowns of response codes, machine load balances, and successful outbound Kafka messages.
scrapy-cluster.readthedocs.io/en/latest/topics/adv...