What toolkit to use for constant data collection and monitoring from a large number of sites?

Question

Perl

It is necessary to set up a service that collects actual data from third-party sites within itself. All data is closed from public access and is in a personal account. Authorization is required to obtain this data. Since there are many sites, the collection needs to be automated somehow, despite the fact that the relevance of the data is very important.
Here also there is a stupor at thought of implementation of the task. How and with what sword can this issue be overcome? =)

Answer 1

usually curl scripts or casper.js

Answer 2

For example, perl with these modules:
WWW::Mechanize to surf the site
Mojo::DOM to parse html
AnyEvent if asynchrony is needed