Data parsing, which language is more practical?

Question

Parsing

Greetings to all guys.
I will try to describe as briefly and as clearly as possible.
You need to write a full-fledged parser that will collect information from sites according to the categories specified by users.
Simple example 1 - It is necessary to collect information from users with a nickname starting with the first letter ( A ) from the site toster.ru, how many questions were solved by these users, the percentage, how many messages they left under which tags the most resolved issues, etc.
Simple example 2 - It is necessary to collect the average cost of work per hour of a php developer from the freelance site in Russia or Ukraine geo. Percentage of positive and negative reviews, etc.
Actually the question is in the implementation. In what language will it be more practical to cut this script?
Somewhere, I heard that such undertakings are well implemented on ASP.NET. Actually, the question is for those who know, is this so?
Classic PHP since the script will be on the server, I think it will take a long time to process with multiple accesses per second. Let's say 500 people parse every second according to (хххх mb) data. Is it a small load for PHP how long will it take to complete the task?
A look at Golang - how does GO handle such tasks?
A look at node js and java - would love to hear your opinion.
Friends, please do not throw unflattering phrases in my direction. Since there was no such experience before, I decided to ask you.
I do not quite understand what is better to use in these situations.
Task: Speed, Reliability, Multithreading, which would withstand a large number of accesses per second.
Thanks to all! :)

Answer 1

on any you can.
php will work, there is a multi-threaded curl, a lot of code on the net, php-fpm works stably.

Answer 2

It seems to me that it is worth paying attention to javascript engines for parsers, such as phantomjs and casperjs.
With their help, extracting data from the page becomes dozens of times easier.
Multi-threading work with these applications can already be implemented in any language and stored in a convenient format in a database or somewhere else.

Answer 3

It doesn't matter which language you choose. The one you know better at the moment, or the one you want to learn in the process. All the same, you will run into not the speed of the parser, but the width of the channel. Well, or you will be banned for too much load on the server.

Answer 4

regex - minimum!
Further - any language.

Classic PHP since the script will be on the server, I think it will take a long time to process with multiple accesses per second. Let's say 500 people parse every second according to (хххх mb) data. Is it a small load for PHP how long will it take to complete the task?

I would advise you to download everything GRADUALLY in the beginning, so as not to strain the donor resource .
And after that - calmly parse on your server.
Try HTTrack

Answer 5

Go is a great language and handles multithreading very well. If high loads and competitive launches are planned, then from all that is listed - only Go.
However, for example 1 and 2, it is not clear how you will use multithreading effectively. Decide first on the volumes, how many requests you will make. How often the datasets will be updated and so on. After that, you can select the tool.

Answer 6

D is a great language and handles multithreading very well. If high loads and competitive launches are planned, then from all that is listed - only D.

Answer 7

I did parsing sites with JavaScript through simulation, tested all this on Linux servers with rabbitmq, the network worked for half a year until I got tired of Chromium and Lazarus-IDE on the server side, with the installation ...