S
S
Sergey Bard2017-09-05 16:32:35
Parsing
Sergey Bard, 2017-09-05 16:32:35

Which parser is the fastest compared to other languages?

Hello.
I can't decide on what to write a parser
. There is a task - to write a parser that will scan about 1M pages (several dozen different sites), periodically repeating the scan, save changes to the database
. I read a lot of things.
And everyone writes in response to the question - “What to write a parser on”, what you need to write on what you know best, but I will ask the question a little differently
. Most of all I work with PHP, but I don’t want to write such a parser on it, because -due to the low speed of work (I checked it myself - wrote the parser)
I have plans to learn a new language and, as you know, learning is best in practice.
Here is my question - in what language is it best to write a parser that will work FAST (compared to other PLs), with a large amount of data.
I repeat, speed and the ability to parse a large amount of data in the shortest possible time are important.
I do not expect ready-made solutions, but just want to know who writes what. Maybe compared the speed of different parsers, etc.
I want to know the latest information because in RuNet I find articles only a year old or more.
I apologize if the question is incorrect)

Answer the question

In order to leave comments, you need to log in

2 answer(s)
I
IllusionTurtle, 2017-09-05
@serg_small_developer

If you mean a parser of web pages on the Internet, then your bottleneck will not be the language, but the network requests to these pages.
It is also worth considering that it is better to solve such tasks in many threads, where php will not be as convenient as go or nodejs.

Y
Yuri Paimurzin, 2020-01-16
@rusellsystems

Here is my old post about parsing sites with JavaScript, and I also tested all this on Linux servers with rabbitmq, the network worked for half a year until I got tired of it. I got acquainted with the site and found a lot of JavaS there...

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question