J
J
Julia Bedrosova2019-07-15 21:19:39
Parsing
Julia Bedrosova, 2019-07-15 21:19:39

What is fashionable / convenient to write parsers in 2019?

Actually, what do you write parsers / web scrapers on and why exactly on this?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
S
sim3x, 2019-07-15
@Bedrosova

python: requests/scrapy or +selenium
requests - push a couple of requests or something back and forth into one thread
Scrapy -
python+selenium combine - parse resources that don't like to be parsed

spoiler
following fashion in 2k19?

E
Evgen, 2019-07-16
@Verz1Lka

Already described a number of times.
Purely subjective opinion - use scrapy under python. Framework is tailored for data collection. It solves multithreading problems out of the box, a lot of extensions, at the first stage, a spider can be created in a couple of dozen lines, and then scaled to infinity.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question