G
G
Gravio2019-05-28 17:09:52
Parsing
Gravio, 2019-05-28 17:09:52

What framework is suitable for web scraping?

At the moment we have such a stack - Front (Vue js) Back (Laravel).
Now we needed to write a script (mini-application) for parsing site data (The site itself is quite simple). We choose which framework to choose for a given task. Requirements: simple, fast, scalable in the future
Now there is a choice between Python (scrapy, pyspider), Node js ( As an option, the usual request + cheerio ), we also considered options on Go, but we have no experience in it at all .. .Need your advice

Answer the question

In order to leave comments, you need to log in

1 answer(s)
S
sim3x, 2019-05-28
@sim3x

Scrapy is a harvester with django integration.
No django, no need for full-scale parsing of several dozen sources, no python - no point in using it
Well, except as a Wishlist
Anyway, if you want to write in python - lxml.
requests if complex interaction with the donor is required

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question