Is there any special tool for web scraping?

Question

data mining

I want to parse several sites (including both simple news sites and more complex ones) just for myself. Usually I solve such problems completely stupidly with my code, which I write from scratch for each such task, but then it became interesting: is there some kind of library specifically for this?

Answer 1

Used simplehtmldom.sourceforge.net , convenient.

Answer 2

PhantomJS - I use it for parsing one-page sites, etc. You can not reverse JS.

Answer 3

Every programming language has libraries for handling the DOM. For example, in python, there is dom.minidom and beutifulsoup for this