M
M
maxxxixxxx2014-08-14 19:18:21
Programming
maxxxixxxx, 2014-08-14 19:18:21

What is the easiest language to write a website scraper in?

Parser for downloading site information in a loop.
The array contains url addresses of sections.
Next, the parser in the loop should run through each page of urls and pull out new urls for posts.
Next, in a new subloop, go to each parsed url and parse the contents.
Take the parsed content and send a post request to the php script.
What is the easiest way to make such a program?

Answer the question

In order to leave comments, you need to log in

4 answer(s)
P
Puma Thailand, 2014-08-14
@opium

almost at any high-level it will be an easy
task for children

M
Marat Amerov, 2014-08-15
@amerov

Ruby example docs.seattlerb.org/mechanize/EXAMPLES_rdoc.html
if you need to parse JS then watir.com/examples & phantomjs

B
barkalov, 2014-08-14
@barkalov

curl / phantom.js depending on the site's AJAX-ness.
UPD: the language is whatever / javascript respectively.

A
Anton Chekhov, 2014-08-16
@chekhovrussia

Whatever you can write on.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question