T
T
The Whiz2016-10-10 13:30:17
Crawling
The Whiz, 2016-10-10 13:30:17

Are there opensource "smart" spiders for crawling/parsing sites?

Interested in web crawlers that can crawl resources using thematic keywords or categories instead of seed url. I suppose with the built-in classifier "ok" - "not ok".
I consider any EP, because. I didn’t find anything like that in my garden near Ruby, but I consider writing from scratch as a plan B.

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question