What are the approaches for extracting data from websites?

Question

Python

Good afternoon, dear development gurus
, tell me, please, what are the general approaches for programmatically extracting data from websites? You need to write a script (for example, in python / groovy) that, having scanned the en.wiktionary.org resource , would save the nouns from it to the file

Answer 1

Scrapy.org is used to collect pages (crawling, scraping), it has a "built-in" html parser - https://pypi.python.org/pypi/lxml/ lxml.de
but for a wiki like resources this is not necessary https:// dumps.wikimedia.org/

Answer 2

The collection process is called site scraping.
Take grab (bs4,requests,mechanize) and read articles on Habré - everything is described there.