A
A
Akigami2018-03-17 11:00:20
Parsing
Akigami, 2018-03-17 11:00:20

What is site scraping (in simple terms)? And in what language is it usually written?

As I understand it, this is a program that selects certain information from the site, for example, any lists, etc. and merges into the document?
And what is the main language of writing?

Answer the question

In order to leave comments, you need to log in

5 answer(s)
C
chromimon, 2018-03-17
@Akigami

Website scraping - extracting information from a website using the same methods as in a browser.
That is, the parser from the point of view of the site does not differ from a regular user.
If the site is not too dependent on JS, then it parses in any language that you know better.
If the site is too dependent on JS, then SlimerJS is used, for example.

M
Matvey Pravosudov, 2018-03-17
@oxyberg

You understand correctly. It is written in any language, if there are convenient libraries.

A
Alexander, 2018-03-17
@NeiroNx

I think the most convenient solution is
Node JS + JQuery https://habrahabr.ru/post/134738/

E
Evgen, 2018-03-17
@Verz1Lka

You can use any AP. The bottom line is to send a request to the web server, and process the response in the right way. Whether php or nodejs.
There is a great Python tool: scrapy .

M
Mikhail Sisin, 2018-03-17
@JabbaHotep

Web scraping (web scraping in English) is the process of taking (crawling) the content of a web resource (for example, a page) and extracting parts of this document into a clearly structured data set. The parser can be written in any programming language, or without knowing any language at all, using cloud services for parsing.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question