R
R
rundll322021-03-12 07:18:17
Parsing
rundll32, 2021-03-12 07:18:17

How are aggregator sites or parser sites arranged?

I worked with desktop parsers or grabbers, whose job was to collect information at the time of launch. But I have come across sites (for example, ka.ru) that are translated (but not necessarily) versions of other sites, there are just aggregators that save data about pages in social networks. Actually the question is: on the server of such sites there is a certain task in the scheduler that launches a parser program that generates html? It is unlikely that at the moment of creating a php session, all these lengthy manipulations take place.
PS I understand that it is unlikely that there are developers of such sites here, but if you cannot answer about the processes in the listed types of sites, then tell me how this can be in theory.

Answer the question

In order to leave comments, you need to log in

3 answer(s)
D
Dr. Bacon, 2021-03-12
@rundll32

Yes, parsing is a background task. Parsing itself is divided into at least two parts, receiving information and processing it. And of course, html is generated not by the parser itself, but by the site engine based on the data received by the parser. All this can be achieved without problems, just by starting to solve a similar problem.

A
alekssamos, 2021-03-12
@alekssamos

Well, yes. The background task parses and writes its results to the database. And the site reads the database and shows what is happening now.

R
Roman, 2021-03-16
@procode

The usual TsMSka, in which a bot works in the background on the crown and drags updates from sites, translates and adds them to the database, and displays them at the request of a browser or search engine.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question