T
T
tltary2017-08-23 04:47:12
PHP
tltary, 2017-08-23 04:47:12

How to get data from another site to your site?

The essence of the problem, there is a website, it displays certain data, I need to take this data, enter it into my database and display it. The data can change dynamically, that is, it is necessary to take data from there with a certain frequency.
The data that must be taken from the site is 1 line of text.

Answer the question

In order to leave comments, you need to log in

4 answer(s)
E
Exploding, 2017-08-23
@Exploding

In principle, the description could not be written. Well, or just write "give a parser"))
Curl and go. What else is there to say...

K
kmg4e, 2017-08-24
@kmg4e

I would caution you against writing this all by hand unless the site is trivial.
Google for the phrases "PHP parser library", "web-spider PHP", etc.
In general, you need to use ready-made tools like
https://github.com/mvdbos/php-spider
Save yourself a lot of time.
But if the site is simple, then on the contrary, without all this you will save a lot of time
;)
However, if the content of the site is formed only through JS, then a simple CURL will not solve your problem.
You will have to work with full browser emulation using headless browsers like SlimerJS or through webDriver with a full browser.

A
Andrey Levchenko, 2017-08-23
@Andrew_Fox

As one of the options to write a bot that will go to the site and rewrite the data to you in the database.

X
xmoonlight, 2017-08-23
@xmoonlight

1. For any external access to your site - check the caching time of the previous data.
2. If it is greater than the set value, we make a request via cURL (a built-in class in PHP) to a third-party site and write the result to our database/file.
If less, we do not access a third-party site and use the cached data (if necessary).
3. GOTO 1
Thus, we do not create an excessive load on the donor site and have constant data updates. The frequency of requests to the donor site (time intervals) - choose based on the goals of the task.
I advise you to set at least 30 minutes if the data rarely changes there.
Best of all, look for a link to an RSS feed on a third-party site - this will be the best indicator of new information.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question