V
V
Valeri Lavrov2017-06-05 16:07:51
PHP
Valeri Lavrov, 2017-06-05 16:07:51

How to quickly parse JSON from a website?

There are 3 sites
https://app.csgoroll.com/v1/market/inventory
https://loot.farm/fullprice.json
https://skinsjar.com/api/v3/load/bots The
last one is quite "heavy"
in the case of the first site, you need to go through all the pages (next_page_url), then through other sites. All this in 10-20 seconds, then pull out the names of items and prices, then add them to the database. The first CURL site parses for about a minute.
What PHP libraries do you recommend? I never parsed on NODE JS, but maybe there are libraries there and faster?
PS I do it for myself

Answer the question

In order to leave comments, you need to log in

2 answer(s)
L
Luther Yezhov, 2017-06-05
@Louter

In nodejs, everything is simple: if the file is downloaded, then open it through if not downloaded, then you can download it through, for example, request: First, install the package, then this code:

var request = require('request');
request(https://skinsjar.com/api/v3/load/bots, function(error, response, body) {
  var json_data = JSON.parse(body);
});

How to work with the database and add all records from json_data is a separate issue, more easily solved

V
Vlad, 2017-06-05
@DaFive

CURL doesn't parse, it gets the content. Actually, no matter what you do, you will not reduce the time for loading data in any way if pagination is not provided for when requesting services. Actually, that through PHP, that through JS, the time will be approximately the same for loading.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question