M
M
makar042019-07-23 10:33:35
API
makar04, 2019-07-23 10:33:35

How to optimize a page with a large number of entries received via API or as a result of parsing?

Good day! There is a closed system that loads suppliers' data, writes them into a js object and displays them immediately on an html page without writing to the database, since the most up-to-date information about this data is required, at any time the supplier can change the cost, mark the goods for return. The JS object is used to search for a product on the page.
For some providers, data is loaded via api, for others, sites are parsed, and others upload information to our database.
After receiving the data for each supplier, a cache is created and deleted only at the moment when an operation was performed with any product, and when the product card is opened, the most relevant data is taken from the supplier's website. Reloading the page takes much less time, because the data is cached and if it is updated, then for a maximum of 1 - 2 suppliers, but the initial generation of the page at the moment for 3 suppliers takes about 2-4 seconds, while about 200 products are loaded . Ideally, there will be 10-15 such suppliers and each with 50-150 products, that is, on average, about 1000-1500 products for each user will be loaded per page.
At the moment, about 6 users use the system at the same time, but their number will grow, so I decided to prepare in advance for a higher load.
I don’t use a record in the database, because, for example, a product can be sent to us, and after we accept it, it is immediately canceled and marked for return to the supplier and does not get to the page with the product that is ready for sale or can be at any time requested by the supplier for return, also the cost or description of the goods can be changed at any time.
Tell me, can anyone come across this way of working or have any ideas for optimizing the system?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
I
Israpil Akhmedov, 2019-07-23
@makar04

Here are some options:

  • 1000-1500 products per page is a lot, 30-50 maximum is enough.
  • Load data with ajax asynchronously as the page loads.
  • Load all the data into your database and make a separate script that will monitor changes and update the data.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question