X
X
xuxubla2015-05-14 06:44:20
PHP
xuxubla, 2015-05-14 06:44:20

What are the ways to speed up parsing?

I would like to know how you can reduce the parsing time? 25,000 products = 25,000 pages. It takes about 4 hours to open the page and collect information.

Answer the question

In order to leave comments, you need to log in

3 answer(s)
M
myfirepukan, 2015-05-14
@myfirepukan

curl cannot be accelerated, only if multi is used, and yes, run several scripts in parallel

R
Roman Webs, 2015-05-14
@roman_vebs

Split into steps, perform steps in parallel, use multicurl and a wide communication channel - to minimize request-response delays.

A
Artur Nurullin, 2015-05-14
@Splo1ter

abandon PHP and rewrite everything in a language that supports multithreading

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question