Parsing the base of more than a million positions, how to do?

Question

PHP

Actually, there is a site with a database of more than a million positions, it is necessary to transfer it to a new site, what is the best way to do it?

Answer 1

If they do not have parsing protection, then write a regular parser that iterates over all positions. Be sure to observe timeouts, be sure to give adequate user-agent, be sure to handle exceptions. If there is some kind of protection, then you already have to go out with all sorts of captcha recognition libraries, an action emulator, and so on.

Answer 2

What format are these items in? Is it something to do with the database? Use its transfer mechanisms.

Answer 3

AftParser (free) or Datacol (paid).

Answer 4

php simple html dom parser