Which of the array iteration options is faster?

Question

PHP

It would be more desirable to consult, than I in difficulty. There is a proxy base that needs to be sorted out (not a very suitable base format) and compared for repetition with the existing one. We are talking about hundreds of thousands of records. Faced with the fact that the script fulfills this business for hours. Hence the question: how to iterate through the array and convert strings as quickly as possible? It should be noted that the database of correct proxies is replenished during the check of new data and the array_flip option does not work at all. Who faced a similar problem and how did you solve it? And it's getting boring waiting.

Answer 1

Two options:
1. Turn the array into a graph type: "tree" and add "branches" (new records) if they are not there.
2. We make 2 lists unique (separately) and perform "merge" ("merge") for these lists (different - will be added, the same - will remain unchanged).

Answer 2

And why is it not convenient to use any DBMS, create a table with a column that stores a textual record of the proxy address, a unique index on this column, and put it there ignoring insert errors (INSERT IGNORE in MySQL, ON CONFLICT DO NOTHING in PostgreSQL)?
And then just make a selection from this table and get unique values. Already DBMS with indexes, trees, hashes and other things can work well. As a bonus, you can store other information along with the address, for example, the date it was added, the date of the last check, etc.