How to compare multiple string arrays?

Question

Delphi

We need to compare string arrays ( more than 10000 strings ) for differences.
The number of arrays is more than two .
Is there a way to do this the fastest way?
Preferably in Delphi.

Answer 1

1. Calculate the hashes of the strings in all arrays using some fast function, so that it does not give a very large number of collisions and the output is Integer or Int64, for comparison speed.
2. Next, compare these hashes, and if they match, you need to perform a full string comparison. At the same time it is possible to count up there was a collision or not. Comparing the number of collisions that the selected hash function gives and the hash time of the strings, it will be possible to draw a conclusion about the acceptability of a particular function.
As a hash function, you can take crc32c from the mORMot framework, they say that it is optimized and very fast: blog.synopse.info/post/2014/05/25/New-crc32c%28%29... They use it for hash tables default.
Here are some comparison tests:https://www.delphitools.info/2014/08/25/string-has...
You can also take a library from Google - CityHash, it can count hashes of 32/64/128/256 bits. Also, they say, fast. We use it for 64-bit hashes, there are binaries and a wrapper for Delphi here: https://bitbucket.org/sas_team/cityhash/src/7c7c73...