How to find all identical elements in an array?

Question

C++ / C#

Hello. There is an array of data, you need to find the same elements. Are there search methods in C#? So far, only a comparison of each file with each has turned out, but we need higher performance. If done through a dictionary, are there methods for searching, so that without a double bypass ( foreach-> for)? The main thing: you can't take a specific key, you can't use a string - the array is just a set of file bytes. Thanks in advance

Answer 1

1. It makes no sense to check already checked pairs and the element itself in each bypass.
2. First compare just the length of the elements.
3. It makes no sense to fully compare elements, check them by bytes.
4. Replace elements with their hashes if they can have the same long start (to optimize point 3).
5. Do not upload or process data again if you suddenly do this.
6. Use Parallel.forEach instead of the usual one.