How to remove duplicates in all files in a folder?

Question

linux

I installed Cygwin on Windows, I delete duplicate lines in the file like this
$ awk '!x[$0]++' 1233.txt > 1233-i.txt
or
$ cat 123.txt | sort | uniq > 1233-i.txt
everything works
but now I want all text files in the folder to be processed:
$ find . -name "*.txt" -exec cat {}\; | sort {}\; | uniq {}\; > 12345.txt
this find does not work, tell me how to process all the files? oak in linux

Answer 1

You can just loop through all the files in the current folder.

for i in *.txt; do sort $i | uniq > $i-sorted; done

Answer 2

look for the
uniq command

Answer 3

fdupes comparison by content