Removing similar words?

Question

JavaScript

Help remove similar words:
For example, the div contains the words: test test.
You need to delete all the words except for one.
Those. we leave one test, delete all other tests, tests, etc.

Answer 1

Use any of the fuzzy search algorithms , for example calculate the Levenshtein Distance for each pair of words. Depending on the received values, decide which words to delete and which to keep.

var str = 'тест тост тест тесто';
var words = str.split(' ');
var result = [];
for (var i = 0; i < words.length; i++) {
    result.push(words[i]);
    for (var j = i + 1; j < words.length; j++) {
        if (levenshtein(words[i], words[j], {}) < 2) {
            words.splice(j--, 1);
        }
    }
}

https://jsfiddle.net/9hz8wmz6/

Answer 2

Well, as an option, you need to break the text into an array of words. And then, in a loop over this array, apply a regular expression to the text in which you indicate that the current word repeated more than 1 time one after another will be replaced by one. Or compare the elements with each other and if they match, remove the duplicate (something like a bubble method; you know what?). Then glue the elements into a string. Only it is very load the client.