H
H
HPositron2022-02-23 16:52:50
go
HPositron, 2022-02-23 16:52:50

How can I implement word comparison taking into account word forms in Golang?

How can words be compared taking into account word forms? Are there ready-made algorithms that allow you to get an abstract function that will work like thiscompareWords(first string, second string) bool

compareWords("автомобиль", "автомобильный") == true
compareWords("автомобиль", "aвиационный") == false

Levenshtein distancedid not fit, since this algorithm, for example, considers the words инициацияand to be similar ИИ. Too many false positives.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
W
WinPooh32, 2022-03-09
@HPositron

As already answered, you need a stemmer to normalize words.
Concrete implementation in Go:
https://github.com/blevesearch/snowballstem

Z
Zolg, 2022-02-23
@Zolg

Keywords for a more or less correct solution to the problem: stemming, lemmatization, and NLP in general

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question