Is there an algorithm for reducing text?

Question

Books

Hello. Is there any algorithm to shrink the text i.e. so that extra words: mataphors, repetitions, epithets are removed, for example, and the meaning of the text remains clear?

Answer 1

It is called automatic text summarization
There are open source programs. The truth is bad, and for English :).
If you want to make your own, you need to define a certain criterion, for example, entropy or perplexity, and try to minimize it. Or create matrices and try to reconstruct low-rank with minimal loss.

Answer 2

I remember there was such a post on Habré, where one journalist talked about such an algorithm. It included all sorts of words that, if the meaning is removed, will not change, the algorithm highlighted these words and showed it to him so that he could remove these words in some places and shorten the text.

Answer 3

The easiest way is to make a large list of regular expressions to replace or remove parts of the text. In s/ style, like //g. s/So //g and so on.