Answer the question
In order to leave comments, you need to log in
How to highlight similar features in the text (ML)?
Good afternoon.
I am new to the ML area, slowly learning the area, with a deep dive into PyTorch.
A task arose in the field of text analysis, which I still don’t know how to approach.
The essence of the task:
There are prepared and marked phrases. For example: "data is being transferred".
If a phrase with a similar essence is found in the text, it is necessary to detect it and then perform a certain pool of actions.
Problem 1:
In a variety of arbitrary texts, the essence of this example can be stated differently.
For example: "data is being transmitted", "data is being transmitted", "data is being sent", "a request is being sent", etc.
Problem 2:
For training, 1-2 variants of the phrase will be available and I cannot yet understand how training can be built on a small volume.
For example: only 2 phrases "data transfer in progress" and "data transfer in progress" will be available to me and refer to one class (id) to which the subsequent action is attached.
I have made some attempts at torchtext trying to classify but they have not been successful.
Thank you in advance for your help.
Answer the question
In order to leave comments, you need to log in
Knowing PyTorch will make you a PyTorch expert, but it won't make you an ML expert.
To learn ML, you need to learn ML, and consider PyTorch as just one (of many) tools for solving (some) problems in the field of ML and in the field of natural language processing in particular. And no more.
Therefore, I recommend not jumping through five steps, but starting from the beginning and moving naturally:
https://www.kdnuggets.com/2020/10/roadmap-natural-...
https://www.lexalytics.com/lexablog /machine-learni...
https://towardsdatascience.com/text-classification...
https://towardsdatascience.com/natural-language-pr...
https://www.kdnuggets.com/2020/07 /5-fantastic-nlp-...
https://www.amazon.com/Applied-Text-Analysis-Pytho...
I share your current view, so I watch a course of lectures from ShAD (Yandex Courses on ML), but I always want to try in practice what I learned about.
Thanks for the links.
Object/subject of action + synonyms.
Find similar "chains" through shingle comparisons and get a cluster.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question