How to search for a match by words in php?

Question

PHP

Hello everyone
There is a large archive of correspondence with clients. The task is to automatically segment the dialogues, collect the emotional coloring of the dialogues (everything is fine, problem, sales, competitors, etc.). Those. actually run all the dialogues for the presence of some keywords. How to do it correctly so as not to sag much in terms of speed?
ps or does it make sense to already use tree with python big data?

Answer 1

Or read, for example, about Elastic Search or Sphinx

Answer 2

It is hardly necessary to take the entire dialogue, because of the possibility of keywords encountered in the course of the conversation that are not related to the essence of the appeal.
It makes sense to take only the first message from the client, in which he stated the essence of the appeal. This means reducing the amount of data being processed. If not, then it will not complicate processing much if you process all messages, even if there are millions of them - This is a single processing. Next, you will process new incoming requests on the fly. Means:
1. Select the first messages of all dialogs from the database. (you can use limit and mark after processing that this dialog has already been processed).
2. Cycle through all the received data. Inside the foreach loop with an array of keywords and inside the foreach the stripos() function. if there is a match with the keyword, we throw the dialog ID and the keyword ID into the array of matches.
3. After going through all the dialogues. Write to a separate table the IDs of dialogs and keywords in one query.
Further, all new questions are processed in the same way. And you get the list of dialogs in the same way as you did, but using a JOIN to a new table to display the appropriate notes

Answer 3

NLP