How to search for several regular expressions in the text at the same time?

Question

Python

There are N texts and there are M regex expressions
For each expression, you need to find the texts corresponding to this expression.
M and N are large enough numbers to sequentially execute REGEX_i in TEXT_j and see the result
I would like to somehow view all regexes at the same time
Well, it is desirable to advise some modules, if not difficult

Answer 1

github.com/dprokoptsev/pire
This library is aimed at checking a huge amount of text against
relatively many regular expressions.

Answer 2

I did not specifically research, but the impression is that there is no ready-made tool. Although the construction seems obvious - there are M finite automata (regexes), we combine them into one (methods are known), plus in the initial and / or final state of each source automaton we set a callback with the identifier of this automaton and the position in the text.
Maybe take some KA library and try to make something out of it? For C, there is libfa (a port of the Java dk.brics.automaton ), it will perform the first half of the task, but I don’t know about the marked states and callback.

Answer 3

It is unlikely that there are ready-made solutions, but you can do it not sequentially, but in parallel, processing in several processes / threads, since these operations are independent and well parallelized. If you haven't considered this option yet, then I think the multiprocessing module will help you.
If this is too slow, you can try rewriting it in C.

Answer 4

I'm working on a similar task: an index search by regular expressions in a database. You can read and watch the presentation here:
www.pgcon.org/2012/schedule/events/383.en.html
As applied to your case, the algorithm will be as follows:
1. Fill N texts into the database.
2. Build an index.
3. Search by index M times.
It was interesting to see what texts and expressions you have, how applicable my approach is to this task.

Answer 5

Хочется как-то одновременно просматривать все регексы

Ну так состряпайте сложный regexp с множественным ИЛИ.

Answer 6

Составное регулярное выражение?
Пайтона не знаю, но в джаваскрипте выглядело бы эвона как:

/(regex1|regex2|regex3|regex4|regex5|regex6|regex7|regex8|…|regexM)/