M
M
Michael2017-03-30 06:22:59
Data processing
Michael, 2017-03-30 06:22:59

What technology to use for data processing?

Initial data:

  1. contents of the payload network package
  2. filtering rules
  3. all data is in RAM, not on disk

According to the filtering rules, you need to decide what to do with the packet.
Problem :
Several hundred thousand packets arrive per second. More than a thousand rules. The processor, of course, "drowns".
We need your fantasies on how to process such a huge amount of data efficiently and how it can be clustered at all.
Most likely, there is not enough initial data: I will clarify in the course of the dialogue.
Thanks in advance for ideas!

Answer the question

In order to leave comments, you need to log in

1 answer(s)
E
Egor Kazantsev, 2017-03-30
@saintbyte

It is clear to you it is necessary to write rules for a firewall. here you need to apply logic and make them as small as possible and make sure that the packet goes through as few rules as possible. Apart from knowing the syntax for writing firewall rules and a bit of logic with brains, you don't need anything.
If you have experience in optimizing database queries - this is something similar in terms of brain tension, only the rules are different =)

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question