Which way to move when creating a system for tracking the status of a large number of devices (Fault Management)?

Question

data mining

Hello.
There are 400'000 devices that send a total of about 1'000 a-la syslog messages per second. Each message contains, among other things, a device ID and a "number of errors". According to some algorithm, it is necessary to issue an alarm (and show the identifier of the device for which it worked) when the number of errors for a certain period of time exceeds a configurable threshold (there will be other algorithms for determining "problems").
While interested in the concept.
Now I use MySQL (with partitioning by auxiliary fields) + Python on a single server, but the system is bent.

Answer 1

At least keep information about devices in memory. Use the database only as a slow persistent storage.

Answer 2

Should I look towards Spark?