How to store netflow?

Question

NetFlow

Hello. Such a question - in Kazakhstan it is necessary to store data on the movement of users on the Internet - 5 years. How can this be organized? At the moment, there is Barsum TEM - and, to put it mildly, it barely works. A terabyte has accumulated in a month and he almost can't cope. It writes normally to the database, but when you turn on sorting for 1 ip per day, at least it will take more than 1 hour. Maybe there are some means? Preferably frivolous.

Answer 1

nfcapd. Free, can compress the database, everything is stored in separate files, you can use nfdump for analysis. There is a graphical part - nfSense, works on files collected by nfcapd. We have a netflow base - about 18 TB for three years (this is compressed data). nfcapd splits them into directories according to the scheme: <gateway_name>/year/month/day/hour. Accordingly, 24 files per day.

Answer 2

Let's define - do you need to collect, analyze, analyze for free or analyze quickly?
There are free collectors - and they work, according to rumors, quite stably. Analyzers are worse.
13 Free Netflow Analyzer
Well, to analyze quickly and free of charge with 1TB of data per month - this is not a question for software, or rather, not only for software, but also for hardware.

Answer 3

Perhaps the hardware is failing.
As a free solution, ntop may be suitable