OS and software for monitoring Internet access?

Question

linux

Hey Habr!
There is an office for 30-40 cars.
The Internet transparently distributes Debian 6 (vuurmuur + dnsmasq)
There was a need to monitor in detail and limit who goes where + draw graphs and tables for the authorities.
I googled a lot of information on how to tie squid + goodies to debian and there will be happiness, but in what I found, I noticed that only http can be tracked. But our traffic is heterogeneous, not only Web. It is necessary to objectively evaluate the channel load for all users and protocols.
Need something like

дата.<br/>
IP пользователя<br/>
протокол (ftp, smtp, pop3 и т.д.)<br/>
объём данных<br/>
--------------<br/>
Далее список посещённых ресурсов (dns имя или IP) отсортированный по объёму.<br/>

Kerio WinRoute would have ideally coped with this task, but it’s a shame for the power of Linux and it’s a pity for money (even if it’s someone else’s).
I ask you to advise a free free analogue, otherwise * tsat kilorubles will go to the Czech Republic to pay for the license.

Answer 1

At different stages of my work, this role was performed by different softwares on different distros, at the moment it is: SF X2100 M2 (1.2 AMD Opteron, 1GB of RAM) + OracleLinux (without support, I pull updates from RPM repositories, in principle, you can choose any)+LAMP+Squid+SAMS+Ipcad+script to fetch from ipcad.
In this configuration, the piece of iron can be anything, distro too, all the salt in the script that makes a selection from the output of Ipcad and puts it in access.log, sams, in turn, parses all this - this puts it in the MySQL database. All this is quite nicely displayed in the web interface, if you are interested, I can tell you in more detail.

Answer 2

traffpro.ru/ - for 30-40 users it is quite a tolerable solution. If you plan to expand, consider buying Kerio Control

Answer 3

You can try ntop, but it's not exactly what you need.
If I did, then I would probably write netflow logs.

Answer 4

Have you tried netams?

Answer 5

I understand that they do not consider money for the purchase of a gateway at all? Or do they still count? We took into account that for Kerio to work, you need to install a Windows Server, for a Windows server it is possible to allocate funds for the purchase of new hardware. A few years ago, Windows with Kerio on a computer with 256 megabytes of RAM, the system simply choked and loaded the swap continuously. Later launched the gateway on FreeBSD on the same hardware. The exchange rate increased by 3 times (grids were 100 megabits). If possible, add additional. costs, maybe the desire to buy Kerio will disappear.
And don't forget to master netams of course.

Answer 6

softflowd/fprobe+flowtools+mysql+rrdtool three and a half scripts, if you want to write something with your Wishlist.
If there is no desire - bandwidthd draws per-IP graphics well, though you don’t know how it works under linux, under BSD - it removes the stat from bpf, which on the one hand is cool and fast enough, and on the other, it can at speeds > 70-100 Mbps per person skip packets and lie with graphs somewhere up to ~ 20%. From this side parsing netflova is more preferable.