How to monitor activity on the network?

Question

linux

Salute comrades!
Share your experience, how can you monitor user activity on the network?
I have Mikrotik as the heart of the network, NetFlow is enabled on it, and there is a server that collects statistics and puts everything into a database. But the peculiarity of Netflow is that it gives information only after the client has downloaded / uploaded the file. But there are situations when someone occupied the channel by downloading a large file, and until he downloads it, NetFlow will not tell me about it. Are there any mechanisms to keep track of such moments. Or, after all, do you have to set a proxy?

Answer 1

What do you mean it won't report? Netflow doesn't know anything about files. It operates on packets transmitted over the network. If someone downloads a "file", then you will have a bunch of records "someone-to-how many_bytes". It if to explain on fingers
If it is necessary to monitor not statistics of use, and loading of the channel, then take away current digits on SNMP.

Answer 2

I am a noob in this, but if we are talking about the http protocol (not https), then you can catch the content-type and content-length headers.

Answer 3

Certainly a proxy. Not Mikrotik in the sense of a built-in proxy, but a normal full-fledged squid. Which, in particular, also solves the problem of breaking off a user who wants to merge a movie into a couple of gigs during working hours :)

Answer 4

+ to all the above unsubscribed. the solution would be qos+proxy.
you can of course send netflow to siem, but this is already from the depth of tasks

Answer 5

I join kinvlad and Dmitry . NetFlow does not care who transmits what. It only distinguishes between packets being transmitted and the type of traffic. It does not distinguish between files. Perhaps you are confusing with SNMP, but this is not particularly observed there either.
1. NetFlow Analyzer works well under Windows.
2. On Mikrotik, in the properties of the interface with the load, click on the Torch button and you will be happy. Of course, there will be no notifications (there is a whole class of programs for this, for example, from the first paragraph), but you can see who and how is clogging the channel now.

Answer 6

You can do a tricky thing through the firewall. Following the link, an example with a time block, you can also make a block not by time, but let's say just make an entry in the log or send a message immediately to the mail. You can again, in the image and likeness of the example, make a list as in the example, but do not cut it at all, but simply throw it into another rule and it will enjoy a speed of 100 kb / s for a whole hour / 6 / day / week.
Another option is to use QoS. Then they can swing calmly, but they will move if someone else wants to shake something too.
Mikrotiks are very cool things and almost any task can be solved with standard tools, I highly recommend climbing their wiki with examples of different tricks.