Monitoring Network Traffic (Downloads)

Forums › LAN (ethernet/Wifi/routers/Bluetooth) › Monitoring Network Traffic (Downloads)

1 | 2

DazRaz

4 posts

Wannabe Geek

#612559 20-Apr-2012 18:58

I use ipcop and it is doing much the same as the trafficbadger teaser.

I have the advanced proxy, urlfilter and traffic control and report addons. Can set and enforce quotas for everyone (I do it by device), view browsing logs, filter sites and report on usage.

Spend a bit of time setting it up last year and I had bored kids home after the earthquake and one Uni student trying to study online. The anti-social users have been reigned in and now everyone has a fair slice of the pie. All you need is an old PC with a couple of nics. A little use of vi may be required setup the traffic control and report but hey, vi is easy.

DonGould

3892 posts

Uber Geek

#612565 20-Apr-2012 19:14

DazRaz: I use ipcop and it is doing much the same as the trafficbadger teaser.

I have the advanced proxy, urlfilter and traffic control and report addons. Can set and enforce quotas for everyone (I do it by device), view browsing logs, filter sites and report on usage.

Spend a bit of time setting it up last year and I had bored kids home after the earthquake and one Uni student trying to study online. The anti-social users have been reigned in and now everyone has a fair slice of the pie. All you need is an old PC with a couple of nics. A little use of vi may be required setup the traffic control and report but hey, vi is easy.

That can be an expensive way of doing things depending on how much power the old pc pulls. At RB750 will only pull ~4w, a PC could pull 100w.

Promote New Zealand - Get yourself a .kiwi.nz domain name!!!

Check out mine - i.am.a.can.do.kiwi.nz - don@i.am.a.can.do.kiwi.nz

DazRaz

4 posts

Wannabe Geek

#612589 20-Apr-2012 20:29

100W is a light bulb. It's not that bad.

DonGould

3892 posts

Uber Geek

#612599 20-Apr-2012 20:37

DazRaz: 100W is a light bulb. It's not that bad.

It's still $10 a month if 24/7, so after 12 months you could have a router that takes less space and uses a fraction of the power. Long term it's greener.

it will also be UFB read, where as how much data will another PC move over it's 2 nics?

Promote New Zealand - Get yourself a .kiwi.nz domain name!!!

Check out mine - i.am.a.can.do.kiwi.nz - don@i.am.a.can.do.kiwi.nz

DazRaz

4 posts

Wannabe Geek

#612638 20-Apr-2012 22:50

DonGould:
DazRaz: 100W is a light bulb. It's not that bad.

It's still $10 a month if 24/7, so after 12 months you could have a router that takes less space and uses a fraction of the power. Long term it's greener.

it will also be UFB read, where as how much data will another PC move over it's 2 nics?

So the the solution I use which uses a PC is terrible because of the extreme power use but the other posters solution using a PC does not attract your notice. It's what I use and I'm happy with it. As for power, maybe it's using half that. Neither of us has metered it so who knows.

I don't get the point with the two nics. It will move as much data as the household uses. They are not teamed. One is the outside network, one for the internal network.

jlct021

3 posts

Wannabe Geek

#613409 22-Apr-2012 22:47

Hello from Sunny South Africa and thanks Colinspocket for the Topic :)

Over the past few years I've (also) tried various ways of achieving this; all with their own pro’s and cons, none (yet) meet my needs exactly. That is not to say they don’t exits but rather I don’t (yet) have the skills / know how to get the results I'm looking for.

Some of the methods I’ve tried include:

1) Installing software on each users PC that records / alerts me via email when quotas are reached - Netlimiter (free) / Dumeter (not free) were two of the better ones I remember using

2) Updating routers Firmware as Tigercorp suggested. I tried DD-WRT at the time. (Though needing a new router just for this I decided to buy a Mikrotik RB750CL instead for its extra capabilities.) The below caught my eyes recently though perhaps it will help you -

http://www.myopenrouter.com/forum/thread/26431/Bandwidth-monitoring-per-IP-and-download- upload-usage-quotas/

3) IPCOP version is 1.4.21 with TCAR add-on for recording users bandwidth per ip, sending email reports to user and admin once a day and capping users if they use up their allocated quota. (please note TCRAR only records downloads, for uploads there is an add-on called xtra graphs. This link can help you install TCAR if you decide to go this route:

http://www.linuxquestions.org/questions/linux-networking-3/internet-quota-for-adsl-using-linux-752598/

4) Smoothwall Express 3.0 has some great Bandwidth monitoring tools right out the box and for Quota's there's a mod called BandviewX -

http://community.smoothwall.org/forum/viewtopic.php?f=26&t=31111

5) PFsense - While it has some bandwidth monitoring tools, none (that I found) were able to match the above mentioned. Here's a link relating to its bandwidth monitoring tools:

http://doc.pfsense.org/index.php/How_can_I_monitor_bandwidth_usage%3F

I found this thread yesterday highlighting the need for just such that:

http://forum.pfsense.org/index.php?action=printpage;topic=32256.0

6) Bandwidthd - Really thought Id found what I was looking for with this one but when I compared its stats to those of my ISP they didnt match - some Googling later and I found it is best used for "trends" rather that dead accurate stats - one of the things I really liked about bandwidthd was the ability to give users access to it via a link on on their browser so they could view their own usage. It was also light (low-bloat.)

http://bandwidthd.sourceforge.net/
http://community.spiceworks.com/how_to/show/1261

7) NTOP with NetFlow - NTOP Seems great for very detailed real time stats but way more info that what I need.

8) Using VM Wares's ESXi4.1 (Hypervisor) I installed IPCOP, Smoothwall, PFsense, my Linux server (Debian Squeeze) and two desktop pc's all on one (Intel Motherboard not Xeon or anything facy) box as Virtual Machines. (only running one firewall at a time obviously) While I realize (seriously) not recomended for any environment where security is crucial as a Firewall shouuld always run on dedicated hardware; it was a great learning curve, I had lot of fun setting it all up and it actually met my needs for a few months. I could install those virtual firewalls as one would normally install programmes all on one "big" computer then just making use of the add-ons I needed. It still blows my mind but then thats virtualisation for you i guess. I should also mention that this option also requires two machines an the virtual machines on the Hypervisor can only be accessed via a second machine on the network. Im my case though because i have a server i would otherwize need three machines; Firewall, Server and desktop(s) to access them all, thus this option meant I needed one less box.

9) Using my Mikrotik RB750CL to send stats via NetFlow (TrafficFlow on Mikrotik) to an (open source) collector like nfsen, pmacct, prtg (not open source and will only run on Windows but free version allows for monitoring of 10 sensors). This is what Im currently trying to get to work and as yet have not been successful - due to my own (current) lack of skills not that of the software etc

http://nfsen.sourceforge.net/

http://www.pmacct.net/

http://www.paessler.com/prtg/?source=adwords&campaign=LOW_PRTG_Search&adgroup=prtg_traffic_grapher&adnum=003&gclid=CP3Fla2ayK8CFZARfAodUAuSYQ

And then have those stats (nicley) graphed using a third party tool (if need be - I think nfsen is all in one?) like pmgraphs, Cacti, Bwstat (2005 - not sure if will still work with current pmacct), pmacct-fe (2006 - not sure if will still work with current pmacct), pmacct-frontend.

http://www.aptivate.org/pmgraph
http://www.cacti.net/
http://projects.celuloza.ro/bwstat/
http://www.pmacct.net/fe/
https://github.com/Daniel15/pmacct-frontend/

The results Im looking to achive now are:

1) Use a router to accurately record WAN / local traffic (by user IP) passing through it

2) Have those stats sent to a pc on my network where they can be analyzed / graphed

3) Have those stats viewable to myself and users on my LAN via a hyperlink (like Bandwidthd) in a simple: In / Out / Total by Hour / Day / Week / Month / Year format.

4) If no simple way of doing the above then at least have the stats available for me to view / keep track of for a start.

My current choice of routers are a Mikrotik RB 750CL because:

1) As DazRaz pointed out they can not be beaten in terms of "bang for your buck."

2) As DonGould mentioned using a dedicated pc just for measuring bandwidth stops making sense (at least to me) when one realizes the same goal (bandwidth monitoring per user ip) (and more) can be reached using a router that uses a fraction of the power.

3) I already have a (file) server on my network thus want to make full use of it / reduce (not add to) my power consumption.

Having said all of the above; if anyone has any insight / experience in getting option 9 (Using Mikrotik) to do the above or any other insight at all into any of the above it would be great to hear it.

Thanks again Colinspocket for the topic and Geekzone for the awesome site.

ArcticSilver

729 posts

Ultimate Geek

#615213 25-Apr-2012 23:13

michaelmurfy: Maybe it's about time we release TrafficBadger...

Runs on a PC with 2x NICS, people dial through PPPoE, a custom project from a few of us geeks at our flat. Also does Youtube caching, monitors b/w usage per user, allows guests to purchase data etc.

Would be extremely interested in a system like this. Its a pitty no one else has really made anything like TrafficBadger.

AliExpress

Shop now on AliExpress (affiliate link).

insane

3239 posts

Uber Geek

ID Verified

Trusted

#615222 25-Apr-2012 23:43

On my one internet connection I've got an IPcop for two flatmates with the TCRAR add-on, it's not ideal as each device has to be setup with a DHCP reservation before it works. By default if you haven't setup the device then there is no internet connectivity!

I'm surprised no one here has just used TomatoUSB with a w/less router. The traffic accounting on TomatoUSB automatically captures new devices which pickup an IP via DHCP, all you have to do is every now and then setup some DHCP reservations to give each new device a name and you're away. It's supposed to be able to pipe the logs off to a samba share or an attached USB flash drive so that your stats survive a router reboot.

The TomatoUSB accounting is missing is chargeback features and daily end user reports though, but is dead simple to setup and costs noting to run.

Ragnor

8219 posts

Uber Geek

Trusted

#615243 26-Apr-2012 04:10

ArcticSilver:
Would be extremely interested in a system like this. Its a pitty no one else has really made anything like TrafficBadger.

There's Web Gauge but it has a monthly fee
http://www.webgauge.co.nz/

However all you really need is static DHCP by mac address then per ip address data usage over a date range, there are plenty of solutions that can do that:

TomatoUSB
Gargoyle Router

pfsense
ipcop
smoothwall
monowall

timbosan

2159 posts

Uber Geek

#615249 26-Apr-2012 07:02

Ragnor:
ArcticSilver:
Would be extremely interested in a system like this. Its a pitty no one else has really made anything like TrafficBadger.

There's Web Gauge but it has a monthly fee
http://www.webgauge.co.nz/

However all you really need is static DHCP by mac address then per ip address data usage over a date range, there are plenty of solutions that can do that:

TomatoUSB
Gargoyle Router

pfsense
ipcop
smoothwall
monowall

I cannot believe no one has mentioned Untangle yet! Its a very mature product, free, and sits on a PC with two network cards (like pfsense) and has very detailed reporting built in, daily/weekly/monthly by user, by site etc. It has firewall, SPAM control, web-filtering (auto blocks porn sites, gambling sites, proxy sites etc) remote access, anti-virus, ad-blocker (sorry admins it automatically blocks ads on geekzone...) etc and I was running it on a P3 for quite a while. It has advanced features such as web cache (based on Squid), load balancer etc that do cost. I have played with the cache (it is free for 30 days) and it does make quite a difference.

I also have it configured to be a DNS server for the most common used sites, makes browsing the web a bit quicker.

For this kind of solution its actually more important to have good network cards rather than a fast PC, I ran Untangle on a 3Ghz P4 for over a year and could still pull files down a full line speed.

Plus there is NO software to install on each PC, it installs as a transparent proxy, just plug it in an go, no configuration needed or anything.

chevrolux

4962 posts

Uber Geek
Inactive user

#615272 26-Apr-2012 08:44

Isn't Untangle crazy expensive? You can download a basic version but all the cool quota stuff is paid?

I've out in a couple of Gargoyle routers for mates in flats. Really really simple.

Colinspocket

30 posts

Geek

#615387 26-Apr-2012 12:42

Oh wow glad this topic took off =]

jlct021 thanks for your reply! Seems like you're been at this for a long time! Which worries me as you say you still haven't found a reliable way to track users data??
Although from the replies about Gargoyle and TomatoUSB, and then the more complex ones like pfsense, IPcop, nfsen etc, it seems like others have achieved this already?

We're looking at grabbing a router (need wireless, whole flat is on laptops/mobile devices) we can install Gargoyle on and see how that goes for us. could be interested in setting up an old PC as a router with a cache and server but I think that's another project in itself, just want to focus on a regular router for now.

I'm surprised until now I'd always had gateways and never separated the whole modem/router as separate components. After I figured this out I understood why we wouldn't be able to install Gargoyle on our gateway.

timbosan

2159 posts

Uber Geek

#615397 26-Apr-2012 12:54

chevrolux: Isn't Untangle crazy expensive? You can download a basic version but all the cool quota stuff is paid?

I've out in a couple of Gargoyle routers for mates in flats. Really really simple.

I have only ever used the Free version (now called Lite) and it does everything I have ever needed. There is a comparison at http://www.untangle.com/software-packages.

The only parts I would pay for as a home user are Web Cache and possible Balancer, although the latter is only of use if you have two Internet connections.

Also, having a central system means when I enter my unblock-us.com DNS address in Untangle as my DNS server, all my machines can use Hulu, Netflix etc, including the PS3, Apple TV etc. Just one place to change to change it all.

I am not sure of the quota (i.e. limit usage by user) as I have never needed it.

1 | 2