Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.



2055 posts

Uber Geek
+1 received by user: 118

Trusted

Topic # 72123 21-Nov-2010 11:52 Send private message

Hi

running a squid proxy at home, mostly for saving bandwidth in windows updates etc.

But now I'd like to see what is using the internet at certain times to track usage.

Anyone know of a linux install/script/report or something that will allow me to see sites used over specified time period?




Previously known as psycik

NextPVR Based HTPC:

Gigabyte AMD A8 Brix --> Samsung LA46A650D via HDMI, SiliconDust HDHomeRun Dual DVB-T Tuner, NextPVR, Plex Plugin 
Clients:
Popcorn Hour A-100, 1xATV2, 1xATV3, Roku3
Windows 7 Ultimate Host (Plex Server)
3x2TB, 1x3TB, 1x4TB + 1x1.5TB using DriveBender, VMWare Workstation 10 with 1xW7, 2xW2k3 1xUbuntu 11.10 Desktop, 1xWHS2011, Plex, Crashplan, NextPVR channel for Plex

UnblockUS - Unblock your freedom

Create new topic
178 posts

Master Geek
+1 received by user: 3

Trusted

  Reply # 407405 21-Nov-2010 20:07 Send private message

Off http://www.squid-cache.org/Scripts/ is a list of squid log file analysis programs that will tell you what you want to know, .
 



2055 posts

Uber Geek
+1 received by user: 118

Trusted

  Reply # 407414 21-Nov-2010 20:52 Send private message

I've got calamaris and webalyser on the system but cannot see it producing the information I want.

Have you used one that will do what I want? if so which one?




Previously known as psycik

NextPVR Based HTPC:

Gigabyte AMD A8 Brix --> Samsung LA46A650D via HDMI, SiliconDust HDHomeRun Dual DVB-T Tuner, NextPVR, Plex Plugin 
Clients:
Popcorn Hour A-100, 1xATV2, 1xATV3, Roku3
Windows 7 Ultimate Host (Plex Server)
3x2TB, 1x3TB, 1x4TB + 1x1.5TB using DriveBender, VMWare Workstation 10 with 1xW7, 2xW2k3 1xUbuntu 11.10 Desktop, 1xWHS2011, Plex, Crashplan, NextPVR channel for Plex

UnblockUS - Unblock your freedom

25 posts

Geek


  Reply # 408246 23-Nov-2010 14:35 Send private message

I use SARG. It has by far the best summary information of any squid report analysis tool I have tried. It will also draw graphs showing when each machine has visited particular sites, as well as report some of this data numerically.



2055 posts

Uber Geek
+1 received by user: 118

Trusted

  Reply # 408302 23-Nov-2010 15:12 Send private message

Yeah I use sarg as well (configured via webmin - be that good or bad).

And while i can get the usage in bytes etc, I can't figure out what i need to set to say xyz machine visited site abc at so and so time.




Previously known as psycik

NextPVR Based HTPC:

Gigabyte AMD A8 Brix --> Samsung LA46A650D via HDMI, SiliconDust HDHomeRun Dual DVB-T Tuner, NextPVR, Plex Plugin 
Clients:
Popcorn Hour A-100, 1xATV2, 1xATV3, Roku3
Windows 7 Ultimate Host (Plex Server)
3x2TB, 1x3TB, 1x4TB + 1x1.5TB using DriveBender, VMWare Workstation 10 with 1xW7, 2xW2k3 1xUbuntu 11.10 Desktop, 1xWHS2011, Plex, Crashplan, NextPVR channel for Plex

UnblockUS - Unblock your freedom

25 posts

Geek


  Reply # 408465 23-Nov-2010 18:50 Send private message

Are you much of a coder in perl or bash or anything that can convert between unix dates and human-readable dates? When I need specific searches such as yours, I usually do my processing against the original squid log. cat /var/log/squid/access.log | filtercommand > anotheraccess.log

Then, run sarg against the filtered output. In your case, you'd just need a filter than reads the start date/time and end date/time, then outputs only lines that fall between the two.



2055 posts

Uber Geek
+1 received by user: 118

Trusted

  Reply # 408801 24-Nov-2010 10:57 Send private message

greaneyr: Are you much of a coder in perl or bash or anything that can convert between unix dates and human-readable dates? When I need specific searches such as yours, I usually do my processing against the original squid log. cat /var/log/squid/access.log | filtercommand > anotheraccess.log

Then, run sarg against the filtered output. In your case, you'd just need a filter than reads the start date/time and end date/time, then outputs only lines that fall between the two.


Not really. I was hoping sarg or webalyser would do it for me.

I've not even really looked at the raw access.log before.




Previously known as psycik

NextPVR Based HTPC:

Gigabyte AMD A8 Brix --> Samsung LA46A650D via HDMI, SiliconDust HDHomeRun Dual DVB-T Tuner, NextPVR, Plex Plugin 
Clients:
Popcorn Hour A-100, 1xATV2, 1xATV3, Roku3
Windows 7 Ultimate Host (Plex Server)
3x2TB, 1x3TB, 1x4TB + 1x1.5TB using DriveBender, VMWare Workstation 10 with 1xW7, 2xW2k3 1xUbuntu 11.10 Desktop, 1xWHS2011, Plex, Crashplan, NextPVR channel for Plex

UnblockUS - Unblock your freedom

25 posts

Geek


  Reply # 409145 24-Nov-2010 20:39 Send private message

Well do you want it for an ongoing thing, or are you just wanting to know where people are going between certain times just for a little while? If you were after an ongoing thing, then you probably do need something a little less manual, but if you just wanted it for say a week, then it wouldn't be much at all to make something like I was talking about. I could probably throw something together for you if you thought it might help.



2055 posts

Uber Geek
+1 received by user: 118

Trusted

  Reply # 409176 24-Nov-2010 21:28 Send private message

if you could, as a manual thing, then I can always hack around with it to make it into a script.

thanks.




Previously known as psycik

NextPVR Based HTPC:

Gigabyte AMD A8 Brix --> Samsung LA46A650D via HDMI, SiliconDust HDHomeRun Dual DVB-T Tuner, NextPVR, Plex Plugin 
Clients:
Popcorn Hour A-100, 1xATV2, 1xATV3, Roku3
Windows 7 Ultimate Host (Plex Server)
3x2TB, 1x3TB, 1x4TB + 1x1.5TB using DriveBender, VMWare Workstation 10 with 1xW7, 2xW2k3 1xUbuntu 11.10 Desktop, 1xWHS2011, Plex, Crashplan, NextPVR channel for Plex

UnblockUS - Unblock your freedom

Create new topic




Twitter »
Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:




News »

Trending now »
Hot discussions in our forums right now:

Windows 10 announced, as well as developer preview
Created by macuser, last reply by KiwiNZ on 2-Oct-2014 19:18 (105 replies)
Pages... 5 6 7


What time will the Apple Store online be selling the iPhone 6?
Created by scotiwis, last reply by zipstacrack on 2-Oct-2014 19:25 (197 replies)
Pages... 12 13 14


Moment of Truth?
Created by BarTender, last reply by JimmyC on 29-Sep-2014 09:16 (441 replies)
Pages... 28 29 30


Can i have 2 ISP's at home?
Created by ReckITT, last reply by Lazarui on 30-Sep-2014 18:15 (49 replies)
Pages... 2 3 4


Samsung Galaxy S4
Created by beenz, last reply by beenz on 2-Oct-2014 19:02 (20 replies)
Pages... 2


Why is your nickname what it is, what are the origins of it?
Created by Presso, last reply by BlueShift on 2-Oct-2014 16:24 (96 replies)
Pages... 5 6 7


iPhone 6 From Spark - Order Dates and Pricing?
Created by Otagolad, last reply by jeeg on 2-Oct-2014 19:12 (353 replies)
Pages... 22 23 24


Harvey Norman's Biggest Ever Retail Sale
Created by DravidDavid, last reply by joker97 on 2-Oct-2014 14:24 (30 replies)
Pages... 2



Geekzone Live »
Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.

Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.