[TelstraClear] Internet usage when modem is off!!

Forums › One NZ (including Farmside) › [TelstraClear] Internet usage when modem is off!!

1 | ... | 9 | 10 | 11 | 12 | 13 | 14 | 15

637 posts

Ultimate Geek
+1 received by user: 2

Trusted

#525714 26-Sep-2011 10:34

DonGould:

Ok 11 terms there that I can only guess what they are, and others not in the click would have no idea about...

So if you have a second, as we've had over 6,000 views on this thread in such a short time, would you might just expanding those terms to something that people can Google the meaning of?

Not really - it's beyond both the scope of this thread and my spare time to go into details about router and forwarding architecture. You can either take my comments at face value as someone who designs and operates broadband networks, or you can challenge me on them if you want - but be prepared to go and do your own research if that's what tickles your fancy.

DonGould: With respect to the section I underlined, can you please either link an explanation or further explain what sampling means in this instance?

http://en.wikipedia.org/wiki/Netflow#Sampled_NetFlow
http://www.cisco.com/en/US/docs/ios/12_0s/feature/guide/12s_sanf.html

For two examples.

DonGould: How is it done?

Does it, for example, just check every 1000th packet header and just assume all the others are the same size and from the same locations?

The routers will only look at the 1 packet in every 1000 and add the data to the flow cache, either randomly or deterministically depending on the platform and implementation. It doesn't make any inferences about any other data, just about the 1/1000 (or 1/250, or 1/100, or whatever you configure) packets that are actually sampled. The rest of the packets are ignored.

DonGould: If you're running your flows at 1/1000th, what is the value of that data at all? Why would you even bother to collect it at that resolution? What would it tell you? What would you be looking for?

Typically just for statistical information for network planning, in some cases used as a form of looking for malicious traffic (e.g. to feed Arbor) or unusual traffic patterns.

E.g. Many operators use sampled netflow on my border routers to analyze which networks they need to pursue peering with - based on destination AS data.

DonGould:
Ok, cool, your view and yet many providers do use this technology in their billing, so how do you suggest they should be doing it?

They can talk to their vendors or systems integrators, but there are many better implementations, usually based around using L4 classification into policer or queues that account accurately for each subscriber session; or by using DPI boxes; or PCMM in the cable world; and so on.

Anyone using netflow for billing is likely going to get it wrong, but it should always under-report not over-report.

PenultimateHop

637 posts

Ultimate Geek
+1 received by user: 2

Trusted

#525715 26-Sep-2011 10:35

jnawk: It appears that TCL don't use session information in their IP accounting. And no, there is no DHCP on the TCL Cable network - we are all assigned a static IP address, given a mask (usually class C), and a gateway.

Ouch!

Skolink

1081 posts

Uber Geek
+1 received by user: 32

#525998 26-Sep-2011 21:46

Has DonGould been 'inactived'? Slightly OT but this is the last thread he posted in I think. I had a question for him but now his user profile doesn't seem to work.

jnawk

176 posts

Master Geek
+1 received by user: 11

#526004 26-Sep-2011 22:06

Skolink: Has DonGould been 'inactived'? Slightly OT but this is the last thread he posted in I think. I had a question for him but now his user profile doesn't seem to work.

He said a naughty word. I'll relay your message if you like.

freitasm

BDFL - Memuneh
80658 posts

Uber Geek
+1 received by user: 41071

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#526006 26-Sep-2011 22:10

I've explained on the other thread. He didn't say a naughty word, but there was a serious breach of privacy. He's been warned and is in suspended for a week.

Referral links: Quic Broadband (free setup code: R587125ERQ6VE) | Samsung | AliExpress | Wise | Sharesies

Support Geekzone by subscribing (browse ads-free), or making a one-off or recurring donation through PressPatron.

Skolink

1081 posts

Uber Geek
+1 received by user: 32

#526011 26-Sep-2011 22:15

freitasm: I've explained on the other thread. He didn't say a naughty word, but there was a serious breach of privacy. He's been warned and is in suspended for a week.

Thanks. I had a look but must have missed that.

Lenovo

Shop now for Lenovo laptops and other devices (affiliate link).

jnawk

176 posts

Master Geek
+1 received by user: 11

#526022 26-Sep-2011 22:26

freitasm: He didn't say a naughty word

I was just poking fun. :-)

Well, I think getting banned for a naughty word is funnier than that which he tried to do...

DonGould

3892 posts

Uber Geek
+1 received by user: 164

#527611 30-Sep-2011 11:31

Quick Update:

My vacation away gave me time to clear my head and chat to a few people about this problem, off line, in real life.

I'm sure most will agree that I was getting more and more wound up about this issue with each post last week.

It's my considered view that I wouldn't be worrying about this issue if I was living in the USA where they either have massive data caps or don't have them at all. If I was over the ditch in Australia, where you can get 1Tb for roughly the cost we're paying for 60Gb, then again I wouldn't be to worried (this is a point that PenultimateHop alluded to in one of his posts).

In New Zealand our data is worth ~16 times more than in Australia, so managing the limited resource we have is more important to me even if it's not important to others.

Clearly I'm a bit of a hot head on this issue. I'm not going to deny this, and it really needs some far more balanced people than me looking over it with clear heads and a national policy interest.

As such, I've contacted the Ministers office, the Greens and have a call in with the Labour IT spokes person as well, to cast an eye over the issues.

I have also talked to a number of technical people about CPE solutions to provide some auditing of data usage so that users can regain some confidence in this space. I think consumer confidence is very important.

Some have said that regulation in this area will not help, while others here seem to feel it could. I don't know, I'm not a policy expert, I'm not a regulatory expert. There are people who are experts in this area and I think we should deffer to them for their guidance.

Bellow is my communication to the ministers office that was copied to the Greens and Labour:

To: Joe Connell | Private Secretary, Communications and Information Technology | Office of Hon Steven Joyce | Minister for Communications and Information Technology

Hi Joe,

Thanks for your time on the phone, I do appreciate you're busy and freed up a good amount of time for this issue.

Internet usage when modem is off!! - http://www.geekzone.co.nz/forums.asp?forumid=44&topicid=89860

Above is the link to cast your eye over.

Key issues:

* Data is counted even when the modem is off. (power meter - when meter is off power doesn't get used).
* Counting of data is not regulated/audited in any way. (butcher, steak, measure example)
* Customer service fob users off when they complain - this is CGA stuff.
* Other industries are expected to measure with accuracy - http://www.geekzone.co.nz/forums.asp?forumid=44&topicid=89860&page_no=13#525525
* Consumers are not able to compare their own measures with that of their providers - http://www.geekzone.co.nz/forums.asp?forumid=44&topicid=89860&page_no=13#525348
* Consumer confidence in this space is eroded - http://www.geekzone.co.nz/forums.asp?forumid=44&topicid=89860&page_no=13#524817
* It is possible to get this right - http://www.geekzone.co.nz/forums.asp?forumid=44&topicid=89860&page_no=13#524634
* Users have no way to use the metering systems as part of their own network security - http://www.geekzone.co.nz/forums.asp?forumid=44&topicid=89860&page_no=1#519582

As I explained on the phone, and I'm sure will be very obvious from the web pages at Geekzone, I can be a bit hot headed about these issues, so it would be good if your office could cast an eye over the whole issue, with some calm reasoning heads and consider what sort of action is needed.

I think that many users have made a good range of quite balanced suggestions in the thread on the web site.

In my mind the key considerations have to be:

* Is FTA, CGA legislation just being mocked by the entire industry? Consumers would not expect this kind service from Kmart or the Warehouse would they?
* Is the Weights and Measures Act out of kilter and does it need attention to ensure measurement of 1's and 0's?
* Are the industry 'working groups' such as TCF working effectively in the interests of consumers?

D

Joe was very helpful, did seem to understand the basic issues, though was unaware of them, and agreed that some officials from his office should look at this issue and that he would follow a response from the minister.

I also talked to Gareth Hughes about the issue yesterday. He reviewed a copy of my email and agree that there is an issue that he thought should be looked at. He very quickly picked up the question of the weights and measure act. He committed to having some more tech savey members of the Green party look at the issue and would also write to the minister.

Clare Curran (Labour) is back in Dunedin today and her office said they would draw the issue to her attention but I have not talked to her as yet.

I also talked to Paul Brislen from Tuanz. He commented that the Weights and Measures Act does not cover 1's and 0's at all.

I also talked with a monitoring vendor who tells me they can deliver consumer CPE that could record traffic usage and provide the ability to do the auditing and monitoring that would help consumer confidence.

Anyway, that's my short update for now :)

BiDi, your latest graphs are very cool and watching your posts last week inspired me to push this along a bit.

D

Promote New Zealand - Get yourself a .kiwi.nz domain name!!!

Check out mine - i.am.a.can.do.kiwi.nz - don@i.am.a.can.do.kiwi.nz

BiDi

72 posts

Master Geek
+1 received by user: 3

#527618 30-Sep-2011 11:49

Good to see you on the forum again Don.

There's not much more I can do with my limited data set. I think it needs a more network-savy geek to collect data for analysis.

However, I hope that it is now pretty clear that TC are not providing useful information to their customers.

I hear more and more buzz about 'smart metering' these days, but see no sign of it here.

BiDi

72 posts

Master Geek
+1 received by user: 3

#527648 30-Sep-2011 12:46

DonGould: I also talked to Paul Brislen from Tuanz. He commented that the Weights and Measures Act does not cover 1's and 0's at all.

No, it doesn't seem to (http://www.legislation.govt.nz/regulation/public/1999/0373/latest/DLM301528.html). It was clearly designed in earlier times (who needs 'leather measuring instruments' these days?) and has had some fairly obvious patches added.

The Act is really about physical quantities.

To extend the Act to cover the measurement of 'information' would seem ambitious (you will note that the Act is more concerned about milk, beer and fuel). I wonder whether any other country regulates and measures information flows.

Nevertheless, Consumer Affairs (http://www.consumeraffairs.govt.nz/legislation-policy/policy-reports-and-papers/disscussion-papers/consumer-law-reform-a-discussion-paper/10.-trader-obligations/10.2-weights-and-measures) does ask if "... there any other areas within the Weights and Measures Act that you think could be improved?" (bottom of page, Q33).

Our data, as Don points out, is expensive and as a consumer, I want to know that I am being treated fairly. From this discussion, it is apparently not so easy for me to verify this.

Historically, that's why things like the Act were needed. It is easy now to re-weigh your bananas when you get home from the supermarket, but that was not always so. And on a commercial scale, one still requires specialised instruments to monitor quantities. Our society would suffer greatly without confidence in measurements. It is one of the traits that distinguishes a developed country from a developing one. We take it all for granted, but we shouldn't.

I applaud Don's initiative in bringing this discussion to the attention of our politicians.

DonGould

3892 posts

Uber Geek
+1 received by user: 164

#527660 30-Sep-2011 13:07

BiDi: Good to see you on the forum again Don.

Thanks.

BiDi:
There's not much more I can do with my limited data set. I think it needs a more network-savy geek to collect data for analysis.

I agree. That's why I've invested some time into getting a bit more geek help.

However, I hope that it is now pretty clear that TC are not providing useful information to their customers.

Ok couple of points I'd like to make.

1. The Firefox plug in thing is actually quite a good tool and in my view every user should be using that tool. If you have a close look at it, you'll see it keeps a history of your usage locally.

2. I now have a data set showing my entire last years metering. So TCL staff are actually being helpful to a degree.

However, in the same way that a builder can look at the box and just tell you it's not square, I can look at data sets and spot obvious issues.

I've forwarded my data set to a number of people now and there seems to be agreement that the data is not very good at all.

It has way to many holes in it. While the graphs I posted earlier show days that have been missed, it seems, even since the upgrades they did in July, that the system misses the counter pick up often. Not as bad as it was, but still far more often that I would be happy to sign off on.

Others I talked to commented about the general lack of data.

The bottom line for me is that the data was so lacking and appeared so far out of whack that I had to decide if I either work with them to attempt to address the issue or just step back and have a rethink about how to approach the whole issue.

It was at that point that I decided that I really need a range of eyes over this problem at many levels to give me a more balanced view.

The work you have done BiDi is amazing and I can see it took some real effort, frankly more effort that it should have.

I hear more and more buzz about 'smart metering' these days, but see no sign of it here.

Ya that's a whole other interesting discussion. I follow the NBN debate in Australia. They did a talk to the power companies about using the NBN for smart metering... I would have liked to be sitting in on that presentation.

Promote New Zealand - Get yourself a .kiwi.nz domain name!!!

Check out mine - i.am.a.can.do.kiwi.nz - don@i.am.a.can.do.kiwi.nz

AliExpress

Shop now on AliExpress (affiliate link).

jpollock

600 posts

Ultimate Geek
+1 received by user: 5

Trusted

#527663 30-Sep-2011 13:11

I went looking for government involvement in electricity measurement. I found this page from the MED:

http://www.med.govt.nz/templates/MultipageDocumentPage____2329.aspx

---
http://blog.jason.pollock.ca

DonGould

3892 posts

Uber Geek
+1 received by user: 164

#527681 30-Sep-2011 13:31

jpollock: I went looking for government involvement in electricity measurement. I found this page from the MED:

http://www.med.govt.nz/templates/MultipageDocumentPage____2329.aspx

Thanks. I now have a call in with MSL. I will report back on that one.

Promote New Zealand - Get yourself a .kiwi.nz domain name!!!

Check out mine - i.am.a.can.do.kiwi.nz - don@i.am.a.can.do.kiwi.nz

BiDi

72 posts

Master Geek
+1 received by user: 3

#527683 30-Sep-2011 13:33

jpollock: I went looking for government involvement in electricity measurement. I found this page from the MED:

http://www.med.govt.nz/templates/MultipageDocumentPage____2329.aspx

Yes, this is a good link. You can see the separate roles of the Measurement Standards Laboratory, dealing with the scientific and technical aspects of measurement (of physical quantities), and the Consumer Affairs involved in regulatory and legal aspects.

One thing caught my eye:

The Act sets down a number of fundamental rights for consumers and those involved in trading goods by weight, measure or number. It underpins the trading environment within New Zealand and gives international credibility to our exporters.

I would say that 'number' is fundamentally what we are dealing in here. So, perhaps the Act is relevant.

DonGould

3892 posts

Uber Geek
+1 received by user: 164

#527687 30-Sep-2011 13:35

BiDi: I applaud Don's initiative in bringing this discussion to the attention of our politicians.

Thanks BiDi. I really can't take the credit on this one. You started this puppy... all I did was make a couple of phone calls to draw their attention to the GZ thread.

Who knows where it will go.

My current thinking is:

There is regulated auditing and declaration of how monitoring systems work and that usages systems must be up to date within 5 minutes.
Users need to be away how they can manage their usages and have some way to suspend traffic such as logging out or turning off the CPE.
Users must have a way of blocking traffic such as a dynamic firewall.
Users must have a way of being able to meter traffic to compare with their ISP - ISPs will be required to provide details of how they can do this.
ComCom must also perform random audits annually on the performance of metering systems.

Anything else you can think of?

Promote New Zealand - Get yourself a .kiwi.nz domain name!!!

Check out mine - i.am.a.can.do.kiwi.nz - don@i.am.a.can.do.kiwi.nz

1 | ... | 9 | 10 | 11 | 12 | 13 | 14 | 15