Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.


JohnButt

374 posts

Ultimate Geek

Trusted

#92827 7-Nov-2011 11:46
Send private message

Latest broadband comparison statistics

October is the third month of data from our series of detailed broadband performance statistics.  There were many improvements in performance in October over both August and September, but the greatest improvement was fromVodafone, moving from last to a position that would have seen them third in September, but for the competitive response from Orcon & Slingshot.

Orcon also improved to take over second position from Telecom with a second significant improvement month on month

This is also the first time TrueNet has had sufficient volunteers to report on TelstraClear DSL as well as TesltraClear Cable.  While the Cable result continues to top the poll, the DSL result is well down the list at seventh.

File 596
  

The Vodafone improvement was the result of changes that affected almost all TrueNet Vodafone probes on the 7th October after some temporary improvements in September.

View this topic in a long page with up to 500 replies per page Create new topic
 1 | 2 | 3
DonGould
3892 posts

Uber Geek


  #541972 7-Nov-2011 11:59
Send private message

More nice work John! Top stuff man... this is very very cool stuff to see. :)




Promote New Zealand - Get yourself a .kiwi.nz domain name!!!

Check out mine - i.am.a.can.do.kiwi.nz - don@i.am.a.can.do.kiwi.nz




michaelmurfy
meow
13242 posts

Uber Geek

Moderator
ID Verified
Trusted
Lifetime subscriber

  #541987 7-Nov-2011 12:39
Send private message

That's pretty awesome to see - all we need now is more probes at more locations.




Michael Murphy | https://murfy.nz
Referral Links: Quic Broadband (use R122101E7CV7Q for free setup)

Are you happy with what you get from Geekzone? Please consider supporting us by subscribing.
Opinions are my own and not the views of my employer.


JohnButt

374 posts

Ultimate Geek

Trusted

  #541991 7-Nov-2011 12:45
Send private message

michaelmurfy: That's pretty awesome to see - all we need now is more probes at more locations.


Actually a few more probes with some of the smaller ISPs would be good also, we have some but not enough with Actrix and Maxnet, but none yet from Woosh or Inspire, which I thought were of reasonable size.

Any others we should definitely cover?



Ragnor
8219 posts

Uber Geek

Trusted

  #542018 7-Nov-2011 13:43
Send private message

I think the default of interleaving on hurts Telecom's average quite a bit.. here's my Telecom connection at home for example:


johnr
19282 posts

Uber Geek
Inactive user


  #542022 7-Nov-2011 13:49
Send private message

Nice to see the hard work paying off

John

michaeln
238 posts

Master Geek

Trusted

  #542042 7-Nov-2011 14:28
Send private message

JohnButt: Latest broadband comparison statistics

This is also the first time TrueNet has had sufficient volunteers to report on TelstraClear DSL as well as TesltraClear Cable.  While the Cable result continues to top the poll, the DSL result is well down the list at seventh.


Do you have an estimate of the error around these figures? I.e. "ISP A" Time to load x seconds [+/- y seconds].

Likewise, any statistical analysis around whether results are indeed statistically different?

JohnButt

374 posts

Ultimate Geek

Trusted

  #542063 7-Nov-2011 15:09
Send private message

michaeln:
JohnButt: Latest broadband comparison statistics

This is also the first time TrueNet has had sufficient volunteers to report on TelstraClear DSL as well as TesltraClear Cable.  While the Cable result continues to top the poll, the DSL result is well down the list at seventh.


Do you have an estimate of the error around these figures? I.e. "ISP A" Time to load x seconds [+/- y seconds].

Likewise, any statistical analysis around whether results are indeed statistically different?


I am reluctant to provide everything.  Let me assure you that the TelstraClear Cable results are very reliable, the worst cable measure has only one DSL probe with better results.  

Note we use the Median probe performances per ISP, ie of the average performance for the month for each probe.  Each probe has little variation, making the average reliable.  That way we drop any outliers (if there are any - mine is) providing more reliable results.  

 
 
 

GoodSync. Easily back up and sync your files with GoodSync. Simple and secure file backup and synchronisation software will ensure that your files are never lost (affiliate link).
jpollock
600 posts

Ultimate Geek

Trusted

  #542445 8-Nov-2011 11:14
Send private message

JohnButt:
michaeln:
JohnButt: Latest broadband comparison statistics

This is also the first time TrueNet has had sufficient volunteers to report on TelstraClear DSL as well as TesltraClear Cable.  While the Cable result continues to top the poll, the DSL result is well down the list at seventh.


Do you have an estimate of the error around these figures? I.e. "ISP A" Time to load x seconds [+/- y seconds].

Likewise, any statistical analysis around whether results are indeed statistically different?


I am reluctant to provide everything.  Let me assure you that the TelstraClear Cable results are very reliable, the worst cable measure has only one DSL probe with better results.  

Note we use the Median probe performances per ISP, ie of the average performance for the month for each probe.  Each probe has little variation, making the average reliable.  That way we drop any outliers (if there are any - mine is) providing more reliable results.  


Is it on port 80/8080, or any other port which will be proxied by a transparent (or even non-transparent) proxy?  That would provide a real ability for ISPs to skew results. 




michaeln
238 posts

Master Geek

Trusted

  #542692 8-Nov-2011 16:41
Send private message

jpollock:

Is it on port 80/8080, or any other port which will be proxied by a transparent (or even non-transparent) proxy?  That would provide a real ability for ISPs to skew results. 


Does that matter? The report is "Time to download a dummy page". I.e., it's explicitly WWW traffic. Anything the ISP does that improves that time is presumably legitimate. After all, that's what most users perceive.


JohnButt

374 posts

Ultimate Geek

Trusted

  #542701 8-Nov-2011 16:54
Send private message

michaeln:
jpollock:

Is it on port 80/8080, or any other port which will be proxied by a transparent (or even non-transparent) proxy?  That would provide a real ability for ISPs to skew results. 


Does that matter? The report is "Time to download a dummy page". I.e., it's explicitly WWW traffic. Anything the ISP does that improves that time is presumably legitimate. After all, that's what most users perceive.



I agree with @michaeln, tis not our problem if the ISP improves the timing for all users to all pages, in fact that is our desire.  We may even publish results to more pages in future to provide greater scope for comparisons.  

We have a logical solution for identifying what we determine is intentional gaming by ISPs and are prepared to publish results when we find any attempt.

insane
3239 posts

Uber Geek

ID Verified
Trusted

  #542705 8-Nov-2011 17:00
Send private message

As far as I know some ISPs do prioritise some widely used benchmarks (speedtest.net)

Hopefully TrueNet uses a decent random sample of webpages for testing as it wouldn't take too much effort for an ISP to sniff the DNS requests from a user/staff member with a TrueNet probe and skew results in their favor.


JohnButt

374 posts

Ultimate Geek

Trusted

  #542708 8-Nov-2011 17:06
Send private message

insane: As far as I know some ISPs do prioritise some widely used benchmarks (speedtest.net)

Hopefully TrueNet uses a decent random sample of webpages for testing as it wouldn't take too much effort for an ISP to sniff the DNS requests from a user/staff member with a TrueNet probe and skew results in their favor.



That is exactly what our solution will detect. 

jpollock
600 posts

Ultimate Geek

Trusted

  #542727 8-Nov-2011 17:49
Send private message

JohnButt:
michaeln:
jpollock:

Is it on port 80/8080, or any other port which will be proxied by a transparent (or even non-transparent) proxy?  That would provide a real ability for ISPs to skew results. 


Does that matter? The report is "Time to download a dummy page". I.e., it's explicitly WWW traffic. Anything the ISP does that improves that time is presumably legitimate. After all, that's what most users perceive.



I agree with @michaeln, tis not our problem if the ISP improves the timing for all users to all pages, in fact that is our desire.  We may even publish results to more pages in future to provide greater scope for comparisons.  

We have a logical solution for identifying what we determine is intentional gaming by ISPs and are prepared to publish results when we find any attempt.


You see, depending on the traffic/probes/etc, it can easily be detected and optimized for.  For example, by increasing the cache hold times for the pages which are used, to the point of ignoring the validity period.  While this "improves" test results because the page is served from the local cache, it results in the wrong page being returned to the customer.

So, without publication of the test methodology, I can only say it's measuring the performance of the http cache, and easily fudged by ISPs, much the same way that video card manufacturers would detect a performance test and run special code paths which produced visibly buggy results faster.

It is the responsibility of the test designer to design a benchmark that is representative and validate that the results aren't actively subverted by ISPs looking to improve their results.  It's good to hear that you've got a solution to identify intentional gaming.  Care to share the test methodology, or do I have to sign up and snoop the traffic?

edit: Even without funny business, if all of the terminals are hitting the same pages, they will all hit the cache.  The first one will be slow, the rest quick, with an average that doesn't represent random surfing.




DonGould
3892 posts

Uber Geek


  #542765 8-Nov-2011 19:16
Send private message

jpollock: Care to share the test methodology, or do I have to sign up and snoop the traffic?

edit: Even without funny business, if all of the terminals are hitting the same pages, they will all hit the cache.  The first one will be slow, the rest quick, with an average that doesn't represent random surfing.


How is the information actually used?

I was driving down the road this afternoon and saw a massive bill board claiming that Telstra are the fastest.

Are you suggesting that Telstra are the fastest just because they fudge the figures and Vodafone have been in the back water because they just don't bother?

Is this information being used by consumers to make provider choices?  Or is it mainly being used by industry guys to help build business cases to get better get in to the network or the likes of Telecom to sell more capacity to slower providers?

If it's consumers, then yes, I can see a point for not having the figures fudged... but who would fudge it anyway that we wouldn't eventually notice and start posting on GZ naming names?  Just look at the long thread about the Telstra link to the US last month as an example.

Do most consumers talk to a trusted tech guy first, and do most of us not see though the rubbish?

If it's ISPs just cheating the stats for industry, then who are they kidding?!  ...and what for?







Promote New Zealand - Get yourself a .kiwi.nz domain name!!!

Check out mine - i.am.a.can.do.kiwi.nz - don@i.am.a.can.do.kiwi.nz


JohnButt

374 posts

Ultimate Geek

Trusted

  #542773 8-Nov-2011 19:35
Send private message

jpollock: 
It is the responsibility of the test designer to design a benchmark that is representative and validate that the results aren't actively subverted by ISPs looking to improve their results.  It's good to hear that you've got a solution to identify intentional gaming.  Care to share the test methodology, or do I have to sign up and snoop the traffic?



Yup we benchmark each ISP - snooping will not assist.  

 1 | 2 | 3
View this topic in a long page with up to 500 replies per page Create new topic





News and reviews »

Air New Zealand Starts AI adoption with OpenAI
Posted 24-Jul-2025 16:00


eero Pro 7 Review
Posted 23-Jul-2025 12:07


BeeStation Plus Review
Posted 21-Jul-2025 14:21


eero Unveils New Wi-Fi 7 Products in New Zealand
Posted 21-Jul-2025 00:01


WiZ Introduces HDMI Sync Box and other Light Devices
Posted 20-Jul-2025 17:32


RedShield Enhances DDoS and Bot Attack Protection
Posted 20-Jul-2025 17:26


Seagate Ships 30TB Drives
Posted 17-Jul-2025 11:24


Oclean AirPump A10 Water Flosser Review
Posted 13-Jul-2025 11:05


Samsung Galaxy Z Fold7: Raising the Bar for Smartphones
Posted 10-Jul-2025 02:01


Samsung Galaxy Z Flip7 Brings New Edge-To-Edge FlexWindow
Posted 10-Jul-2025 02:01


Epson Launches New AM-C550Z WorkForce Enterprise printer
Posted 9-Jul-2025 18:22


Samsung Releases Smart Monitor M9
Posted 9-Jul-2025 17:46


Nearly Half of Older Kiwis Still Write their Passwords on Paper
Posted 9-Jul-2025 08:42


D-Link 4G+ Cat6 Wi-Fi 6 DWR-933M Mobile Hotspot Review
Posted 1-Jul-2025 11:34


Oppo A5 Series Launches With New Levels of Durability
Posted 30-Jun-2025 10:15









Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.