Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.




374 posts

Ultimate Geek

Trusted

#92827 7-Nov-2011 11:46
Send private message

Latest broadband comparison statistics

October is the third month of data from our series of detailed broadband performance statistics.  There were many improvements in performance in October over both August and September, but the greatest improvement was fromVodafone, moving from last to a position that would have seen them third in September, but for the competitive response from Orcon & Slingshot.

Orcon also improved to take over second position from Telecom with a second significant improvement month on month

This is also the first time TrueNet has had sufficient volunteers to report on TelstraClear DSL as well as TesltraClear Cable.  While the Cable result continues to top the poll, the DSL result is well down the list at seventh.

File 596
  

The Vodafone improvement was the result of changes that affected almost all TrueNet Vodafone probes on the 7th October after some temporary improvements in September.

View this topic in a long page with up to 500 replies per page Create new topic
 1 | 2 | 3
3892 posts

Uber Geek


  #541972 7-Nov-2011 11:59
Send private message

More nice work John! Top stuff man... this is very very cool stuff to see. :)




Promote New Zealand - Get yourself a .kiwi.nz domain name!!!

Check out mine - i.am.a.can.do.kiwi.nz - don@i.am.a.can.do.kiwi.nz


/dev/null
9150 posts

Uber Geek

Moderator
Trusted
Lifetime subscriber

  #541987 7-Nov-2011 12:39
Send private message

That's pretty awesome to see - all we need now is more probes at more locations.




 
 
 
 




374 posts

Ultimate Geek

Trusted

  #541991 7-Nov-2011 12:45
Send private message

michaelmurfy: That's pretty awesome to see - all we need now is more probes at more locations.


Actually a few more probes with some of the smaller ISPs would be good also, we have some but not enough with Actrix and Maxnet, but none yet from Woosh or Inspire, which I thought were of reasonable size.

Any others we should definitely cover?

8035 posts

Uber Geek

Trusted

  #542018 7-Nov-2011 13:43
Send private message

I think the default of interleaving on hurts Telecom's average quite a bit.. here's my Telecom connection at home for example:


19282 posts

Uber Geek
Inactive user


  #542022 7-Nov-2011 13:49
Send private message

Nice to see the hard work paying off

John

220 posts

Master Geek

Trusted

  #542042 7-Nov-2011 14:28
Send private message

JohnButt: Latest broadband comparison statistics

This is also the first time TrueNet has had sufficient volunteers to report on TelstraClear DSL as well as TesltraClear Cable.  While the Cable result continues to top the poll, the DSL result is well down the list at seventh.


Do you have an estimate of the error around these figures? I.e. "ISP A" Time to load x seconds [+/- y seconds].

Likewise, any statistical analysis around whether results are indeed statistically different?



374 posts

Ultimate Geek

Trusted

  #542063 7-Nov-2011 15:09
Send private message

michaeln:
JohnButt: Latest broadband comparison statistics

This is also the first time TrueNet has had sufficient volunteers to report on TelstraClear DSL as well as TesltraClear Cable.  While the Cable result continues to top the poll, the DSL result is well down the list at seventh.


Do you have an estimate of the error around these figures? I.e. "ISP A" Time to load x seconds [+/- y seconds].

Likewise, any statistical analysis around whether results are indeed statistically different?


I am reluctant to provide everything.  Let me assure you that the TelstraClear Cable results are very reliable, the worst cable measure has only one DSL probe with better results.  

Note we use the Median probe performances per ISP, ie of the average performance for the month for each probe.  Each probe has little variation, making the average reliable.  That way we drop any outliers (if there are any - mine is) providing more reliable results.  

 
 
 
 


600 posts

Ultimate Geek

Trusted

  #542445 8-Nov-2011 11:14
Send private message

JohnButt:
michaeln:
JohnButt: Latest broadband comparison statistics

This is also the first time TrueNet has had sufficient volunteers to report on TelstraClear DSL as well as TesltraClear Cable.  While the Cable result continues to top the poll, the DSL result is well down the list at seventh.


Do you have an estimate of the error around these figures? I.e. "ISP A" Time to load x seconds [+/- y seconds].

Likewise, any statistical analysis around whether results are indeed statistically different?


I am reluctant to provide everything.  Let me assure you that the TelstraClear Cable results are very reliable, the worst cable measure has only one DSL probe with better results.  

Note we use the Median probe performances per ISP, ie of the average performance for the month for each probe.  Each probe has little variation, making the average reliable.  That way we drop any outliers (if there are any - mine is) providing more reliable results.  


Is it on port 80/8080, or any other port which will be proxied by a transparent (or even non-transparent) proxy?  That would provide a real ability for ISPs to skew results. 




220 posts

Master Geek

Trusted

  #542692 8-Nov-2011 16:41
Send private message

jpollock:

Is it on port 80/8080, or any other port which will be proxied by a transparent (or even non-transparent) proxy?  That would provide a real ability for ISPs to skew results. 


Does that matter? The report is "Time to download a dummy page". I.e., it's explicitly WWW traffic. Anything the ISP does that improves that time is presumably legitimate. After all, that's what most users perceive.




374 posts

Ultimate Geek

Trusted

  #542701 8-Nov-2011 16:54
Send private message

michaeln:
jpollock:

Is it on port 80/8080, or any other port which will be proxied by a transparent (or even non-transparent) proxy?  That would provide a real ability for ISPs to skew results. 


Does that matter? The report is "Time to download a dummy page". I.e., it's explicitly WWW traffic. Anything the ISP does that improves that time is presumably legitimate. After all, that's what most users perceive.



I agree with @michaeln, tis not our problem if the ISP improves the timing for all users to all pages, in fact that is our desire.  We may even publish results to more pages in future to provide greater scope for comparisons.  

We have a logical solution for identifying what we determine is intentional gaming by ISPs and are prepared to publish results when we find any attempt.

2415 posts

Uber Geek

Trusted
Subscriber

  #542705 8-Nov-2011 17:00
Send private message

As far as I know some ISPs do prioritise some widely used benchmarks (speedtest.net)

Hopefully TrueNet uses a decent random sample of webpages for testing as it wouldn't take too much effort for an ISP to sniff the DNS requests from a user/staff member with a TrueNet probe and skew results in their favor.




374 posts

Ultimate Geek

Trusted

  #542708 8-Nov-2011 17:06
Send private message

insane: As far as I know some ISPs do prioritise some widely used benchmarks (speedtest.net)

Hopefully TrueNet uses a decent random sample of webpages for testing as it wouldn't take too much effort for an ISP to sniff the DNS requests from a user/staff member with a TrueNet probe and skew results in their favor.



That is exactly what our solution will detect. 

600 posts

Ultimate Geek

Trusted

  #542727 8-Nov-2011 17:49
Send private message

JohnButt:
michaeln:
jpollock:

Is it on port 80/8080, or any other port which will be proxied by a transparent (or even non-transparent) proxy?  That would provide a real ability for ISPs to skew results. 


Does that matter? The report is "Time to download a dummy page". I.e., it's explicitly WWW traffic. Anything the ISP does that improves that time is presumably legitimate. After all, that's what most users perceive.



I agree with @michaeln, tis not our problem if the ISP improves the timing for all users to all pages, in fact that is our desire.  We may even publish results to more pages in future to provide greater scope for comparisons.  

We have a logical solution for identifying what we determine is intentional gaming by ISPs and are prepared to publish results when we find any attempt.


You see, depending on the traffic/probes/etc, it can easily be detected and optimized for.  For example, by increasing the cache hold times for the pages which are used, to the point of ignoring the validity period.  While this "improves" test results because the page is served from the local cache, it results in the wrong page being returned to the customer.

So, without publication of the test methodology, I can only say it's measuring the performance of the http cache, and easily fudged by ISPs, much the same way that video card manufacturers would detect a performance test and run special code paths which produced visibly buggy results faster.

It is the responsibility of the test designer to design a benchmark that is representative and validate that the results aren't actively subverted by ISPs looking to improve their results.  It's good to hear that you've got a solution to identify intentional gaming.  Care to share the test methodology, or do I have to sign up and snoop the traffic?

edit: Even without funny business, if all of the terminals are hitting the same pages, they will all hit the cache.  The first one will be slow, the rest quick, with an average that doesn't represent random surfing.




3892 posts

Uber Geek


  #542765 8-Nov-2011 19:16
Send private message

jpollock: Care to share the test methodology, or do I have to sign up and snoop the traffic?

edit: Even without funny business, if all of the terminals are hitting the same pages, they will all hit the cache.  The first one will be slow, the rest quick, with an average that doesn't represent random surfing.


How is the information actually used?

I was driving down the road this afternoon and saw a massive bill board claiming that Telstra are the fastest.

Are you suggesting that Telstra are the fastest just because they fudge the figures and Vodafone have been in the back water because they just don't bother?

Is this information being used by consumers to make provider choices?  Or is it mainly being used by industry guys to help build business cases to get better get in to the network or the likes of Telecom to sell more capacity to slower providers?

If it's consumers, then yes, I can see a point for not having the figures fudged... but who would fudge it anyway that we wouldn't eventually notice and start posting on GZ naming names?  Just look at the long thread about the Telstra link to the US last month as an example.

Do most consumers talk to a trusted tech guy first, and do most of us not see though the rubbish?

If it's ISPs just cheating the stats for industry, then who are they kidding?!  ...and what for?







Promote New Zealand - Get yourself a .kiwi.nz domain name!!!

Check out mine - i.am.a.can.do.kiwi.nz - don@i.am.a.can.do.kiwi.nz




374 posts

Ultimate Geek

Trusted

  #542773 8-Nov-2011 19:35
Send private message

jpollock: 
It is the responsibility of the test designer to design a benchmark that is representative and validate that the results aren't actively subverted by ISPs looking to improve their results.  It's good to hear that you've got a solution to identify intentional gaming.  Care to share the test methodology, or do I have to sign up and snoop the traffic?



Yup we benchmark each ISP - snooping will not assist.  

 1 | 2 | 3
View this topic in a long page with up to 500 replies per page Create new topic



Twitter and LinkedIn »



Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:





News »

Huawei introduces the P40 Series of Android-based smartphones
Posted 31-Mar-2020 17:03


Samsung Galaxy Z Flip now available for pre-order in New Zealand
Posted 31-Mar-2020 16:39


New online learning platform for kids stuck at home during COVID-19 lockdown
Posted 26-Mar-2020 21:35


New 5G Nokia smartphone unveiled as portfolio expands
Posted 26-Mar-2020 17:11


D-Link ANZ launches wireless AC1200 4G LTE router
Posted 26-Mar-2020 16:32


Ring introduces two new video doorbells and new pre-roll technology
Posted 17-Mar-2020 16:59


OPPO uncovers flagship Find X2 Pro smartphone
Posted 17-Mar-2020 16:54


D-Link COVR-2202 mesh Wi-Fi system now protected by McAfee
Posted 17-Mar-2020 16:00


Spark Sport opens its platform up to all New Zealanders at no charge
Posted 17-Mar-2020 10:04


Spark launches 5G Starter Fund
Posted 8-Mar-2020 19:19


TRENDnet launches high-performance WiFi Mesh Router System
Posted 5-Mar-2020 08:48


Sony boosts full-frame lens line-up with introduction of FE 20mm F1.8 G large-aperture ultra-wide-angle prime Lens
Posted 5-Mar-2020 08:44


Vector and Spark teamed up on smart metering initiative
Posted 5-Mar-2020 08:42


Schneider Electric launches new PDL Pro Series designed specifically for the commercial building market
Posted 5-Mar-2020 08:39


Kiwi app Pedigree DentaStix Studios uses pet images to counter impact of negative social media Content
Posted 5-Mar-2020 08:32



Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.


Support Geekzone »

Our community of supporters help make Geekzone possible. Click the button below to join them.

Support Geezone on PressPatron



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.