July 2012 Broadband Report - lots of changes

Forums › New Zealand Broadband › July 2012 Broadband Report - lots of changes

JohnButt

374 posts

Ultimate Geek

Trusted

#108485 30-Aug-2012 09:16

Orcon Spurred To Silver

Orcon emerged second only to Xnet for maintaining file download speeds throughout the day on the TrueNet network of probes

A major improvement in Website download performance by Orcon, Slingshot and Snap on 11th July resulted in changes to rankings this month.

TrueNet provides more details to background the issue in a comprehensive explanation to encourage debate.

Speed by Time of Day (File Download)

Orcon improved peak evening performance to become second to Xnet.

TelstraClear are still showing the worst performance during the evening.

Browsing Performance (Webpage Download)

There were some significant changes to webpage download speeds this month. Slingshot, Orcon and Snap improved while Telecom, Maxnet and Actrix all saw a drop in performance.
https://www.truenet.co.nz/articles/july-2012-broadband-report

quakeguy

111 posts

Master Geek

Trusted

#679147 30-Aug-2012 10:11

Hi John,

We appreciate the in-depth analysis you're providing here and these results are among some of the most comprehensive so far.

However I cannot help but notice the results are being prejudiced because you're only testing webpage download performance to one location: http://www.trademe.co.nz/Community/cctn345156/index.html

Regarding your comment on the 11th of July - this could just be TradeMe making changes!

Can we ask that you test in a more agnostic fashion? Can you have the probes rotate through different websites? Perhaps sites which are of mixed geographical location? This would produce a more meaningful result.

“I do not think there is any thrill that can go through the human heart like that felt by the inventor as he sees some creation of the brain unfolding to success... Such emotions make a man forget food, sleep, friends, love, everything.” - Nikola Tesla

Disclaimer: Views expressed in my posts do not necessarily reflect those views of my employer.

hamish225

1418 posts

Uber Geek

ID Verified

#679180 30-Aug-2012 10:57

maybe you could have the probes test several websites on different servers, one in chch, wgtn and auckland one after another, then send the average of the results back to you.

*Insert big spe*dtest result here*

JohnButt

374 posts

Ultimate Geek

Trusted

#679181 30-Aug-2012 10:57

We are testing multiple locations and I checked the changes against those. There is a small problem with caps in NZ that holds us back from more regular testing of more sites. Note that only some ISPs had a change, so it is unlikely that the webpage server was the cause.

We published a banking survey recently, based on other webpages :-)

quakeguy

111 posts

Master Geek

Trusted

#679218 30-Aug-2012 12:26

(John's asked me privately to not publish his testing URL; but I want this reply to be public.)

John,

Regarding your testing URL - why can't I publish this -- in fact, why can't you publish this?

Sure, you might have concerns about someone replicating your work; but realistically, you already have your appliances working in a large number of households and across a number of providers. It's not going to happen.

If providers want to see/analyse where the probes are on their networks, you are going to have a hard time stopping them. I expect a number of providers have done this already.

I'm not discounting the fantastic work on your part, it must have taken to get to this point.

But let's face it - there's no reason not to be transparent about your monitoring practices, except that it would stop you drawing conclusions about 'New Zealand's broadband performance' when you're only monitoring select sites.

New Zealanders aren't browsing your testing URL!

They're looking at sites like facebook.com, google.co.nz, www.youtube.co.nz, stuff.co.nz, trademe.co.nz (the main page, not a hidden URL), nzherald.co.nz, wikipedia.org, ... the list goes on.

In fact, the list I've used is openly available at http://www.alexa.com/topsites/countries/NZ . There are many more; if I was being serious, I'd base testing on a hash of several of these lists.

When you post test results claiming to be representing 'New Zealand's broadband performance', you really aren't; you're doing a synthetic test, loading a page you won't even give out.

Disclaimer: Views expressed in my posts do not necessarily reflect those views of my employer.

jnimmo

1097 posts

Uber Geek

#679244 30-Aug-2012 13:24

Xnet are probably in the lead because all the torrentors are moving over other ISPs!

Yeah I think some international testing is really important in all of this, and also the testing of pages/files that could not be cached by ISPs

JohnButt

374 posts

Ultimate Geek

Trusted

#679256 30-Aug-2012 13:43

quakeguy - Tim ??

I am not sure you have read my message. It is not that we have a need to keep the site hidden, but to avoid large numbers of downloads from the public creating traffic issues that impact the testing. Our probes test hourly based on a random sequence to avoid this impact. ie we do not want it to be a common testing location.

JohnButt

374 posts

Ultimate Geek

Trusted

#679257 30-Aug-2012 13:45

jnimmo: Xnet are probably in the lead because all the torrentors are moving over other ISPs!

Yeah I think some international testing is really important in all of this, and also the testing of pages/files that could not be cached by ISPs

The issue with testing a live site is that you have to be careful that your tests are not being compromised by many other issues that may alter the results, eg site content changes, site specific traffic volumes etc. Since you have no control over these factors, testing to a live site makes little sense when comparing providers.

The fixed site, on a server with few traffic issues makes the best testing regime.

Yes it would be better to have multiple sites, but they would also need to be non-live. We do have tests from an international site - cache forced off, but we are still checking validity and that takes resource