NZ ISPs download com.au sites faster than Auzzie ISP

Forums › New Zealand Broadband › NZ ISPs download com.au sites faster than Auzzie ISP

JohnButt

374 posts

Ultimate Geek
+1 received by user: 47

Trusted

#205695 23-Nov-2016 15:04

There are three interesting special features to this month's report:

Australian ISP TPG downloads Auzzie websites slower than NZ ISPs - for the same sites
ADSL spread continues with a 2:1 ratio, with Vodafone at the bottom and Slingshot at the top
A special chart showing that with popular video-rich USA websites all technologies are equal

PS, we need more Rural volunteers from any ISP - volunteer FAQs here

We are also recruiting for Spark, Vodafone, Orcon and 2Degrees in Urban sectors on the 3 major technologies.

https://truenet.nz/story/2016/11/october-2016-urban-broadband-report

ADSL average time to download popular Australian websites

ADSL average time to download popular NZ websites

ghettomaster

387 posts

Ultimate Geek
+1 received by user: 130

#1676181 23-Nov-2016 15:41

Are these sites hosted in AU or are they on a CDN?

JohnButt

374 posts

Ultimate Geek
+1 received by user: 47

Trusted

#1676191 23-Nov-2016 15:58

ghettomaster:

Are these sites hosted in AU or are they on a CDN?

The NZ sites are: NZTA, PriceMe, The Warehouse, Radio NZ, Real Estate, Scoop, Kiwibank

The AU sites are: ninemsn - Nine News, SMH, AU Bureau of Meteorology and on a CDN a site which we do not use but please comment - Google AU

ghettomaster

387 posts

Ultimate Geek
+1 received by user: 130

#1676273 23-Nov-2016 17:32

All the AU sites are Akamai based which means the NZ testers are not connecting across the ditch. It's not really a surprise that in-country Akamai connections perform better in New Zealand than in Australia considering how the NBN rollout is going, but I wonder how things would go if you analyzed how AU hosted sites compare between the countries. Didn't you guys have your own servers in AU and the US at one stage? Were these tested?

It's great to see testers in other countries coming on stream as this should make for more interesting reading in the future. Keep up the good work guys!

BarTender

3629 posts

Uber Geek
+1 received by user: 2572

ID Verified

Trusted

Lifetime subscriber

#1676282 23-Nov-2016 17:55

But really. A 1mb file or even a 5mb file isn't enough to accurately test the throughput. Perhaps on sub 10mbit ADSL. But certainly not anything over a 30mbit connection.

The changes between last month and this month in DSL speeds is very noticeable and consistent across all ISPs in NZ.

Perhaps it's your testing methodology that has changed and not the ISPs.

JohnButt

374 posts

Ultimate Geek
+1 received by user: 47

Trusted

#1676546 24-Nov-2016 09:16

ghettomaster:

All the AU sites are Akamai based which means the NZ testers are not connecting across the ditch. It's not really a surprise that in-country Akamai connections perform better in New Zealand than in Australia considering how the NBN rollout is going, but I wonder how things would go if you analyzed how AU hosted sites compare between the countries. Didn't you guys have your own servers in AU and the US at one stage? Were these tested?

It's great to see testers in other countries coming on stream as this should make for more interesting reading in the future. Keep up the good work guys!

Actually this is full equivalence, although potentially a NZ handicap - all panelists are on ADSL for this comparison, no NBN (we do have a few but not in this story) The potential handicap comes from the fact that all AU probes are in major cities, while the NZ probes are all Urban ADSL - and we include small towns as Urban.

Akamai or Cloudfare or any other CDN is not an issue, at the end of the day these are popular sites from the Alexa list, ie they are frequently used in each country - likely from the panelists own connections.

Our own server pages were being cached too much to be useful for testing, so we now only use popular live websites with a lot of activity. We also reduced the number of sites in Australia after our survey data said most people were less interested than our US websites, which we increased. I suspect we went too far so I plan to increase the AU sites again now we have so many panelists on unlimited data.

We expect to be able to increase our presence in Australia once we complete our replacement cycle with the GigE MikroTik's so we are keen to enlist new AU volunteers. Thanks for the support.

BarTender

3629 posts

Uber Geek
+1 received by user: 2572

ID Verified

Trusted

Lifetime subscriber

#1676553 24-Nov-2016 09:26

I would really love an explaination between these two graphs:

October Chart 9: VDSL ToD performance from Sydney, Australia (1MB & 2MB Files used)

September Chart 9

Since are you seriously trying to say over the course of 1 month all the ISPs had a sigificant change in their NZ->SYD Speed. These two graphs are a complete nonsense to me.

HP

Shop now for HP laptops and other devices (affiliate link).

yitz

2238 posts

Uber Geek
+1 received by user: 594

#1676555 24-Nov-2016 09:30

What is 'NZ Best Speed' ?

JohnButt

374 posts

Ultimate Geek
+1 received by user: 47

Trusted

#1676571 24-Nov-2016 09:39

BarTender: But really. A 1mb file or even a 5mb file isn't enough to accurately test the throughput. Perhaps on sub 10mbit ADSL. But certainly not anything over a 30mbit connection.

The changes between last month and this month in DSL speeds is very noticeable and consistent across all ISPs in NZ.

Perhaps it's your testing methodology that has changed and not the ISPs.

We have had this discussion before, which I reported in our August report, maybe you could read it - https://truenet.nz/story/2016/09/august-2016-urban-broadband-report#Intl

Rampup does reach over 100Mb/s within 1MB on many probes (although we measure quartiles so miss that point unless we use a 2MB file) ISP averages are quite different from each other as shown in the article. Rampup is clearly not totally predictable and may be influenced by many factors.

This forum topic is all about websites, not speed, which comes from downloads between the websites and the probe directly. Speed is only part of the influence on website downloads. Your comment about the changes in speed between months is somewhat misplaced, we never, ever publish ISPs DSL speeds, simply because the actual speed of DSL is related to distance and we could never get a fair average due to the differences in average distance by ISPs due to marketing issues in the past.

The difference between website downloads (in case you confuse speed with time to download) will be due to the website changes - we use live websites as we very often quote. Since we use the same websites for all probes, and changes impact all probes the moment the change occurs, this is not a change in testing methodology (unless there is inappropriate caching by any ISP)

JohnButt

374 posts

Ultimate Geek
+1 received by user: 47

Trusted

#1676575 24-Nov-2016 09:45

yitz: What is 'NZ Best Speed' ?

from the box on every monthly report just above the chart:

Copper connections (ADSL & VDSL) have a speed that is dependent on the distance between the home modem, and the exchange equipment which means that ISPs do not have any influence on the peak speed of each connection. To overcome this limitation, the Australia, and USA speeds are referenced to the average NZ download speed of the respective ISPs. The speed reported is from the 8pm-10pm evening period.

The NZ Best Speed, is a recognition that DSL has a maximum speed related to sync speed and the best speed measured on any probe in 720+ tests per month is likely to be as close as possible to the line's sync speed. This chart is intended to compare the AU speed with the fastest the line will allow - a reasonable measure.

ghettomaster

387 posts

Ultimate Geek
+1 received by user: 130

#1676581 24-Nov-2016 09:58

JohnButt:

Akamai or Cloudfare or any other CDN is not an issue, at the end of the day these are popular sites from the Alexa list, ie they are frequently used in each country - likely from the panelists own connections.

I'm with you now - you're more worried about real-world performance for typical use cases than about delving into hosting details etc. This makes a lot of sense.

BarTender

3629 posts

Uber Geek
+1 received by user: 2572

ID Verified

Trusted

Lifetime subscriber

#1676584 24-Nov-2016 10:02

JohnButt:

BarTender: But really. A 1mb file or even a 5mb file isn't enough to accurately test the throughput. Perhaps on sub 10mbit ADSL. But certainly not anything over a 30mbit connection.

The changes between last month and this month in DSL speeds is very noticeable and consistent across all ISPs in NZ.

Perhaps it's your testing methodology that has changed and not the ISPs.

We have had this discussion before, which I reported in our August report, maybe you could read it - https://truenet.nz/story/2016/09/august-2016-urban-broadband-report#Intl

Rampup does reach over 100Mb/s within 1MB on many probes (although we measure quartiles so miss that point unless we use a 2MB file) ISP averages are quite different from each other as shown in the article. Rampup is clearly not totally predictable and may be influenced by many factors.

This forum topic is all about websites, not speed, which comes from downloads between the websites and the probe directly. Speed is only part of the influence on website downloads. Your comment about the changes in speed between months is somewhat misplaced, we never, ever publish ISPs DSL speeds, simply because the actual speed of DSL is related to distance and we could never get a fair average due to the differences in average distance by ISPs due to marketing issues in the past.

The difference between website downloads (in case you confuse speed with time to download) will be due to the website changes - we use live websites as we very often quote. Since we use the same websites for all probes, and changes impact all probes the moment the change occurs, this is not a change in testing methodology (unless there is inappropriate caching by any ISP)

I still don't get it. You first say that you can reach 100Mb/s with a 1MB file then in the same sentice say that Rampup is clearly not totally predictable and may be influenced by many factors (such as latency).

But you're reporting on speeds with the graphs shown above and month to month have wildly different results for ALL ISPs. Some in a positive and others in a negative way month to month yet provide no explaination on why those numbers have changed so much. And it's not just one ISP it's ALL of them with the exception of Orcon.

People are making decisions on which ISP to use based on the Truenet results and the way I read the above VDSL graphs one month 2Degrees was 50% of the advertised speed to AU, and the next it was 80%. I just can't believe the numbers from the above graphs when 3 ISPs have a greater than 20% increase in speed, Voyager drops by over 10% and Orcon stays solid.

You've also completely dismissed any relevance of CDNs and how they are served up locally depending on which ISP you are with and if that ISP has a CDN node in their network or not. That has a siginificant impact on the time it takes to serve up a full page. Yet there is no mention of it at all.

If you know the numbers aren't good due to "many factors" you shouldn't be presenting them.

Lego

Shop now for Lego sets and other gifts (affiliate link).

JohnButt

374 posts

Ultimate Geek
+1 received by user: 47

Trusted

#1676616 24-Nov-2016 10:50

BarTender:

JohnButt:

BarTender: But really. A 1mb file or even a 5mb file isn't enough to accurately test the throughput. Perhaps on sub 10mbit ADSL. But certainly not anything over a 30mbit connection.

The changes between last month and this month in DSL speeds is very noticeable and consistent across all ISPs in NZ.

Perhaps it's your testing methodology that has changed and not the ISPs.

We have had this discussion before, which I reported in our August report, maybe you could read it - https://truenet.nz/story/2016/09/august-2016-urban-broadband-report#Intl

Rampup does reach over 100Mb/s within 1MB on many probes (although we measure quartiles so miss that point unless we use a 2MB file) ISP averages are quite different from each other as shown in the article. Rampup is clearly not totally predictable and may be influenced by many factors.

This forum topic is all about websites, not speed, which comes from downloads between the websites and the probe directly. Speed is only part of the influence on website downloads. Your comment about the changes in speed between months is somewhat misplaced, we never, ever publish ISPs DSL speeds, simply because the actual speed of DSL is related to distance and we could never get a fair average due to the differences in average distance by ISPs due to marketing issues in the past.

The difference between website downloads (in case you confuse speed with time to download) will be due to the website changes - we use live websites as we very often quote. Since we use the same websites for all probes, and changes impact all probes the moment the change occurs, this is not a change in testing methodology (unless there is inappropriate caching by any ISP)

I still don't get it. You first say that you can reach 100Mb/s with a 1MB file then in the same sentice say that Rampup is clearly not totally predictable and may be influenced by many factors (such as latency).

But you're reporting on speeds with the graphs shown above and month to month have wildly different results for ALL ISPs. Some in a positive and others in a negative way month to month yet provide no explaination on why those numbers have changed so much. And it's not just one ISP it's ALL of them with the exception of Orcon.

People are making decisions on which ISP to use based on the Truenet results and the way I read the above VDSL graphs one month 2Degrees was 50% of the advertised speed to AU, and the next it was 80%. I just can't believe the numbers from the above graphs when 3 ISPs have a greater than 20% increase in speed, Voyager drops by over 10% and Orcon stays solid.

You've also completely dismissed any relevance of CDNs and how they are served up locally depending on which ISP you are with and if that ISP has a CDN node in their network or not. That has a siginificant impact on the time it takes to serve up a full page. Yet there is no mention of it at all.

If you know the numbers aren't good due to "many factors" you shouldn't be presenting them.

BarTender, It would be considerate to know who you are - you are hiding behind anonymous.

We know the numbers are good, we are simply reporting a lot of test results from a lot of locations, why should we care where the files are? why should we care what is the cause of the changes month to month? We care that there are changes and report them. The public are not expected to know if the files are located in NZ, AU or a CDN, so what purpose would reporting on these separations possibly serve?

Consumers do know if a website is an Australian or a NZ website however so we report on those distinctions

Rampup is an interesting topic, and I am interested in finding how it impacts users, in case we can find a way it impacts them that is directly relevant. The details behind performance is interesting for experts, but for others it is confusing when they simply want to know which ISP suits them best. Since the best ISP is a moving feast depending on a users practices, we do have a problem trying to provide a lot of information that users can absorb and use.

We would love to do a more comprehensive study of rampup and "Saw-tooth" like speed changes we see on many downloads, I suspect there is a story there but we do not have the resources.