TrueNet Latency Rankings

Forums › New Zealand Broadband › TrueNet Latency Rankings

JohnButt

374 posts

Ultimate Geek
+1 received by user: 47

Trusted

#92667 3-Nov-2011 15:12

TrueNet measures both National Latency and International Latency, this article compares average results for all probes by ISP for September.

The best are Vodafone for national latency and TelstraClear Cable for international latency.

This TrueNet measure of International Latency is to a server in the USA. Distance travelled affects performance, however the differences between ISPs shows other influences because the international distance is roughly the same for all ISPs.

It is interesting to compare this result with our webpage download times here
File 598

Orcon are able to manage fast webpage downloads despite these latency results, note that excluding mobile, they were last in both national and international latency measures.

Technical Note: Some network equipment is known to give ICMP a much lower priority than other traffic, this could make the results for ISPs that use that equipment appear to be slower than they really are. However, we perform a large number of pings, which should reduce the affects of routing equipment giving ICMP traffic a low priority.

Comments?

1 | 2

BDFL - Memuneh
80646 posts

Uber Geek
+1 received by user: 41030

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#540756 3-Nov-2011 15:14

Thanks John for the update (and I haven't replied to your email yet, sorry).

Referral links: Quic Broadband (free setup code: R587125ERQ6VE) | Samsung | AliExpress | Wise | Sharesies

Support Geekzone by subscribing (browse ads-free), or making a one-off or recurring donation through PressPatron.

NonprayingMantis

6434 posts

Uber Geek
+1 received by user: 1528

#540757 3-Nov-2011 15:15

ranking is slightly misleading for this result since everybody did pretty much as well as each other IMHO (apart from mobile of course, but that is more just for aninteresting comparison)

i.e. the range of results is very narrow.

These tests are, of course, a marketers wet dream when they come top of the rankings and can crow about being the best even when the difference is not material and well within any margin of error.

John, what would you say is the margin of error on these tests? +/- 5%?

Ragnor

8279 posts

Uber Geek
+1 received by user: 585

Trusted

#540763 3-Nov-2011 15:23

JohnButt:

Comments?

You could use tcptraceroute, icmp is not really reliable at all.

Wouldn't you want more than one US server in different locations on different hosting providers to average out the effect of a specific routing advantage to a specific hosting provider?

Interleaving on or off (on by default on some ISP's, off by default on others) could have a big effect on your results. Though, I guess the type of people that signed up for Truenet (like me!) are the type of people that would already have asked for interleaving off.

Could be possible to split the results by interleaving off or on if you look at the latency of the first hop for national tracert, of just make it an option in the TrueNet website settings so user's can indicate whether they have interleaving on or off.

Regs

4066 posts

Uber Geek
+1 received by user: 206

Trusted

Snowflake

#540765 3-Nov-2011 15:26

JohnButt:
The best are Vodafone for national latency and TelstraClear Cable for international latency.

in my recent testing experience, Vodafone is the *worst* for latency to servers in singapore, which is where a number of data centers targetting asia pacific are HQ'd - especially Microsoft and Salesforce. TelstraClear were one of the best.

Sales Engineer
Snowflake
www.snowflake.com

about.me/nzregs
Twitter: @nzregs

JohnButt

374 posts

Ultimate Geek
+1 received by user: 47

Trusted

#540766 3-Nov-2011 15:27

Interesting that you see the "range of results is very narrow". My advisors told me that a percentage figure would be too hard to understand, hence the actual. But the difference for National Latency between best & last fixed line is actually 50%.

Test numbers are at least 5,000 and up to 15,000 tests per ISP.

NonprayingMantis

6434 posts

Uber Geek
+1 received by user: 1528

#540770 3-Nov-2011 15:40

JohnButt: Interesting that you see the "range of results is very narrow". My advisors told me that a percentage figure would be too hard to understand, hence the actual. But the difference for National Latency between best & last fixed line is actually 50%.

Test numbers are at least 5,000 and up to 15,000 tests per ISP.

Sorry, I was referring to the international ones there.

The national ones might be up to 50% difference, but even the worst performer was still fast enough to be not be noticeable for any normal usage scenarios.
(I’m thinking something like an online gamer. Latency is important, but the difference between 30ms and 60ms is not noticeable unless you are some kind of Pro, and even then it’s debatable. Now if one of the ISPs was showing 100ms+ for national that would be much more of interest for people who cared that much about latency.

Geekzone support

Support Geekzone with one-off or recurring donations Donate via PressPatron.

Regs

4066 posts

Uber Geek
+1 received by user: 206

Trusted

Snowflake

#540777 3-Nov-2011 15:45

NonprayingMantis:
(I’m thinking something like an online gamer. Latency is important, but the difference between 30ms and 60ms is not noticeable unless you are some kind of Pro, and even then it’s debatable.

the difference between 100ms and 300ms to load a complex webpage can be huge. especially if there are lots of components - images, javascripts, iframes etc.

Sales Engineer
Snowflake
www.snowflake.com

about.me/nzregs
Twitter: @nzregs

NonprayingMantis

6434 posts

Uber Geek
+1 received by user: 1528

#540779 3-Nov-2011 15:48

Regs:
NonprayingMantis:
(I’m thinking something like an online gamer. Latency is important, but the difference between 30ms and 60ms is not noticeable unless you are some kind of Pro, and even then it’s debatable.

the difference between 100ms and 300ms to load a complex webpage can be huge. especially if there are lots of components - images, javascripts, iframes etc.

right, but according to the chart above the national latnecy variance from best to worst was only about 15ms (just going by eye as the bars do not have valueon them), and the international latency variance from best to worst was only about 40ms. I would contend that neither of those are material enough differences for the vast majority of peopl, and a far cry from the 200ms difference you mention in your post.

JohnButt

374 posts

Ultimate Geek
+1 received by user: 47

Trusted

#540792 3-Nov-2011 16:11

NonprayingMantis:
Regs:
NonprayingMantis:
(I’m thinking something like an online gamer. Latency is important, but the difference between 30ms and 60ms is not noticeable unless you are some kind of Pro, and even then it’s debatable.

the difference between 100ms and 300ms to load a complex webpage can be huge. especially if there are lots of components - images, javascripts, iframes etc.

right, but according to the chart above the national latnecy variance from best to worst was only about 15ms (just going by eye as the bars do not have valueon them), and the international latency variance from best to worst was only about 40ms. I would contend that neither of those are material enough differences for the vast majority of peopl, and a far cry from the 200ms difference you mention in your post.

Complex webpages can chew up a lot of time and while a 40ms difference may not be big, lots of 40ms becomes significant.

Variances are 31 to 46ms and 187 to 229ms, so yes your estimates are almost exact. The differences are more real though when you drill down to individual probes. I was not actually looking for differences in latency by ISP, it is just easy to produce. I would prefer a more useful comparison - any ideas? It probably needs to be something an ISP or an individual can influence to be useful.

Ragnor:

Wouldn't you want more than one US server in different locations on different hosting providers to average out the effect of a specific routing advantage to a specific hosting provider?

Yes I would love to have more international sites, although the routing advantage/disadvantage should be similar for all. We will have them ASAP.

michaeln

238 posts

Master Geek
+1 received by user: 44

Trusted

#540797 3-Nov-2011 16:15

JohnButt: TrueNet measures both National Latency and International Latency, this article compares average results for all probes by ISP for September.

Technical Note: Some network equipment is known to give ICMP a much lower priority than other traffic, this could make the results for ISPs that use that equipment appear to be slower than they really are. However, we perform a large number of pings, which should reduce the affects of routing equipment giving ICMP traffic a low priority.

Comments?

It is usually just the end point equipment that deprioritises ICMP Echo. I don't know of any switch that delays pings in transit. Thus, as a comparative measure between local ISPs, this should be a fair test.

Regs

4066 posts

Uber Geek
+1 received by user: 206

Trusted

Snowflake

#540807 3-Nov-2011 17:00

NonprayingMantis:
Regs:
NonprayingMantis:
(I’m thinking something like an online gamer. Latency is important, but the difference between 30ms and 60ms is not noticeable unless you are some kind of Pro, and even then it’s debatable.

the difference between 100ms and 300ms to load a complex webpage can be huge. especially if there are lots of components - images, javascripts, iframes etc.

right, but according to the chart above the national latnecy variance from best to worst was only about 15ms (just going by eye as the bars do not have valueon them), and the international latency variance from best to worst was only about 40ms. I would contend that neither of those are material enough differences for the vast majority of peopl, and a far cry from the 200ms difference you mention in your post.

unfortunately a latency test to a US server is NOT representative of latency accross the world.

testing over several ISPs, i have seen the traffic to a single IP address in Singapore travel over different routes ranging from 130ms to > 500ms. Thats a whopping 370ms latency difference between best and worst. I've seen better latency from 3g connections on telecom and 2degrees than on some wired connections!

examples of some of the routes i've seen. the actual cities/countries might not 100% accurate... but the latency is:

Auckland-Sydney-Perth-Singapore (130ms)
Auckland-Sydney-Guam-Tokyo-Singapore (185ms)
Auckland-Los Angeles-Narito (Japan)-???-Singapore (330ms)
Auckland-Sydney-Los Angeles-Tokyo-Singapore (380ms)

Sales Engineer
Snowflake
www.snowflake.com

about.me/nzregs
Twitter: @nzregs

Quic Broadband

Move to New Zealand's best fibre broadband service (affiliate link). Free setup code: R587125ERQ6VE. Note that to use Quic Broadband you must be comfortable with configuring your own router.

Screeb

698 posts

Ultimate Geek
+1 received by user: 52

#540863 3-Nov-2011 19:42

So I guess the national test didn't include any Wellington servers, because TelstraClear would probably come last if it did.

Ragnor

8279 posts

Uber Geek
+1 received by user: 585

Trusted

#540957 3-Nov-2011 22:58

michaeln:

It is usually just the end point equipment that deprioritises ICMP Echo. I don't know of any switch that delays pings in transit. Thus, as a comparative measure between local ISPs, this should be a fair test.

Unfortunately practically every router in the chain between you and the destination and back (routes are not always asymmetric) will have rate limits on ICMP responses.

If you happen to do a ping or tracert using the ICMP protocol at the same time as lots of other people are you will sometimes get delayed or even dropped packets, when UDP and TCP flows/latency are perfectly fine.

LennonNZ

2459 posts

Uber Geek
+1 received by user: 411

ID Verified

Trusted

#540968 3-Nov-2011 23:20

For National Tests what are you actually testing it against?

For example if you are testing it against a machine hosted at ICONZ for example from an ICONZ connection you would expect (normally) the ping to be a tiny bit lower than an IHUG connection thus making ICONZ artificially lower ping rate.

Also you say "Some network equipment is known to give ICMP a much lower priority than other traffic, this could make the results for ISPs that use that equipment appear to be slower than they really are. However, we perform a large number of pings, which should reduce the affects of routing equipment giving ICMP traffic a low priority."

Can you comment on this? I know that some equipment has ICMP Rate limits TO the routing engine of the routers but not thru it but this won't affect ICMP to whatever you use to test the ping (unless you are pinging the customers default gateway)

Are all the tests are ADSL Connections or are they an average of all connections of the ISP. If you are using the average speed from all connections on the ISP and say Vodafone has more higher speed connections with much less ping times the average would be skewed.

As Telecom/2Degrees/Vodafone networks are completely different networks why are you classing them all as the same (as the ping rates are quite different)

How are the tests done on the mobile networks?

Morph

190 posts

Master Geek
Inactive user

#540980 4-Nov-2011 00:04

Screeb: So I guess the national test didn't include any Wellington servers, because TelstraClear would probably come last if it did.

TelstraClear have fiber lines up and down the country and in so connect with many companies in Wellington .. Citylink is 1 they do not peer with , There are many they do.

1 | 2