Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.


374 posts

Ultimate Geek
+1 received by user: 47

Trusted

Topic # 92667 3-Nov-2011 15:12
Send private message

TrueNet measures both National Latency and International Latency, this article compares average results for all probes by ISP for September.

The best are Vodafone for national latency and TelstraClear Cable for international latency.

This TrueNet measure of International Latency is to a server in the USA. Distance travelled affects performance, however the differences between ISPs shows other influences because the international distance is roughly the same for all ISPs.

It is interesting to compare this result with our webpage download times here 
File 598

Orcon are able to manage fast webpage downloads despite these latency results, note that excluding mobile, they were last in both national and international latency measures.

Technical Note: Some network equipment is known to give ICMP a much lower priority than other traffic, this could make the results for ISPs that use that equipment appear to be slower than they really are.  However, we perform a large number of pings, which should reduce the affects of routing equipment giving ICMP traffic a low priority.

Comments?

 

View this topic in a long page with up to 500 replies per page Create new topic
 1 | 2
BDFL - Memuneh
61489 posts

Uber Geek
+1 received by user: 12209

Administrator
Trusted
Geekzone
Lifetime subscriber

6434 posts

Uber Geek
+1 received by user: 1571


  Reply # 540757 3-Nov-2011 15:15
Send private message

ranking is slightly misleading for this result since everybody did pretty much as well as each other IMHO (apart from mobile of course, but that is more just for aninteresting comparison)

i.e. the range of results is very narrow.

These tests are, of course, a marketers wet dream when they come top of the rankings and can crow about being the best even when the difference is not material and well within any margin of error.


John, what would you say is the margin of error on these tests?  +/- 5%?

8027 posts

Uber Geek
+1 received by user: 387

Trusted
Subscriber

  Reply # 540763 3-Nov-2011 15:23
Send private message

JohnButt:

Comments?
 


You could use tcptraceroute, icmp is not really reliable at all.

Wouldn't you want more than one US server in different locations on different hosting providers to average out the effect of a specific routing advantage to a specific hosting provider?

Interleaving on or off (on by default on some ISP's, off by default on others) could have a big effect on your results. Though, I guess the type of people that signed up for Truenet (like me!) are the type of people that would already have asked for interleaving off.

Could be possible to split the results by interleaving off or on if you look at the latency of the first hop for national tracert, of just make it an option in the TrueNet website settings so user's can indicate whether they have interleaving on or off.

Infrastructure Geek
4057 posts

Uber Geek
+1 received by user: 195

Trusted
Microsoft NZ
Subscriber

  Reply # 540765 3-Nov-2011 15:26
Send private message

JohnButt:
The best are Vodafone for national latency and TelstraClear Cable for international latency.



in my recent testing experience, Vodafone is the *worst* for latency to servers in singapore, which is where a number of data centers targetting asia pacific are HQ'd - especially Microsoft and Salesforce.  TelstraClear were one of the best.




Technical Evangelist
Microsoft NZ
about.me/nzregs
Twitter: @nzregs




374 posts

Ultimate Geek
+1 received by user: 47

Trusted

  Reply # 540766 3-Nov-2011 15:27
Send private message

Interesting that you see the "range of results is very narrow".  My advisors told me that a percentage figure would be too hard to understand, hence the actual.  But the difference for National Latency between best & last fixed line is actually 50%.

Test numbers are at least 5,000 and up to 15,000 tests per ISP.

6434 posts

Uber Geek
+1 received by user: 1571


  Reply # 540770 3-Nov-2011 15:40
Send private message

JohnButt: Interesting that you see the "range of results is very narrow".  My advisors told me that a percentage figure would be too hard to understand, hence the actual.  But the difference for National Latency between best & last fixed line is actually 50%.

Test numbers are at least 5,000 and up to 15,000 tests per ISP.


Sorry, I was referring to the international ones there.

The national ones might be up to 50% difference, but even the worst performer was still fast enough to be not be noticeable for any normal usage scenarios.  
(I’m thinking something like an online gamer. Latency is important, but the difference between 30ms and 60ms is not noticeable unless you are some kind of Pro, and even then it’s debatable. Now if one of the ISPs was showing 100ms+ for national that would be much more of interest for people who cared that much about latency.

Infrastructure Geek
4057 posts

Uber Geek
+1 received by user: 195

Trusted
Microsoft NZ
Subscriber

  Reply # 540777 3-Nov-2011 15:45
Send private message

NonprayingMantis:
(I’m thinking something like an online gamer. Latency is important, but the difference between 30ms and 60ms is not noticeable unless you are some kind of Pro, and even then it’s debatable.


the difference between 100ms and 300ms to load a complex webpage can be huge.  especially if there are lots of components - images, javascripts, iframes etc.




Technical Evangelist
Microsoft NZ
about.me/nzregs
Twitter: @nzregs


6434 posts

Uber Geek
+1 received by user: 1571


  Reply # 540779 3-Nov-2011 15:48
Send private message

Regs:
NonprayingMantis:
(I’m thinking something like an online gamer. Latency is important, but the difference between 30ms and 60ms is not noticeable unless you are some kind of Pro, and even then it’s debatable.


the difference between 100ms and 300ms to load a complex webpage can be huge.  especially if there are lots of components - images, javascripts, iframes etc.


right, but according to the chart above the national latnecy variance from best to worst was only about 15ms (just going by eye as the bars do not have valueon them), and the international latency variance from best to worst was only about 40ms.  I would contend that neither of those are material enough differences for the vast majority of peopl, and a far cry from the 200ms difference you mention in your post.



374 posts

Ultimate Geek
+1 received by user: 47

Trusted

  Reply # 540792 3-Nov-2011 16:11
Send private message

NonprayingMantis:
Regs:
NonprayingMantis:
(I’m thinking something like an online gamer. Latency is important, but the difference between 30ms and 60ms is not noticeable unless you are some kind of Pro, and even then it’s debatable.


the difference between 100ms and 300ms to load a complex webpage can be huge.  especially if there are lots of components - images, javascripts, iframes etc.


right, but according to the chart above the national latnecy variance from best to worst was only about 15ms (just going by eye as the bars do not have valueon them), and the international latency variance from best to worst was only about 40ms.  I would contend that neither of those are material enough differences for the vast majority of peopl, and a far cry from the 200ms difference you mention in your post.


Complex webpages can chew up a lot of time and while a 40ms difference may not be big, lots of 40ms becomes significant.

Variances are 31 to 46ms and  187 to 229ms, so yes your estimates are almost exact.  The differences are more real though when you drill down to individual probes.  I was not actually looking for differences in latency by ISP, it is just easy to produce.  I would prefer a more useful comparison - any ideas?  It probably needs to be something an ISP or an individual can influence to be useful.

Ragnor: 

Wouldn't you want more than one US server in different locations on different hosting providers to average out the effect of a specific routing advantage to a specific hosting provider? 



Yes I would love to have more international sites, although the routing advantage/disadvantage should be similar for all. We will have them ASAP. 

211 posts

Master Geek
+1 received by user: 21

Trusted

  Reply # 540797 3-Nov-2011 16:15
Send private message

JohnButt: TrueNet measures both National Latency and International Latency, this article compares average results for all probes by ISP for September.

Technical Note: Some network equipment is known to give ICMP a much lower priority than other traffic, this could make the results for ISPs that use that equipment appear to be slower than they really are.  However, we perform a large number of pings, which should reduce the affects of routing equipment giving ICMP traffic a low priority.

Comments?

 


It is usually just the end point equipment that deprioritises ICMP Echo. I don't know of any switch that delays pings in transit. Thus, as a comparative measure between local ISPs, this should be a fair test.

Infrastructure Geek
4057 posts

Uber Geek
+1 received by user: 195

Trusted
Microsoft NZ
Subscriber

  Reply # 540807 3-Nov-2011 17:00
Send private message

NonprayingMantis:
Regs:
NonprayingMantis:
(I’m thinking something like an online gamer. Latency is important, but the difference between 30ms and 60ms is not noticeable unless you are some kind of Pro, and even then it’s debatable.


the difference between 100ms and 300ms to load a complex webpage can be huge.  especially if there are lots of components - images, javascripts, iframes etc.


right, but according to the chart above the national latnecy variance from best to worst was only about 15ms (just going by eye as the bars do not have valueon them), and the international latency variance from best to worst was only about 40ms.  I would contend that neither of those are material enough differences for the vast majority of peopl, and a far cry from the 200ms difference you mention in your post.


unfortunately a latency test to a US server is NOT representative of latency accross the world.  

testing over several ISPs, i have seen the traffic to a single IP address in Singapore travel over different routes ranging from 130ms  to > 500ms.  Thats a whopping 370ms latency difference between best and worst.  I've seen better latency from 3g connections on telecom and 2degrees than on some wired connections!

examples of some of the routes i've seen.  the actual cities/countries might not 100% accurate... but the latency is:

Auckland-Sydney-Perth-Singapore  (130ms)
Auckland-Sydney-Guam-Tokyo-Singapore (185ms)
Auckland-Los Angeles-Narito (Japan)-???-Singapore (330ms)
Auckland-Sydney-Los Angeles-Tokyo-Singapore (380ms)




Technical Evangelist
Microsoft NZ
about.me/nzregs
Twitter: @nzregs


671 posts

Ultimate Geek
+1 received by user: 10


  Reply # 540863 3-Nov-2011 19:42
Send private message

So I guess the national test didn't include any Wellington servers, because TelstraClear would probably come last if it did.

8027 posts

Uber Geek
+1 received by user: 387

Trusted
Subscriber

  Reply # 540957 3-Nov-2011 22:58
Send private message

michaeln: 

It is usually just the end point equipment that deprioritises ICMP Echo. I don't know of any switch that delays pings in transit. Thus, as a comparative measure between local ISPs, this should be a fair test.



Unfortunately practically every router in the chain between you and the destination and back (routes are not always asymmetric) will have rate limits on ICMP responses.

If you happen to do a ping or tracert using the ICMP protocol at the same time as lots of other people are you will sometimes get delayed or even dropped packets, when UDP and TCP flows/latency are perfectly fine.


2356 posts

Uber Geek
+1 received by user: 374

Trusted

  Reply # 540968 3-Nov-2011 23:20
Send private message

For National Tests what are you actually testing it against?

For example if you are testing it against a machine hosted at ICONZ for example from an ICONZ connection you would expect (normally) the ping to be a tiny bit lower than an IHUG connection thus making ICONZ artificially lower ping rate.

Also you say "Some network equipment is known to give ICMP a much lower priority than other traffic, this could make the results for ISPs that use that equipment appear to be slower than they really are. However, we perform a large number of pings, which should reduce the affects of routing equipment giving ICMP traffic a low priority."

Can you comment on this? I know that some equipment has ICMP Rate limits TO the routing engine of the routers but not thru it but this won't affect ICMP to whatever you use to test the ping (unless you are pinging the customers default gateway)

Are all the tests are ADSL Connections or are they an average of all connections of the ISP. If you are using the average speed from all connections on the ISP and say Vodafone has more higher speed connections with much less ping times the average would be skewed.

As Telecom/2Degrees/Vodafone networks are completely different networks why are you classing them all as the same (as the ping rates are quite different)

How are the tests done on the mobile networks?




190 posts

Master Geek
Inactive user


  Reply # 540980 4-Nov-2011 00:04
Send private message

Screeb: So I guess the national test didn't include any Wellington servers, because TelstraClear would probably come last if it did.


TelstraClear have fiber lines up and down the country and in so connect with many companies in Wellington .. Citylink is 1 they do not peer with , There are many they do.

 1 | 2
View this topic in a long page with up to 500 replies per page Create new topic

Twitter »

Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:



Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.