Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.



197 posts

Master Geek
+1 received by user: 2


Topic # 110899 18-Oct-2012 16:46 Send private message

I'm wondering if some clueful poeple here can help me.

I've been looking at a few traceroutes from NZ to various overseas destinations and what been bugging me is the times involved with the undersea cable legs.

For example round-trip tracing from Auckland to Sydney (host905.rimuhosting.com)
5. vocus1.ape.nzix.net 3.4
6. ten-0-2-0-400.bdr01.akl01.akl.VOCUS.net.au 43.1

So the real-world one-way distance would be 19.85ms
According to wolfram|alpha "distance auckland sydney" is 10.1ms
According to Southern Cross Networks themselves the auckland sydney cable leg is 11.38ms.

So 19.85 real world, 11.38 for the distance. We have a missing 8.47ms which is huge!

Can anyone explain why? Don't tell me its routers because i'm measuring the between two routers in a path.
Maybe network equipment we can't see? In this trace its within the Vocus network before it heads across the sea and stays within the Vocus network for a good number of hops after so not likely that its bad network design (ie surely all things within their control).

I should add that this is true for Auckland->US etc, and the above trace is not a problem but an example of a generalisation of why Auckland-Sydney (or any other place) seems twice as far away as it should be.

View this topic in a long page with up to 500 replies per page Create new topic
 1 | 2
1742 posts

Uber Geek
+1 received by user: 337

Trusted
Spark NZ

  Reply # 703085 18-Oct-2012 16:51 Send private message

You could be getting routed via the US depending on which ISP you have and the handover points they have.

Plus it's very likely that you could be getting hidden from some core routers / hops depending on the cleverness they have in their core.

How about listing the ISP you are with and a copy specific trace route's you're taking?




I work for Spark, but as always my views are my own.

178 posts

Master Geek
+1 received by user: 3

Trusted

  Reply # 703099 18-Oct-2012 17:39 Send private message

plambrechtsen: You could be getting routed via the US depending on which ISP you have and the handover points they have.

Plus it's very likely that you could be getting hidden from some core routers / hops depending on the cleverness they have in their core.

How about listing the ISP you are with and a copy specific trace route's you're taking?


Going via the US would add a lot more than another 8ms.

In respect of the original question:

Bear in mind that traceroute can be a terrible indicator of latency. For a start, sending ICMP messages back is often done at low priority, so is not indicative of the experience of a normal customer packet. Also, the return path may be asymmetric---there is no way to tell from traceroute.

Did you run the test multiple times and take an average? Was it just an outlier?

Have you pinged the relevant hosts? Did you get a similar answer?

Are you sure the host in question is actually in Sydney and not in Melbourne?


40 posts

Geek


  Reply # 703111 18-Oct-2012 18:04 Send private message

you will not see every device that a packet travels through
reply packets have to be forward routed through core routers in order to return a reply to the source.
depending on the exit point the return traffic (reply) may not follow the same path as was said previously
each device adds some (small..very) time to the process
control plane traffic which a trace is has low priority and is affected by how busy a device is doing meaningful stuff like forwarding packets
8ms is NOT huge perspective?

2007 posts

Uber Geek
+1 received by user: 267

Trusted
Subscriber

  Reply # 703112 18-Oct-2012 18:13 Send private message

Publius: I'm wondering if some clueful poeple here can help me.

I've been looking at a few traceroutes from NZ to various overseas destinations and what been bugging me is the times involved with the undersea cable legs.

For example round-trip tracing from Auckland to Sydney (host905.rimuhosting.com)
5. vocus1.ape.nzix.net 3.4
6. ten-0-2-0-400.bdr01.akl01.akl.VOCUS.net.au 43.1

So the real-world one-way distance would be 19.85ms
According to wolfram|alpha "distance auckland sydney" is 10.1ms
According to Southern Cross Networks themselves the auckland sydney cable leg is 11.38ms.

So 19.85 real world, 11.38 for the distance. We have a missing 8.47ms which is huge!

Can anyone explain why? Don't tell me its routers because i'm measuring the between two routers in a path.
Maybe network equipment we can't see? In this trace its within the Vocus network before it heads across the sea and stays within the Vocus network for a good number of hops after so not likely that its bad network design (ie surely all things within their control).

I should add that this is true for Auckland->US etc, and the above trace is not a problem but an example of a generalisation of why Auckland-Sydney (or any other place) seems twice as far away as it should be.


Yes there are sometimes a LOT more devices involved than what you see in a traceroute, and there are several good reasons for that which won't make an awful lot of sense to 99% of people on this forum.

Also as mentioned there is the whole asymmetric routing thing, so your ICMP traffic could take a different path to actual traffic going end-to-end. Vocus uses Cisco gear, so the routers process ICMP by the supervisor which is also tasked with many other things such as route updates etc, whereas actual traffic is processed in hardware by dedicated ASICs.

Some may say I'm a bit biased as I work for Vocus



197 posts

Master Geek
+1 received by user: 2


  Reply # 703293 19-Oct-2012 00:16 Send private message

And so to do calculations for the US to prove the same point:

SCC website quotes (segments C+D+E1) 62.44ms one way actual latency between Auckland-SanJose

[Unnecessary parts of mtr output omitted, ms given is avg of many pings]
mtr host112.rimuhosting.com
4. 121.99.12.0  4.9
5. te4-4.ccr01.lax05.atlas.cogentco.com 188.6

different host, mtr host112.rimuhosting.com
7. 203.97.2.37    21.1
8. te7-3.ccr01.sjc05.atlas.cogentco.com 191.3

So we have here 91.85ms and 85.1ms respectively as one-way real-world latencies between auckland-lax/sjc. Somewhere somehow more than 20ms is being used up which is a huge amount of time.

Again, i'm not saying this is a fault or unusual because it's not, i'm just hoping someone can enlighten me as to why in the real world there is far more latency than can be explained by the just the physical speed-of-light distances between two routers on other sides of an ocean.


835 posts

Ultimate Geek

Trusted

  Reply # 703316 19-Oct-2012 01:40 Send private message

I'm not too sure if all your figures are based on the submarine cable operators figures or your own calculations.

As previously mentioned by others there are multiple devices on the traffic path between you and your destination, many of which will not show up on a traceroute. Each of these introduces processing delay, each may also introduce delay due to queuing of traffic and prioritisation (ICMP is typically viewed by operators as lower priority).

In addition to this if you are calculating figures based on straight line and speed of light then you must realise that the speed of light figure most often given is within a vacuum, transmission within other mediums is slower. Also cables rarely run in completely straight lines, sources such as wolfram alpha may be calculating simple straight line distance rather then an actual cable path between locations.

2329 posts

Uber Geek
+1 received by user: 79


  Reply # 703366 19-Oct-2012 08:41 Send private message

Remember, ping times are round trip!

627 posts

Ultimate Geek

Trusted

  Reply # 703392 19-Oct-2012 09:36 Send private message

Publius:Can anyone explain why? Don't tell me its routers because i'm measuring the between two routers in a path.
Maybe network equipment we can't see? In this trace its within the Vocus network before it heads across the sea and stays within the Vocus network for a good number of hops after so not likely that its bad network design (ie surely all things within their control).

1. You are not measuring anything of the sort with a single asymmetric traceroute
2. You are not taking into account any photonic or IP switching delay
3. MPLS ICMP/TTL propagation has differing quirks depending on vendor implementation, and since the destination host is in Brisbane, the latency shown for the entire path will look like it is originating from Brisbane since the intermediate P routers will have to tunnel ICMP back to you via the far-end PE.


225 posts

Master Geek
+1 received by user: 13


  Reply # 703418 19-Oct-2012 10:39 Send private message

Perhaps some national security agency is monitoring all internet traffic in and out of the country....ask Dotcom about his ping.

Damn where is my tinfoil hat...Wink

1465 posts

Uber Geek
+1 received by user: 111

Trusted

  Reply # 703423 19-Oct-2012 10:47 Send private message

I think perhaps a better question to ask is:

- What is your problem?I assume it's your issue with ICMS Ping times taking longer than you expect which may or may not have anything to do with your actual internet performance as detailed above.
Based on that... The next logical question to ask is:

- Are you prepared to either
 - A) Pay more than what you already pay for your current internet service
 or
 - B) Move to another internet provider paying either more or less
 or
 - C) Engage a wholesale internet provider to get a faster route to the internet.
  or
 - D) Purchase your own dedicated international link and organise handover in the US or SYD?

To get what you're actually wanting?

How deep are your pockets and you know you can't change the speed of light or the time which core nodes route traffic?

But that really comes back to the first question.. What is the problem you're trying to solve?




Check out my Mobile Cell Site Google Maps KML Files in my blog.
Now using Google Fusion Tables or Address Lookup or GPS using Smartphone
I update it on a monthly basis automatically from RSM.



197 posts

Master Geek
+1 received by user: 2


  Reply # 703436 19-Oct-2012 11:19 Send private message

Not trying to solve or fix anything, just trying to get a better understanding of why it seems like (to me) a large fraction of the latency between our island nation and anywhere else is not simply explained only by our distance and the speed of light in fibre.

For all the talk of PIPE networks building a cable to the US that would be a few ms quicker than Southern Cross, it seems to me that there is other lower hanging fruit for fixing our nations latency problem.

Latency is a big problem for us in NZ as its has an effect on how quick we can download from the rest of the world even if there is unlimited undersea bandwidth.

1742 posts

Uber Geek
+1 received by user: 337

Trusted
Spark NZ

  Reply # 703448 19-Oct-2012 11:43 Send private message

Publius: Not trying to solve or fix anything, just trying to get a better understanding of why it seems like (to me) a large fraction of the latency between our island nation and anywhere else is not simply explained only by our distance and the speed of light in fibre.

For all the talk of PIPE networks building a cable to the US that would be a few ms quicker than Southern Cross, it seems to me that there is other lower hanging fruit for fixing our nations latency problem.

Latency is a big problem for us in NZ as its has an effect on how quick we can download from the rest of the world even if there is unlimited undersea bandwidth.


You're right in one respect that there are many other areas apart from the speed of light over to the US impact our download speeds.

Latency is one part of it, as is using your ISPs DNS server and pulling data off a local CDN, as is checking the wiring in your home to ensure your last hop is performing at its best, as is what has already been said about the unreliability of doing a TraceRoute to measure the true latency of your connection since it uses ICMP rather than TCP/UDP.

IMHO the major impacts on your latency will be in order of impact:

- The physical transport you have to your home, UFB or Cable will have less latency than VDSL, which has less latency than ADSL.
- For xDSL the distance from your home to the Exchange/Cabinet and the quality of your internal wiring and if you have interleaving turn on or off depending on your distance all have a significant impact on latency.
- The quality of your ISPs BNG hardware and handover interconnect from your home to the BNG, and BNG capacity / how over-subscribed they are to their wholesale internet provider.
- The skills of your ISPs techs on how they configure & manage the above BNGs and interconnect.
- Availability of ISP provided CDNs either Google or Akamai or other CDN services so your downloads are local rather than pulling them internationally.
- The speed of light to the US.

Make sure the first 5 are ok... And you're doing the best that can be done.




I work for Spark, but as always my views are my own.



197 posts

Master Geek
+1 received by user: 2


  Reply # 703451 19-Oct-2012 11:51 Send private message

I'm not speaking so much about my connection which is UFB so I have nothing to complain about, i get ~4ms to media.nzherald.co.nz, 2.7ms to www.trademe.co.nz, etc.

What i'm wondering about is will things between NZ-AU or NZ-US get any better latency wise, or is the latency we are all getting (its not just my isp or me in particular its all of us) due to some physical law like speed of light.

What i've calculated is that maybe 2/3rds of the latency between NZ-US NZ-AS (and im measuring just the latency between the two routers in a traceroute which are on either sides of the undersea cable) is due to physical limitations, but ~1/3rd isn't and it surprises me that its as high as that.

It surprises me anyway :)

1742 posts

Uber Geek
+1 received by user: 337

Trusted
Spark NZ

  Reply # 703468 19-Oct-2012 12:23 Send private message

So using your example:

traceroute to host905.rimuhosting.com (117.20.11.52), 30 hops max, 60 byte packets
 1  x (x)  0.735 ms  0.713 ms  0.695 ms
 2  x (x)  9.962 ms  9.929 ms  9.856 ms
 3  x (x)  1.200 ms  1.067 ms  1.114 ms
 4  x (x)  1.404 ms  1.177 ms  1.310 ms
 5  xe2-1-0.akbr4.global-gateway.net.nz (202.50.232.30)  1.518 ms  1.455 ms  1.663 ms = NZ
 6  xe6-0-0.sgbr1.global-gateway.net.nz (202.50.232.10)  25.661 ms  25.654 ms  25.502 ms = AU
 7  ae0-10.sgbr2.global-gateway.net.nz (202.50.232.246)  25.564 ms  25.668 ms  25.688 ms
 8  * vlan556.52gdc76f02.optus.net.au (59.154.21.137)  27.081 ms *
 9  te3-3-3927.sg4.optus.net.au (61.88.241.255)  28.198 ms  28.188 ms  28.278 ms
10  119.225.5.238 (119.225.5.238)  27.518 ms  27.229 ms  27.249 ms
11  bri-sot-tpgn-bras2-ge-0-1.tpgi.com.au (203.29.142.50)  41.831 ms  42.079 ms  41.896 ms
12  bri-sot-tpgn-bras2-ge-0-1.tpgi.com.au (203.29.142.50)  41.859 ms  41.796 ms  41.708 ms
13  220-245-236-170.tpgi.com.au (220.245.236.170)  43.165 ms  43.159 ms  43.678 ms
14  tpg-border.v17.xe4.l1.bne.ozservers.net.au (223.252.0.20)  45.223 ms  45.182 ms  45.611 ms
15  internal-mesh.v478.xe6.h1.bne.ozservers.net.au (223.252.0.108)  45.737 ms  45.722 ms  46.350 ms
16  g1-0-10.core0.bne.ozservers.net.au (117.20.0.22)  45.184 ms  44.539 ms  44.820 ms
17  host905.rimuhosting.com (117.20.11.52)  47.946 ms  47.211 ms  46.153 ms

Using a ICMP Traceroute (tcptraceroute isn't installed on my firewall so I couldn't test with that)

The difference from 1.5 ms and 25.6 ms is..

11.38 * 2 (NZ-AU SSC) = 22.76
25.661 - 1.518= 24.143

So the total difference is 1.383 ms.... which is just a bit under the 1.5ms that you have as a hop to akbr4... So where exactly is the problem?

A lot of this will depend on the ISP you're using and the quality of their network and interconnect.




I work for Spark, but as always my views are my own.

1910 posts

Uber Geek
+1 received by user: 453

Trusted
Spark NZ

  Reply # 703483 19-Oct-2012 12:36 Send private message

Publius: I'm not speaking so much about my connection which is UFB so I have nothing to complain about, i get ~4ms to media.nzherald.co.nz, 2.7ms to www.trademe.co.nz, etc.

What i'm wondering about is will things between NZ-AU or NZ-US get any better latency wise, or is the latency we are all getting (its not just my isp or me in particular its all of us) due to some physical law like speed of light.

What i've calculated is that maybe 2/3rds of the latency between NZ-US NZ-AS (and im measuring just the latency between the two routers in a traceroute which are on either sides of the undersea cable) is due to physical limitations, but ~1/3rd isn't and it surprises me that its as high as that.

It surprises me anyway :)


The main issue is that unless you are sitting in the operational centre of your ISP with a great understanding of MPLS and how the network has been stitched together, you simply can't know the path your traffic is taking - so while you can build a theoretical A-B model using speed of light in a fibre, you don't know the actual distance or traffic path involved.

There are more bits of equipment involved that you can see from a tracert. Often the paths (if you could see them) would appear non-optimal, but that's one of the prices people pay for having a resilient network.

Sorry I can't be more specific. There's a little bit of optimisation that could be done, but it could easily be at the expense of sensible load balancing across alternate paths.

Cheers - N


 1 | 2
View this topic in a long page with up to 500 replies per page Create new topic




Twitter »
Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:





Trending now »

Hot discussions in our forums right now:

Snap have failed our company!
Created by dafman, last reply by kornflake on 21-Oct-2014 23:04 (23 replies)
Pages... 2


Spark Socialiser
Created by freitasm, last reply by Flickky on 21-Oct-2014 22:10 (21 replies)
Pages... 2


Another Trade Me competitor: SellShed
Created by freitasm, last reply by jonathan18 on 21-Oct-2014 23:12 (32 replies)
Pages... 2 3


American legal jurisdiction in New Zealand
Created by ajobbins, last reply by gzt on 21-Oct-2014 14:58 (30 replies)
Pages... 2


Overcharged by Slingshot for months - warning to existing customers
Created by dusty42, last reply by richms on 21-Oct-2014 19:15 (27 replies)
Pages... 2


Why would Suresignal calls be worse quality than non-Suresignal calls from the same location?
Created by Geektastic, last reply by froob on 21-Oct-2014 08:21 (41 replies)
Pages... 2 3


Just bought a TiVo online. No wireless adaptor. Will a standard one work? Or do I need the TiVo one ?
Created by Limerick, last reply by graemeh on 20-Oct-2014 16:03 (11 replies)

Spark Socialiser and new plan
Created by saeran, last reply by eXDee on 21-Oct-2014 21:52 (10 replies)


Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.

Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.