Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.


14209 posts

Uber Geek
+1 received by user: 2570

Trusted
Subscriber

Topic # 129415 14-Sep-2013 22:48
Send private message

It's been like this all day... Stuff loads eventually, but it takes 45 seconds. Not kidding, 45 seconds, waiting for static2.stuff.co.nz



Here's the traceroute, going via Tokyo it seems. This should be a 40ms ping, not 280ms. Is this another Vodafone routing issue, or is it a stuff issue?

tracert static2.stuff.co.nz

Tracing route to a1784.g.akamai.net [125.56.218.19]
over a maximum of 30 hops:

1 <1 ms 1 ms 1 ms 192.168.1.1
2 * * * Request timed out.
3 11 ms 10 ms 9 ms lo0.internet.ivpn.pexx.telstraclear.net 
4 19 ms 20 ms 21 ms xx-g-0-0-0.telstraclear.net 
5 19 ms 20 ms 21 ms ge-0-2-0-1.xcore1.acld.telstraclear.net
6 39 ms 20 ms 33 ms unknown.telstraglobal.net [134.159.174.37]
7 149 ms 147 ms 146 ms i-0-0-4-0.tlot-core01.bx.telstraglobal.net [202.84.142.118]
8 149 ms 145 ms 147 ms i-0-4-0-0.tlot02.bi.telstraglobal.net [202.84.251.238]
9 144 ms 144 ms 147 ms 208.173.53.141
10 152 ms 152 ms 151 ms 208.175.201.14
11 153 ms 153 ms 173 ms ae-6.r21.lsanca03.us.bb.gin.ntt.net [129.250.5.69]
12 300 ms 277 ms 280 ms as-1.r21.tokyjp01.jp.bb.gin.ntt.net [129.250.3.146]
13 253 ms 254 ms 266 ms ae-3.r24.tokyjp05.jp.bb.gin.ntt.net [129.250.6.188]
14 299 ms 291 ms 267 ms ae-3.r00.tokyjp03.jp.bb.gin.ntt.net [129.250.4.233]
15 270 ms 269 ms 268 ms ae-0.akamai.tokyjp03.jp.bb.gin.ntt.net [61.120.145.202]
16 276 ms 280 ms 283 ms a125-56.218-19.deploy.akamaitechnologies.com [125.56.218.19]






AWS Certified Solution Architect Professional, Sysop Administrator Associate, and Developer Associate
TOGAF certified enterprise architect
Professional photographer


View this topic in a long page with up to 500 replies per page Create new topic
 1 | 2 | 3


14209 posts

Uber Geek
+1 received by user: 2570

Trusted
Subscriber

  Reply # 895799 15-Sep-2013 10:54
Send private message

It was still doing it this morning, but it's come right in the past half hour. I wonder if it's a Vodafone or a stuff issue.

Here are a couple of interesting speed test results on stuff.co.nz
From NZ server: Test results.
From USA server: Test results.

tracert cdn.gigya.com

Tracing route to a221.g.akamai.net [125.56.201.123]
over a maximum of 30 hops:

1 <1 ms 1 ms <1 ms 192.168.1.1
2 * * * Request timed out.
3 11 ms 10 ms 10 ms lo0.internet.ivpn.pe25.telstraclear.net [218.101.61.124]
4 19 ms 20 ms 18 ms ie2-g-0-0-0.telstraclear.net [203.98.50.2]
5 20 ms 19 ms 18 ms ge-0-2-0-1.xcore1.acld.telstraclear.net [203.98.50.251]
6 22 ms 20 ms 29 ms unknown.telstraglobal.net [134.159.174.37]
7 147 ms 147 ms 147 ms i-0-0-1-1.1wlt-core01.bx.telstraglobal.net [202.84.223.85]
8 145 ms 145 ms 148 ms i-0-4-0-0.tlot02.bi.telstraglobal.net [202.84.251.238]
9 145 ms 151 ms 144 ms 208.175.201.5
10 153 ms 151 ms 153 ms 208.175.201.14
11 150 ms 151 ms 151 ms ae-8.r20.lsanca03.us.bb.gin.ntt.net [129.250.5.1]
12 260 ms 279 ms 267 ms ae-7.r20.tokyjp05.jp.bb.gin.ntt.net [129.250.3.15]
13 260 ms 259 ms 275 ms ae-19.r25.tokyjp05.jp.bb.gin.ntt.net [129.250.6.213]
14 289 ms 289 ms 298 ms ae-2.r00.tokyjp03.jp.bb.gin.ntt.net [129.250.2.5]
15 260 ms 268 ms 260 ms ae-0.akamai.tokyjp03.jp.bb.gin.ntt.net [61.120.145.202]
16 289 ms 287 ms 282 ms a125-56.201-123.deploy.akamaitechnologies.com [125.56.201.123]

tracert static2.stuff.co.nz

Tracing route to a1784.g.akamai.net [125.56.218.19]
over a maximum of 30 hops:

1 <1 ms 1 ms <1 ms 192.168.1.1
2 * * * Request timed out.
3 324 ms 53 ms 269 ms lo0.internet.ivpn.pe25.telstraclear.net [218.101.61.124]
4 20 ms 18 ms 34 ms ie2-g-0-0-0.telstraclear.net [203.98.50.2]
5 28 ms 21 ms 19 ms ge-0-2-0-1.xcore1.acld.telstraclear.net [203.98.50.251]
6 22 ms 21 ms 21 ms unknown.telstraglobal.net [134.159.174.37]
7 147 ms 147 ms 147 ms i-0-0-4-0.tlot-core01.bx.telstraglobal.net [202.84.142.118]
8 144 ms 145 ms 145 ms i-0-4-0-0.tlot02.bi.telstraglobal.net [202.84.251.238]
9 145 ms 146 ms 144 ms 208.175.201.5
10 150 ms 149 ms 150 ms 208.175.201.14
11 149 ms 157 ms * ae-6.r21.lsanca03.us.bb.gin.ntt.net [129.250.5.69]
12 266 ms 283 ms 267 ms ae-7.r20.tokyjp05.jp.bb.gin.ntt.net [129.250.3.15]
13 284 ms 284 ms 257 ms ae-19.r24.tokyjp05.jp.bb.gin.ntt.net [129.250.6.209]
14 263 ms 268 ms 269 ms ae-3.r00.tokyjp03.jp.bb.gin.ntt.net [129.250.4.233]
15 261 ms 270 ms 270 ms ae-0.akamai.tokyjp03.jp.bb.gin.ntt.net [61.120.145.202]
16 284 ms 286 ms 273 ms a125-56.218-19.deploy.akamaitechnologies.com [125.56.218.19]




AWS Certified Solution Architect Professional, Sysop Administrator Associate, and Developer Associate
TOGAF certified enterprise architect
Professional photographer


596 posts

Ultimate Geek
+1 received by user: 96


  Reply # 895871 15-Sep-2013 13:56
Send private message

What DNS servers are you using? For Akamai CDN to work correctly you need to be using Vodafone's DNS servers.



14209 posts

Uber Geek
+1 received by user: 2570

Trusted
Subscriber

  Reply # 895889 15-Sep-2013 14:48
Send private message

DNS1: 203.97.78.43
DNS2: 203.97.78.44
DNS3: 8.8.8.8

Those are the ones Mauricio recommended in the TC DNS thread.




AWS Certified Solution Architect Professional, Sysop Administrator Associate, and Developer Associate
TOGAF certified enterprise architect
Professional photographer


I fix stuff!
1702 posts

Uber Geek
+1 received by user: 378

Trusted
Vocus
Subscriber

  Reply # 895952 15-Sep-2013 16:54
Send private message

Having 8.8.8.8 in your DNS could cause odd experiences when it comes to CDN's.

Whats probably happened is Akamai couldn't tell where you were located based on a DNS going to 8.8.8.8 (for some reason) and your akamai traffic came from some random akamai server.

I would recommended not using 8.8.8.8 as a resolver.




14209 posts

Uber Geek
+1 received by user: 2570

Trusted
Subscriber

  Reply # 895958 15-Sep-2013 17:11
Send private message

It's the third resolver, so long as the TC name servers work it'll never get used, right? It's in my router, which is running DD-WRT.




AWS Certified Solution Architect Professional, Sysop Administrator Associate, and Developer Associate
TOGAF certified enterprise architect
Professional photographer


I fix stuff!
1702 posts

Uber Geek
+1 received by user: 378

Trusted
Vocus
Subscriber

  Reply # 895978 15-Sep-2013 17:43
One person supports this post
Send private message

timmmay: It's the third resolver, so long as the TC name servers work it'll never get used, right? It's in my router, which is running DD-WRT.


Depends on the software. It could be round robin'ing or using 1 until it fails then moves to the next, so eventually you get stuck on 8.8.8.8 until it fails and it moves back to #1.



1387 posts

Uber Geek
+1 received by user: 134


  Reply # 895979 15-Sep-2013 17:56
Send private message

really it's stuff to blame for using akamai.

cdn's like cachefly don't have these issues, and have much better average performance, even to NZ when their nearest node is in sydney.


the pages won't necessarily load faster in nz, as akamai doesn't have dns servers in new zealand, and only really works well for very high volume web sites as every single akamai node has to cache the resource individually to get good performance.

anyway, you could in theory complain to akamai, as even if using 8.8.8.8 it should pass on location data of the user.

that said, 45 seconds is nowhere near what you should expect for 240 msec ping.  bbc is higher ping away than that, and loads in about 4 seconds uncached, and about 2 seconds cached. (from memory)  there most likely is some kind of dns or timeout issue happening.  maybe a resource that can't load at all.  maybe try pulling up the file on static2.stuff.co.nz individually and see how long it takes.

like from the uk, the first image i see on main page is 30k big and takes 1.1 second to load:

 


curl -O http://static2.stuff.co.nz/1379223534/138/9168138.jpg 0.00s user 0.01s system 1% cpu 1.162 total

1387 posts

Uber Geek
+1 received by user: 134


  Reply # 895983 15-Sep-2013 18:08
Send private message

Sounddude:
timmmay: It's the third resolver, so long as the TC name servers work it'll never get used, right? It's in my router, which is running DD-WRT.


Depends on the software. It could be round robin'ing or using 1 until it fails then moves to the next, so eventually you get stuck on 8.8.8.8 until it fails and it moves back to #1.




it's likely to be dnsmasq, which can round robin a bit. :(

it can also parallel request, which is probably actually safer for this problem, but it seems that it is an issue with 8.8.8.8 and akamai in this instance.

unfortunately if he was to set both the top two dns servers on their own, he could hit problems like how orcon's primary dns servers both went down at once today.  and i'd probably recommend setting 203.96.152.4 in the mix instead of 8.8.8.8.

it's likely to perform worse than the two normal clear dns servers, but it's still clear ip space and will still be regionalised as NZ  -- it is the legacy paradise name server that most cable users are still using as was configured statically, and so isn't likely to stop working any time soon, 

and is in wellington rather than anycast/akld (i'm not sure which clear is doing, but it's akld from akld)

you would probably do ok using a vodafone dns server as well, but i'm not sure if they're allowed/blocked from legacy clear users or not.



14209 posts

Uber Geek
+1 received by user: 2570

Trusted
Subscriber

  Reply # 895990 15-Sep-2013 18:18
Send private message

Thanks guys, I've removed Google's DNS server and I've added in the other clear DNS server. I'll report back if I see anything different. If what's said above is correct it could explain why some websites really have problems at times - ebay and amazon for example.




AWS Certified Solution Architect Professional, Sysop Administrator Associate, and Developer Associate
TOGAF certified enterprise architect
Professional photographer


1387 posts

Uber Geek
+1 received by user: 134


  Reply # 895991 15-Sep-2013 18:21
Send private message

i just realised the most likely reason for a severe degradation rather than a minor delay is that you hit telstraclear's bug with their proxy servers, where if you go to an international site, and it returns a NZ IP, then the traffic is blackholed, and you can't reach it.

this issue has been going on for years, and it's negatively effected many users and sites over time.  i hope with the vodafone takeover that this issue is resolved.



1387 posts

Uber Geek
+1 received by user: 134


  Reply # 895993 15-Sep-2013 18:23
Send private message

timmmay: Thanks guys, I've removed Google's DNS server and I've added in the other clear DNS server. I'll report back if I see anything different. If what's said above is correct it could explain why some websites really have problems at times - ebay and amazon for example.


you can also use opera turbo, hola unblocker, or google's new proxy service to work around these issues.  it's telstraclear transparent proxy issues, which effects http sites but not https.


1387 posts

Uber Geek
+1 received by user: 134


  Reply # 895994 15-Sep-2013 18:25
Send private message

timmmay: Thanks guys, I've removed Google's DNS server and I've added in the other clear DNS server. I'll report back if I see anything different. If what's said above is correct it could explain why some websites really have problems at times - ebay and amazon for example.


ebay and amazon have both had a few issues this year in general as well.   if ebay is showing some error page it's unrelated.  amazon has had issues as recently as last month:

http://gigaom.com/2013/08/19/amazons-u-s-front-end-web-site-is-down/

27128 posts

Uber Geek
+1 received by user: 6572

Moderator
Trusted
Biddle Corp
Lifetime subscriber

  Reply # 895997 15-Sep-2013 18:33
Send private message

A nslookup on 8.8.8.8 on my cable connection points to 125.56.201.121/123 whereas using the TCL DNS servers points to two different 203.97.86.235/200 which are both local.

Your router is definitely using Google DNS to get those results.

1267 posts

Uber Geek
+1 received by user: 291


  Reply # 895998 15-Sep-2013 18:37
Send private message

mercutio: i just realised the most likely reason for a severe degradation rather than a minor delay is that you hit telstraclear's bug with their proxy servers, where if you go to an international site, and it returns a NZ IP, then the traffic is blackholed, and you can't reach it.
Asymmetric routing is the term you're looking for 

1387 posts

Uber Geek
+1 received by user: 134


  Reply # 895999 15-Sep-2013 18:39
Send private message

yitz:
mercutio: i just realised the most likely reason for a severe degradation rather than a minor delay is that you hit telstraclear's bug with their proxy servers, where if you go to an international site, and it returns a NZ IP, then the traffic is blackholed, and you can't reach it.
Asymmetric routing is the term you're looking for 


<rant>
i was hoping to try to convey it in a general way.  aymmetric routing is a normal part of the internet, and doesn't really explain the weakness of their segregation.

it's kind of like having a passport stamp when you leave new zealand unless you go to australia, and a passport stamp when you come back from anywhere but australia.  but if you come back from australia they don't stamp the passport, but they won't let you back in the country because you didn't have a return passport stamp, but you had one when you left.

actually maybe it'd be clearer, if you had a passport stamp when coming out or leaving from auckland, but not from wellington, but some international flights terminate in wellington rather than auckland.  so normally if returning to auckland, your flight would leave auckland and come back on auckland, but it could be rerouted to wellington if there is hazardous landing conditions in auckland, or a flight problem to auckland.

most international traffic is assymetric, and a significant amount of national traffic is assymetric, but as long as the path is advertised to telecom or telstraclear at the time of making the request, then traffic to the website will work.  but if that connection is severed and it falls back on international routes, and it doesn't take an international path both ways (such as when people use vocus international plus domestic peering this can happen)  but if dns load balances between two locations one with paid domestic transit and the other without, or one in nz and one overseas then the site will invariably fail unless the load balancing is very consistent such as with nzherald.

anyway, it's easy to create the problem, you just make a web site that has multiple ip's, one in nz, one overseas, and and ip addresses for both of them to dns.  then it'll often fail on telstraclear, and work on other isp's.

</rant>

telstraclear's network is broken for http web sites, and works most of the time.  but will not work reliably with some less common situations, in the event of partial failures.  if web sites use https instead they can bypass telstraclear's transparent proxy.  and if they always use US routing or paid domestic peeering consistently for the same hostname they will work.  anycast cdn's are more reliable than smart geoip based dns solutions.

 1 | 2 | 3
View this topic in a long page with up to 500 replies per page Create new topic

Twitter »

Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:



Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.