Vodafone APE/WIX peering broken

Forums › One New Zealand (including Vodafone, WxC, Farmside) › Vodafone APE/WIX peering broken

Zeon

3918 posts

Uber Geek

Trusted

#70848 1-Nov-2010 14:38

Hey guys,
Noticed some weird routing going on from ICONZ to vodafone. Rather than going via APE as normal its going via Vocus so much slower:

Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\jonathan.spence>tracert vodafone.co.nz

Tracing route to vodafone.co.nz [202.73.199.35]
over a maximum of 30 hops:

  1 6 ms 1 ms 6 ms 10.2.253.1
  2 5 ms 6 ms 6 ms ip-210-48-87-185.iconz.net.nz [210.48.87.185]
  3 35 ms 10 ms 4 ms ge-2-1.cat03-sky-int-vocus.iconz.net [210.48.5.8
6]
  4 97 ms 175 ms 61 ms ip-113.203.31.114.vocus.net.au [114.31.203.113]

  5 75 ms 68 ms 149 ms ge-0-0-0.bdr02.akl02.akl.vocus.net.au [114.31.20
2.37]
  6 9 ms 24 ms 8 ms as7657.cust.bdr02.akl02.akl.vocus.net.au [114.31
.203.106]
  7 7 ms 7 ms 9 ms gi5-0-0.akl-grafton-edge1.ihug.net [203.109.130.
161]
  8 6 ms 22 ms 19 ms gi5-1-0.akl-grafton-edge2.ihug.net [203.109.130.
162]
  9 10 ms 10 ms 9 ms 221.143.109.203.core.vf.net.nz [203.109.143.221]

10 38 ms 7 ms 11 ms unassigned.core.vf.net.nz [203.109.143.222]
11 * * * Request timed out.
12 * ^C
C:\Users\jonathan.spence>

Performance for customers connecting over Vodafone to us @ ICONZ atm is crap as we only have 512kbps of international and I think, even though this isn't going international is still applying because its going via Vocus.

Speedtest 2019-10-14

Kraven

729 posts

Ultimate Geek

#398605 1-Nov-2010 15:58

At a guess, they have dropped peering... Mines going via Sydney!

Tracing route to www.vodafone.co.nz [202.73.199.35]
over a maximum of 30 hops:

1 * * * Request timed out.
2 * * * Request timed out.
3 10 ms 10 ms 8 ms pmr-inspire-br-3.inspire.net.nz [121.79.197.169]
4 6 ms 5 ms 5 ms pmr-inspire-br-1-ge0-3.core.sta.inspire.net.nz [203.114.128.254]
5 14 ms 10 ms 25 ms gigabitethernet0-3-237.akkin-rt2.fx.net.nz [131.203.253.65]
6 12 ms 11 ms 21 ms fxn-0001.gw1.akl1.asianetcom.net [203.192.167.162]
7 12 ms 11 ms 14 ms gi-1-0.gw1.akl1.asianetcom.net [203.192.167.161]
8 12 ms 21 ms 12 ms ge-3-3-0-0.gw4.akl1.asianetcom.net [202.147.55.210]
9 39 ms 35 ms 39 ms po2-0.cr4.syd1.asianetcom.net [202.147.55.254]
10 36 ms 38 ms 35 ms ge-1-0-0-0.gw4.syd1.asianetcom.net [202.147.40.150]
11 38 ms 39 ms 38 ms gi1-0-0.gw1.syd2.asianetcom.net [202.147.55.90]
12 38 ms 37 ms 35 ms gi1-0-4.gw1.syd3.asianetcom.net [202.147.14.154]
13 36 ms 40 ms 36 ms gi8-8.switch1.syd.pacific.net.au [125.255.112.253]
14 38 ms 38 ms 41 ms as4826-2.sydney.pipenetworks.com [218.100.2.45]
15 37 ms 37 ms 37 ms ip-108.192.31.114.vocus.net.au [114.31.192.108]
16 39 ms 40 ms 38 ms ge-0-3-0-131.bdr01.akl01.akl.vocus.net.au [114.31.192.97]
17 37 ms 38 ms 37 ms ip-35.202.31.114.vocus.net.au [114.31.202.35]
18 40 ms 36 ms 38 ms ten-0-1-0-62.bdr02.akl02.akl.vocus.net.au [114.31.202.43]
19 37 ms 38 ms 38 ms as7657.cust.bdr02.akl02.akl.vocus.net.au [114.31.203.106]
20 37 ms 37 ms 38 ms gi5-0-0.akl-grafton-edge1.ihug.net [203.109.130.161]
21 41 ms 38 ms 39 ms gi5-1-0.akl-grafton-edge2.ihug.net [203.109.130.162]
22 47 ms 38 ms 43 ms 221.143.109.203.core.vf.net.nz [203.109.143.221]
23 39 ms 44 ms 39 ms unassigned.core.vf.net.nz [203.109.143.222]
24 * * * Request timed out.

muppet

2571 posts

Uber Geek

Trusted

#398612 1-Nov-2010 16:07

We peer at APE with the route servers. This is me asking the APE Route Server what routes it's teaching our router that's doing the peering, with the routes filtered to just show anything matching 203.109.x.x:

user@router> show route receive-protocol bgp 192.203.154.1 | match 203.109
* 203.109.128.0/19 192.203.154.36 9560 7657 I
* 203.109.133.0/24 192.203.154.37 9560 7657 I
* 203.109.146.0/24 192.203.154.37 9560 7657 I
* 203.109.158.0/24 192.203.154.123 9560 4826 7657 I
* 203.109.160.0/19 192.203.154.36 9560 7657 I
* 203.109.192.0/19 192.203.154.36 9560 7657 I
* 203.109.224.0/19 192.203.154.36 9560 7657 I

AS7657 is Vodafone.

Summary: We're still learning Vodafone's routes via APE (and traffic is going that way)

So something else is broken.

Audiophiles are such twits! They buy such pointless stuff: Gold plated cables, $2000 power cords. Idiots.

OOOHHHH HYPERFIBRE!

BlakJak

1275 posts

Uber Geek

Trusted

#398616 1-Nov-2010 16:15

Hi guys, from an ICONZ perspective (noting original Traceroute) specific customers who are downstream of ICONZ will see Vodafone via our International transit at the moment,

However the vast majority of our customers (and our NOC itself) are still going across the APE as it should be.

Zeon should probably contact ICONZ Customer Care as it appears there's a unique set of circumstances at work here. And I don't have reason to believe that Inspire's situation is in any way related.

Cheers
BJ.
(In my day job I help keep ICONZ's NOC ticking, however, I don't post on their behalf.)

No signature to see here, move along...

Zeon

3918 posts

Uber Geek

Trusted

#398617 1-Nov-2010 16:17

muppet: We peer at APE with the route servers. This is me asking the APE Route Server what routes it's teaching our router that's doing the peering, with the routes filtered to just show anything matching 203.109.x.x:

user@router> show route receive-protocol bgp 192.203.154.1 | match 203.109
* 203.109.128.0/19 192.203.154.36 9560 7657 I
* 203.109.133.0/24 192.203.154.37 9560 7657 I
* 203.109.146.0/24 192.203.154.37 9560 7657 I
* 203.109.158.0/24 192.203.154.123 9560 4826 7657 I
* 203.109.160.0/19 192.203.154.36 9560 7657 I
* 203.109.192.0/19 192.203.154.36 9560 7657 I
* 203.109.224.0/19 192.203.154.36 9560 7657 I

AS7657 is Vodafone.

Summary: We're still learning Vodafone's routes via APE (and traffic is going that way)

So something else is broken.

Can you tell us all routes for Vodafone's AS?

Speedtest 2019-10-14

muppet

2571 posts

Uber Geek

Trusted

#398628 1-Nov-2010 16:25

Well, you can see every route that VF *might* announce right here:

http://ape.net.nz/cgi-bin/ShowRoutes.cgi

Just put in VF's AS.

But here's the current list of routes with VF's AS in it that we're learning from the Route Servers. Enjoy, for whatever it's worth.

user@router> show route receive-protocol bgp 192.203.154.1 | match 7657
* 27.252.0.0/16 192.203.154.36 9560 7657 I
* 118.92.0.0/16 192.203.154.36 9560 7657 I
* 118.93.0.0/16 192.203.154.36 9560 7657 I
* 121.90.0.0/16 192.203.154.36 9560 7657 I
* 124.6.192.0/19 192.203.154.36 9560 7657 I
* 124.6.220.0/22 192.203.154.36 9560 7657 38748 I
* 192.190.108.0/24 192.203.154.36 9560 7657 38329 I
* 202.37.182.0/24 192.203.154.36 9560 7657 ?
* 202.49.33.0/24 192.203.154.36 9560 7657 I
* 202.49.63.0/24 192.203.154.36 9560 7657 I
* 202.49.148.0/24 192.203.154.36 9560 7657 ?
* 202.49.149.0/24 192.203.154.36 9560 7657 ?
* 202.49.178.0/24 192.203.154.36 9560 7657 I
* 202.50.84.0/24 192.203.154.36 9560 7657 I
* 202.50.188.0/24 192.203.154.36 9560 7657 I
* 202.73.192.0/20 192.203.154.36 9560 7657 I
* 203.96.192.0/20 192.203.154.36 9560 7657 I
* 203.96.208.0/20 192.203.154.36 9560 7657 I
* 203.96.218.0/24 192.203.154.36 9560 7657 ?
* 203.96.219.0/24 192.203.154.36 9560 7657 ?
* 203.109.128.0/19 192.203.154.36 9560 7657 I
* 203.109.133.0/24 192.203.154.37 9560 7657 I
* 203.109.146.0/24 192.203.154.37 9560 7657 I
* 203.109.158.0/24 192.203.154.123 9560 4826 7657 I
* 203.109.160.0/19 192.203.154.36 9560 7657 I
* 203.109.192.0/19 192.203.154.36 9560 7657 I
* 203.109.224.0/19 192.203.154.36 9560 7657 I
* 203.118.128.0/20 192.203.154.36 9560 7657 I
* 203.118.144.0/20 192.203.154.36 9560 7657 I
* 203.118.145.0/24 192.203.154.37 9560 7657 I
* 203.118.147.0/24 192.203.154.37 9560 7657 I
* 203.118.150.0/24 192.203.154.36 9560 7657 I
* 203.118.151.0/24 192.203.154.36 9560 7657 I
* 203.118.152.0/24 192.203.154.36 9560 7657 I
* 203.118.153.0/24 192.203.154.36 9560 7657 I
* 203.118.160.0/20 192.203.154.36 9560 7657 I
* 203.118.176.0/20 192.203.154.36 9560 7657 I
* 203.147.109.0/24 192.203.154.36 9560 7657 23754 23754 45163 I
* 203.171.4.0/24 192.203.154.37 9560 7657 38605 I
* 203.173.128.0/19 192.203.154.36 9560 7657 I
* 203.173.160.0/19 192.203.154.36 9560 7657 I
* 203.173.192.0/19 192.203.154.36 9560 7657 I
* 203.173.224.0/20 192.203.154.123 9560 4826 7657 I
* 203.173.240.0/21 192.203.154.36 9560 7657 I
* 203.173.248.0/21 192.203.154.36 9560 7657 I
* 205.191.208.0/24 192.203.154.36 9560 7657 ?
* 205.233.128.0/24 192.203.154.36 9560 7657 23754 23754 45633 I
* 210.4.208.0/22 192.203.154.37 9560 7657 9733 9733 9733 9733 9733 I
* 210.4.214.0/23 192.203.154.37 9560 7657 9733 9733 9733 9733 9733 I

Audiophiles are such twits! They buy such pointless stuff: Gold plated cables, $2000 power cords. Idiots.

OOOHHHH HYPERFIBRE!

Zeon

3918 posts

Uber Geek

Trusted

#398630 1-Nov-2010 16:26

BlakJak: Hi guys, from an ICONZ perspective (noting original Traceroute) specific customers who are downstream of ICONZ will see Vodafone via our International transit at the moment,

However the vast majority of our customers (and our NOC itself) are still going across the APE as it should be.

Zeon should probably contact ICONZ Customer Care as it appears there's a unique set of circumstances at work here. And I don't have reason to believe that Inspire's situation is in any way related.

Cheers
BJ.
(In my day job I help keep ICONZ's NOC ticking, however, I don't post on their behalf.)

Hey BJ, OK its all back to normal now. You could actually tell who we are based on the 2nd hop :) (if you are @ ICONZ that is)!

Speedtest 2019-10-14

BlakJak

1275 posts

Uber Geek

Trusted

#398641 1-Nov-2010 16:40

Yeah, we've made a change which has sorted things out for you and the small pool of customers who were similarly affected.

And yes, was able to find out who you were, but still, Customer Care are worth talking to when this sorta thing comes up so we can take an 'official' look. ;-) It would not have been corrected so fast if I hadn't spied this thread :)

No signature to see here, move along...