Routes to SG and changes to routes

Forums › Spark New Zealand (including Skinny and BigPipe) › Routes to SG and changes to routes

1 | 2

Talkiet

4792 posts

Uber Geek

Trusted

#1142861 27-Sep-2014 23:44

insane:
Talkiet: Slower routes don't always mean something's broken, and therefore, there may be no need to 'fix'.

There are many reasons traffic may not be taking the shortest route somewhere. Roading analogies again come in useful. If every single person took the shortest route in Auckland to work at peak time, there would be a LOT of wasted roading infrastructure not being used and the shortest routes would get congested, causing even bigger issues. Things work best when the traffic spreads itself around.

Internet - Same deal.

Cheers _ N

While technically you're not wrong, I think that traditional Telecom/GGI line of thinking is a bit out dated now. In the past users accepted whatever they got, today users have options and if some ISPs are willing to accommodate then it's only a matter of time before users will vote with their feet, and they should.

The last ISP I worked for at one point had really passionate network engineers, who so happened to be gamers who took it upon themselves to always ensure they engineered gaming traffic accordingly. It can be done, it just takes passion/creativity, and a attitude of enablement which looks for ways to improve things, not ways to justify the status quo.

And in this case, while you're not technically wrong either, it takes a lot more than that to sustainably do what you suggest - and its a lot easier for a small ISP to do it than someone with real scale.

Check my replies carefully - I have never denied that the optimisations have happened before, and they will continue to happen from time to time. However expecting that efforts will be made to optimise all games, all the time, for all users is not realistic.

I promise you the engineers in Spark (or VF for example) are just as passionate as those in [small isp] - but because they deal with orders of magnitude more users and traffic, they simply have to take a more controlled approach to this sort of traffic engineering.

When someone optimised WOT Asia traffic for example in [small isp], where did they record the change, the reason for the change, and what the traffic impact was? How will the next engineer know why that range has been plucked out of a much larger set of prefixes and sent the other way? Now, multiply that by a lot of games, and a lot of regions, and add in applications like Office 365, various voice and other apps - then consider how many of those may add (or delete!) prefixes from time to time. How are those changes picked up?

What you advocate is possible for sure, and on a small scale it's even a good idea. But as a general approach for a large ISP, is it a scalable and supportable solution? I'm not sure it is.

Cheers - N

Please note all comments are from my own brain and don't necessarily represent the position or opinions of my employer, previous employers, colleagues, friends or pets.

slingynz

154 posts

Master Geek

#1142975 28-Sep-2014 13:49

michaelmurfy: In future remember just how far Singapore is from us

2500km closer than California?

yitz

2074 posts

Uber Geek

#1142992 28-Sep-2014 15:05

Is Global-Gateway's peering out of Perth now gone for good?
Since the sale of the AAPT Australian backbone, seems it's just dump as much traffic off as possible in Sydney at Equinix or transit partner NTT.

insane

3236 posts

Uber Geek

ID Verified

Trusted

#1143862 29-Sep-2014 22:17

Talkiet:
And in this case, while you're not technically wrong either, it takes a lot more than that to sustainably do what you suggest - and its a lot easier for a small ISP to do it than someone with real scale.

Check my replies carefully - I have never denied that the optimisations have happened before, and they will continue to happen from time to time. However expecting that efforts will be made to optimise all games, all the time, for all users is not realistic.

I promise you the engineers in Spark (or VF for example) are just as passionate as those in [small isp] - but because they deal with orders of magnitude more users and traffic, they simply have to take a more controlled approach to this sort of traffic engineering.

When someone optimised WOT Asia traffic for example in [small isp], where did they record the change, the reason for the change, and what the traffic impact was? How will the next engineer know why that range has been plucked out of a much larger set of prefixes and sent the other way? Now, multiply that by a lot of games, and a lot of regions, and add in applications like Office 365, various voice and other apps - then consider how many of those may add (or delete!) prefixes from time to time. How are those changes picked up?

What you advocate is possible for sure, and on a small scale it's even a good idea. But as a general approach for a large ISP, is it a scalable and supportable solution? I'm not sure it is.

Cheers - N

I don't doubt for a minute that in larger teams that it's harder to keep track of changes/impact, but am sure those same pain points aren't unique to one area of the business. On the flip-side larger teams, more users, more traffic, more revenue should also hopefully translate into better Engineers, monitoring/visability and automation.

I think it's also fair to assume that network engineer numbers don't increase linearly with traffic growth, so once you reach a certain number of Snr IP Engineers they can take care of magnitudes more traffic. I do concede that swinging traffic around for 500,000 users isn't something you'd want to do without careful consideration, the media would have a field-day if it went wrong.

Personally I'd like to see transit providers (not just GGI) offer a far more complete sets of BGP communities (and maintain them) which their customers can use to engineer traffic themselves, that's just one way a larger team could show real value.

I guess what I'm trying to say is that a larger network alone shouldn't be an excuse to not innovate, and that looking to new ways to retain share in a market with declining revenues should be rather important. I'll let the guys at Vocus know you think they are 'small' ;)

/my personal views of course :)

yitz: Is Global-Gateway's peering out of Perth now gone for good?
Since the sale of the AAPT Australian backbone, seems it's just dump as much traffic off as possible in Sydney at Equinix or transit partner NTT.

Think they sold their share in the cable system Sea-Me-We 3 out of Perth to Vocus

Regs

4066 posts

Uber Geek

Trusted

Snowflake

#1144521 30-Sep-2014 21:05

the capacity of the smw3 cable is so small that it couldn't possibly host all the potential NZ/AU to SG traffic. Because its a scarce resource, the price per MBps goes up. Other routes have the price per MBps going down. Users want Unlimited traffic for less dollars. End shot is that ISPs will buy other routes.

If a new cable goes in between AU & SG -e.g. the ASC Cable that was supposed to be up and running by now, or others - then you might see return of optimal routes. Until then, don't hold your breath (but you should still log the request) :)

Sales Engineer
Snowflake
www.snowflake.com

about.me/nzregs
Twitter: @nzregs

insane

3236 posts

Uber Geek

ID Verified

Trusted

#1144545 30-Sep-2014 21:24

Sea-Me-We 3 currently has a capacity of 480Gbps using 10Gbps wave lengths, that's a lot really. If History is anything to go by then like other operators they will look to upgrade their optics to 100GBps too (assuming the repeaters can handle it).

Now in your defense, traffic patterns have been changing over recent years, however I'm not sure whether the trend is continuing now that Azure/AWS have an AU presence. Perhaps you can speak on that last part, I genuinely am interested!

pchs

185 posts

Master Geek

#1144550 30-Sep-2014 21:33

Regs: the route that gives you only 130ms is via the SEA-ME-WE 3 undersea cable that runs from Perth to Singapore via Indonesia. This is a very low capacity cable, e.g. compared to Southern Cross, and it is significantly more expensive than other routes. There is also no redundancy as we found out a year or so ago when it broke and was out of action for a few months forcing all traffic to reroute.

There is a good chance your routes will never get back to what you had - the traffic via sea-me-we 3 may be too expensive for anything but commercial services now. I know that Microsoft (and probably others) pay extra to buy capacity on this cable and have their traffic (office 365, crm, azure etc) go via this optimal route.

So Microsoft peers at WIX/APE and provides routes to O365 etc in Singapore which is provided by Spark (which must be layer 2 access as it goes through MS L3 network), but for their GGI International they are not using this route? it does seem coincidental that they have just sold their IRU to Vocus and then this happens, presumably this does point to Spark now having limited capacity on this route and are now only allowing certain traffic over it.

http://en.wikipedia.org/wiki/SEA-ME-WE_3 is 480Gbps, so not much less than the SSCN cable was not so long ago. (more than enough to service NZ many times over)

Equinox IT

Cloud spending continues to surge globally, but most organisations haven’t made the changes necessary to maximise the value and cost-efficiency benefits of their cloud investments. Download the whitepaper From Overspend to Advantage now.

insane

3236 posts

Uber Geek

ID Verified

Trusted

#1144561 30-Sep-2014 21:57

pchs:

So Microsoft peers at WIX/APE and provides routes to O365 etc in Singapore which is provided by Spark (which must be layer 2 access as it goes through MS L3 network), but for their GGI International they are not using this route? it does seem coincidental that they have just sold their IRU to Vocus and then this happens, presumably this does point to Spark now having limited capacity on this route and are now only allowing certain traffic over it.

http://en.wikipedia.org/wiki/SEA-ME-WE_3 is 480Gbps, so not much less than the SSCN cable was not so long ago. (more than enough to service NZ many times over)

Last time I checked it was another provider that was providing L2 services to MS for that traffic. I can't remember how much capacity they were buying, but presumably MS manages traffic like any other network and could influence traffic flows if they wanted to by prepending AS numbers etc.

yitz

2074 posts

Uber Geek

#1144587 30-Sep-2014 23:16

pchs: it does seem coincidental that they have just sold their IRU to Vocus and then this happens, presumably this does point to Spark now having limited capacity on this route and are now only allowing certain traffic over it.

Can't be sure about voice, but they don't have any GGI (i.e. Internet) traffic over it anymore, their Perth border phbr2.global-gateway.net.nz no longer exists.

GGI used to peer with (most also hold IRUs on that segment of SMW3) Most SEA gaming traffic traversed the first two networks:
Starhub (singapore)
Singtel (singapore)
China Unicom (mainland china via hk)
Twgate (taiwan chunghwa enterprise)
CAT Telecom (thailand)
PLDT (philippines)
KDDI (japan)

SMW3 is 2.4 Gb SDH on that segment, can't do 10G so wouldn't say it's high capacity.

1 | 2