National & International with 2 different Carriers

Forums › New Zealand Broadband › National & International with 2 different Carriers

Zeon

3916 posts

Uber Geek

Trusted

#76475 2-Feb-2011 23:23

Hey guys,
As I've posted before, we have just moved into the Orcon datacenter so we now have a 10mbps fibre and half rack colcoation with them. Speeds have been great and the service is very reliable. The only problem is the cost of international. Let's take a look at a couple of examples:

20GB: $100/month ($5/GB overage)
50GB: $200/month ($5/GB overage)
256kbps CIR / 1mbps PIR: $450/month

I saw that Hosting Direct is selling 1mbps CIR international for $99/month:
http://hd.net.nz/carrier-grade-bandwidth

Anyway I have just spoken to the guy from there, Ben on the phone (he got banned from here as some may remember). Because hosting direct itself is Orcon's biggest customer, he thinks we can setup something along these lines:

We maintain our current IPv4 and IPv6 subnets with Orcon

We have 2 feeds into our cabinet, one for Orcon and one for Hosting Direct

Hosting Direct and Orcon figure out routing of that subnet between them so Orcon route our national and Hosting Direct route international.

We have a router with OSPF running that plugs into each of the two sockets, Orcon advertises NZ routes to and Hosting Direct advertises international routes

It seems pretty complicated but could save serious $$ on a monthly basis.

What are people's thoughts or do they even understand how this works (I'm struggling myself).

Speedtest 2019-10-14

1 | 2

4066 posts

Uber Geek

Trusted

Snowflake

#434576 2-Feb-2011 23:52

technically it might be possible, although your routing table for international vs national would possibly require manual maintenance and some guesswork to determine which is which.

Would the extra costs in setting up and maintaining the router - equipment, rack costs, skilled people to configure it - negate the savings you might make? Also, i expect that orcon would probably charge some sort of setup fee if you're using their subnets and messing with the routing of them.

Does Hosting Direct buy their internation bandwidth from Orcon, or do they have their own transit? If they're buying it off of orcon, then perhaps you might be able to get some better pricing out of orcon?

Sales Engineer
Snowflake
www.snowflake.com

about.me/nzregs
Twitter: @nzregs

Beccara

1469 posts

Uber Geek

ID Verified

#434581 3-Feb-2011 00:00

Well for a start "CIR International" for $99/mb is a joke, They'd have to be buying 500mbit+ and selling it at cost to their clients to hit that price point (I've seen the rackrate cards for a couple of International Transit providers)

Next up, You're going to need your own allocation and ASN to really do this well, You can't take a /26 or /27 from Orcon and expect HD.net to announce that internationally, Alot of places wont accept anything small than a /24 and even then you'll run into issues where a International host will see your /26 as part of a /21 or something coming from Orcon and use that path always rather than the other /26 path from HD.net except now your still using Orcon's bandwidth.

It's late and I'm tired but long story short is yes you can do it, It's a setup nightmare that's going to lead to async routing hell and tie up alot of engineers time on both Orcon and HD.net's side to put in place which will negate any saving's you may get. What your talking about is multi-homing in a crude sense and it only really works with your own IP space and ASN

Ragnor

8218 posts

Uber Geek

Trusted

#434584 3-Feb-2011 00:10

That's Hosting Direct for you, their business model seems to be very sharp prices low overheads (it's a very small team from what I gather).

You can see from their pricing for other services lik domain name registration, adsl and hosting they price their services with wafer thin margins.

Great for consumers and forces prices down in general but I would weigh that up with the expectation that you will not get same level of service/support as you would get from the more expensive options like Maxnet, ICONZ etc.

Often you do get what you pay for in terms of quality of service.

With two upstream carriers you should look at getting an ASN... it's a weird situation where HD's international service is actually via Orcon though.

wjw

162 posts

Master Geek

#434642 3-Feb-2011 08:41

Beccara: Well for a start "CIR International" for $99/mb is a joke, They'd have to be buying 500mbit+ and selling it at cost to their clients to hit that price point (I've seen the rackrate cards for a couple of International Transit providers)

Next up, You're going to need your own allocation and ASN to really do this well, You can't take a /26 or /27 from Orcon and expect HD.net to announce that internationally, Alot of places wont accept anything small than a /24 and even then you'll run into issues where a International host will see your /26 as part of a /21 or something coming from Orcon and use that path always rather than the other /26 path from HD.net except now your still using Orcon's bandwidth.

It's late and I'm tired but long story short is yes you can do it, It's a setup nightmare that's going to lead to async routing hell and tie up alot of engineers time on both Orcon and HD.net's side to put in place which will negate any saving's you may get. What your talking about is multi-homing in a crude sense and it only really works with your own IP space and ASN

What he said, you will need your OWN /24 and your own ASN these could be supplied by either Orcon or Hosting Direct. The best setup would be:

Orcon Deliver you a BGP feed with a full national table, about 1600 routes from memory
You setup a static default route to hosting direct.
You advertise your /24 to both Orcon and Hosting Direct.

Zeon

3916 posts

Uber Geek

Trusted

#434761 3-Feb-2011 12:16

Hey guys, thanks for the feedback. Yea that was the first thing I said to HD that do I need my own ASN and BGP session to do this. Ben initially said we could just route everything through them which is possible at the datacenter as we can just run a cable from their rack to ours but would be tricky for our office internet connection. Also I wouldn't rely on them with national which is vital to our business - international isn't as important so we can live with more risk on that front.

So we can get our own ASN from either Orcon or HD? I thought we had to go directly to APNIC for that? Also while we conceivably could use a /24 in a couple of years I'm not sure if we could qualify for that at the moment.

Yea just seems really complicated without an ASN, especially with our office internet connection too. I'll talk to our account manager at Orcon about getting a better deal as their pricing is over 100% of other even reputable suppliers e.g. ICONZ so they may need to better their game.

If things stagnate on that front I may take this further. Would be pretty whack though for a company of only 30 staff to have its own ASN, /24 and BGP session. Anyone know what APNIC would charge us for memebrship etc.?

Speedtest 2019-10-14

wjw

162 posts

Master Geek

#434789 3-Feb-2011 12:58

So how does your office hook in?

webwat

2036 posts

Uber Geek

Trusted

#434804 3-Feb-2011 13:21

$99 per meg suggests that hes got a contract for several megs and desperate to sell them, perhaps after losing some clients or having yet another dispute, which wouldn't suprise me in this case.

Time to find a new industry!

Equinox IT

Cloud spending continues to surge globally, but most organisations haven’t made the changes necessary to maximise the value and cost-efficiency benefits of their cloud investments. Download the whitepaper From Overspend to Advantage now.

Zeon

3916 posts

Uber Geek

Trusted

#434817 3-Feb-2011 13:44

wjw: So how does your office hook in?

We have a 10mbps down / 10mbps+ fibre connection with orcon (Telstra fibre). I presume Telstra give this its own VLAN which Orcon then hook into.

And yea seems pretty cheap. He said he had around 500mbit with 270mbps unallocated so could be he just wants to get something for it. I checked out APNIC's fees:

http://www.apnic.net/services/become-a-member/how-much-does-it-cost

We would have to be saving an awful lot to make it worthwhile to get our own ASN and IP space.

Speedtest 2019-10-14

Regs

4066 posts

Uber Geek

Trusted

Snowflake

#434822 3-Feb-2011 13:59

Zeon:
wjw: So how does your office hook in?

We have a 10mbps down / 10mbps+ fibre connection with orcon (Telstra fibre). I presume Telstra give this its own VLAN which Orcon then hook into.

And yea seems pretty cheap. He said he had around 500mbit with 270mbps unallocated so could be he just wants to get something for it. I checked out APNIC's fees:

http://www.apnic.net/services/become-a-member/how-much-does-it-cost

We would have to be saving an awful lot to make it worthwhile to get our own ASN and IP space.

well, you wouldnt get your own IP space anyway... there's none left!

Sales Engineer
Snowflake
www.snowflake.com

about.me/nzregs
Twitter: @nzregs

nate

6473 posts

Uber Geek

Retired Mod

Trusted

Lifetime subscriber

#434853 3-Feb-2011 14:33

If I can jump in, this sounds like a very mickey mouse solution. You've got with a good data centre, good support yet you're willing to risk it to save some dollars.

Not sure what the saving is, but for me, it would have to be very significant to risk your data connectivity. I wouldn't risk it, and go with Orcon entirely.

/my 2 cents.

Zeon

3916 posts

Uber Geek

Trusted

#434889 3-Feb-2011 15:38

nate: If I can jump in, this sounds like a very mickey mouse solution. You've got with a good data centre, good support yet you're willing to risk it to save some dollars.

Not sure what the saving is, but for me, it would have to be very significant to risk your data connectivity. I wouldn't risk it, and go with Orcon entirely.

/my 2 cents.

Yea I'm having second thoughts after figuring out the complexity and as you say, the risk. TBH I think its only workable if we have our own ASN and subnet which belongs to us as we can use standard BGP then rather than some cut together solution.

Not sure exactly who HD is targeting with that service considering the costs of having different providers for national and international.

Speedtest 2019-10-14

wjw

162 posts

Master Geek

#434914 3-Feb-2011 16:00

If HD are primarily a hosting provider most of their traffic usage should be in the opposite direction to a normal internet service. So they should have spare capacity

freitasm

BDFL - Memuneh
79250 posts

Uber Geek

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#434927 3-Feb-2011 16:25

Zeon: Because hosting direct itself is Orcon's biggest customer, he thinks we can setup something along these lines:

We maintain our current IPv4 and IPv6 subnets with Orcon

We have 2 feeds into our cabinet, one for Orcon and one for Hosting Direct

Hosting Direct and Orcon figure out routing of that subnet between them so Orcon route our national and Hosting Direct route international.

We have a router with OSPF running that plugs into each of the two sockets, Orcon advertises NZ routes to and Hosting Direct advertises international routes

Is Orcon a carrier neutral datacentre? I didn't it would be... If it is then fine, otherwise, how do you think Orcon would feel about "Hosting Direct and Orcon figure out routing of that subnet between them so Orcon route our national and Hosting Direct route international."

They wouldn't do it for free, surely? And lose revenue in the process?

If you want a carrier neutral datacentre then perhaps should look at Data Centre.

mercutio

1392 posts

Uber Geek

#437513 9-Feb-2011 21:28

I'd say just get all your bandwidth through hosting direct for your servers if you want to save money and risk your connectivity on them.

It simplifys things. If you're sharing the same bandwidth segment for office, then you could run a proxy at data centre for web and cut your usage a bit down.

Another option would be to go assymetric and only send outgoing via hosting direct. If doing this I'd suggest doing BGP on a private AS number sending a full route table, in case they go down. That way you can just keep your default route at orcon and not accept default route from hosting direct.

That said, you have got another option too. Try talking to your Orcon account manager and see if you can get some sharper pricing.

You could also consider carrier-neutral datacentre. But I'd imagine you'd only do that if Orcon dropped the ball considering you just moved into orcon dc...

mercutio

1392 posts

Uber Geek

#437517 9-Feb-2011 21:34

Zeon:
nate: If I can jump in, this sounds like a very mickey mouse solution. You've got with a good data centre, good support yet you're willing to risk it to save some dollars.

Not sure what the saving is, but for me, it would have to be very significant to risk your data connectivity. I wouldn't risk it, and go with Orcon entirely.

/my 2 cents.

Yea I'm having second thoughts after figuring out the complexity and as you say, the risk. TBH I think its only workable if we have our own ASN and subnet which belongs to us as we can use standard BGP then rather than some cut together solution.

What kind of traffic are you trying to do? And what kind of level of reliability are you looking for?

Not sure exactly who HD is targeting with that service considering the costs of having different providers for national and international.

Anyone who'll bite I imagine.

1 | 2