Orcon DC down?

Forums › 2degrees (including Slingshot, Orcon, Flip, Stuff Fibre, MyRepublic, 2talk and Vocus) › Orcon DC down?

Zeon

3916 posts

Uber Geek

Trusted

#103624 12-Jun-2012 12:17

Hey guys,
Haven't been able toa ccess our colocated servers at Orcon for the last 15 mins from an Orcon fibre connection. Company downstairs lost all their servers at the orcon DC too.

What's going on?

Speedtest 2019-10-14

1 | 2

305 posts

Ultimate Geek

Trusted

2degrees

#639602 12-Jun-2012 12:27

Zeon: Hey guys,
Haven't been able toa ccess our colocated servers at Orcon for the last 15 mins from an Orcon fibre connection. Company downstairs lost all their servers at the orcon DC too.

What's going on?

We experienced a brief issue with a link at our datacentre - service has been restored now and is under observation.

Apologies for any inconvenience.

xpd

Geek @ Coastguard NZ
13765 posts

Uber Geek

Retired Mod

ID Verified

Trusted

Lifetime subscriber

#639612 12-Jun-2012 12:50

Didnt notice ours going down but then Im not on them all the time - is running now. Reminds me, must add Orcon DC to our alert system...

Gavin / xpd / FastRaccoon / Geek of Coastguard New Zealand

LinkTree

Zeon

3916 posts

Uber Geek

Trusted

#639616 12-Jun-2012 12:51

***RESOLVED***Loss of Secondary link at Northcote Datacentre

Update: Service has now been restored and is under observation

We have experienced a loss of our secondary link at our Northcote Datacentre. Some services are impacted, others will be running on reduced capacity.
Loss of secondary link at Datacentre

AFFECTED CUSTOMERS/SERVICES:Any services hosted at our datacentre
SYMPTOMS:Loss of connectivity
DURATION:25 minutes

CURRENT STATUS: Active: Resolved (Under observation)

TYPE: Unscheduled Outage

VENDOR: Orcon Internet

EVENT WINDOW: 2012-06-12 11:50:00 to 0000-00-00 00:00:00

Speedtest 2019-10-14

xpd

Geek @ Coastguard NZ
13765 posts

Uber Geek

Retired Mod

ID Verified

Trusted

Lifetime subscriber

#639641 12-Jun-2012 13:09

Hmm Im suppose to get those emails but didnt get it.....

Gavin / xpd / FastRaccoon / Geek of Coastguard New Zealand

LinkTree

detonate

45 posts

Geek

#639715 12-Jun-2012 15:09

Link between CRE2 and DSW2 died.

Services that are built redundantly weren't affected, sadly a lot of services are single homed for various reasons.
(ie a single service provider port or handover from an upstream, or single physical port in the colo, etc.)

Zeon

3916 posts

Uber Geek

Trusted

#640071 13-Jun-2012 10:48

detonate: Link between CRE2 and DSW2 died.

Services that are built redundantly weren't affected, sadly a lot of services are single homed for various reasons.
(ie a single service provider port or handover from an upstream, or single physical port in the colo, etc.)

Can the distribution switch not reroute through CRE1 in case of those types of failures? Or is only possible if we were to run say BGP with Orcon against CRE1 and CRE2 via each of the 2 switches?

Speedtest 2019-10-14

detonate

45 posts

Geek

#640080 13-Jun-2012 10:51

Exactly, as above, if the service is built with redundancy, then it would've failed over fine.

ie. with VRRP or multiple BGP sessions as you mentioned.

Lenovo

Shop now for Lenovo laptops and other devices (affiliate link).

Zeon

3916 posts

Uber Geek

Trusted

#642685 18-Jun-2012 17:48

Just happened again. WTF is going on?!

Speedtest 2019-10-14

martyyates

68 posts

Master Geek

#642691 18-Jun-2012 18:03

Likewise. on Orcon BizOnline fibre in the CBD. Out since 1745. Phone lines to many places are overloaded too. Orcon's site also down; nice!

wazzageek

1093 posts

Uber Geek

ID Verified

Trusted

Lifetime subscriber

#642692 18-Jun-2012 18:03

Zeon: Just happened again. WTF is going on?!

I've not heard anything yet - must be a biggy as the website is down, phones are claiming congestion, no updates on twitter yet ... (@orcon)

Viaan

2 posts

Wannabe Geek

#642694 18-Jun-2012 18:05

I have been having problems from ADSL connection as well.
-Email down
-SIP Down
-Website down

Sounds like a major meltdown

Beccara

1469 posts

Uber Geek

ID Verified

#642699 18-Jun-2012 18:11

Whatever it is it's quite major, Sounds like the apocalypse just his Orcon and Orcon provided Kordia services

Most problems are the result of previous solutions...

All comment's I make are my own personal opinion and do not in any way, shape or form reflect the views of current or former employers unless specifically stated

martyyates

68 posts

Master Geek

#642701 18-Jun-2012 18:13

Finally a tweet from them about 30mins after the event:
Orcon ‏@OrconWe are experiencing issues across multiple services and our contact centre is unreachable. Engineers are working to resolve ASAP. Apologies.

Zeon

3916 posts

Uber Geek

Trusted

#642706 18-Jun-2012 18:18

It's funny but our internet connection via Telstra Fibre is still up. Perhaps a power outage but their core routers are on seperate UPS?

Speedtest 2019-10-14

bameron

305 posts

Ultimate Geek

Trusted

2degrees

#642708 18-Jun-2012 18:21

Sorry for the delay guys - as the tweet mentioned we're having issues across multiple services at the moment and our contact centre is unreachable at this time.

I have no further details at this stage however our engineers are definitely aware of these problems and working to fix them ASAP. Sincere apologies for the disruption.

Cheers,

Cam

1 | 2