[WXC/VFX] XNET / VFX Outage 07 Sept 2008

Forums › One New Zealand (including Vodafone, WxC, Farmside) › [WXC/VFX] XNET / VFX Outage 07 Sept 2008

maverick

3594 posts

Uber Geek

Trusted

WorldxChange

#25947 7-Sep-2008 16:26

There was a major power problem with the supply to the Auckland Exchange at 13.28 this afternoon and emergency services were called , the building was required to be evacuated and due to the emergency services being on site we also needed to confirm that the power facilities were determined to be in a safe condition before service restoration procedures could begin.

The majority of Services were restored at 15:00, work is ongoing to restore the remaining services.

Yes I am a employee of WxC (My Profile) ... but I do have my own opinions as well

https://www.facebook.com/wxccommunications

1 | 2 | 3

675 posts

Ultimate Geek

Trusted

#162829 7-Sep-2008 16:43

Thanks for the update Mav

CRKD

19 posts

Geek

#162836 7-Sep-2008 16:46

When you say "Auckland Exchange", you mean the WXC Datacentre / Office?

Sounds like a electrical fire / fault in one of the power circuits feeding your racks? Did you not have Inergen / Seconday UPS's on core network servers and routers?

andrewcnz

961 posts

Ultimate Geek

#162837 7-Sep-2008 16:47

Hi can anything be done to make it easy to find out what is going on? Can all the contact numbers have a recorded message like "We have a problem,yes are sorting it! No it's not just you its everyone!"Had to use my N95 to check the forum goto be the best place to find you ain't alone with issues!

CRKD

19 posts

Geek

#162840 7-Sep-2008 16:49

andrewcnz: Hi can anything be done to make it easy to find out what is going on? Can all the contact numbers have a recorded message like "We have a problem,yes are sorting it! No it's not just you its everyone!"Had to use my N95 to check the forum goto be the best place to find you ain't alone with issues!

Change ISP perhaps.

phindmarsh

117 posts

Master Geek

#162843 7-Sep-2008 16:52

CRKD, all you have had to say about this outage is sarcasm and negativeness. No one appreciates you trolling around. Why not offer something constructive or not offer anything at all.

xpd

Geek @ Coastguard NZ
13767 posts

Uber Geek

Retired Mod

ID Verified

Trusted

Lifetime subscriber

#162845 7-Sep-2008 16:53

Do you want to be the one to sit round in a potentially burning building to setup a voicemail mesage ?

We used to do the same at IHUG and QSI whenever there was an alarm, just hang up the phones and leave. At EDS we had a system the manager could activate that just redirected all the phones to the Australian office.....

WXC/Xnet made the right decsision, worry about the consequences later in this case.

All the users screaming because Xnet were offline for a few hours need to get out more.... its their first major outage (since Ive been with them) in months and could not be avoided.

Gavin / xpd / FastRaccoon / Geek of Coastguard New Zealand

LinkTree

andrewcnz

961 posts

Ultimate Geek

#162846 7-Sep-2008 16:54

CRKD:
andrewcnz: Hi can anything be done to make it easy to find out what is going on? Can all the contact numbers have a recorded message like "We have a problem,yes are sorting it! No it's not just you its everyone!"Had to use my N95 to check the forum goto be the best place to find you ain't alone with issues!

Change ISP perhaps.

umm well, I don't need the wife sayin the phones out Again! Will have to think about it.

Sharesies

Trade NZ and US shares and funds with Sharesies (affiliate link).

msian

93 posts

Master Geek

#162847 7-Sep-2008 16:55

i just got back internet! missed alot of important emails and online discussions due to this. I hope we will be credited/compensated for this! very frustrating experience. Wanted to destroy my router as I thought that was what the problem was at one point of time. Sighhh

JayADee

2148 posts

Uber Geek

#162850 7-Sep-2008 16:56

Thanks for letting us know what happened. First thing the man asked, "Are they going to tell us what happened?" so upon finding the xnet home page down, of course I came here. He is somewhat mollified now. (not completely, but somewhat). :)

andrewcnz

961 posts

Ultimate Geek

#162851 7-Sep-2008 16:58

xpd: Do you want to be the one to sit round in a potentially burning building to setup a voicemail mesage ?

We used to do the same at IHUG and QSI whenever there was an alarm, just hang up the phones and leave. At EDS we had a system the manager could activate that just redirected all the phones to the Australian office.....

WXC/Xnet made the right decsision, worry about the consequences later in this case.

All the users screaming because Xnet were offline for a few hours need to get out more.... its their first major outage (since Ive been with them) in months and could not be avoided.

I don't expect anyone to hang around if there is a fire. But there must be an easy way to either hit one button on the way out to put a basic pre recorded message up. Or remote access with a laptop or cell phone?

boby55

1539 posts

Uber Geek

Trusted

#162852 7-Sep-2008 17:00

Ive been with xnet comming up 8 months and this is the first mager fault we have had

BlakJak

1275 posts

Uber Geek

Trusted

#162854 7-Sep-2008 17:02

xpd:

All the users screaming because Xnet were offline for a few hours need to get out more.... its their first major outage (since Ive been with them) in months and could not be avoided.

To be fair, not entirely true on my part...

And it was a bit suprising to not even have the phones be answered by 'something' - but that speaks volumes in itself, so I figured it must've been big and left it alone for a while.

Came back after being out for a bit and noted my DSL came back, website was still down until a few moments ago.

Website is now up but Network Status screen doesn't yet explain what happened. Geekzone FTW, I guess.

BJ.

(PS: Previous ISPs ive had exposure to have at least had an emergency divert-pilot-phone-to-cellphone option, and it's not hard to set a voicemail on a cellphone...)

No signature to see here, move along...

CRKD

19 posts

Geek

#162857 7-Sep-2008 17:03

phindmarsh:
CRKD, all you have had to say about this outage is sarcasm and negativeness. No one appreciates you trolling around. Why not offer something constructive or not offer anything at all.

Ok, no problem, go tell WXC to get a second DC, and co-locate the RAIDUS servers and routers that handle DSL and Voice authentication / traffic, so customers aren’t affected by their repetitively poor operations management, and put secondary inline UPS's on critical servers, and install inergen like the leading ISP's.

sbiddle

30853 posts

Uber Geek

Retired Mod

Trusted

Biddle Corp

Lifetime subscriber

#162858 7-Sep-2008 17:03

andrewcnz:
xpd: Do you want to be the one to sit round in a potentially burning building to setup a voicemail mesage ?

We used to do the same at IHUG and QSI whenever there was an alarm, just hang up the phones and leave. At EDS we had a system the manager could activate that just redirected all the phones to the Australian office.....

WXC/Xnet made the right decsision, worry about the consequences later in this case.

All the users screaming because Xnet were offline for a few hours need to get out more.... its their first major outage (since Ive been with them) in months and could not be avoided.

I don't expect anyone to hang around if there is a fire. But there must be an easy way to either hit one button on the way out to put a basic pre recorded message up. Or remote access with a laptop or cell phone?

That's all fine if the machine that would/could play you such a message was still running!

exportgoldman

1202 posts

Uber Geek

Trusted

#162859 7-Sep-2008 17:05

andrewcnz:
I don't expect anyone to hang around if there is a fire. But there must be an easy way to either hit one button on the way out to put a basic pre recorded message up. Or remote access with a laptop or cell phone?

Like a Big Red Button? Hmmmmm... Always wondered what happened if you pressed that.

Tyler - Parnell Geek - iPhone 3G - Lenovo X301 - Kaseya - Great Western Steak House, these are some of my favourite things.

1 | 2 | 3