Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.
View this topic in a long page with up to 500 replies per page Create new topic
1 | 2 | 3 | 4 | 5
2006 posts

Uber Geek
+1 received by user: 761

Trusted

  Reply # 372229 24-Aug-2010 15:59
Send private message

Zeon: We lost:
- Internet with ICONZ through Araneo
- Backups vodafone data sticks
- International connecitity at home via Xnet

So did the APE stay online during the outage? I couldn't get out on the APE (but probably related to Araneo/Vodafone more than anything) nor was there any international.


Yea. None of our peers went down. We saw a bit of a drop off in APE traffic, but nothing I'd call serious.

262 posts

Ultimate Geek
Inactive user


  Reply # 372231 24-Aug-2010 16:01
Send private message

I think what Maurico is saying is that Vodafone were on their high horse about single points of failure on the XT network, yet they're no better with a single point of failure at APE.

2006 posts

Uber Geek
+1 received by user: 761

Trusted

  Reply # 372232 24-Aug-2010 16:01
Send private message

aperfectcircle: Isn't Vodafone the "reliable" network hmmm? Gotta love the excuses that it's "an hour outage".


Of their data network only. Caused by circumstances outside their control. That affected more than one provider.

Strange they didn't have some sort of backup/DR failover, but hardly comparable to XT's problems.


992 posts

Ultimate Geek
+1 received by user: 159

UberGroup

  Reply # 372244 24-Aug-2010 16:20
Send private message

A little apple and oranges here since XT was major voice outages and Vodafone was a minor Data outage. Had Vodafone's voice services gone down I would agree that it's a bit hypocritical




Most problems are the result of previous solutions...

All comment's I make are my own personal opinion and do not in any way, shape or form reflect the views of current or former employers unless specifically stated 

641 posts

Ultimate Geek
+1 received by user: 117


  Reply # 372247 24-Aug-2010 16:25
Send private message

What a great little cut throat world we live in.

When Telcom goes belly-up, the VF fans toss the $4it

When VF goes belly-up, the Telecom fans toss it :)

Who is keeping the score?

4 - 1??

2078 posts

Uber Geek
+1 received by user: 230

Subscriber

  Reply # 372249 24-Aug-2010 16:28
Send private message

muppet:
aperfectcircle: Isn't Vodafone the "reliable" network hmmm? Gotta love the excuses that it's "an hour outage".


Of their data network only. Caused by circumstances outside their control. That affected more than one provider.

Strange they didn't have some sort of backup/DR failover, but hardly comparable to XT's problems.



Using that same logic then Telecom could say the issues with the XT network were caused by issues outside their control.

In both cases the backup/DR systems either didn't work or didn't kick in quick enough.  The only differences here are the duration of the outage and the type of traffic affected.

Meow
7881 posts

Uber Geek
+1 received by user: 3913

Moderator
Trusted
Lifetime subscriber

  Reply # 372250 24-Aug-2010 16:31
Send private message

Zippity: What a great little cut throat world we live in.

When Telcom goes belly-up, the VF fans toss the $4it

When VF goes belly-up, the Telecom fans toss it :)

Who is keeping the score?

4 - 1??


Something like that :P - Except people tend to complain about Vodafone quite a bit around here, more people seem to complain about Telecom. YOU CAN NOT WIN! >.<

The APE didn't go down during this outage, it was up the whole time, but as Muppit said, there was a drop of traffic. We were just running on UPS power for a hour or so :)

 




2696 posts

Uber Geek
+1 received by user: 688


  Reply # 372258 24-Aug-2010 16:42
Send private message

full story from the herald,


"SkyCity was evacuated this afternoon as a precaution after smoke was seen coming from a plant room. Photo / NZPA


Vodafone internet users were cut off today after a power outage at SkyCity.

Smoke was seen coming from a plant room on the Federal Street side of the SkyCity complex this afternoon, so around 200 staff and patrons on the gaming floor and at the bars and restaurants were evacuated and the fire service called.

The SkyTower itself was not evacuated, but due to the power outage Vodafone and other web providers which transmit from the tower were cut off.

Vodafone spokesman Paul Brislen said as the complex had been evacuated, no one was allowed in to switch on a back-up generator to restore the connection.

Vodafone broadband and data users had no access for about an hour.

Services were now running as usual, he said.

SkyCity staff and patrons have now been allowed back into the building and technicians are working on finding out where the smoke came from"




Common sense is not as common as you think.


2006 posts

Uber Geek
+1 received by user: 761

Trusted

  Reply # 372259 24-Aug-2010 16:43
Send private message

graemeh: Using that same logic then Telecom could say the issues with the XT network were caused by issues outside their control.

Fair call. Personally I think a fire is much more serious and uncontrollable (force majeure) than any software glitch coded by your supplier, which could be found by testing and review and those sort of things. But you could argue the toss on that I guess.

graemeh: In both cases the backup/DR systems either didn't work or didn't kick in quick enough. The only differences here are the duration of the outage and the type of traffic affected.

I disagree. The difference as I see it is a full network outage (On XT, nothing worked) vs a partial outage (only Data). A large majority of customers wouldn't have noticed the outage at all.
But again, you could argue that if you're a data customer, XT vs VFNZ outages are equally impacting to you.
If it happens another 3 times though, I think they'll be mocked/pitied as much as XT was, because of the backup situation.




It looks like I'm using an adblocker. I should consider whitelisting Geekzone in my adblocker or a subscription. The Quick Reply box will appear for me when Geekzone is whitelisted. Hooray for me! If I want to reply to this topic I should click on Compose Reply.


2078 posts

Uber Geek
+1 received by user: 230

Subscriber

  Reply # 372261 24-Aug-2010 16:48
Send private message

muppet:
graemeh: Using that same logic then Telecom could say the issues with the XT network were caused by issues outside their control.

Fair call. Personally I think a fire is much more serious and uncontrollable (force majeure) than any software glitch coded by your supplier, which could be found by testing and review and those sort of things. But you could argue the toss on that I guess.


The issue is not the fire, the issue is that there was no effective backup in place.

muppet:
graemeh: In both cases the backup/DR systems either didn't work or didn't kick in quick enough. The only differences here are the duration of the outage and the type of traffic affected.

I disagree. The difference as I see it is a full network outage (On XT, nothing worked) vs a partial outage (only Data). A large majority of customers wouldn't have noticed the outage at all.
But again, you could argue that if you're a data customer, XT vs VFNZ outages are equally impacting to you.
If it happens another 3 times though, I think they'll be mocked/pitied as much as XT was, because of the backup situation.


XT was NOT a full network outage.  My phone started working as soon as I got off the plane in Auckland, I just couldn't use it at home in Wellington to chase the taxi company Wink

You are right though, the Vodafone outage affected less customers.

970 posts

Ultimate Geek

Trusted
Vodafone

  Reply # 372291 24-Aug-2010 17:33

Hi all,

we're still investigating just what happened and why - the facility isn't directly operated by Vodafone (or any of the local telcos as I understand it) so we're trying to find out why our back-up generator didn't kick in directly that the first power feed was cut off. I've been told that the back-up supply did indeed kick in but was switched off to allow the fire service guys in. However, that's yet to be confirmed.

Please do not misconstrue this as me blaming anyone for anything - I'm simply sharing the information I have to hand.

Our internet access was cut for about 45-50 minutes and would have affected customers nationwide. Voice and TXT were not affected.

Our plan is to build additional redundancy into the system by opening a second centre at the Telecom Mayoral Drive exchange - that's still being put together.

Paul




Paul Brislen
Head of Corporate Communications
Vodafone

http://forum.vodafone.co.nz


189 posts

Master Geek


  Reply # 372340 24-Aug-2010 19:34
Send private message

Spawneh: 
an hour outage which was caused by a power surge/spike Vs multiple outages for multiple hours. I dont think it benefits this post trying to compare the two personally, as I think they are far different.


Yes, very different. I'd argue this Vodafone outage is far more embarrassing than XT's outages from a technical perspective. Honestly, this is the internet we're talking about. It was designed from the ground up to explicitly avoid localised failures taking out massive parts of the network.

Frankly it's pretty negligent that a large carrier such as Vodafone has a single point of failure like this in their IP network.

Voice networks are a very different beast, never really designed for failure in the same way IP networks have been. I don't know all the technical details behind the XT failures, but this Vodafone outage seems a total rookie mistake, trivial to avoid.

But who knows, I may be totally wrong, and it may have just been bad timing for Vodafone, maybe their alternate routes were down for some sort of maintenance or something ;-) 

637 posts

Ultimate Geek
+1 received by user: 2

Trusted

  Reply # 372501 25-Aug-2010 10:19
Send private message

That is pretty poor - there is no excuse for being 100% dependent on the Sky Tower for Internet connectivity. It is a facility which is known and proven to be unreliable - particularly for power.

Also, the Sky Tower and APE are not 1:1 related: as muppet pointed out, APE is a L2 virtual instance on the Citylink PublicLAN infrastructure, and it appears anywhere that Citylink have service in Auckland, so it can easily survive an outage at Sky Tower. It also doesn't tend to have much to do with transit connectivity... Only peering.

2 posts

Wannabe Geek


  Reply # 372524 25-Aug-2010 10:51
Send private message

I guess it looks even worse as it comes during the middle of several other issues Vodafone is having with Latency (over a month), Connection Speed issues, red network issues (apparently unrelated), email unreliability (this has been ongoing for months but far worse recently), dodgy usage meters and the billing Fiasco. 

2078 posts

Uber Geek
+1 received by user: 230

Subscriber

  Reply # 372527 25-Aug-2010 10:56
Send private message

PaulBrislen: Our plan is to build additional redundancy into the system by opening a second centre at the Telecom Mayoral Drive exchange - that's still being put together.


Thanks for letting us all know.  I assume by now Telecom have gotten over their power shortages in Mayoral Drive :)

1 | 2 | 3 | 4 | 5
View this topic in a long page with up to 500 replies per page Create new topic

Twitter »

Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:



Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.