XT Network down (outage 14 DEC 2009)?

Forums › Spark New Zealand (including Skinny and BigPipe) › XT Network down (outage 14 DEC 2009)?

1 | 2 | 3 | 4 | 5 | 6 | 7 | ... | 14

69 posts

Master Geek
+1 received by user: 2

#282113 14-Dec-2009 09:08

Some of us don't have twitter, can you keep us updated via Geekzone......

Coffee loves you, it makes no discrimination by race, sex, age, social position or creed.
And is equally available to all. Pour yourself a cup today!

oldmaknz

536 posts

Ultimate Geek

#282116 14-Dec-2009 09:12

Latest from twitter:

XT Fault Update: Looking at resolution closer to 10am, will let you know further details once they come in ^AC

http://twitter.com/TelecomNZ/status/6637710609

stubbed

GIS Geek
149 posts

Master Geek
+1 received by user: 93

Trusted

Lifetime subscriber

#282122 14-Dec-2009 09:24

Is anyone from Telecom willing to give any technical explanation on what has caused this fault? I just can't understand how an outage at a single location, even a major one, can cause total network loss for the entire South Island and the South of the North Island?

Why is there no redundancy? I just can't imagine how anyone got away with designing a system that in the event of a single point of outage has the potential bring down the entire network with the potential to disrupt over a million people.

https://gisgeek.app

Oggie

69 posts

Master Geek
+1 received by user: 2

#282125 14-Dec-2009 09:30

Here here, or at least some sort of credit for the disruption's

Coffee loves you, it makes no discrimination by race, sex, age, social position or creed.
And is equally available to all. Pour yourself a cup today!

sbiddle

30853 posts

Uber Geek
+1 received by user: 10000

Retired Mod

Trusted

Biddle Corp

Lifetime subscriber

#282127 14-Dec-2009 09:32

stubbed: Is anyone from Telecom willing to give any technical explanation on what has caused this fault? I just can't understand how an outage at a single location, even a major one, can cause total network loss for the entire South Island and the South of the North Island?

Why is there no redundancy? I just can't imagine how anyone got away with designing a system that in the event of a single point of outage has the potential bring down the entire network with the potential to disrupt over a million people.

One can only presume it's an RNC failure. This is the resourse that controls all the cellsites (node B's) within a network.

freitasm

BDFL - Memuneh
81348 posts

Uber Geek
+1 received by user: 42712

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#282129 14-Dec-2009 09:33

It seems some sites are up and others being brought online soon.

Referral links: Quic Broadband (free setup code: R587125ERQ6VE) | Samsung | AliExpress | Wise | Sharesies

Support Geekzone by subscribing (browse ads-free), or making a one-off or recurring donation through PressPatron.

LEGAL video, movies and TV download and streaming options in New Zealand

Quic Broadband

Move to New Zealand's best fibre broadband service (affiliate link). Free setup code: R587125ERQ6VE. Note that to use Quic Broadband you must be comfortable with configuring your own router.

freitasm

BDFL - Memuneh
81348 posts

Uber Geek
+1 received by user: 42712

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#282131 14-Dec-2009 09:34

sbiddle: One can only presume it's an RNC failure. This is the resourse that controls all the cellsites (node B's) within a network.

While a network has mutliple RNC's there are no "backups" as such.

Interesting that every time I read about a XT outage the official word is "down south of Taupo".

What makes their network above Taupo more resilient?

Referral links: Quic Broadband (free setup code: R587125ERQ6VE) | Samsung | AliExpress | Wise | Sharesies

Support Geekzone by subscribing (browse ads-free), or making a one-off or recurring donation through PressPatron.

LEGAL video, movies and TV download and streaming options in New Zealand

zonky

62 posts

Master Geek

#282132 14-Dec-2009 09:36

I first noticed it down around 5:45am, FWIW.

stubbed

GIS Geek
149 posts

Master Geek
+1 received by user: 93

Trusted

Lifetime subscriber

#282133 14-Dec-2009 09:37

One can only presume it's an RNC failure. This is the resourse that controls all the cellsites (node B's) within a network.

That's fine, but why isn't there a backup? Surely if the Taupo site is responsible for controlling everything South of it in the country then the logical thing to do would be to have redundancy at this location.

https://gisgeek.app

kiwitrc

4123 posts

Uber Geek
+1 received by user: 833
Inactive user

#282136 14-Dec-2009 09:47

Just as I was about to fire up my new T Stick :)

Oggie

69 posts

Master Geek
+1 received by user: 2

#282137 14-Dec-2009 09:50

What's the latest on Twitter? ...... man, I need a coffee.

Coffee loves you, it makes no discrimination by race, sex, age, social position or creed.
And is equally available to all. Pour yourself a cup today!

Dell

Shop now for Dell laptops and other devices (affiliate link).

stubbed

GIS Geek
149 posts

Master Geek
+1 received by user: 93

Trusted

Lifetime subscriber

#282139 14-Dec-2009 09:52

Twitter:

# @nzmatts yep - all sites for XT south of taupo are down, currently working on it, some sites are back up, we're working through the rest ^AC 5 minutes ago from CoTweet in reply to nzmatts

# @JonoHale we currently have a network issue with XT users south of taupo unable to get reception, look at our updates for further info ^AC 6 minutes ago from CoTweet in reply to JonoHale

https://gisgeek.app

lambo

9 posts

Wannabe Geek

#282143 14-Dec-2009 09:55

See here if you can't get to Twitter for updates - http://www.telecom.co.nz/help/alerts

Oggie

69 posts

Master Geek
+1 received by user: 2

#282144 14-Dec-2009 09:56

Thanks, keep the updates coming guys.

Coffee loves you, it makes no discrimination by race, sex, age, social position or creed.
And is equally available to all. Pour yourself a cup today!

NealR

428 posts

Ultimate Geek
+1 received by user: 20

ID Verified

Trusted

Lifetime subscriber

#282148 14-Dec-2009 09:59

This is not a single element (server) failure. A failure earlier this morning had a cascade effect on a number of elements which resulted in the major issue we have now.

Luckily this did not affect customer service for the northern part of our network.

This was not a "single point of failure" issue.

FYI, the XT network is divided into two based on number of cell sites and number of customers.

The comments I write on this forum do not necessarily reflect the views of my employer and as such cannot be taken as official statements of my employer.

1 | 2 | 3 | 4 | 5 | 6 | 7 | ... | 14