Enable: NETWORK OUTAGE NOTIFICATION – Update 15:30, 14 March 2016

Forums › New Zealand Broadband › Enable: NETWORK OUTAGE NOTIFICATION – Update 15:30, 14 March 2016

Beccara

990 posts

Ultimate Geek
+1 received by user: 157

UberGroup

Topic # 193537 14-Mar-2016 15:42

One person supports this post

FYI:

http://www.enable.net.nz/about-enable/news-and-media-centre/network-outage-notification-14-march-2016.html

Update 15:30

Technicians are onsite and workng to restore services to customers in the broader St Albans area, and Rangiora, Kaiapoi and Woodend. Restoration of services is expected by 17:00.

We continue to assess service restoration requirements for other parts of Christchurch - and will provide updated timeframes for restoration as soon as the information is available.

Update 14:00

Enable has experienced a major network outage impacting a significant number of customers across parts of Christchurch, and in Rangiora, Kaiapoi and Woodend.

The outage is a result of a third-party contractor damaging underground cable.

Enable is working to understand the extent of the outage – including reach and number of customers impacted, and the timeframe for resolution.

Repair resources have been mobilised.

Enable is working closely with its retail service providers to provide them with further details as these become available.

Most problems are the result of previous solutions...

All comment's I make are my own personal opinion and do not in any way, shape or form reflect the views of current or former employers unless specifically stated

1 | 2

211 posts

Master Geek
+1 received by user: 69

Reply # 1513229 14-Mar-2016 18:57

Update 17:30

Services have now been restored to customers in Rangiora and Woodend.

Customers still impacted by the outage are in central and sourthern parts of Christchurch. With restoration work underway for these customers - with this work expected to be completed early this evening.

Update 17:15

Services have now been restored to customers in Kaiapoi.

We continue to work on restoration to other areas - with all services expected to be restored this evening.

Update 16:10

Services have now been restored to almost all customers in the broader St Albans area.

We continue to work on restoration to other areas.

robcreid

211 posts

Master Geek
+1 received by user: 69

Reply # 1513237 14-Mar-2016 19:28

Seems like quite a big area to be taken out by one stray contractor. I hope they are building enough redundancy into their network.

jnimmo

What does this tag do
966 posts

Ultimate Geek
+1 received by user: 200

Subscriber

Reply # 1513240 14-Mar-2016 19:40

One person supports this post

It is indeed concerning, I assumed with the amount of fibre they've got running around the city that they would have better redundancy in place

robcreid

211 posts

Master Geek
+1 received by user: 69

Reply # 1513274 14-Mar-2016 20:08

Update 19:10

We are continuing to work to restore services to the remaing customers impacted by the outage - largely customers in central and sourthern parts of Christchurch. We now expect to have restoration complete to almost all customers by 22:00 tonight.

Mspec

1119 posts

Uber Geek
+1 received by user: 388

Subscriber

Reply # 1513300 14-Mar-2016 21:05

We lost connection from midday until 8pm tonight, was a major problem for work as all our phones went dead.

Ding Ding Ding Ding Ding : Ice cream man , Ice cream man

chevrolux

3562 posts

Uber Geek
+1 received by user: 1307

Subscriber

Reply # 1513332 14-Mar-2016 21:57

jnimmo:

It is indeed concerning, I assumed with the amount of fibre they've got running around the city that they would have better redundancy in place

Yep quite concerning. We had people calling us from Addington/Sydnenham area too without service. For that bigger an area to be affected I would of thought it would have to be more than just one case of spade fade.

Paul1977

2317 posts

Uber Geek
+1 received by user: 655

Reply # 1513500 15-Mar-2016 09:15

Update 09:00, 15 March

Most of the remaining services were restored by 5am.

Less than 10 services are still impacted and we are working to restore these this morning.

Well, my home internet is still down

EDIT: In fact, mine didn't go down until late last night after Enable reported they had almost everyone back up!

noroad

371 posts

Ultimate Geek
+1 received by user: 203

Trusted

Reply # 1513535 15-Mar-2016 09:59

One person supports this post

chevrolux:

jnimmo:

It is indeed concerning, I assumed with the amount of fibre they've got running around the city that they would have better redundancy in place

Yep quite concerning. We had people calling us from Addington/Sydnenham area too without service. For that bigger an area to be affected I would of thought it would have to be more than just one case of spade fade.

Dark Fiber services don't have redundancy, you build that yourself by buying diverse fibers. GPON is not a redundant service in any way, a single Central office (OLT) can service end customers (ONT's) 15+ KM away if desired. So a fiber cut that hits a feeder between the Central office and other areas can easily cause Dark Zones (no Internet, no mobile (if your mobile provider is using GPON backhaul)). Welcome to the new world! (yes, this is less diverse than a ring fed roadside cabinet). This is why a large business who wants diversity will still need deliberately planned diverse routes costing several hundred or a few thousand dollars, you don't get last mile diversity with a $100 GPON UFB connection, that's just not financially viable. GPON is cheap and fast, but not at all diverse in the last mile.

Paul1977

2317 posts

Uber Geek
+1 received by user: 655

Reply # 1513562 15-Mar-2016 10:32

Paul1977:

Update 09:00, 15 March

Most of the remaining services were restored by 5am.

Less than 10 services are still impacted and we are working to restore these this morning.

Well, my home internet is still down

EDIT: In fact, mine didn't go down until late last night after Enable reported they had almost everyone back up!

Hooray, I'm back up.

chevrolux

3562 posts

Uber Geek
+1 received by user: 1307

Subscriber

Reply # 1513663 15-Mar-2016 12:42

noroad:

chevrolux:

jnimmo:

It is indeed concerning, I assumed with the amount of fibre they've got running around the city that they would have better redundancy in place

Yep quite concerning. We had people calling us from Addington/Sydnenham area too without service. For that bigger an area to be affected I would of thought it would have to be more than just one case of spade fade.

Dark Fiber services don't have redundancy, you build that yourself by buying diverse fibers. GPON is not a redundant service in any way, a single Central office (OLT) can service end customers (ONT's) 15+ KM away if desired. So a fiber cut that hits a feeder between the Central office and other areas can easily cause Dark Zones (no Internet, no mobile (if your mobile provider is using GPON backhaul)). Welcome to the new world! (yes, this is less diverse than a ring fed roadside cabinet). This is why a large business who wants diversity will still need deliberately planned diverse routes costing several hundred or a few thousand dollars, you don't get last mile diversity with a $100 GPON UFB connection, that's just not financially viable. GPON is cheap and fast, but not at all diverse in the last mile.

I guess I never really thought about it that way.

We had a couple of customers that were quite happy in Kaiapoi as we had installed a DSL circuit for redundancy - we got a few more calls from people wanting that sort of back up today haha.

noroad

371 posts

Ultimate Geek
+1 received by user: 203

Trusted

Reply # 1513683 15-Mar-2016 13:04

jnimmo:

I guess I never really thought about it that way.

We had a couple of customers that were quite happy in Kaiapoi as we had installed a DSL circuit for redundancy - we got a few more calls from people wanting that sort of back up today haha.

You are not alone, we are still in the honeymoon period where people are so happy they can now get affordable, fast and mostly reliable fibre that few realize its got its own design implications. The first time we have half a mid sized city go dark due to the reliance on a single OLT / Fibre path it will be interesting.

robcreid

211 posts

Master Geek
+1 received by user: 69

Reply # 1513697 15-Mar-2016 13:35

Do any of the LFCs release details of their fibre network layouts? Like you can see maps of Chorus cabinets for the copper network.

Darren0

70 posts

Master Geek
+1 received by user: 12

Reply # 1513781 15-Mar-2016 15:18

robcreid:

Do any of the LFCs release details of their fibre network layouts? Like you can see maps of Chorus cabinets for the copper network.

Don't think so, but they all operate cable location services.

Mspec

1119 posts

Uber Geek
+1 received by user: 388

Subscriber

Reply # 1513797 15-Mar-2016 15:41

Bet the guy on the digger was, Hey guys I think we are in the s*it.

Ding Ding Ding Ding Ding : Ice cream man , Ice cream man

jnimmo

What does this tag do
966 posts

Ultimate Geek
+1 received by user: 200

Subscriber

Reply # 1513810 15-Mar-2016 16:11

noroad:

So a fiber cut that hits a feeder between the Central office and other areas can easily cause Dark Zones (no Internet, no mobile (if your mobile provider is using GPON backhaul)).

Yeah - but surely each of their aggregation points (not sure on the correct terminology for where the GPON switches live) should have diversity for the backhaul...

I've put a lot of thought into ISP selection and which ISPs have redundant paths but always assumed Enable would build some level of this into their network.

We're switching our secondary dedicated fibre connection to GPON at work shortly as consider this risk to be fairly low given the primary is dedicated and has physical path/PoP diversity.

1 | 2