Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.


BigPipeNZ

1170 posts

Uber Geek
+1 received by user: 778

Trusted
BigPipe

#145468 19-May-2014 09:38
Send private message

Surprised nobody has posted this here yet, but in the interest of transparency I think it is useful to note that chorus is having some major issues right now with provisioning and faults systems, which means no new connections can be processed and no faults can be logged or worked on.  This is affecting all ISPs who interact with Chorus - which is basically everybody.

Sucks for us as an ISP, and sucks for anybody in the midst of transitioning broadband providers as this will either be delayed, or, if not done properly, cannot be escalated and fixed.


Email from Chorus:

Good morning,   Chorus continue to experience an outage of various internal systems, including OFM, SPM, SSP, Chorus Portal and our inbound phone queues company-wide.   Currently we are unable to complete any provisioning or fault restoration work. Please phone through any medical emergencies via the following escalation channels;

We do not have an ETA for service restoration, however our technical teams are investigating and we endeavour to keep you updated


ETA: Confirmed that this is indeed infrastructure that Chorus and Telecom still share post demerger that has gone haywire, so it's definitely related.

Not trying to bag Chorus here, just trying to be transparent about what things aren't working and why.
Looks like the issue is (mostly) fixed now anyway (as at 5.50pm Monday 19.05).


(we're not impacted by the Telecom component of the outage as we are on totally separate systems, but we are impacted by this Chorus outage)




bigpipe.co.nz
https://www.facebook.com/BigPipeNZ
https://twitter.com/BigPipeNZ


Filter this topic showing only the reply marked as answer Create new topic

dan

dan
1134 posts

Uber Geek
+1 received by user: 105

Lifetime subscriber

  #1047831 19-May-2014 09:42
Send private message

hmm, got a line and broadband install happening in 20 mins, chrous tech didnt mention anything about it





BigPipeNZ

1170 posts

Uber Geek
+1 received by user: 778

Trusted
BigPipe

  #1047833 19-May-2014 09:45
Send private message

dan: hmm, got a line and broadband install happening in 20 mins, chrous tech didnt mention anything about it




I'm guessing here,  but physical work already planned can likely still go ahead (i.e. techs in vans doing stuff)  but faults cannot be logged  if he stuffs it up,  and new orders cannot be placed.

Also note that the techs aren't directly employed by Chorus, they are employed by a sub-contractor (e.g. Visionstream) who have their own systems, so potentially may not be affected as much.




bigpipe.co.nz
https://www.facebook.com/BigPipeNZ
https://twitter.com/BigPipeNZ


sbiddle
30853 posts

Uber Geek
+1 received by user: 9996

Retired Mod
Trusted
Biddle Corp
Lifetime subscriber

  #1047834 19-May-2014 09:45
Send private message

It's worth noting it's not actually a Chorus fault - it's a total meltdown of Telecom's internal network which has taken Telecom pretty much offline.

Many Chorus systems are still legacy and hosted by Telecom hence the problem.






Inphinity
2780 posts

Uber Geek
+1 received by user: 1184


  #1047836 19-May-2014 09:46
Send private message

Sounds like someone's gonna be crying themselves to sleep over their stuffup :P

DjShadow
4222 posts

Uber Geek
+1 received by user: 1322

ID Verified
Trusted
Subscriber

  #1047839 19-May-2014 09:49
Send private message

Would it be related to that upgrade in the weekend affecting mobile?

sbiddle
30853 posts

Uber Geek
+1 received by user: 9996

Retired Mod
Trusted
Biddle Corp
Lifetime subscriber

  #1047842 19-May-2014 09:51
Send private message

DjShadow: Would it be related to that upgrade in the weekend affecting mobile?


No idea. Telecom internally has pretty much ground to a halt with pretty much every system down from what I've heard.




 
 
 

Move to New Zealand's best fibre broadband service (affiliate link). Free setup code: R587125ERQ6VE. Note that to use Quic Broadband you must be comfortable with configuring your own router.
freitasm
BDFL - Memuneh
80646 posts

Uber Geek
+1 received by user: 41029

Administrator
ID Verified
Trusted
Geekzone
Lifetime subscriber

  #1047843 19-May-2014 09:52
Send private message

sbiddle: It's worth noting it's not actually a Chorus fault - it's a total meltdown of Telecom's internal network which has taken Telecom pretty much offline.

Many Chorus systems are still legacy and hosted by Telecom hence the problem.



Please follow the Telecom Meltdown here.




Referral links: Quic Broadband (free setup code: R587125ERQ6VE) | Samsung | AliExpress | Wise | Sharesies 

 

Support Geekzone by subscribing (browse ads-free), or making a one-off or recurring donation through PressPatron.

 


surfisup1000
5288 posts

Uber Geek
+1 received by user: 2159


  #1047844 19-May-2014 09:53
Send private message


Maybe you mean all subscribers connected via chorus?

I'm with UFF (via snap) and they seem just fine. 



sbiddle
30853 posts

Uber Geek
+1 received by user: 9996

Retired Mod
Trusted
Biddle Corp
Lifetime subscriber

  #1047847 19-May-2014 09:57
Send private message

surfisup1000:
Maybe you mean all subscribers connected via chorus?

I'm with UFF (via snap) and they seem just fine. 




The issue is nothing to do with Chorus provided ISP services. If you have internet through a Chorus wholesale connection you will not be affected unless you're with Telecom where you may be affected due to their DNS outage.

The issue is with Telecom's internal network affecting Chorus backend systems. Most of the Chorus systems are legacy products still run on Telecom infrastructure so are affected by this outage.


DarthKermit
5346 posts

Uber Geek
+1 received by user: 3317

Trusted

  #1047950 19-May-2014 12:29
Send private message

Have Telecom installed a master splitter yet? That'll fix everything.




Whatifthespacekeyhadneverbeeninvented?


BigPipeNZ

1170 posts

Uber Geek
+1 received by user: 778

Trusted
BigPipe

  #1048268 19-May-2014 17:49
Send private message

sbiddle:
surfisup1000:
Maybe you mean all subscribers connected via chorus?

I'm with UFF (via snap) and they seem just fine. 




The issue is nothing to do with Chorus provided ISP services. If you have internet through a Chorus wholesale connection you will not be affected unless you're with Telecom where you may be affected due to their DNS outage.

The issue is with Telecom's internal network affecting Chorus backend systems. Most of the Chorus systems are legacy products still run on Telecom infrastructure so are affected by this outage.



Yep, confirmed that this is the case. It's infrastructure that Chorus and Telecom still share post demerger that has gone haywire.


Not trying to bag Chorus here, just trying to be transparent about what things aren't working and why.


Looks like the issue is (mostly) fixed now.






bigpipe.co.nz
https://www.facebook.com/BigPipeNZ
https://twitter.com/BigPipeNZ


Filter this topic showing only the reply marked as answer Create new topic








Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.