Orcon Down?

Forums › 2degrees (including Slingshot, Orcon, Flip, Stuff Fibre, MyRepublic, 2talk and Vocus) › Orcon Down?

1 | 2

Sounddude

I fix stuff!
1928 posts

Uber Geek

Trusted

2degrees

Lifetime subscriber

#2070409 9-Aug-2018 09:21

sonyxperiageek:

Sounddude:

Yes we had a bad night. Sorry for any inconvenience :-(

So who dug through another cable (what happened) ?

Once we have written up our incident reports and communicated the root cause to our staff/major customers&partners. I will come and explain here what happened.

dt

1152 posts

Uber Geek
Inactive user

#2070798 9-Aug-2018 16:01

that doesn't sound good at all!

ronw

1222 posts

Uber Geek

#2070800 9-Aug-2018 16:06

I don't mind outage so much as the fact that the Orcon supplier Modems Netcom Nf4V do not automatically reconnect when problem is fixed. I have to reboot the Netcom and of course if I am away from home it stays down until I return which is a real pain

Nokia 7 Plus
Nexus 6P 32Gb
Nexus 6 Phone
Nexus 5 Phone
Nexus 7 2013 Tablet
Samsung TAB A 8"
Samsung TAB A 10"

& many Windows laptops, Desktops etc

hio77

12999 posts

Uber Geek

ID Verified

Trusted

Lizard Networks

#2070801 9-Aug-2018 16:06

dt:

that doesn't sound good at all!

Pretty standard practice for any high impact incident to have a root cause analysis... learn from your mistakes rather than hope for the best and repeat?

I see it as a good sign, or a bad sign for @sounddude if he did trip over the cables! :P

#include <std_disclaimer>

Any comments made are personal opinion and do not reflect directly on the position my current or past employers may have.

dt

1152 posts

Uber Geek
Inactive user

#2070808 9-Aug-2018 16:11

hio77:

Pretty standard practice for any high impact incident to have a root cause analysis... learn from your mistakes rather than hope for the best and repeat?

I see it as a good sign, or a bad sign for @sounddude if he did trip over the cables! :P

Oh yeah mate completely, we have the whole critical incident management process at work as well

Just meant sounded like something very serious.. possibly a tech error ? (sounddude tripping over a cable, while downing a beer!)

Nil Einne

469 posts

Ultimate Geek

#2070810 9-Aug-2018 16:17

Well whatever happened, an incident that lost service for a lot of customers both direct and indirect (main reason I'm following this is because I was affected on MyRepublic) for I think over an hour and at a fairly peak time of usage for the service is IMO by definition a serious incident whatever lessons there may be to learn.

KiwiSurfer

1456 posts

Uber Geek

ID Verified

Lifetime subscriber

#2070817 9-Aug-2018 16:25

The outage definitely wasn't that long for me. 45mins tops for me and after quite a long period of reliable service. IMHO thats acceptable for a residential grade connection. It never will be 100% with any RSP. No complaints from me.

Sharesies

Trade NZ and US shares and funds with Sharesies (affiliate link).

Sounddude

I fix stuff!
1928 posts

Uber Geek

Trusted

2degrees

Lifetime subscriber

#2070820 9-Aug-2018 16:31

hah no cables or beer involved :-)

Just not good practice to announce what happened on here before I tell the business :-)

Sounddude

I fix stuff!
1928 posts

Uber Geek

Trusted

2degrees

Lifetime subscriber

#2080408 28-Aug-2018 19:40

OK Guys

I did promise to come on here and tell you guys what happened.

Basically we hit a router bug. This bug stopped our line cards from forwarding packets....silently.....while leaving the control plane traffic to flow correctly. This meant the router didn't know anything was wrong and carried on as usual, however no traffic was passing over certian line cards.

In the network world this is worst case scneario, as it doesn't matter how much money you spend on redundancy (we have spent ALOT!) it won't help.

We have now upgraded the router to a new "golden" version which this is now fixed, so it shouldn't reoccur.

Any questions sing out :-) You can go back to your very stable Orcon UFB connections! :)

1 | 2