I can report that Kiwiwebhost is back now. 

And I can report our landline phone came on🙂

Super effort everyone involved.

The country’s third-largest internet provider, Vocus NZ, says a cyber attack on one of its customers and a subsequent error was the reason many internet users were knocked offline around lunchtime on Friday.
Vocus owns the Orcon, Slingshot and Stuff Fibre internet brands and also provides the internet infrastructure for Sky Broadband which is also believed to have been impacted.

The problems began shortly after 1pm, but appear to have been resolved at about 2pm.

Vocus NZ initially reported that its network had been impacted by a denial of service (DDos) attack but chief executive Mark Callander later clarified that it was not its own network but a customer that had been attacked.
But that process had not gone as it should, instead impacting its own service to other customers.

https://www.stuff.co.nz/business/300398873/cyberattack-on-customer-and-an-error-took-down-nzs-thirdlargest-isp

wow, what  day.....

I think I am going to need a few beers.

https://www.vocus.co.nz/networkstatus for more official updates 

Sounddude:

wow, what  day.....

 

I think I am going to need a few beers.

Sounddude:

 

wow, what  day.....

 

 

 

I think I am going to need a few beers.

 

not one of your customers ... but totally understand ... ill buy you one of those when we are back at lvl 2!

Sounddude:

 

wow, what  day.....

 

I think I am going to need a few beers.

 

https://www.vocus.co.nz/networkstatus for more official updates 

 

Slingshot IPv6 still not restored at 4pm.... Northern Hawkes Bay.

Nothing worse than a fix causing bigger and more issues :(

Sometimes I hate computers lol

Clint

Sounddude:

 

wow, what  day.....

 

 

 

I think I am going to need a few beers.

 

 

 

https://www.vocus.co.nz/networkstatus for more official updates 

 

well done!

though it does reiterate why redundancy providers are needed.

Status page says resolved at 2:30pm. Support phone line started working around 3:20pm. Got through at 4:00pm - not working? restarted modem? how about factory reset? Obviously I didn't reset, just kept rebooting hundred times. We just got online at 4:15pm. I wish the ISP were telling truth on their status pages. Orcon, Stuff-Fibre, Slingshot and Flip - I'm talking about you.

chevrolux:

 

Yep obviously someone tripped over a pretty big cable!!

 

Or maybe the cleaner unplugged it to do the vacuuming 

 

Not as funny as you think.

The Cleaner @ RNZ in Timaru in the Mid 80's used to unplug the IBM Server, do her vacuuming then plug it back in.

Took a few days to work out why the server was off in the morning!

@sounddude did you change the arbor firewall rule ? 😝

Thanks @sounddude for the updates.  I've got several thousand WFH staff who are BYOD on their own internet, a huge number I'm sure are Vocus behind the scenes.  It was a tricky disruption to manage.  

So what can we learn from this event, and I'm not talking about human error or how this specific event was managed?

Mod Edit: Removed link. Post your comments here if you want them known, please don't link to external sources.