Yep voice services seemed to be down first..... again!

But they had that notification up before all the DDoS notifications were up. 

Wonder if it the same as last time - to many SIP Re-registrations overloading the system after an outage

Clint

Who would bother to DDOS a smaller NZ ISP??
You'd think that they have some form of functional DDOS protection as well?
Made me remember this cool site: https://globe.akamai.com/

According to the status page the DDOS was mitigated by 9.32 ( detected at 9.20) so not a bad response time

But Voice is still down :(

Clint

edit : Changed detected time to correct time

Yeah our voice playing up still as well.

@VygrNetworkMonkey able to give update ?

Thanks for the ping @xpd, 

Yes, the DDoS was mitigated and has not yet returned.

Voice issues however are still ongoing, and are being worked on by the Voice team.
I understand that the vendor has been called in also.

I'm not a voice person, so I can't give anything super helpful sorry - but if I hear anything, I'll be sure to let you know.

clinty:

 

Wonder if it the same as last time - to many SIP Re-registrations overloading the system after an outage

 

 

 

Clint

 

Supposedly they fixed that last time with "more capacity" right?....

VygrNetworkMonkey:

 

Thanks for the ping @xpd, 

 

Yes, the DDoS was mitigated and has not yet returned.

 

Voice issues however are still ongoing, and are being worked on by the Voice team.
I understand that the vendor has been called in also.

 

I'm not a voice person, so I can't give anything super helpful sorry - but if I hear anything, I'll be sure to let you know.

 

Cheers for that, long as its being worked on :D Means my folks cant ring with their PC problems, woohoo!!! ;)

chevrolux:

 

clinty:

 

Wonder if it the same as last time - to many SIP Re-registrations overloading the system after an outage

 

 

 

Clint

 

 

 

 

Supposedly they fixed that last time with "more capacity" right?....

 

yeah - possibly not enough - if the vendor is involved they may not have spec'd it out enough- or its another issue

clint

chevrolux:

 

clinty:

 

Wonder if it the same as last time - to many SIP Re-registrations overloading the system after an outage

 

 

 

Clint

 

 

 

 

Supposedly they fixed that last time with "more capacity" right?....

 

It's amazing hold badly those re registrations can hammer a system..

Hopefully Voyager can get it sorted soon, Voice issues can be quite some pain..

Ufone customers are affected as well, so I assume there is some tie-up between the companies. 

No phones for us all day at work. Makes a quiet day bar my cell which has had all incoming manually redirected to it. Yay me.

hio77:

 

It's amazing hold badly those re registrations can hammer a system..

 

Hopefully Voyager can get it sorted soon, Voice issues can be quite some pain..

 

If the provider decouples their registration services from the main SIP signalling switch and media servers then the number of re-registrations post-outage can be made to be irrelevant.

Registration isn't required for outbound calling (nor inbound termination if you have a static IP), so it should be possible for a good VoIP platform to continue working, as far as outbound calls are concerned. And also for inbound calls for those that have static IP trunking set up.

I have no knowledge as to whether there is any truth in the claim of a registration storm being at fault; I'm just saying that if the platform is engineered well, a registration storm should not stop outbound calling from working.

Hokay ... quick update: Both Inbound and Outbound calls are now proceeding, and being closely monitored
Still seeing issues with 0800, but thats being tackled.

It's not a registration storm/issue :)

Thank you very much for the update, Mr Monkey.  I hope your day is improving rapidly!

We would love to hear about a post-mortem (is that the right spelling?) in time so all NZ infrastructure providers can learn from the issue.