They sure are, its been down and up all morning

There is an ongoing outage affecting VNGS telephony services. ETR unknown.

Edit: I hate to link to the Herald but they've got more detail: https://www.nzherald.co.nz/nz/phone-woes-as-major-outage-hits-some-vodafone-call-centres-across-new-zealand/HKFZBRAYSB5L2MP3675NGVFRBA/

Juast had ths via our account manager:

 

 

 

This morning at around 5:40am we had an intermittent failure of one of our routers that interfaces to the Call Centre systems.  Engineers were trouble shooting the issue by 6.20. Services are beginning to restore as at 10.10am, however the teams are still working to resolve some intermittent call quality issues and stabilise services.

 

I want to sincerely apologise for any inconvenience that this is causing, both for you and your customers and wanted to explain a bit more about what has happened.

 

 

 

We targeted the issue to two devices in the network and manually failed these over to the redundant systems, bypassing traffic from the affected hardware.  Usually failover is automatic, but this did not occur because the fault was intermittent in nature and did not affect all customers.   Once traffic was bypassed, we restored the primary systems and then switched routed traffic back through them.

 

 

 

We take service uptime and customer service extremely seriously, as we know you and your customers rely on us. I’ll share  more details as we get to the bottom of the ongoing quality issues and a full  post incident analysis will be provided to you when complete.

 

 

 

Best Regards

 

Tony Baird

 

Director Wholesale and Infrastructure

 

 

 

The website is not accessible, and my clients are still having trouble - anyone else still having issues?

Clint

It's still up and down like a yo yo

We're still having issues, but not all sites.

Coincidentally, we also use a call centre on AWS (not through Vodafone, but a company out of Aus), and it's behaving realy badly today too. Still can't get into the self service portal either.

They have found the router (hopefully) and tech is on way to troubleshoot

VF:

 

This morning at around 5:40am..

 

It's nearly 2 pm.  That's a lot of customer SLAs breached...and some service credits.  Poor SDMs' weekend ruined writing PIRs

 

..we had an intermittent failure of one of our routers that interfaces to the Call Centre systems.....

 

No redundancy? No automatic fail-over?    Tut-tut Vodafone

3.30pm and VFX still down for me.

SIP looks like it's registered but I don't have any lines.

Fantastic, finally able to call through to the support line, but now out of hours... 😂

Telephony services should all be back up (resolved at 4ish) but management access (VFX/Xport) is still down.

We have an emergency change set for tonight to carry out further diagnostics with the vendor and to restore management.

Hi,

I have a WXC\VFX SIP trunk on FreePBX.
The SIP host is configured with pan.wxnz.net which is not DNS resolvable for me. Is this related to todays problems or is this some kind of migrations Vodafone are doing which requires a new SIP host?
The trunk was working as of yesterday.

Cheers

Looks like pan.wxnz.net is resolving this morning. I typically only use IP addresses anyway since Asterisk had it's long standing issue with DNS issues.

If it's not resolving for you add a DNS record for 182.154.16.150