Looks like they may have an issue with their DNS servers

Possibly they were the Vocus client that were subject to the DDOS that caused the issue last week

Clint

I suspect their DNS servers are under attack as they're both down. We saw this last week when Vocus went down.

Seems to be back now, Internet Banking and App working for me.

Kraven:

 

Seems to be back now, Internet Banking and App working for me.

DNS servers are still down for me.

michaelmurfy:

 

I suspect their DNS servers are under attack as they're both down. We saw this last week when Vocus went down.

 

Hmm.. looks like they are Redshield's customer. hopefully they will be back online soon!

➜  ~ host ib.kiwibank.co.nz

ib.kiwibank.co.nz is an alias for ib-ak-kiwibank-co-nz.cdn.red-shield.net.

ib-ak-kiwibank-co-nz.cdn.red-shield.net has address 185.71.230.171

back up for me at 13:30
:)

https://www.nzherald.co.nz/business/kiwibank-customers-fume-at-banking-access-outages/AH52YNWTVPIMTQWBNMSXVCH6Z4/ 

Seems to be busted again

$ whois kiwibank.co.nz | grep ns_ip4
ns_ip4_01: 103.140.196.86
ns_ip4_02: 131.203.107.34

$ dig kiwibank.co.nz @131.203.107.34

; <<>> DiG 9.9.5-3ubuntu0.19-Ubuntu <<>> kiwibank.co.nz @131.203.107.34
;; global options: +cmd
;; connection timed out; no servers could be reached

Hmm yeah strange, seems like there dns servers are broken :| (or ddos or some other kinda attack)

Seems like homepage works using cloudflare dns, though kinda slow. But www.ib.kiwibank.co.nz still a "ERR_NAME_NOT_RESOLVED"

Also, for those that can't load the site, here's the latest from 5pm.

Somebody has just added a couple more nameserver a few minutes ago

ns_name_01: ns7.kiwibank.co.nz
ns_ip4_01: 52.223.37.62

ns_name_02: ns8.kiwibank.co.nz
ns_ip4_02: 35.71.186.36

those IP addresses are Amazon ones, Route53 to the rescue maybe.

oh damn they did.

I feel sorry for the IT guys trying to fix this behind the scenes, going at it for over 12 hours now.

I've been there (albeit in a much smaller scale) and it sucks, sucks big time. As I'm sure most here have.

As somebody who works in this industry on customer facing banking platforms I can tell you now the stress is real.

I've been there, done that with a long production outage and had to take risks, deal with beurocrats telling me "this needs a production change", deal with the stress of customers not having access to their money and the contact centre and staff getting slammed and abused by customers all while being asked by a whole bunch of staff members "what is the ETA" and all while in a meeting with service managers while I actively work on the production issue in-front of me.

It takes a crazy amount of multitasking, keeping your cool and self management. I also remember once being in the office a number of years back with the CTO a few desks down, vaping at our desks during a P1 incident as we couldn't leave. Nobody cared and that production incident was the worst in my career (full loss of network, mainframe and banking services).

The stress is on a whole other level in the banking industry because customers don't have access to their money + you're dealing with money.

So I really feel for the Kiwibank guys working on this (I assume P1) incident - #hugops

Yeah.. the stress is real, for sure. But not only for those who work on banking platforms.

The demand for uptime has been increasing and not only by the customers, but from the business itself too. I work for a SaaS company and if we ever go down, the stakeholders go craaazy... mainly because we almost never go down.. so they're used to that good uptime.

Anyways... keep it up Kiwibank IT guys.. You'll get this fixed!

Looks to all be working again this morning.

Looks like they ended up bailing on the original 2 name servers and are now running on the 2 new ones they added last night.