https://www.westpac.co.nz/foreign-exchange/

A bank or a foreign exchange place (often found in malls).  Shop around for the best deal, and watch the fees.  

Thanks. :) 

Last time I tried it was around covid and no banks were taking foreign cash even though the cash itself was pre Covid vintage.

Wasn't sure if banks were taking it now or not.

Edit: my bank, ANZ, does not. So now I'm looking for a bank that does without needing an account with them.

The banks, Bar westpac to their own customers. Have all withdrawn cash and tell you to use the eftpos machine at arrival if you need it. Or cash up a debit card.

You're now at the mercy of the last remaining sole exchange traders and their fees.