You don't get GST refunds if you don't charge international customers GST.
This differs from the advice I was given from the IRD.
If you don't charge GST why would you get a refund? I think GST refunds come from buying products for business use, GST is refunded on those.
Normally (as in, if you do all your business in NZ) you would only get refunds if your expenses are greater than your income. Which could happen when you start up a business.
But, if you are invoicing zero rated services (like businesses outside of NZ), then you will be invoicing with 0% GST. Which means the GST on your business expenses are still deductible, but since the GST you have invoiced is 0, the end result is the IRD paying you back the business expense GST.