E3xtc: The amp is just one that is plugged into the outlet/wall plate

That sounds like it might be the power supply injector to a mast head amplifier, which is good.  You want to take the signal at the aerial and make it stronger.

If it's honestly just some amp that sits at the wall plate though then this is bad, as you'd be amplifying a crap signal to make a really big crap signal that way.

You sound like you're in a dodgy reception area for some of the frequencies.  Unfortunately that's your fault and not TVNZ's.  Yes, if it was me I would remove the splitter for testing purposes and connect the joints to make one straight length to your TV.

Really though i'd be looking to invest in a high gain UHF aerial and run new RG6 coax cabling down to your TV.  If you are still in a weak signal area then you might need a mast head amplifier and really a good site survey too by a professional to check if there are any areas on your property that get the signal better.